Configuring and Building Version 13 of Unicon

Jafar Al-Gharaibeh and Clinton Jeffery


Unicon Technical Report #21
April 16, 2019

Abstract

Unicon Version 13 features a fully automatic configuration and build mechanism based on GNU autoconf. The new model has fewer manual steps, and configuration parameters selected via command line options instead of preprocessor #define's. This report describes the configuration system.

1. Background

There presently are implementations of Unicon only for platforms that host UNIX-compatible shell scripting and configuration tools such as autoconf. This includes most versions of UNIX such as Linux, *BSD and MacOS, as well as Microsoft Windows 10 via its Ubuntu shell or MSYS and MinGW development environment.

All implementations of Unicon are obtained from the same source code, using conditional compilation and defined constants to select and configure platform-dependent code. Appropriate values for these constants are determined automatically by a configure shell script, as is common on UNIX-based software. The configure script is run automatically the first time a build is performed, or when any configuration files may have been changed. The configure script can also be run manually by typing ./configure or on windows sh configure A number of features in a Unicon build are optional, selected by command line arguments to the configure script. Consequently, installing Unicon on a new platform is largely a matter of selecting configuration options (usually the defaults are adequate), and typing make.

The purpose of this document is to describe aspects of the process of configuring Version 13 of the Unicon source code, in the event that optional arguments are needed or if make does not work out of the box. For example, make might not work on a platform on which Unicon has not previously been installed.

Building Unicon with a new C compiler on an operating system where Unicon has previously been installed may be a fairly simple task and ideally will be handled by the configure script. Porting Unicon to a new operating system or to an environment in which the configure script does not run is more complex; read this report carefully before undertaking such a project.

2. Requirements

C Data Sizes

• char must be 8 bits.
• int must be 32 or 64 bits.
• long and pointers must be 32 or 64 bits.
• All pointers must be the same length.
• long and pointers should be the same length.

The C Compiler

Memory

File Space

3. File Structure

               |-bin------    executable binaries and support files
               |-config---    configurations
     |-unicon--|-src------    source code
               |-tests----    tests
               |-uni------    unicon

               |-editor---
               |-port-----
     --config--|-scripts--
               |-unix-----
               |-win32----

               |-asm-------   assembler (co-expression) source
               |-common----   common source
               |-gdbm------   GDBM local indexed file library source
               |-h---------   header files
               |-iconc-----   Optimizing compiler source
     -src------|-icont-----   VM bytecode translator source
               |-lib-------   libraries
               |-preproc---   C preprocessor source
               |-rtt-------   run-time translator source
               |-runtime---   run-time support
               |-xpm-------   XPM image-file format support

               |-bench-----   benchmarks
               |-calling---   calling C functions from Unicon
               |-coexpr----   co-expressions
               |-container-   container
               |-general---   general tests
               |-graphics--   graphics facilities
               |-ipl-------   Icon program library
               |-lib-------   library
     -tests----|-mt--------   multi-thread?
               |-pattern---   pattern matching
               |-posix-----   POSIX facilities
               |-preproc---   C preprocessor tests
               |-samples---   short sample programs
               |-special---   special features
               |-thread----   threads
               |-udb-------   debugger
               |-unicon----   unicon language features

               |-3d--------   3D graphics class libraries
               |-cint------   C interface support
               |-gprogs----   graphics programs
               |-gui-------   graphical user interface class library
               |-ide-------   ui integrated development environment
               |-ivib------   version 1 improved visual interface builder (deprecated)
               |-iyacc-----   Icon yacc, a parser generator
               |-lib-------   general purpose Unicon libraries
               |-monvis----   monitoring and visualization tools
               |-native----   native something
     -uni------|-parser----   a unicon parser library
               |-progs-----   programs
               |-shell-----   a simple shell
               |-udb-------   Unicon debugger
               |-ulex------   Unicon lexical analyzer generator
               |-unicon----   Unicon translator
               |-unidep----   Unicon dependency generator
               |-unidoc----   Unicon documentation generator
               |-util------   utilities
               |-xml-------   XML library

4. Parameters and Definitions

Makefile	updated from Makefile.in
Makedefs	generated from Makedefs.in. It holds parameters and symbols used with the C language compiler.
Makedefs.uni	generated from Makedefs.uni.in. It holds parameters symbols used with the Unicon language translator and compiler.

In the unfortunate event that you need to build Unicon on a system where the configure script does not run, src/h/define.h can still be used and you can still manually copy in or tweak makefiles or the Makedefs files and such, however this should be considered a last resort. An example src/h/define.h for a "vanilla" 32-bit platform is:

#define HostStr "new host"
#define NoCoexpr
#define PORT 1

5. Configuring Unicon for a UNIX Platform

  ./configure
  make

The biggest issue with building Unicon on a new machine is usually the question of what optional language features are required, and what packages of C libraries and header files must be built or installed in order for Unicon to provide those features.

5.1 Configure Arguments

./configure --help

  --disable-graphics      No graphics subsystem
  --disable-graphics3d    No 3D graphics support
  --disable-concurrency   No concurrent thread support
  --disable-pattern       No pattern type support
  --disable-database      No database support
  --disable-ssl           No SSL support
  --disable-audio         No audio support
  --disable-voip          No VOIP support
  --disable-plugins       No loadfunc or plugins support
  --disable-iconc         Build Unicon Compiler (Iconc/Uniconc)
  --enable-iconcurrency   enable thread support in Unicon Compiler
  --enable-ovld           enable operator overloading
  --enable-udbtools       enable Unicon debugger tools
  --enable-progs          enable Unicon programs
  --enable-verbosebuild   Show full CC build lines with all compiler arguments
  --enable-thin           Do a minimalist build disabling non critical
                          features

  --with-zlib[=DIR]       Use zlib package (DIR: custom library path)
  --with-xlib[=DIR]       Use xlib package (DIR: custom library path)
  --with-freetype[=DIR]   Use freetype package (DIR: custom library path)
  --with-Xft[=DIR]        Use Xft package (DIR: custom library path)
  --with-jpeg[=DIR]       Use jpeg package (DIR: custom library path)
  --with-png[=DIR]        Use png package (DIR: custom library path)
  --with-opengl[=DIR]     Use opengl package (DIR: custom library path)
  --with-ftgl[=DIR]       Use ftgl package (DIR: custom library path)
  --with-ogg[=DIR]        Use ogg package (DIR: custom library path)
  --with-SDL[=DIR]        Use SDL package (DIR: custom library path)
  --with-smpeg[=DIR]      Use smpeg package (DIR: custom library path)
  --with-openal[=DIR]     Use openal package (DIR: custom library path)
  --with-jvoip[=DIR]      Use jvoip package (DIR: custom library path)
  --with-odbc[=DIR]       Use odbc package (DIR: custom library path)
  --with-pthread[=DIR]    Use pthread package (DIR: custom library path)
  --with-ssl[=DIR]        Use ssl package (DIR: custom library path)

5.2 Example: Enabling Graphics Facilities

In order to build Unicon with these X Window functions, you will need to know what library or libraries are required to link in the X facilities into C programs; this library information is needed when iconx is built and when iconc links a compiled Unicon executable. Normally, the answer will be -lX11, but on some platforms additional libraries or alternate paths are required. Consult appropriate manuals to find out what libraries are needed.

The configure will find the X11 library if your platform has it at one of the most common locations. If not, there are two possibilities. If it is installed at an unusual location, you may end up using a command line option to configure to specify its location:

./configure --with-xlib=/my/unusual/location

Historically, the files xiconx.mak and xiconc.def, if they are present, were used during Unicon configuration to supply non-default library information to the interpreter and the compiler. Although these files are not normally used any more, the description below is retained it is needed on some non-UNIX build at some point.

If a platform requires an additional pseudo-terminal library and a BSD-compatibility package in order to link X applications, you would edit the Makedefs top level files and change the XLIB line to include the new -lbsd library as follows:

     XLIB= -L../../bin -lX11 -lpt -lbsd

     #define ICONC XLIB "-lX11 -lpt -lbsd"

For the Unicon compiler, this option is added via the COpts macro in define.h for your configuration. The COpts macro must define a quoted C string. For the interpreter, the option is added to the CFLAGS argument of the common.hdr, icont.hdr, runtime.hdr, and xpm.hdr Makefile headers for your configuration.

6. Configuring Unicon for an MS Windows Platform

• the 64-bit Mingw version of GCC, when the version 1 of MSYS UNIX-like tools and shell are installed.
• the Windows Ubuntu shell allows a Unix-like Unicon to be built that runs well other than known issues in the threads facilities.

For the 64-bit Mingw version of GCC, the configuration and build step is performed by invoking

  make WUnicon64
  make

  sh configure --build=x86_64-w64-ming32 --disable-iconc

This begs the question of how to build Unicon's iconc on Windows. Undoubtedly, many Bothan spies will die to bring us this information. A Windows iconc port was performed awhile back, but building it does not occur by default.

The rest of this section should speculate on what it would take to build Unicon using the last-known-but-currently-unsupported Windows Compiler, Microsoft's C/C++ compiler.

Unicon's autoconf-based configuration script, configure, has not been tested as to whether it would find or use a Microsoft C compiler if it was on the path. It is thus expected that a traditional Icon-style manual configuration process would be required. Feel free to update the files in the config/win32/msvc/ directory for us and have a go at it.

An MS Windows configuration includes Makefiles, batch scripts, and response files for linking. These files should be modified for the new compiler as appropriate. According to config/win32/msvc/status, the last time MSVC was used in a built was 2014 under Visual C++ 18.0 from Visual Studio 12.0.

7. Configuring Unicon for a New Operating System

   ...
#if NT
   ...
/* code for MS Windows */
   ...
#endif
#if UNIX || VMS
   ...
/* code for UNIX and VMS */
   ...
#endif
   ...

Logical conditionals with #if are used instead of defined or undefined names with #ifdef to avoid nested conditionals, which become very complicated and difficult to understand when there are several alternative operating systems. Note that it is important not to use #ifdef in place of #if, since all the names are defined.

The file define.h for a different operating system should initially contain

#define PORT 1

Note: The PORT sections contain deliberate syntax errors (so marked) to prevent sections from being overlooked during configuration. These syntax errors must, of course, be removed before compilation.

To make it easy to locate places where there is code that may be dependent on the operating system, such code usually is bracketed by unique comments of the following form:

/*
* The following code is operating-system dependent.
*/
   ...
/*
* End of operating-system specific code.
*/

Look through some of the files for such segments to get an idea of what is involved. Each segment contains comments that describe the purpose of the code. In some cases, the most likely code or a suggestion is given in the conditional code under PORT. In some cases, no code will be needed. In others, code for an existing operating system may suffice for the new one.

In any event, code for the new operating system name must be added to each such segment, either by adding it to a logical disjunction to take advantage of existing code for other operating systems, as in

#if MSDOS || UNIX || PORT
   ...
#endif

#if VMS
   ...
#endif

#if PORT
   ...
   /* code for the new operating system */
   ...
#endif

8. Trouble Reports and Feedback

Unicon Project, c/o Clinton Jeffery
Department of Computer Science
The University of Idaho
875 Perimeter Drive
Moscow, ID 83844-1010
U.S.A.

(208) 885-4789 (voice)

Once you have completed your installation, please send us copies of any files that you modified so that we can make corresponding changes in the central version of the source code. Once this is done, you can get a new copy of the source code whenever changes or extensions are made to the implementation. Be sure to include documentation on any features that are not implemented in your installation or any changes that would affect users.

References

Appendix A -- Configuration Parameters and Definitions

C Compiler Considerations

typedef huge void *pointer;

#define PointerDef

Sometimes computing the difference of two pointers causes problems. Pointer differences are computed using the macro DiffPtrs(p1, p2), which has the default definition:

#define DiffPtrs(p1, p2) (word)((p1)-(p2))

This definition can be overridden in define.h. For example, at one time Microsoft C used

#define DiffPtrs(p1, p2) ((word)(p1)-(word)(p2))

Character Set

#define EBCDIC 1

Data Sizing and Alignment

WordBits    (default: 32)
IntBits     (default: WordBits)

If your C library expects doubles to be aligned at double-word boundaries, add

#define Double

The word alignment of stacks used by co-expressions is controlled by

StackAlign   (default: 2)

Most computers have downward-growing C stacks, for which stack addresses decrease as values are pushed. If you have an upward-growing stack, for which stack addresses increase as values are pushed, add

#define UpStack

Floating-Point Arithmetic

     Big        (default: 9007199254740092.)
     LogHuge    (default: 309)
     Precision  (default: 10)

Large Integers

     #define NoLargeInts

Storage Region Sizes

     MaxAbrSize   (default: 2% of available memory)
     MaxStrSize   (default: 2% of available memory)

The sizes for Unicon's main interpreter stack and co-expression stacks also can be set:

     MStackSize   (default: 0.5% of available memory)
     StackSize    (default: 0.02% of available memory)

Finally, a list used for pointers to strings during garbage collection, can be sized:

     QualLstSize  (default: 5000)

Allocation Sizing

malloc() is used to allocate space for Unicon's storage regions. This limits region sizes to the value of the largest unsigned int. Historically, some platforms provided alternative allocation routines for allocating larger regions. To change the allocation procedure for regions, add a definition for AllocReg to define.h. For example, an ancient huge-memory-model implementation of Icon for Microsoft C used the following:

     #define AllocReg(n) halloc((long)n, sizeof(char))

     #define malloc lmalloc

     #define AllocType unsigned long int

     #define MaxBlock n

File Name Suffixes

#define SourceSuffix  (default: ".icn")
#define U1Suffix      (default: ".u1")
#define U2Suffix      (default: ".u2")
#define USuffix       (default: ".u")
#define IcodeSuffix   (default: "")
#define IcodeASuffix  (default: "")

Paths

     #define TargetDir SourceDir

Command-Line Options

Similarly, the error message produced for erroneous command lines is defined by TUsage for icont and CUsage for iconc. The default values, which should correspond to the value of IconOptions, are in config.h, but may be overridden by definitions in define.h.

If your C library includes getopt(), you can add

     #define SysOpt

Host Identification

     #define HostStr "identification"

Directory Reading

     #define ReadDirectory

Keyboard Functions

     #define KeyboardFncs

You can also define KeyboardFncs if you supply your own keyboard functions; see src/runtime/rlocal.r for examples.

Dynamic Loading

     #define LoadFunc

Co-Expressions

     #define NoCoexpr

X Window Facilities

Some early versions of X Window Systems, notably X11R3, do not support the attribute iconic. If this is the case for your platform, add

     #define NoIconify

Compiler Options

     #define CComp "name"

     #define CComp "gcc"

     #define COpts "options"

     #define COpts "-O"

     #define NoRanlib

Dynamic Hashing Constants

HSlots	Initial number of hash buckets; it must be a power of 2
HSegs	Maximum number of hash bucket segments
MaxHLoad	Maximum allowable loading factor
MinHLoad	Minimum loading factor for new structures

Every set or table starts with HSlots hash buckets, using one bucket segment. When the average hash bucket exceeds MaxHLoad entries, the number of buckets is doubled and one more segment is consumed. This repeats until HSegs segments are in use; after that, structure still grows but no more hash buckets are added.

MinHLoad is used only when copying a set or table or when creating a new set through the intersection, union, or difference of two other sets. In these cases a new set may be more lightly loaded than otherwise, but it is never less than MinHLoad if it exceeds a single bucket segment.

For all machines, the default load factors are 5 for MaxHLoad and 1 for MinHLoad. Because splitting or combining buckets halves or doubles the load factor, MinHLoad should be no more than half MaxHLoad. The average number of elements in a hash bucket over the life of a structure is about (2/3)*MaxHLoad, assuming the structure is not so huge as to be limited by HSegs. Increasing MaxHLoad delays the creation of new hash buckets, reducing memory demands at the expense of increased search times. It has no effect on the memory requirements of minimally-sized structures.

HSlots and HSegs interact to determine the minimum size of a structure and its maximum efficient capacity. The size of an empty set or table is directly related to HSegs+HSlots; smaller values of these parameters reduce the memory needs of programs using many small structures. Doubling HSlots delays the onset of the first structure reorganization until twice as many elements have been inserted. It also doubles the capacity of a structure, as does increasing HSegs by 1.

The maximum number of hash buckets is HSlots*(2^(HSegs-1)). A structure can be considered "full" when it contains MaxHLoad times that many entries; beyond that, lookup times gradually increase as more elements are added. Until a structure becomes full, the values of HSlots and HSegs do not affect lookup times.

For machines with 16-bit ints, the defaults are 4 for HSlots and 6 for HSegs. Sets and tables grow from 4 hash buckets to a maximum of 128, and become full at 640 elements. For other machines, the defaults are 8 for HSlots and 10 for HSegs. Sets and tables grow from 8 hash buckets to a maximum of 4096, and become full at 20480 elements.

Implementation Debugging Code

#define DeBugTrans  /* debugging code for the translator in icont */
#define DeBugLinker /* debugging code for the linker in icont */
#define DeBugIconx  /* debugging code for the run-time */

The debugging code for the translator consists of functions for dumping symbol tables (see icont/tsym.c). These functions are rarely needed and there are no calls to them in the source code as it is distributed.

The debugging code for the linker consists of a function for dumping the code region (see icont/lcode.c) and code for generating a debugging file that is a printable image of the icode file produced by the linker. This debugging file, which is produced if the option -L is given on the command line when icont is run, may be useful if icode files are incorrect.

The debugging code for the executor consists of a few validity checks at places where problems have been encountered in the past. It also provides functions for dumping Unicon values. See runtime/rmisc.r and runtime/rmemmgt.r.

When installing Unicon on a new operating system, it is advisable to enable the debugging code until Unicon is known to be running properly. The code produced is innocuous and adds only a few percent to the size of the executable files. It should be removed by deleting the definition listed above from define.h as the final step in the implementation for a new operating system.

Appendix B -- Implementing a Co-Expression Context Switch

The routine coswitch() is needed for context switching. This routine requires assembly language, since it must manipulate hardware registers. It either can be written as a C routine with asm directives or directly as an assembly language routine.

Calls to the context switch have the form coswitch(old_cs,new_cs,first), where old_cs is a pointer to an array of words (C longs) that contain C state information for the current co-expression, new_cs is a pointer to an array of words that hold C state information for a co-expression to be activated, and first is 1 or 0, depending on whether or not the new co-expression has or has not been activated before. The zeroth element of a C state array always contains the hardware stack pointer (sp) for that co-expression. The other elements can be used to save any C frame pointers and any other registers your C compiler expects to be preserved across calls.

The default size of the array for saving the C state is 15. This number may be changed by adding

#define CStateSize n

/*
* coswitch
*/
coswitch(old_cs, new_cs, first)
long *old_cs, *new_cs;
int first;
{
   ...
   /* save sp, frame pointers, and other registers in old_cs */
      ...
   if (first == 0) { /* this is first activation */
         ...
      /* load sp from new_cs[0] and clear frame pointers */
         ...
      new_context(0, 0);
      syserr("new_context() returned in coswitch");
      }
   else {
         ...
      /* load sp, frame pointers, and other registers from new cs */
         ...
      }
   }

To test your context switch, run the programs in tests/general/coexpr.lst. Ideally, there should be no differences in the comparison of outputs.

If you have trouble with your context switch, the first thing to do is double-check the registers that your C compiler expects to be preserved across calls -- different C compilers on the same computer may have different requirements.

Another possible source of problems is built-in stack checking. Co-expressions rely on being able to specify an arbitrary region of memory for the C stack. If your C compiler generates code for stack probes that expects the C stack to be at a specific location, you may need to disable this code or replace it with something more appropriate.

Appendix C -- Obtaining Packages to Enable Optional Unicon Features

Linux

sudo apt-get install lib

sudo dnf install lib

Windows

MacOS