**********************
Jam-based Build System
**********************

This document describes the Jam-based build system for Open Dylan.

Introduction
============

The purpose of the Open Dylan ``build-system`` is to
coordinate the final stages in the building of a Dylan library project.
The Open Dylan compiler takes a Dylan library in the form of
Dylan source files, and directly generates x86 or PPC machine language
object files (in either COFF or ELF format, depending on the target
platform). These object files need to be linked together to construct
either an executable object file, or a loadable dynamically-linked
library (also called a shared library). This link needs to performed by
an external tool such as the Microsoft or GNU linker. The
``build-system`` component, controlled by a user-specified script file,
directs the execution of these external tools.

Why Jam-based?
==============

Use of the Jam scripting language increases the flexibility of
Open Dylan for users, and makes it more maintainable by
removing hard-coded assumptions spread throughout various layers of the
compiler. Previous versions of the Open Dylan ``build-system``
component permitted users to choose either the Microsoft or GNU Binutils
linker on the Win32 platform, and always used the GNU Binutils ELF
linker on Linux platforms. New versions of these tools require
modifications that go beyond the current flexibility of the
``build-system`` component, as will new Open Dylan target
platforms.

Though the logic of ``build-system`` (and a former companion library,
``linker-support``) was hard-coded, it did allow a limited amount of
parameterization using script files. A typical parameterization, from
the Microsoft linker script, looked like this::

    #
    linkdll
    link /NODEFAULTLIB /INCREMENTAL:NO /PDB:NONE /NOLOGO \
      /ENTRY:$(mangled-dllname)Dll@12 -debug:full -debugtype:cv \
      /nologo /dll /out:$(full-dll-name) kernel32.lib \
      @$(dllname).link $(objects) $(c-libs) /base:$(base) \
      /version:$(image-version) $(linkopts)

Though these script files were poorly documented and insufficiently
flexible, they did inspire the introduction of a real scripting language
to direct the final stages of compilation in Open Dylan.

`Jam`_ is a build tool designed by Christopher Seiwald, founder of
`Perforce Software`_. It is similar in some ways to
``make``, the traditional Unix build tool. However, instead of using
only simple declarative rules to define build targets and the
dependencies between them, Jam contains a full scripting language,
allowing build script authors to define high-level build instructions
that match particular applications. The Jam program also includes
Jambase, a library of rules (functions) for building executables and
libraries from C, C++, and Fortran sources.

The original Jam tool is a standalone program written in C and YACC.
Peter Housel re-implemented the Jam language interpreter and build logic
as a reusable Dylan library for use in the Open Dylan
``build-system``.

Choosing Build Scripts
======================

Normally you can simply use the build script supplied with Open Dylan
that corresponds to the external linker you will be using. The
supplied build scripts include the following:

``x86-win32-vc6-build.jam``
    Build script for Microsoft Visual C++ 6.0.
``x86-win32-vc7-build.jam``
    Build script for Microsoft Visual C++ .NET.
``x86-linux-build.jam``
    Build script for x86 Linux systems using gcc.

The default build script is ``platform-name-build.jam``. You can select
a different build script from the Link page of the Environment Options
dialog in the IDE, or using the ``-build-script`` option on the console
compiler or console environment command-line.

Build scripts are written using the Jam script language, as described in
the `Jam manual page`_. Most Open Dylan build scripts ``include`` the
``mini-jambase.jam`` file, which contains excerpts from the `Jambase`_
file included with Perforce Jam and described in the `Jambase Reference`_.
They can also make use of additional built-in rules defined by the
Open Dylan build system, as described in `Additional Built-In Jam Rules`_
and `Built-In Jam Variables`_.

How the Compiler Uses the Build System
======================================

When you compile a library, the Open Dylan compiler constructs
a new ``build`` directory and places the generated object files in it.
It also constructs a text file called ``dylanmakefile.mkf`` to be read
by the build system. This file contains information imported from the
original LID or HDP project file, as well as information generated
during compilation. Here is a sample ``dylanmakefile.mkf``, in this case
the one generated for the ``build-system`` component itself:

::

    comment:        This build file is generated, please don't edit
    library:        build-system
    base-address:   0x63F20000
    major-version:  0
    minor-version:  0
    library-pack:   0
    compilation-mode:       tight
    target-type:    executable
    files:  library
            paths
            variables
            build
            jam-build
    used-projects:  functional-dylan
            dummy
            ..\functional-dylan\
            io
            dummy
            ..\io\
            system
            dummy
            ..\system\
            file-source-records
            dummy
            ..\file-source-records\
            release-info
            dummy
            ..\release-info\
            dfmc-mangling
            dummy
            ..\dfmc-mangling\
            jam
            dummy
            ..\jam\
    all-c-libraries: advapi32.lib
            shell32.lib

External files are used to communicate with the build system in order
for the information to persist between invocations of the compiler. On
the Win32 platform, ``dylanmakefile.mkf`` files are also copied into the
``lib`` directory on installation so that other libraries can link
against the actual DLL (whose name might not be identical to the library
name).

When Open Dylan needs to link a project, it calls the
``build-system``, passing the name of the build directory and a list of
targets to be built. The build system reads the ``dylanmakefile.mkf``
file and builds the targets accordingly.

The Open Dylan compiler's project manager expects the build
script to define the following pseudo (``NotFile``) targets:

``exports``
    Describe exports.
``unify-dll``
    Describe unify-dll.
``dll``
    Link the project as a dynamically-linked library.
``unify-exe``
    Describe unify-exe.
``exe``
    Link the project as an executable program.
``release``
    Describe release.
``clean-all``
    Remove build products in the top-level project, and in all of the
    non-system libraries that it uses.
``clean``
    Remove build products in the top-level project.

Automatically-invoked Jam Rules
===============================

When the build system reads a ``dylanmakefile.mkf`` file, it invokes
several of the Jam rules (functions) defined in the user's build script.
These rules in turn register the necessary targets and their
dependencies with the Jam build mechanism.

All of the rules described below take *image* as their first parameter;
this is a list whose first element is the library name (from the
``Library:`` keyword of the ``.mkf`` file) and whose optional second
component is the base name of the executable or shared library (from the
``Executable:`` keyword of the ``.mkf`` file).

``DylanLibrary *image* : *version* ;``
    Link a Dylan library as a shared library or executable image. This is
    always the first rule invoked for a given library, and it is usually
    charged with establishing the library target and setting global and
    target-specific variables.

    The *version* argument normally contains two components, the first
    obtained from the ``Major-version:`` keyword of the ``.mkf`` file, and
    the second from the ``Minor-version:`` keyword.

``DylanLibraryLinkerOptions *image* : *options* ;``
    Add the given options to the link command line of the shared library and
    executable images. The link options provided in the ``Linker-options:``
    keyword of the ``.mkf`` file are expanded using the usual Jam variable
    expansion rules before being passed to this rule. (This allows
    ``Linker-options:`` keywords in LID and HDP files to refer to
    platform-specific variables such as ``$(guilflags)``).

``DylanLibraryBaseAddress *image* : *address* ;``
    Set the base address of the shared library. The compiler-computed base
    addresses are probably only usable on the Win32 platform.

``DylanLibraryCLibraries *image* : *libraries* ;``
    Link C (or other externally-derived) libraries into the shared library.
    The link options provided in the ``C-libraries:`` keyword of the
    ``.mkf`` file are expanded using the usual Jam variable expansion rules
    before being passed to this rule.

``DylanLibraryCObjects *image* : *objects* ;``
    Link C (or other externally-derived) object files into the shared
    library.

``DylanLibraryCSources *image* : *sources* ;``
    Link C source files into the shared library.

``DylanLibraryCHeaders *image* : *headers* ;``
    This rule normally does nothing. The ``C-header-files:`` HDP/LID file is
    normally used to ensure that files of various sorts (not just C header
    files) are copied into the build directory.

``DylanLibraryRCFiles *image* : *rcfiles* ;``
    Link Win32 resource files into the shared library and executable.

``DylanLibraryJamIncludes *image* : *includes* ;``
    Include other Jam files into the build definition. This is typically
    used via the ``jam-includes:`` keyword in the HDP/LID file. It is
    useful for setting up extensions to library or include search
    paths.

``DylanLibraryUses *image* : *library* : *dir* ;``
    Link other Dylan libraries into the shared library. The *library*
    argument gives the name of the other library, and the *dir* argument
    gives the name of the other library's build directory. If *dir* is
    ``system``, then the library is an installed system library.

Additional Built-In Jam Rules
=============================

The build system defines the following additional built-in rules.

``IncludeMKF *includes* ;``
    Read each of the given ``.mkf`` files and invoke Jam rules as described
    in `Automatically-invoked Jam Rules`_.

``DFMCMangle *name* ;``
    Mangle the given *name* according to the Open Dylan compiler's
    mangling rules. If *name* has a single component, it is considered to be
    a raw name; if there are three components they correspond to the
    variable-name, module-name, and library-name respectively.

Built-In Jam Variables
======================

By default, the Jam build system is provided with some values. Some of these
are derived from the base Jam implementation and are documented in the
`Jam manual page`_ while others are Open Dylan extensions.

``.``
   The build directory.

   *Open Dylan extension.*

``COMPILER_BACKEND``
   The name of the compiler back-end currently in use. Typically one ``c``,
   ``harp`` or ``llvm``.

   *Open Dylan extension.*

``JAMDATE``
   The current date, in ISO-8601 format.

``NT``
   True on Windows.

``OS``
   The OS of the build host, not the target. This will typically be something
   like ``linux``, ``freebsd``, ``darwin`` or ``win32``.

``OSPLAT``
   The CPU architecture of the build host, not the target. This will
   typically be something like ``x86`` or ``x86_64``.

``PERSONAL_ROOT``
   The root of the destination build path, when the ``-personal-root`` compiler
   option or the ``OPEN_DYLAN_USER_ROOT`` environment variable is set.

   *Open Dylan extension.*

``SYSTEM_BUILD_SCRIPTS``
   The path where the installed build scripts can be found.

   *Open Dylan extension.*

``SYSTEM_ROOT``
   The path where the installation of Open Dylan can be found.

   *Open Dylan extension.*

``TARGET_PLATFORM``
   The Open Dylan identifier for the target platform. This is something
   like ``x86-linux`` or ``x86_64-darwin``.

   *Open Dylan extension.*

``UNIX``
   True on non-Windows platforms, like Linux, FreeBSD and Mac OS X.

Editing Jam Files
=================

There is an `Emacs major mode`_ for editing Jam files.

.. _Jam: http://www.perforce.com/resources/documentation/jam
.. _Perforce Software: http://www.perforce.com/
.. _Jam manual page: https://swarm.workshop.perforce.com/view/guest/perforce_software/jam/src/Jam.html
.. _Jambase: https://swarm.workshop.perforce.com/files/guest/perforce_software/jam/src/Jambase
.. _Jambase reference: https://swarm.workshop.perforce.com/view/guest/perforce_software/jam/src/Jambase.html
.. _Emacs major mode: https://github.com/emacsmirror/jam-mode
