--- The Detailed Node Listing ---

Installing GNU Software

Writing Good Programs

Using GNU Emacs

Compiling with Makefiles

The GNU build system

Using Automake and Autoconf

Using Libtool

Using Autotoolset

Using C and C++ effectively

Using Fortran effectively

Internationalization

Maintaining Documentation

Portable shell programming

Writing Autoconf macros

Legal issues with Free Software

Philosophical issues

Licensing Free Software

GNU GENERAL PUBLIC LICENSE

Preface

The purpose of this document is to introduce you to the GNU build system, and show you how to use it to write good code. It also discusses peripheral topics such as how to use GNU Emacs as a source code navigator, how to write good software, and the philosophical concerns behind the free software movement. The intended reader should be a software developer who knows his programming languages, and wants to learn how to package his programs in a way that follows the GNU coding standards.

This manual introduces you to the GNU build system and shows you how to develop high-quality

This manual shows you how to develop high-quality software on GNU using the GNU build system that conforms to the GNU coding standards. These techniques are also useful for software development on GNU/Linux and most variants of the Unix system. In fact, one of the reasons for the elaborate GNU build system was to make software portable between GNU and other similar operating systems. We also discuss peripheral topics such as how to use GNU Emacs as an IDE (integrated development environment), and the various practical, legal and philosophical concerns behind software development.

When we speak of the GNU build system we refer primarily to the following four packages:

• Autoconf produces a configuration shell script, named `configure', which probes the installer platform for portability related information which is required to customize makefiles, configuration header files, and other application specific files. Then it proceeds to generate customized versions of these files from generic templates. This way, the user will not need to customize these files manually.
• Automake produces makefile templates, `Makefile.in' to be used by Autoconf, from a very high level specification stored in a file called `Makefile.am'. Automake produces makefiles that conform to the GNU makefile standards, taking away the extraordinary effort required to produce them by hand. Automake requires Autoconf in order to be used properly.
• Libtool makes it possible to compile position independent code and build shared libraries in a portable manner. It does not require either Autoconf, or Automake and can be used independently. Automake however supports libtool and interoperates with it in a seamless manner.
• Autotoolset FIXME: Add content

The GNU build system has two goals. The first is to simplify the development of portable programs. The second is to simplify the building of programs that are distributed as source code. The first goal is achieved by the automatic generation of a `configure' shell script. The second goal is achieved by the automatic generation of Makefiles and other shell scripts that are typically used in the building process. This way the developer can concentrate on debugging his source code, instead of his overly complex Makefiles. And the installer can compile and install the program directly from the source code distribution by a simple and automatic procedure.

The GNU build system needs to be installed only when you are developing programs that are meant to be distributed. To build a program from distributed source code, you only need a working make, a compiler, a shell, and sometimes standard Unix utilities like sed, awk, yacc, lex. The objective is to make software installation as simple and as automatic as possible for the installer. Also, by setting up the GNU build system such that it creates programs that don't require the build system to be present during their installation, it becomes possible to use the build system to bootstrap itself.

Some tasks that are simplified by the GNU build system include:

• Building multi directory software packages. It is much more difficult to use raw make recursively. Having simplified this step, the developer is encouraged to organize his source code in a deep directory tree rather than lump everything under the same directory. Developers that use raw make often can't justify the inconvenience of recursive make and prefer to disorganize their source code. With the GNU tools this is no longer necessary.
• Automatic configuration. You will never have to tell your users that they need to edit your Makefile. You yourself will not have to edit your Makefiles as you move new versions of your code back and forth between different machines.
• Automatic makefile generation. Writing makefiles involves a lot of repetition, and in large projects it will get on your nerves. The GNU build system instead requires you to write `Makefile.am' files that are much more terse and easy to maintain.
• Support for test suites. You can very easily write test suite code, and by adding one extra line in your `Makefile.am' make a check target available such that you can compile and run the entire test suite by running make check.
• Automatic distribution building. The GNU build tools are meant to be used in the development of free software, therefore if you have a working build system in place for your programs, you can create a source code distribution out of it by running make distcheck.
• Shared libraries. Building shared libraries becomes as easy as building static libraries.

The Autotoolset package complements the GNU build system by providing the following additional features:

• Automatic generation of legal notices that are needed in order to apply the GNU GPL license.
• Automatic generation of directory trees for new software packages, such that they conform to the GNITS standard (more or less).
• Some rudimentary portability framework for C++ programs. There is a lot of room for improvement here, in the future. Also a framework for embedding text into your executable and handling include files across multiple directories.
• Support for writing portable software that uses both Fortran and C++.
• Additional support for writing software documentation in Texinfo, but also in LaTeX.

Autotoolset is still under development and there may still be bugs. At the moment Autotoolset doesn't do shared libraries, but that will change in the future.

This effort began by my attempt to write a tutorial for Autoconf. It involved into "Learning Autoconf and Automake". Along the way I developed Autotoolset to deal with things that annoyed me or to cover needs from my own work. Ultimately I want this document to be both a unified introduction of the GNU build system as well as documentation for the Autotoolset package.

I believe that knowing these tools and having this know-how is very important, and should not be missed from engineering or science students who will one day go out and do software development for academic or industrial research. Many students are incredibly undertrained in software engineering and write a lot of bad code. This is very very sad because of all people, it is them that have the greatest need to write portable, robust and reliable code. I found from my own experience that moving away from Fortran and C, and towards C++ is the first step in writing better code. The second step is to use the sophisticated GNU build system and use it properly, as described in this document. Ultimately, I am hoping that this document will help people get over the learning curve of the second step, so they can be productive and ready to study the reference manuals that are distributed with all these tools.

This manual of course is still under construction. When I am done constructing it some paragraph somewhere will be inserted with the traditional run-down of summaries about each chapter. I write this manual in a highly non-linear way, so while it is under construction you will find that some parts are better-developed than others. If you wish to contribute sections of the manual that I haven't written or haven't yet developed fully, please contact me.

Chapters 1,2,3,4 are okay. Chapter 5 is okay to, but needs a little more work. I removed the other chapters to minimize confusion, but the sources for them are still being distributed as part of the Autotoolset package for those that found them useful. The other chapters need a lot of rewriting and they would do more harm than good at this point to the unsuspecting reader. Please contact me if you have any suggestions for improving this manual.

Remarks by Marcelo: I am currently updating this manual to the last release of the autoconf/automake tools.

Acknowledgements

This document and the Autotools package have originally been written by Eleftherios Gkioulekas. Many people have further contributed to this effort, directly or indirectly, in various way. Here is a list of these people. Please help me keep it complete and exempt of errors.

• Philosophical issues, and Licensing Free Software, were written by Richard Stallman. Richard has also contributed many useful review comments and helped me with the legal paperwork. The appendix Philosophical issues has been written by Richard Stallman. (see section Philosophical issues)
• The chapter on Fortran, and the Autotools support for developing software that is partly written in Fortran is derived from the work of John Eaton on GNU Octave, which I mainly generalized for use in other programs. (see section Using Fortran effectively).
• Mark Galassi was the first person, to the best of my knowledge, who tried to write an Autoconf tutorial. It is thanks to his work that I was inspired to begin this work.

FIXME: I need to start keeping track of acknowledgements here

Copying

This book that you are now reading is actually free. The information in it is freely available to anyone. The machine readable source code for the book is freely distributed on the internet and anyone may take this book and make as many copies as they like. (take a moment to check the copying permissions on the Copyright page). If you paid money for this book, what you actually paid for was the book's nice printing and binding, and the publisher's associated costs to produce it.

The following notice refers to the Autotoolset package, which includes this documentation, as well as the source code for utilities like `acmkdir' and for additional Autoconf macros. The complete GNU development tools involves other packages also, such as Autoconf, Automake, Libtool, Make, Emacs, Texinfo, the GNU C and C++ compilers and a few other accessories. These packages are free software, and you can obtain them from the Free Software Foundation. For details on doing so, please visit their web site http://www.fsf.org/. Although Autotoolset has been designed to work with the GNU build system, it is not yet an official part of the GNU project.

The Autotoolset package is also "free"; this means that everyone is free to use it and free to redistribute it on a free basis. The Autotoolset package is not in the public domain; it is copyrighted and there are restrictions on its distribution, but these restrictions are designed to permit everything that a good cooperating citizen would want to do. What is not allowed is to try to prevent others from further sharing any version of this package that they might get from you.

Specifically, we want to make sure that you have the right to give away copies of the programs that relate to Autotoolset, that you receive source code or else can get it if you want it, that you can change these programs or use pieces of them in new free programs, and that you know you can do these things.

To make sure that everyone has such rights, we have to forbid you to deprive anyone else of these rights. For example, if you distribute copies of the Autotoolset-related code, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must tell them their rights.

Also, for our own protection, we must make certain that everyone finds out that there is no warranty for the programs that relate to Autotoolset. If these programs are modified by someone else and passed on, we want their recipients to know that what they have is not what we distributed, so that any problems introduced by others will not reflect on our reputation.

The precise conditions of the licenses for the programs currently being distributed that relate to Autotoolset are found in the General Public Licenses that accompany them.

1. Installing GNU software

Free software is distributed in source code distributions. Many of these programs are difficult to install because they use system dependent features, and they require the user to edit makefiles and configuration headers. By contrast, the software distributed by the GNU project is autoconfiguring; it is possible to compile it from source code and install it automatically, without any tedious user intervention.

In this chapter we discuss how to compile and install autoconfiguring software written by others. In the subsequent chapters we discuss how to use the development tools that allow you to make your software autoconfiguring as well.

1.1 Installing a GNU package

Autoconfiguring software is distributed with packaged source code distributions. These are big files with filenames of the form:

package-version.tar.gz

For example, the file `autoconf-2.57.tar.gz' contains version 2.57 of GNU Autoconf. We often call these files source distributions; sometimes we simply call them packages.

The steps for installing an autoconfiguring source code distribution are simple, and if the distribution is not buggy, can be carried out without substantial user intervention.

1. First, you have to unpack the package to a directory:

   % gunzip foo-1.0.tar.gz % tar xf foo-1.0.tar

   This will create the directory `foo-1.0' which contains the package's source code and documentation. Look for the files `README' to see if there's anything that you should do next. The `README' file might suggest that you need to install other packages before installing this one, or it might suggest that you have to do unusual things to install this package. If the source distribution conforms to the GNU coding standards, you will find many other documentation files like `README'. See section Maintaining the documentation files, for an explanation of what these files mean.

2. Configure the source code. Once upon a time that used to mean that you have to edit makefiles and header files. In the wonderful world of Autoconf, source distributions provide a `configure' script that will do that for you automatically. To run the script type:

   % cd foo-1.0 % ./configure

3. Now you can compile the source code. Type:

   % make

   and if the program is big, you can make some coffee. After the program compiles, you can run its regression test-suite, if it has one, by typing

   % make check

4. If everything is okey, you can install the compiled distribution with:

   % su # make install

The `make' program launches the shell commands necessary for compiling, testing and installing the package from source code. However, `make' has no knowledge of what it is really doing. It takes its orders from makefiles, files called `Makefile' that have to be present in every subdirectory of your source code directory tree. From the installer perspective, the makefiles define a set of targets that correspond to things that the installer wants to do. The default target is always compiling the source code, which is what gets invoked when you simply run make. Other targets, such as `install', `check' need to be mentioned explicitly. Because `make' takes its orders from the makefile in the current directory, it is important to run it from the correct directory. See section Compiling with Makefiles, for the full story behind `make'.

The `configure' program is a shell script that probes your system through a set of tests to determine things that it needs to know, and then uses the results to generate `Makefile' files from templates stored in files called `Makefile.in'. In the early days of the GNU project, developers used to write `configure' scripts by hand. Now, no-one ever does that any more. Now, `configure' scripts are automatically generated by GNU Autoconf from an input file `configure.in'. GNU Autoconf is part of the GNU build system and we first introduce in in The GNU build system.

As it turns out, you don't have to write the `Makefile.in' templates by hand either. Instead you can use another program, GNU Automake, to generate `Makefile.in' templates from higher-level descriptions stored in files called `Makefile.am'. In these files you describe what is being created by your source code, and Automake computes the makefile targets for compiling, installing and uninstalling it. Automake also computes targets for compiling and running test suites, and targets for recursively calling make in subdirectories. The details about Automake are first introduced in Using Automake and Autoconf.

1.2 The Makefile standards

The GNU coding standards are a document that describes the requirements that must be satisfied by all GNU programs. These requirements are driven mainly by technical considerations, and they are excellent advice for writing good software. The makefile standards, a part of the GNU coding standards, require that your makefiles do a lot more than simply compile and install the software.

One requirement is cleaning targets; these targets remove the files that were generated while installing the package and restore the source distribution to a previous state. There are three cleaning targets that corresponds to three levels of cleaning: clean, distclean, maintainer-clean.

clean

Cleans up all the files that were generated by make and make check, but not the files that were generated by running configure. This targets cleans the build, but does not undo the source configuration by the configure script.

distclean

Cleans up all the files generated by make and make check, but also cleans the files that were generated by running configure. As a result, you can not invoke any other make targets until you run the configure script again. This target reverts your source directory tree back to the state in which it was when you first unpacked it.

maintainer-clean

Cleans up all the files that distclean cleans. However it also removes files that the developers have automatically generated with the GNU build system. Because users shouldn't need the entire GNU build system to install a package, these files should not be removed in the final source distribution. However, it is occasionally useful for the maintainer to remove and regenerate these files.

Another type of cleaning that is required is erasing the package itself from the installation directory; uninstalling the package. To uninstall the package, you must call

% make uninstall

from the top level directory of the source distribution. This will work only if the source distribution is configured first. It will work best only if you do it from the same source distribution, with the same configuration, that you've used to install the package in the first place.

When you install GNU software, archive the source code to all the packages that you install in a directory like `/usr/src' or `/usr/local/src'. To do that, first run make clean on the source distribution, and then use a recursive copy to copy it to `/usr/src'. The presence of a source distribution in one of these directories should be a signal to you that the corresponding package is currently installed.

Francois Pinard came up with a cute rule for remembering what the cleaning targets do:

• If configure or make did it, make distclean undoes it.
• If make did it, make clean undoes it.
• If make install did it, make uninstall undoes it.
• If you did it, make maintainer-clean undoes it.

GNU standard compliant makefiles also have a target for generating tags. Tags are files, called `TAGS', that are used by GNU Emacs to allow you to navigate your source distribution more efficiently. More specifically, Emacs uses tags to take you from a place where a C function is being used in a file, to the file and line number where the function is defined. To generate the tags call:

% make tags

Tags are particularly useful when you are not the original author of the code you are working on, and you haven't yet memorized where everything is. See section Navigating source code, for all the details about navigating large source code trees with Emacs.

Finally, in the spirit of free redistributable code, there must be targets for cutting a source code distribution. If you type

% make dist

it will rebuild the `foo-1.0.tar.gz' file that you started with. If you modified the source, the modifications will be included in the distribution (and you should probably change the version number). Before putting a distribution up on FTP, you can test its integrity with:

% make distcheck

This makes the distribution, then unpacks it in a temporary subdirectory and tries to configure it, build it, run the test-suite, and check if the installation script works. If everything is okey then you're told that your distribution is ready.

Writing reliable makefiles that support all of these targets is a very difficult undertaking. This is why we prefer to generate our makefiles instead with GNU Automake.

1.3 Configuration options

The `configure' script accepts many command-line flags that modify its behaviour and the configuration of your source distribution. To obtain a list of all the options that are available type

% ./configure --help

on the shell prompt.

The most useful parameter that the installer controls during configuration is the directory where they want the package to be installed. During installation, the following files go to the following directories:

Executables   → /usr/local/bin
Libraries     → /usr/local/lib
Header files  → /usr/local/include
Man pages     → /usr/local/man/man?
Info files    → /usr/local/info

The `/usr/local' directory is called the prefix. The default prefix is always `/usr/local' but you can set it to anything you like when you call `configure' by adding a `--prefix' option. For example, suppose that you are not a privileged user, so you can not install anything in `/usr/local', but you would still like to install the package for your own use. Then you can tell the `configure' script to install the package in your home directory `/home/username':

% ./configure --prefix=/home/username
% make
% make check
% make install

The `--prefix' argument tells `configure' where you want to install your package, and `configure' will take that into account and build the proper makefile automatically.

If you are installing the package on a filesystem that is shared by computers that run variations of GNU or Unix, you need to install the files that are independent of the operating system in a shared directory, but separate the files that are dependent on the operating systems in different directories. Header files and documentation can be shared. However, libraries and executables must be installed separately. Usually the scheme used to handle such situations is:

Executables   → /usr/local/system/bin
Libraries     → /usr/local/system/lib
Header files  → /usr/local/include
Man pages     → /usr/local/man/mann
Info files    → /usr/local/info

The directory `/var/local/system' is called the executable prefix, and it is usually a subdirectory of the prefix. In general, it can be any directory. If you don't specify the executable prefix, it defaults to being equal to the prefix. To change that, use the `--exec-prefix' flag. For example, to configure for a GNU/Linux system, you would run:

% configure --exec-prefix=/usr/local/linux

To configure for GNU/Hurd, you would run:

% configure --exec-prefix=/usr/local/hurd

In general, there are many directories where a package may want to install files. Some of these directories are controlled by the prefix, where others are controlled by the executable prefix. See section Installation standard directories, for a complete discussion of what these directories are, and what they are for.

Some packages allow you to enable or disable certain features while you configure the source code. They do that with flags of the form:

   --with-package   --enable-feature
--without-package  --disable-feature

The --enable flags usually control whether to enable certain optional features of the package. Support for international languages, debugging features, and shared libraries are features that are usually controlled by these options. The --with flags instead control whether to compile and install certain optional components of the package. The specific flags that are available for a particular source distribution should be documented in the `README' file.

Finally, configure scripts can be passed parameters via environment variables. One of the things that configure does is decide what compiler to use and what flags to pass to that compiler. You can overrule the decisions that configure makes by setting the flags CC and CFLAGS. For example, to specify that you want the package to compile with full optimization and without any debugging symbols (which is a bad idea, yet people want to do it):

% export CFLAGS="-O3"
% ./configure

To tell configure to use the system's native compiler instead of gcc, and compile without optimization and with debugging symbols:

% export CC="cc"
% export CFLAGS="-g"
% ./configure

This assumes that you are using the bash shell as your default shell. If you use the csh or tcsh shells, you need to assign environment variables with the setenv command instead. For example:

% setenv CFLAGS "-O3"
% ./configure 

Similarly, the flags CXX, CXXFLAGS control the C++ compiler.

1.4 Doing a VPATH build

Autoconfiguring source distributions also support vpath builds. In a vpath build, the source distribution is stored in a, possibly read-only, directory, and the actual building takes place in a different directory where all the generated files are being stored. We call the first directory, the source tree, and the second directory the build tree. The build tree may be a subdirectory of the source tree, but it is better if it is a completely separate directory.

If you, the developer, use the standard features of the GNU build system, you don't need to do anything special to allow your packages to support vpath builds. The only exception to this is when you define your own make rules (see section General Automake principles). Then you have to follow certain conventions to allow vpath to work correctly.

You, the installer, however do need to do something special. You need to install and use GNU make. Most Unix make utilities do not support vpath builds, or their support doesn't work. GNU make is extremely portable, and if vpath is important to you, there is no excuse for not installing it.

Suppose that `/sources/foo-0.1' contains a source distribution, and you want to build it in the directory `/build/foo-0.1'. Assuming that both directories exist, all you have to do is:

% cd /build/foo-0.1
% /sources/foo-0.1/configure ...options...
% make
% make check
% su
# make install

The configure script and the generated makefiles will take care of the rest.

vpath builds are preferred by some people for the following reasons:

1. They prevent the build process form cluttering your source directory with all sorts of build files.
2. To remove a build, all you have to do is remove the build directory.
3. You can build the same source multiple times using different options. This is very useful if you would like to write a script that will run the test suite for a package while the package is configured in many different ways (e.g. different features, different compiler optimization, and so on). It is also useful if you would like to do the same with releasing binary distributions of the source.

Some developers like to use vpath builds all the time. Others use them only when necessary. In general, if a source distribution builds with a vpath build, it also builds under the ordinary build. The opposite is not true however. This is why the distcheck target checks if your distribution is correct by attempting a vpath build.

1.5 Making a binary distribution

After compiling a source distribution, instead of installing it, you can make a snapshot of the files that it would install and package that snapshot in a tarball. It is often convenient to the installers to install from such snapshots rather than compile from source, especially when the source is extremely large, or when the amount of packages that they need to install is large.

To create a binary distribution run the following commands as root:

# make install DESTDIR=/tmp/dist
# tar -C /tmp/dist -cvf package-version.tar
# gzip -9 package-version.tar

The variable DESTDIR specifies a directory, alternative to root, for installing the compiled package. The directory tree under that directory is the exact same tree that would have normally been installed. Why not just specify a different prefix? Because very often, the prefix that you use to install the software affects the contents of the files that actually get installed.

Please note that under the terms of the GNU General Public License, if you distribute your software as a binary distribution, you also need to provide the corresponding source distribution. The simplest way to comply with this requirement is to distribute both distributions together.

2. Writing Good Programs

2.1 Why good code is important

When you work on a software project, one of your short-term goals is to solve a problem at hand. If you are doing this because someone asked you to solve the problem, then all you need to do to look good in per eyes is to deliver a program that works. Nevertheless, regardless of how little person may appreciate this, doing just that is not good enough. Once you have code that gives the right answer to a specific set of problems, you will want to make improvements to it. As you make these improvements, you would like to have proof that your code's known reliability hasn't regressed. Also, tomorrow you will want to move on to a different set of related problems by repeating as little work as possible. Finally, one day you may want to pass the project on to someone else or recruit another developer to help you out with certain parts. You need to make it possible for the other person to get up to speed without reinventing your efforts. To accomplish these equally important goals you need to write good code.

2.2 Choosing a good programming language

To write a good software, you must use the appropriate programming language and use it well. To make your software free, it should be possible to compile it with free tools on a free operating system. Therefore, you should avoid using programming languages that do not have a free compiler.

The C programming language is the native language of GNU, and the GNU coding standards encourage you to program in C. The main advantages of C are that it can be compiled with the system's native compiler, many people know C, and it is easy to learn. Nevertheless, C has weaknesses: it forces you to manually manage memory allocation, and any mistakes you might make can lead to very difficult bugs. Also C forces you to program at a low level. Sometimes it is desirable to program at a low level, but there are also cases where you want to build on a higher level.

For projects where you would like a higher-level compiled language, the recommended choice is to use C++. The GNU project distributes a free C++ compiler and nowadays most GNU systems that have a C compiler also have the free C++ compiler. The main advantage of C++ is that it will automatically manage dynamic memory allocation for you. C++ also has a lot of powerful features that allow you to program at a higher level than C, bringing you closer to the algorithms and the concepts involved, and making it easier to write robust programs. At the same time, C++ does not hide low-level details from you and you have the freedom to do the same low-level hacks that you had in C, if you choose to. In fact C++ is 99% backwards compatible with C and it is very easy to mix C and C++ code. Finally, C++ is an industry standard. As a result, it has been used to solve a variety of real-world problems and its specification has evolved for many years to make it a powerful and mature language that can tackle such problems effectively. The C++ specification was frozen and became an ANSI standard in 1998.

One of the disadvantages of C++ is that C++ object files compiled by different C++ compilers can not be linked together. In order to compile C++ to machine language, a lot of compilation issues need to be deferred to the linking stage. Because object file formats are not traditionally sophisticated enough to handle these issues, C++ compilers do various ugly kludges. The problem is that different compilers do these kludges differently, making object files across compilers incompatible. This is not a terrible problem, since object files are incompatible across different platforms anyways. It is only a problem when you want to use more than one compiler on the same platform. Another disadvantage of C++ is that it is harder to interface a C++ library to another language, than it is to interface a C library. Finally not as many people know C++ as well as they know C, and C++ is a very extensive and difficult language to master. However these disadvantages must be weighted against the advantages. There is a price to using C++ but the price comes with a reward.

If you need a higher-level interpreted language, then the recommended choice is to use Guile. Guile is the GNU variant of Scheme, a LISP-like programming language. Guile is an interpreted language, and you can write full programs in Guile, or use the Guile interpreter interactively. Guile is compatible with the R4RS standard but provides a lot of GNU extensions. The GNU extensions are so extensive that it is possible to write entire applications in Guile. Most of the low-level facilities that are available in C, are also available in Guile.

What makes the Guile implementation of Scheme special is not the extensions themselves, but the fact that it it is very easy for any developer to add their own extensions to Guile, by implementing them in C. By combining C and Guile you leverage the advantages of both compiled and interpreted languages. Performance critical functionality can be implemented in C and higher-level software development can be done in Guile. Also, because Guile is interpreted, when you make your C code available through an extended Guile interpreter, then the user can also use the functionality of that code interactively through the interpreter.

The idea of extensible interpreted languages is not new. Other examples of extensible interpreted languages are Perl, Python and Tcl. What sets Guile apart from these languages is the elegance of Scheme. Scheme is the holy grail in the quest for a programming language that can be extended to support any programming paradigm by using the least amount of syntax. Scheme has natural support for both arbitrary precision integer arithmetic and floating point arithmetic. The simplicity of Scheme syntax, and the completeness of Guile, make it very easy to implement specialized scripting languages simply by translating them to Scheme. In Scheme algorithms and data are interchangeable. As a result, it is easy to write Scheme programs that manipulate Scheme source code. This makes Scheme an ideal language for writing programs that manipulate algorithms instead of data, such as programs that do symbolic algebra. Because Scheme can manipulate its own source code, a Scheme program can save its state by writing Scheme source code into a file, and by parsing it later to load it back up again. This feature alone is one reason why engineers should use Guile to configure and drive numerical simulations.

Some people like to use Fortran 77. This is in many ways a good language for developing the computational core of scientific applications. We do have free compilers for Fortran 77, so using it does not restrict our freedom. (see section Using Fortran effectively) Also, Fortran 77 is an aggressively optimized language, and this makes it very attractive to engineers that want to write code optimized for speed. Unfortunately, Fortran 77 can not do well anything except array-oriented numerical computations. Managing input/output is unnecessarily difficult with Fortran, and there's even computational areas, such as infinite precision integer arithmetic and symbolic computation that are not supported.

There are many variants of Fortran like Fortran 90, and HPF. Fortran 90 attempts, quite miserably, to make Fortran 77 more like C++. HPF allows engineers to write numerical code that runs on parallel computers. These variants should be avoided for two reasons:

1. There are no free compilers for Fortran 90 or HPF. If you happen to use a proprietary operating system, you might as well make use of proprietary compilers if they generate highly optimized code and that is important to you. Nevertheless, in order for your software to be freed, it should be possible to compile it with free tools on a free operating system. Because it is possible to make parallel computers using GNU/Linux (see the Beowulf project), parallelized software can also be free. Therefore both Fortran 90 and HPF should be avoided.
2. Another problem with these variants is that they are ad hoc languages that have been invented to enable Fortran to do things that it can not do by design. Eventually, when engineers will like to do things that Fortran 90 can't do either, it will be necessary to extend Fortran again, rewrite the compilers and produce yet another variant. What engineers need is a programming language that has the ability to self extend itself by writing software in the same programming language. The C++ programming language can do this without loss of performance. The departmentalization of disciplines in academia has made it very difficult for such a project to take off. Despite that, there is ongoing research in this area. (for example, see the Blitz++ project)

It is almost impossible to write good programs entirely in Fortran, so please use Fortran only for the numerical core of your application and do the bookkeeping tasks, including input/output using a more appropriate language.

If you have written a program entirely in Fortran, please do not ask anyone else to maintain your code, unless person is like you and also knows only Fortran. If Fortran is the only language that you know, then please learn at least C and C++ and use Fortran only when necessary. Please do not hold the opinion that contributions in science and engineering are "true" contributions and software development is just a "tool". This bigoted attitude is behind the thousands of lines of ugly unmaintainable code that goes around in many places. Good software development can be an important contribution in its own right, and regardless of what your goals are, please appreciate it and encourage it. To maximize the benefits of good software, please make your software free. (FIXME: Cross reference copyright section in this chapter)

2.3 Developing libraries

The key to better code is to focus away from developing monolithic throw-away hacks that do only one job, and focus on developing libraries (FIXME: cross reference). Break down the original problem to parts, and the parts to smaller parts, until you get down to simple subproblems that can be easily tested, and from which you can construct solutions for both the original problem and future variants. Every library that you write is a legacy that you can share with other developers, that want to solve similar problems. Each library will allow these other developers to focus on their problem and not have to reinvent the parts that are common with your work from scratch. You should definitely make libraries out of subproblems that are likely to be broadly useful. Please be very liberal in what you consider "broadly useful". Please program in a defensive way that renders reusable as much code as possible, regardless of whether or not you plan to reuse it in the near future. The final application should merely have to assemble all the libraries together and make their functionality accessible to the user through a good interface.

It is very important for each of your libraries to have a complete test suite. The purpose of the test suite is to detect bugs in the library and to prove to you or convince you, the developer, that the library works. A test suite is composed of a collection of test programs that link with your libraries and experiment with the features provided by the library. These test programs should return with

exit(0);

if they do not detect anything wrong with the library and with

exit(1);

if they detect problems. The test programs should not be installed with the rest of the package. They are meant to be run after your software is compiled and before it is installed. Therefore, they should be written so that they can run using the compiled but uninstalled files of the library. Test programs should not output messages by default. They should run completely quietly and communicate with the environment in a yes or no fashion using the exit code. However, it is useful for test programs to output debugging information when they fail during development. Statements that output such information should be surrounded by conditional directives like this:

#if INSPECT_ERRORS
 printf("Division by zero: %d / %d\n",a,b);
#endif

This way it becomes easy to switch them on or off upon demand. The preferred way to manipulate a macro like this INSPECT_ERRORS is by adding a switch to your `configure' script. You can do this by adding the following lines to `configure.in':

AC_ARG_WITH(inspect,
  [  --with-inspect           Inspect test suite errors],
  [ AC_DEFINE(INSPECT_ERRORS, 1, "Inspect test suite errors")],
  [ AC_DEFINE(INSPECT_ERRORS, 0, "Inspect test suite errors")])

After the library is debugged, the debug statements should not be removed. If a future version of the library regresses and an old test begins to fail again, it will be useful to be able to reactivate the same error messages that were useful in debugging the test when it was first put together, and it may be necessary to add a few new ones.

The best time to write each test program is as soon as it is possible!. You should not be lazy, and you should not just keep throwing in code after code after code. The minute there is enough code in there to put together some kind of test program, just do it! When you write new code, it is easy to think that you are producing work with every new line of code that is written. The reality is that you know you have produced new work every time you write working a test program for new features, and not a minute before. Another time when you should definitely write a test program is when you find a bug while ordinarily using the library. Then, write a test program that triggers the bug, fix the bug, and keep the test in your test suite. This way, if a future modification reintroduces the same bug it will be detected.

Please document your library as you go. The best time to update your documentation is immediately after you get new test programs checking out new futures. You might feel that you are too busy to write documentation, but the truth of the matter is that you will always be too busy. In fact, if you are a busy person, you are likely to have many other obligations bugging you around for your attention. There may be times that you have to stay away from a project for a large amount of time. If you have consistently been maintaining documentation, it will help you refocus on your project even after many months of absence.

2.4 Developing applications

Applications are complete executable programs that can be run by the end-user. With library-oriented development the actual functionality is developed by writing libraries and debugged by developing test-suites for each library. With command-line oriented applications, the application source code parses the arguments that are passed to it by the user, and calls up the right functions in the library to carry out the user's requests. With GUI (1) applications, the application source code creates the widgets that compose the interface, binds them to actions, and then enters an event loop. Each action is implemented in terms of the functionality provided by the appropriate library.

It should be possible to implement applications by using relatively few application-specific source files, since most of the functionality is actually done in libraries. In some cases, the application is simple enough that it would be an overkill to package its functionality as a library. Nevertheless, in such cases please separate the source code that handles actual functionality from the source code that handles the user interface. Also, please always separate the code that handles input/output with the code that does actual computations. If these aspects of your source code are sufficiently separated then you make it easier for other people to reuse parts of your code in their applications. You also make it easier of yourself to switch to library-oriented development when your application grows and is no longer "simple enough".

Library-oriented development allows you to write good and robust applications. In return it requires discipline. Sometimes you may need to add experimental functionality that is not available through your libraries. The right thing to do is to extend the appropriate library. The easy thing to do is to implement it as part of your application-specific source code. If the feature is experimental and undergoing many changes, it may be best to go with the easy approach at first. Still, when the feature matures, please migrate it to the appropriate library, document it, and take it out of the application source code. What we mean by discipline is doing these migrations, when the time is right, despite pressures from "real life", such as deadlines, pointy-haired bosses, and nuclear terrorism. A rule of thumb for deciding when to migrate code to a library is when you find yourself cut-n-pasting chunks of code from application to application. If you do not do the right thing, your code will become increasingly harder to debug, harder to maintain, and less reliable.

Applications should also be documented, especially the ones that are command-line oriented. Application documentation should be thorough in explaining to the user all the things that he needs to know to use the application effectively and should be distributed separately from the application itself. Nevertheless, applications should recognize the --help switch and output a synopsis of how the application is used. Applications should also recognize the --version switch and state their version number. The easiest way to make applications understand these two switches is to use the GNU Argp library (FIXME: cross reference).

2.5 Free software is good software

One of the reasons why you should write good code is because it allows you to make your code robust, reliable and most useful to your needs. Another reason is to make it useful to other people too, and make it easier for them to work with your code and reuse it for their own work. In order for this to be possible, you need to give worry about a few obnoxious legal issues.

2.6 Invoking the `gpl' utility

Maintaining these legalese notices can be quite painful after some time. To ease the burden, Autotools distributes a utility called `gpl'. This utility will conveniently generate for you all the legal wording you will ever want to use. It is important to know that this application is not approved in any way by the Free Software Foundation. By this I mean that I haven't asked their opinion of it yet.

To create the file `COPYING' type:

% gpl -l COPYING

If you want to include a copy of the GPL in your documentation, you can generate a copy in texinfo format like this:

% gpl -lt gpl.texi

Also, every time you want to create a new file, use the `gpl' to generate the copyright notice. If you want it covered by the GPL use the standard notice. If you want to invoke the Guile-like permissions, then also use the library notice. If you want to grant unlimited permissions, meaning no copyleft, use the special notice. The `gpl' utility takes many different flags to take into account the different commenting conventions.

• For a C file, create the standard notice with

  % gpl -c file.c

  the library notice with

  % gpl -cL file.c

  and the special notice with

  % gpl -cS file.c

• For a C++ file, create the standard notice with

  % gpl -cc file.cc

  the library notice with

  % gpl -ccL file.cc

  and the special notice with

  % gpl -ccS file.cc

• For a shells script (BASH, Perl) that uses hash marks for commenting, create the standard notice with

  % gpl -sh foo.pl

  the library notice with

  % gpl -shL foo.tcl

  and the special notice with

  % gpl -shS foo.pl

  It does not make sense to use the library notice, if no executable is being formed from this file. If however, you parse that file into C code that is then compiled into object code, then you may consider using the library notice on it instead of the special notice. One of the features provided by Autotools allows you to embed text, such as Tcl scripts, into the executable. In that case, you can use the library notice to license the original text.

• For files that define autoconf macros:

  % gpl -m4 file.m4

  In general, we exempt autoconf macro files from the GNU GPL because the terms of autoconf also exclude its output, the `configure' script, from the GPL.

• For `Makefile.am', or files that describe targets:

  % gpl -am Makefile.am

  For these we also exempt them from the GPL because they are so trivial that it makes no sense to add copyleft protection.

2.7 Inserting notices with Emacs

If you are using GNU Emacs, then you can insert these copyright notices on-demand while you're editing your source code. Autotools bundles two Emacs packages: gpl and gpl-copying which provide you with equivalents of the `gpl' command that can be run under Emacs. These packages will be byte-compiled and installed automatically for you while installing Autotools.

To use these packages, in your `.emacs' you must declare your identity by adding the following commands:

(setq user-mail-address "me@here.com")
(setq user-full-name "My Name")

Then you must require the packages to be loaded:

(require 'gpl)
(require 'gpl-copying)

These packages introduce a set of Emacs commands all of which are prefixed as gpl-. To invoke any of these commands press M-x, type the name of the command and press enter.

The following commands will generate notices for your source code:

`gpl-c'

Insert the standard GPL copyright notice using C commenting.

`gpl-cL'

lnsert the standard GPL copyright notice using C commenting, followed by a Guile-like library exception. This notice is used by the Guile library. You may want to use it for libraries that you write that implement some type of a standard that you wish to encourage. You will be prompted for the name of your package.

`gpl-cc'

Insert the standard GPL copyright notice using C++ commenting.

`gpl-ccL'

Insert the standard GPL copyright notice using C++ commenting, followed by a Guile-like library exception. You will be prompted for the name of your package

`gpl-sh'

Insert the standard GPL copyright notice using shell commenting (i.e. has marks).

`gpl-shL'

Insert the standard GPL copyright notice using shell commenting, followed by a Guile-like library exception. This can be useful for source files, like Tcl files, which are executable code that gets linked in to form an executable, and which use hash marks for commenting.

`gpl-shS'

Insert the standard GPL notice using shell commenting, followed by the special Autoconf exception. This is useful for small shell scripts that are distributed as part of a build system.

`gpl-m4'

Insert the standard GPL copyright notice using m4 commenting (i.e. dnl) and the special Autoconf exception. This is the preferred notice for new Autoconf macros.

`gpl-el'

Insert the standard GPL copyright notice using Elisp commenting. This is useful for writing Emacs extension files in Elisp.

The following commands will generate notices for your source code:

`gpl-insert-copying-texinfo'

Insert a set of paragraphs very similar to the ones appearing at the Copying section of this manual. It is a good idea to include this notice in an unnumbered chapter titled "Copying" in the Texinfo documentation of your source code. You will be prompted for the title of your package. That title will substitute the word Autotools as it appears in the corresponding section in this manual.

`gpl-insert-license-texinfo'

Insert the full text of the GNU General Public License in Texinfo format. If your documentation is very extensive, it may be a good idea to include this notice either at the very beginning of your manual, or at the end. You should include the full license, if you plan to distribute the manual separately from the package as a printed book.

3. Using GNU Emacs

Emacs is an environment for running Lisp programs that manipulate text interactively. To call Emacs merely an editor does not do it justice, unless you redefine the word "editor" to the broadest meaning possible. Emacs is so extensive, powerful and flexible, that you can almost think of it as a self-contained "operating system" in its own right.

Emacs is a very important part of the GNU development tools because it provides an integrated environment for software development. The simplest thing you can do with Emacs is edit your source code. However, you can do a lot more than that. You can run a debugger, and step through your program while Emacs shows you the corresponding sources that you are stepping through. You can browse on-line Info documentation and man pages, download and read your email off-line, and follow discussions on newsgroups. Emacs is particularly helpful with writing documentation with the Texinfo documentation system. You will find it harder to use Texinfo, if you don't use Emacs. It is also very helpful with editing files on remote machines over FTP, especially when your connection to the internet is over a slow modem. Finally, and most importantly, Emacs is programmable. You can write Emacs functions in Emacs Lisp to automate any chore that you find particularly useful in your own work. Because Emacs Lisp is a full programming language, there is no practical limit to what you can do with it.

If you already know a lot about Emacs, you can skip this chapter and move on. If you are a "vi" user, then we will assimilate you: See section Using vi emulation, for details. (2) This chapter will be most useful to the novice user who would like to set per Emacs up and running for software development, however it is not by any means comprehensive. See section Further reading on Emacs, for references to more comprehensive Emacs documentation.

3.1 Introduction to Emacs

Emacs is an environment for running Lisp programs that manipulate text interactively. Because Emacs is completely programmable, it can be used to implement not only editors, but a full integrated development environment for software development. Emacs can also browse info documentation, run email clients, a newsgroup reader, a sophisticated xterm, and an understanding psychotherapist.

Under the X window system, Emacs controls multiple x-windows called frames. Each frame has a menu bar and the main editing area. The editing area is divided into windows with horizontal bars. You can grab these bars and move them around with the first mouse button. (3) Each window is bound to a buffer. A buffer is an Emacs data structure that contains text. Most editing commands operate on buffers, modifying their contents. When a buffer is bound to a window, then you can see its contents as they are being changed. It is possible for a buffer to be bound to two windows, on different frames or on the same frame. Then whenever a change is made to the buffer, it is reflected on both windows. It is not necessary for a buffer to be bound to a window, in order to operate on it. In a typical Emacs session you may be manipulating more buffers than the windows that you have on your screen.

A buffer can be visiting files. In that case, the contents of the buffer reflect the contents of a file that is being edited. But buffers can be associated with anything you like, so long as you program it up. For example, under the Dired directory editor, a buffer is bound to a directory, showing you the contents of the directory. When you press Enter while the cursor is over a file name, Emacs creates a new buffer, visits the file, and rebinds the window with that buffer. From the user's perspective, by pressing Enter he "opened" the file for editing. If the file has already been "opened" then Emacs simply rebinds the existing buffer for that file.

Emacs uses a variant of LISP, called Emacs LISP, as its programming language. Every time you press a key, click the mouse, or select an entry from the menu bar, an Emacs LISP function is evaluated. The mode of the buffer determines, among many other things, what function to evaluate. This way, every buffer can be associated with functionality that defines what you do in that buffer. For example you can program your buffer to edit text, to edit source code, to read news, and so on. You can also run LISP functions directly on the current buffer by typing M-x and the name of the function that you want to run. (4)

What is known as the "Emacs editor" is the default implementation of an editor under the Emacs system. If you prefer the vi editor, then you can instead run a vi clone, Viper (see section Using vi emulation). The main reason why you should use Emacs, is not the particular editor, but the way Emacs integrates editing with all the other functions that you like to do as a software developer. For example:

• You can edit multiple files under one program. From the user perspective, you can edit two different parts of a file under two different x-windows. And when you revisit a file, the cursor is placed where it was the last time you were editing the file.
• You can quickly browse a directory and navigate from file to file. You can also do simple operations on files, without needing to go to a shell.
• You can transparently edit files over FTP. This is extremely valuable if you are editing source code on a remote computer and you are connected through a modem link.
• You can have a running shell for typing unix commands, and access the same shell from any Emacs frame. You can use that shell to run `reconf', `configure' and `make'. You can also save the contents of your session to a file.
• Color is used to highlight syntactic information about the text. This makes browsing more pleasing to the eye, and it can also help you catch syntactic mistakes. Emacs understands the syntax of most types of files you are likely to edit and will color them up for you accordingly.
• When you edit source code under Emacs, it will automatically be formatted for you to conform to the GNU coding standards. At your request, appropriate copyright notices can be inserted. (see section Inserting notices with Emacs)
• When you make changes to a file, Emacs can automatically warp you to the appropriate `ChangeLog' file to record your changes. It will handle formatting details for you allowing you to focus on content. (see section Maintaining the documentation files)
• Emacs is invaluably helpful for writing Texinfo documentation. In fact, it is excruciatingly painful to maintain Texinfo documentation without using Emacs. (see section GNU Emacs support for Texinfo)
• You can run the `gdb' debugger under Emacs and use it to step through your code. As you do that, Emacs will show you on a separate buffer the code that is currently being stepped through.
• You can read email and newsgroups. If you are connected over a modem, all your editing is done locally, so you do not get bogged down by the speed of your connection. You can apply patches that you get through email or news to your source code directly, without needing to save the message to a file.
• Emacs currently supports almost every international language, even languages that do not use the Roman alphabet, like Greek, Chinese, Hebrew, Tibetan,etc.

All of these features make Emacs a very powerful, albeit unusual, integrated development environment. Many users of proprietary operating systems, like Lose95 (5), complain that GNU (and Unix) does not have an integrated development environment. As a matter of fact it does. All of the above features make Emacs a very powerful IDE.

Emacs has its own very extensive documentation (see section Further reading on Emacs). In this manual we will only go over the fundamentals for using Emacs effectively as an integrated development environment.

3.2 Installing GNU Emacs

If Emacs is not installed on your system, you will need to get a source code distribution and compile it yourself. Installing Emacs is not difficult. If Emacs is already installed on your GNU/Linux system, you might still need to reinstall it: you might not have the most recent version, you might have XEmacs instead, you might not have support for internationalization, or your Emacs might not have compiled support for reading mail over POP (a feature very useful to developers that hook up over modem). If any of these is the case, then uninstall that version of Emacs, and reinstall Emacs from a source code distribution.

The entire Emacs source code is distributed in three separate files:

`emacs-21.2.tar.gz'

This is the main Emacs distribution. If you do not care about international language support, you can install this by itself.

`leim-21.2.tar.gz'

This supplements the Emacs distribution with support for multiple languages. If you develop internationalized software, it is likely that you will need this.

`intlfonts-1.1.tar.gz'

This file contains the fonts that Emacs uses to support international languages. If you want international language support, you will definitely need this.

Get a copy of these three files, place them under the same directory and unpack them with the following commands:

% gunzip emacs-21.2.tar.gz
% tar xf emacs-21.2.tar
% gunzip leim-21.2.tar.gz
% tar xf leim-21.2.tar

Both tarballs will unpack under the `emacs-21.2' directory. When this is finished, configure the source code with the following commands:

% cd emacs-21.2
% ./configure --with-pop --with-gssapi
% make

The `--with-pop' flag is almost always a good idea, especially if you are running Emacs from a home computer that is connected to the internet over modem. It will let you use Emacs to download your email from your internet provider and read it off-line (see section Using Emacs as an email client). Most internet providers use GSSAPI-authenticated POP. If you need to support other authentication protocols however, you may also want to add one of the following flags:

--with-kerberos

support Kerberos-authenticated POP

--with-kerberos5

support Kerberos version 5 authenticated POP

--with-hesiod

support Hesiod to get the POP server host

Then compile and install Emacs with:

$ make
# make install

Emacs is a very large program, so this will take a while.

To install `intlfonts-1.1.tar.gz' unpack it, and follow the instructions in the `README' file. Alternatively, you may find it more straightforward to install it from a Debian package. Packages for `intlfonts' exist as of Debian 2.1.

3.3 Basic Emacs concepts

In this section we describe what Emacs is and what it does. We will not yet discuss how to make Emacs work. That discussion is taken up in the subsequent sections, starting with Configuring GNU Emacs. This section instead covers the fundamental ideas that you need to understand in order to make sense out of Emacs.

You can run Emacs from a text terminal, such as a vt100 terminal, but it is usually nicer to run Emacs under the X-windows system. To start Emacs type

% emacs &

on your shell prompt. The seasoned GNU developer usually sets up per X configuration such that it starts Emacs when person logs in. Then, person uses that Emacs process for all of per work until person logs out. To quit Emacs press C-x C-c, or select

Files → Exit Emacs

from the menu. The notation C-c means CTRL-c. The separating dash `-' means that you press the key after the dash while holding down the key before the dash. Be sure to quit Emacs before logging out, to ensure that your work is properly saved. If there are any files that you haven't yet saved, Emacs will prompt you and ask you if you want to save them, before quiting. If at any time you want Emacs to stop doing what it's doing, press C-g.

Under the X window system, Emacs controls multiple x-windows which are called frames. Each frame has a menu bar and the main editing area. The editing area is divided into windows (6) by horizontal bars, called status bars. Every status bar contains concise information about the status of the window above the status bar. The minimal editing area has at least one big window, where editing takes place, and a small one-line window called the minibuffer. Emacs uses the minibuffer to display brief messages and to prompt the user to enter commands or other input. The minibuffer has no status bar of its own.

Each window is bound to a buffer. A buffer is an Emacs data structure that contains text. Most editing commands operate on buffers, modifying their contents. When a buffer is bound to a window, then you can see its contents as they are being changed. It is possible for a buffer to be bound to two windows, on different frames or on the same frame. Then whenever a change is made to the buffer, it is reflected on both windows. It is not necessary for a buffer to be bound to a window, in order to operate on it. In a typical Emacs session you may be manipulating more buffers than the windows that you actually have on your screen.

A buffer can be visiting files. In that case, the contents of the buffer reflect the contents of a file that is being edited. But buffers can be associated with anything you like, so long as you program it up. For example, under the Dired directory editor, a buffer is bound to a directory, showing you the contents of the directory. When you press RET while the cursor is over a file name, Emacs creates a new buffer, visits the file, and rebinds the window with that buffer. From the user's perspective, by pressing RET person "opened" the file for editing. If the file has already been "opened" then Emacs simply rebinds the existing buffer for that file.

Sometimes Emacs will divide a frame to two or more windows. You can switch from one window to another by clicking the 1st mouse button, while the mouse is inside the destination window. To resize these windows, grab the status bar with the 1st mouse button and move it up or down. Pressing the 2nd mouse button, while the mouse is on a status bar, will bury the window bellow the status bar. Pressing the 3rd mouse button will bury the window above the status bar, instead. Buried windows are not killed; they still exist and you can get back to them by selecting them from the menu bar, under:

Buffers → name-of-buffer

Buffers, with some exceptions, are usually named after the filenames of the files that they correspond to.

Once you visit a file for editing, then all you need to do is to edit it! The best way to learn how to edit files using the standard Emacs editor is by working through the on-line Emacs tutorial. To start the on-line tutorial type C-h t or select:

Help → Emacs Tutorial

If you are a vi user, or you simply prefer to use `vi' key bindings, then read Using vi emulation.

In Emacs, every event causes a Lisp function to be executed. An event can be any keystroke, mouse movement, mouse clicking or dragging, or a menu bar selection. The function implements the appropriate response to the event. Almost all of these functions are written in a variant of Lisp called Emacs Lisp. The actual Emacs program, the executable, is an Emacs Lisp interpreter with the implementation of frames, buffers, and so on. However, the actual functionality that makes Emacs usable is implemented in Emacs Lisp.

Sometimes, Emacs will bind a few words of text to an Emacs function. For example, when you use Emacs to browse Info documentation, certain words that corresponds to hyperlinks to other nodes are bound to a function that makes Emacs follow the hyperlink. When such a binding is actually installed, moving the mouse over the bound text highlights it momentarily. While the text is highlighted, you can invoke the binding by clicking the 2nd mouse button.

Sometimes, an Emacs function might go into an infinite loop, or it might start doing something that you want to stop. You can always make Emacs abort (7) the function it is currently running by pressing C-g.

Emacs functions are usually spawned by Emacs itself in response to an event. However, the user can also spawn an Emacs function by typing:

ALT-x function-name RET

These functions can also be aborted with C-g.

It is standard in Emacs documentation to refer to the ALT key with the letter `M'. So, in the future, we will be referring to function invocations as:

M-x function-name

Because Emacs functionality is implemented in an event-driven fashion, the Emacs developer has to write Lisp functions that implement functionality, and then bind these functions to events. Tables of such bindings are called keymaps.

Emacs has a global keymap, which is in effect at all times, and then it has specialized keymaps depending on what editing mode you use. Editing modes are selected when you visit a file depending on the name of the file. So, for example, if you visit a C file, Emacs goes into the C mode. If you visit `Makefile', Emacs goes into makefile mode. The reason for associating different modes with different types of files is that the user's editing needs depend on the type of file that person is editing.

You can also enter a mode by running the Emacs function that initializes the mode. Here are the most commonly used modes:

M-x c-mode

Mode for editing C programs according to the GNU coding standards.

M-x c++-mode

Mode for editing C++ programs

M-x sh-mode

Mode for editing shell scripts.

M-x m4-mode

Mode for editing Autoconf macros.

M-x texinfo-mode

Mode for editing documentation written in the Texinfo formatting language. See section Introduction to Texinfo.

M-x makefile-mode

Mode for editing makefiles.

As a user you shouldn't have to worry too much about the modes. The defaults do the right thing. However, you might want to enhance Emacs to suit your needs better.

3.4 Configuring GNU Emacs

To use Emacs effectively for software development you need to configure it. Part of the configuration needs to be done in your X-resources file. On a Debian GNU/Linux system, the X-resources can be configured by editing

/etc/X11/Xresources

In many systems, you can configure X-resources by editing a file called `.Xresources' or `.Xdefaults' on your home directory, but that is system-dependent. The configuration that I use on my system is:

! Emacs defaults
emacs*Background: Black
emacs*Foreground: White
emacs*pointerColor: White
emacs*cursorColor: White
emacs*bitmapIcon: on
emacs*font: fixed
emacs*geometry: 80x40

In general I favor dark backgrounds and `fixed' fonts. Dark backgrounds make it easier to sit in front of the monitor for a prolonged period of time. `fixed' fonts looks nice and it's small enough to make efficient use of your screen space. Some people might prefer larger fonts however.

When Emacs starts up, it looks for a file called `.emacs' at the user's home directory, and evaluates it's contents through the Emacs Lisp interpreter. You can customize and modify Emacs' behaviour by adding commands, written in Emacs Lisp, to this file. Here's a brief outline of the ways in which you can customize Emacs:

1. A common change to the standard configuration is assigning global variables to non-default values. Many Emacs features and behaviours can be controlled and customized this way. This is done with the `setq' command, which accepts the following syntax:

   (setq variable value)

   For example:

   (setq viper-mode t)

   You can access on-line documentation for global variables by running:

   M-x describe-variable

2. In some cases, Emacs depends on the values of shell environment variables. These can be manipulated with `setenv':

   (setenv "variable" "value")

   For example:

   (setenv "INFOPATH" "/usr/info:/usr/local/info")

   `setenv' does not affect the shell that invoked Emacs, but it does affect Emacs itself, and shells that are run under Emacs.

3. Another way to enhance your Emacs configuration is by modifying the global keymap. This can be done with the `global-set-key' command, which follows the following syntax:

   (global-set-key [key sequence] 'function)

   For example, adding:

   (global-set-key [F12 d] 'doctor)

   to `.emacs' makes the key sequence F12 d equivalent to running `M-x doctor'. Emacs has many functions that provide all sorts of features. To find out about specific functions, consult the Emacs user manual. Once you know that a function exists, you can also get on-line documentation for it by running:

   M-x describe-function

   You can also write your own functions in Emacs Lisp.

4. It is not always good to introduce bindings to the global map. Any bindings that are useful only within a certain mode should be added only to the local keymap of that mode. Consider for example the following Emacs Lisp function:

   (defun texi-insert-@example () "Insert an @example @end example block" (interactive) (beginning-of-line) (insert "\n@example\n") (save-excursion (insert "\n") (insert "@end example\n") (insert "\n@noindent\n")))

   We would like to bind this function to the key `F9', however we would like this binding to be in effect only when we are within `texinfo-mode'. To do that, first we must define a hook function that establishes the local bindings using `define-key':

   (defun texinfo-elef-hook () (define-key texinfo-mode-map [F9] 'texi-insert-@example))

   The syntax of `define-key' is similar to `global-set-key' except it takes the name of the local keymap as an additional argument. The local keymap of any `name-mode' is `name-mode-map'. Finally, we must ask `texinfo-mode' to call the function `texinfo-elef-hook'. To do that use the `add-hook' command:

   (add-hook 'texinfo-mode-hook 'texinfo-elef-hook)

   In some cases, Emacs itself will provide you with a few optional hooks that you can attach to your modes.

5. You can write your own modes! If you write a program whose use involves editing some type of input files, it is very much appreciated by the community if you also write an Emacs mode for that file and distribute it with your program.

With the exception of simple customizations, most of the more complicated ones require that you write new Emacs Lisp functions, distribute them with your software and somehow make them visible to the installer's Emacs when person installs your software. See section Emacs Lisp with Automake, for more details on how to include Emacs Lisp packages to your software.

Here are some simple customizations that you might want to add to your `.emacs' file:

• Set your default background and foreground color for all your Emacs frames:

  (set-background-color "black") (set-foreground-color "white")

  You can change the colors to your liking.

• Tell Emacs your name and your email address. This is particularly useful when you work on an off-line home system but you want Emacs to use the email address of your internet provider, and your real name. Specifying your real name is necessary if you call yourself "Skeletor" or "Dude" on your home computer.

  (setq user-mail-address "karl@whitehouse.com") (setq user-full-name "President Karl Marx")

  Make sure the name is your real name, and the email address that you include can receive email 24 hours per day.

• Add a few toys to the status bar. These commands tell Emacs to display a clock, and the line and column number of your cursor's position at all times.

  (display-time) (line-number-mode 1) (column-number-mode 1)

• When you use the mouse to cut and paste text with Emacs, mouse button 1 will select text and mouse button 2 will paste it. Unfortunately, when you click mouse button 2, emacs will first move the cursor at the location of the mouse, and then insert the text in that location. If you are used to editing with vi under xterm, you will instead prefer to position the cursor yourself, and use mouse button 2 to simply cause the text to be pasted without changing the position of the cursor. If you prefer this behaviour, then add the following line to your `.emacs':

  (global-set-key [mouse-2] 'yank)

  By default, selected text in Emacs buffers is highlighted with blue color. However, you can also select and paste into an Emacs buffer text that you select from other applications, like your web browser, or your xterm.

• Use font-lock. Font-lock decorates your edited text with colors that make it easier to read text with complicated syntax, such as software source codes. This is one of the coolest features of Emacs. To use it, add the following lines to your `.emacs':

  (global-font-lock-mode t) (setq font-lock-maximum-size nil)

• To get rid of the scrollbar at the left of your Emacs window, type

  (setq scroll-bar-mode nil)

  The only reason that the scrollbar is default is to make Emacs more similar to what lusers are used to. It is assumed that seasoned hacker, who will be glad to see the scrollbar bite it, will figure out how to make it go away.

• With most versions of Emacs, you should add the following to your `.emacs' to make sure that editing `configure.in' takes you to m4-mode and editing `Makefile.am' takes you to makefile-mode.

  (setq auto-mode-alist (append '( ("configure.in" . m4-mode) ("\\.m4\\'" . m4-mode) ("\\.am\\'" . makefile-mode)) auto-mode-alist))

  You will have to edit such files if you use the GNU build system. See section The GNU build system, for more details.

• If you have installed Emacs packages in non-standard directories, you need to add them to the `load-path' variable. For example, here's how to add a couple of directories:

  (setq load-path (append "/usr/share/emacs/site-lisp" "/usr/local/share/emacs/site-site" (expand-file-name "~lf/lisp") load-path))

  Note the use of `expand-file-name' for dealing with non-absolute directories. If you are a user in an account where you don't have root privilege, you are very likely to need to install your Emacs packages in a non-standard directory.

• See section Using vi emulation, if you would like to customize Emacs to run a vi editor under the Emacs system.
• See section Navigating source code, for more details on how to customize Emacs to make navigating a source code directory tree easier.
• See section Using Emacs as an email client, if you would like to set up Emacs to process your email.
• Autotoolset distributes two Emacs packages. One for handling copyright notices, and another one for handling Texinfo documentation. See section Inserting copyright notices with Emacs, and See section GNU Emacs support for Texinfo, for more details.

Emacs now has a graphical user interface to customization that will write `.emacs' for you automatically. To use it, select:

Help → Customize → Browse Customization Groups

from the menu bar. You can also manipulate some common settings from:

Help → Options

3.5 Using vi emulation

Many hackers prefer to use the `vi' editor. The `vi' editor is the standard editor on Unix. It is also always available on GNU/Linux. Many system administrators find it necessary to use vi, especially when they are in the middle of setting up a system in which Emacs has not been installed yet. Besides that, there are many compelling reasons why people like vi.

• Vi requires only two special keys: the SHIFT key and the ESC key. All the other keys that you need are standard on all keyboards. You do not need CTRL,ALT,the cursor keys or any of the function keys. Some terminals that miss the escape key, usually have the control key and you can get escape with: CTRL-[
• Vi was designed to deal with terminals that connect to mainframes over a very slow line. So it has been optimized to allow you to do the most editing possible with the fewest keystrokes. This allows users to edit text very efficiently.
• Vi allows your fingers to stay at the center of the keyboard, with the occasional hop to the escape key. It does not require you to stretch your fingers in funny control combinations, which makes typing less tiring and more comfortable.

Because most rearrangements of finger habits are not as optimal as the vi finger habits, most vi users react very unpleasantly to other editors. For the benefit of these users, in this section we describe how to run a vi editor under the Emacs system. Similarly, users of other editors find the vi finger habits strange and unintuitive. For the benefit of these users we describe briefly how to use the vi editor, so they can try it out if they like.

The vi emulation package for the Emacs system is called Viper. To use Viper, add the following lines in your `.emacs':

(setq viper-mode t)
(setq viper-inhibit-startup-message 't)
(setq viper-expert-level '3)
(require 'viper)

We recommend expert level 3, as the most balanced blend of the vi editor with the Emacs system. Most editing modes are aware of Viper, and when you begin editing the text you are immediately thrown into Viper. Some modes however do not do that. In some modes, like the Dired mode, this is very appropriate. In other modes however, especially custom modes that you have added to your system, Viper does not know about them, so it does not configure them to enter Viper mode by default. To tell a mode to enter Viper by default, add a line like the following to your `.emacs' file:

(add-hook 'm4-mode-hook 'viper-mode)

The modes that you are most likely to use during software development are

c-mode  , c++-mode , texinfo-mode
sh-mode , m4-mode  , makefile-mode

Sometimes, Emacs will enter Viper mode by default in modes where you prefer to get Emacs modes. In some versions of Emacs, the compilation-mode is such a mode. To tell a mode not to enter Viper by default, add a line like the following to your `.emacs' file:

(add-hook 'compilation-mode-hook 'viper-change-state-to-emacs)

The Emacs distribution has a Viper manual. For more details on setting Viper up, you should read that manual.

The vi editor has these things called editing modes. An editing mode defines how the editor responds to your keystrokes. Vi has three editing modes: insert mode, replace mode and command mode. If you run Viper, there is also the Emacs mode. Emacs indicates which mode you are in by showing one of `<I>', `<R>', `<V>', `<E>' on the statusbar correspondingly for the Insert, Replace, Command and Emacs modes. Emacs also shows you the mode by the color of the cursor. This makes it easy for you to keep track of which mode you are in.

• Insert mode: When you are in insert mode, the editor simply inserts the things that you type into the text that is being edited. If there are any characters in front of your cursor, these characters are pushed ahead and they are not overwritten. Under Viper, when you are in insert mode, the color of your cursor is green. The only key that has special meaning, while you are in insert mode is ESC. If you press the escape key, you are taken to command mode.
• Replace mode: When you are in replace mode, the editor replaces the text under the cursor with the text that is being typed. So, you want insert mode when you want to write over what's already written. Under Viper, when you are in replace mode, the color of your cursor is red. The ESC will take you to command mode.
• Command mode: When you are in command mode, every letter key that you press is a command and has a special meaning. Some of these keys allow you to navigate the text. Other keys allow you to enter either insert or replace mode. And other keys do various special things. Under Viper, when you are in command mode, the color of your cursor is white.
• Emacs mode: When you are in Emacs mode, then Viper is turned off on the specific buffer, and Emacs behaves as the default Emacs editor. You can switch between Emacs mode and Command mode by pressing CTRL-z. So to go to Emacs mode, from Insert of Replace mode, you need to go through Command mode. When you are dealing with a buffer that runs a special editing mode, like Dired, Emacs defines a specialized "command mode" for manipulating that buffer, that can be completely different from the canonical Viper command mode. You want to be in that mode to access the intended functionality. Occasionally however, you may like to hop to viper's command mode to navigate the buffer, do a search or save the buffer's contents. When you hop to one of the other three modes, the buffer will suddenly be just text to your editor.

While you are in Command mode, you can prepend keystrokes with a number. Then the subsequent keystroke will be executed as many times as the number. We now list the most important keystrokes that are available to you, while you are in Viper's command mode:

• The following keystrokes allow you to navigate the cursor around your text without making any changes on the text itself

  h

  moves one character to the left

  j

  moves down one line

  k

  moves up one line

  l

  moves one character to the left

  w

  moves forward one word

  5w

  moves forward five words (get the idea?)

  b

  moves back one word

  0

  moves to the beginning of the current line

  $

  moves to the end of the current line

  G

  moves to the last line in the file

  1G

  moves to the first line in the file

  :10

  moves to line 10 in the file (get the idea?)

  {

  moves up one paragraph

  }

  moves down one paragraph

• The following keystrokes allow you to delete text

  x

  Deletes the character under the cursor

  dd

  Deletes the current line

  4dd

  Deletes four lines

  dw

  Deletes the current word

  8dw

  Deletes the next eight words

• The following keystrokes allow you to enter Insert mode

  a

  Append text after the cursor position

  i

  Insert text at the current cursor position

  o

  Insert text on a new line bellow the current line

  O

  Insert text on a new line above the current line

• The following keystrokes allow you to enter Replace mode.

  R

  Replace text at the cursor position and stay in Replace mode.

  s

  Replace (substitute) only the character at the cursor position, and enter Insert mode for all subsequent characters.

• The following commands handle file input/output. All of these commands are prepended by the : character. The : character is used for commands that require many characters to be properly expressed. The full text of these commands is entered in the minibuffer. Under viper, the minibuffer itself can run under insert, replace and command mode. By default you get insert mode, but you can switch to command mode by pressing ESC.

  :w

  Save the file to the disk

  :w!

  Force the file to be saved to disk even when file permissions do not allow it but you have the power to overrule the permissions.

  :w filename <RET>

  Save the file to the disk under a specific filename. When you press SPACE Emacs inserts the full pathname of the current directory for you, which you can edit if you like.

  :w! filename <RET>

  Force the file to be saved to the disk under a specific filename.

  :r filename <RET>

  Paste a file from the disk at the cursor's current position.

  :W

  Save all the files on all the Emacs buffers that correspond to open files.

  :q

  Kill the buffer. This does not quite the editor at expert level 3.

  :q!

  Kill the buffer even if the contents are not saved. Use with caution!

• The following commands handle search and replace

  /string <RET>

  Search for string.

  n

  Go to the next occurence of string.

  N

  Go to the previous occurence of string.

  :%s/string1/string2/g <RET>

  Replace all occurences of string1 with string2. Use this with extreme caution!

• The following commands handle undo

  u

  Undo the previous change. Press again to undo the undo

  .

  Press this if you want to repeat the undo further.

These are enough to get you started. Getting used to dealing with the modes and learning the commands is a matter of building finger habits. It may take you a week or two before you become comfortable with Viper. When Viper becomes second nature to you however, you won't want to tolerate what you used to use before.

3.6 Navigating source code

When you develop software, you need to edit many files at the same time, and you need an efficient way to switch from one file to another. The most general solution in Emacs is by going through Dired, the Emacs Directory Editor.

To use Dired effectively, we recommend that you add the following customizations to your `.emacs' file: First, add

(add-hook 'dired-load-hook (function (lambda () (load "dired-x"))))
(setq dired-omit-files-p t)

to activate the extended features of Dired. Then add the following key-bindings to the global keymap:

(global-set-key [f1] 'dired)
(global-set-key [f2] 'dired-omit-toggle)
(global-set-key [f3] 'shell)
(global-set-key [f4] 'find-file)
(global-set-key [f5] 'compile)
(global-set-key [f6] 'visit-tags-table)
(global-set-key [f8] 'add-change-log-entry-other-window)
(global-set-key [f12] 'make-frame)

If you use viper (see section Using vi emulation), you should also add the following customization to your `.emacs':

(add-hook 'compilation-mode-hook 'viper-change-state-to-emacs)

With these bindings, you can navigate from file to file or switch between editing and the shell simply by pressing the right function keys. Here's what these key bindings do:

f1

Enter the directory editor.

f2

Toggle the omission of boring files.

f3

Get a shell at the current Emacs window.

f4

Jump to a file, by filename.

f5

Run a compilation job.

f6

Load a `TAGS' file.

f8

Update the `ChangeLog' file.

f12

Make a new frame.

When you first start Emacs, you should create a few frames with f12 and move them around on your screen. Then press f1 to enter the directory editor and begin navigating the file system. To select a file for editing, move the cursor over the filename and press enter. You can select the same file from more than one emacs window, and edit different parts of it in every different window, or use the mouse to cut and paste text from one part of the file to another. If you want to take a direct jump to a specific file, and you know the filename of that file, it may be faster to press f4 and enter the filename rather than navigate your way there through the directory editor.

To go down a directory, move the cursor over the directory filename and press RET. To go up a few directories, press f1 and when you are prompted for the new directory, with the current directory as the default choice, erase your way up the hierarchy and press RET. To jump to a substantially different directory that you have visited recently, press f1 and then when prompted for the destination directory name, use the cursor keys to select the directory that you want among the list of directories that you have recently visited.

While in the directory navigator, you can use the cursor keys to move to another file. Pressing <RET> will bring that file up for editing. However there are many other things that Dired will let you do instead:

Z

Compress the file. If already compressed, uncompress it.

L

Parse the file through the Emacs Lisp interpreter. Use this only on files that contain Emacs Lisp code.

I, N

Visit the current file as an Info file, or as a man page. See section Browsing documentation.

d

Mark the file for deletion

u

Remove a mark on the file for deletion

x

Delete all the files marked for deletion

C destination <RET>

Copy the file to destination.

R filename <RET>

Rename the file to filename.

+ directoryname <RET>

Create a directory with name directoryname.

Dired has many other features. See the GNU Emacs User Manual, for more details.

Emacs provides another method for jumping from file to file: tags. Suppose that you are editing a C program whose source code is distributed in many files, and while editing the source for the function foo, you note that it is calling another function gleep. If you move your cursor on gleep, then Emacs will let you jump to the file where gleep is defined by pressing M-.. You can also jump to other occurences in your code where gleep is invoked by pressing M-,. In order for this to work, you need to do two things: you need to generate a tags file, and you need to tell emacs to load the file. If your source code is maintained with the GNU build system, you can create that tags files by typing:

% make tags

from the top-level directory of your source tree. Then load the tags file in Emacs by navigating Dired to the top-level directory of your source code, and pressing f6.

While editing a file, you may want to hop to the shell prompt to run a program. You can do that at any time, on any frame, by pressing f3. To get out of the shell, and back into the file that you were editing, enter the directory editor by pressing f1, and then press <RET> repeatedly. The default selections will take you back to the file that you were most recently editing on that frame.

One very nice feature of Emacs is that it understands tar files. If you have a tar file `foo.tar' and you select it under Dired, then Emacs will load the entire file, parse it, and let you edit the individual files that it includes directly. This only works, however, when the tar file is not compressed. Usually tar files are distributed compressed, so you should uncompress them first with Z before entering them. Also, be careful not to load an extremely huge tar file. Emacs may mean "eating memory and constantly swapping" to some people, but don't push it!

Another very powerful feature of Emacs is the Ange-FTP package: it allows you to edit files on other computers, remotely, over an FTP connection. From a user perspective, remote files behave just like local files. All you have to do is press f1 or f4 and request a directory or file with filename following this form:

/username@host:/pathname

Then Emacs will access for you the file `/pathname' on the remote machine host by logging in over FTP as username. You will be prompted for a password, but that will happen only once per host. Emacs will then download the file that you want to edit and let you make your changes locally. When you save your changes, Emacs will use an FTP connection again to upload the new version back to the remote machine, replacing the older version of the file there. When you develop software on a remote computer, this feature can be very useful, especially if your connection to the Net is over a slow modem line. This way you can edit remote files just like you do with local files. You will still have to telnet to the remote computer to get a shell prompt. In Emacs, you can do this with M-x telnet. An advantage to telneting under Emacs is that it records your session, and you can save it to a file to browse it later.

While you are making changes to your files, you should also be keeping a diary of these changes in a `ChangeLog' file (see section Maintaining the documentation files). Whenever you are done with a modification that you would like to log, press f8, while the cursor is still at the same file, and preferably near the modification (for example, if you are editing a C program, be inside the same C function). Emacs will split the frame to two windows. The new window brings up your `ChangeLog' file. Record your changes and click on the status bar that separates the two windows with the 2nd mouse button to get rid of the `ChangeLog' file. Because updating the log is a frequent chore, this Emacs help is invaluable.

If you would like to compile your program, you can use the shell prompt to run `make'. However, the Emacs way is to use the M-x compile command. Press f5. Emacs will prompt you for the command that you would like to run. You can enter something like: `configure', `make', `make dvi', and so on (see section Installing a GNU package). The directory on which this command will run is the current directory of the current buffer. If your current buffer is visiting a file, then your command will run on the same directory as the file. If your current buffer is the directory editor, then your command will run on that directory. When you press <RET>, Emacs will split the frame into another window, and it will show you the command's output on that window. If there are error messages, then Emacs converts these messages to hyperlinks and you can follow them by pressing <RET> while the cursor is on them, or by clicking on them with the 2nd mouse button. When you are done, click on the status bar with the 2nd mouse button to get the compilation window off your screen.

3.7 Using Emacs as an email client

You can use Emacs to read your email. If you maintain free software, or in general maintain a very active internet life, you will get a lot of email. The Emacs mail readers have been designed to address the needs of software developers who get endless tons of email every day.

Emacs has two email programs: Rmail and Gnus. Rmail is simpler to learn, and it is similar to many other mail readers. The philosophy behind Rmail is that instead of separating messages to different folders, you attach labels to each message but leave the messages on the same folder. Then you can tell Rmail to browse only messages that have specific labels. Gnus, on the other hand, has a rather eccentric approach to email. It is a news-reader, so it makes your email look like another newsgroup! This is actually very nice if you are subscribed to many mailing lists and want to sort your email messages automatically. To learn more about Gnus, read the excellent Gnus manual. In this manual, we will only describe Rmail.

When you start Rmail, it moves any new mail from your mailboxes to the file `~/RMAIL' in your home directory. So, the first thing you need to tell Rmail is where your mailboxes are. To do that, add the following to your `.emacs':

(require 'rmail)
(setq rmail-primary-inbox-list
      (list "mailbox1" "mailbox2" ...))

If your mailboxes are on a filesystem that is mounted to your computer, then you just have to list the corresponding filenames. If your mailbox is on a remote computer, then you have to use the POP protocol to download it to your own computer. In order for this to work, the remote computer must support POP. Many hobbyist developers receive their email on an internet provider computer that is connected to the network 24/7 and download it on their personal computer whenever they dial up.

For example, if karl@whitehouse.gov is your email address at your internet provider, and they support POP, you would have to add the following to your `.emacs':

(require 'rmail)
(setq rmail-primary-inbox-list
      (list "po:karl"))
(setenv "MAILHOST" "whitehouse.gov")
(setq rmail-pop-password-required t)
(setq user-mail-address "karl@whitehouse.gov")
(setq user-full-name "President Karl Marx")

The string `"po:username"' is used to tell the POP daemon which mailbox you want to download. The environment variable MAILHOST tells Emacs which machine to connect to, to talk with a POP daemon. We also tell Emacs to prompt in the minibuffer to request the password for logging in with the POP daemon. The alternative is to hardcode the password into the `.emacs' file, but doing so is not a very good idea: if the security of your home computer is compromised, the cracker also gets your password for another system. Emacs will remember the password however, after the first time you enter it, so you won't have to enter it again later, during the same Emacs session. Finally, we tell Emacs our internet provider's email address and our "real name" in the internet provider's account. This way, when you send email from your home computer, Emacs will spoof it to make it look like it was sent from the internet provider's computer.

In addition to telling Rmail where to find your email, you may also want to add the following configuration options:

1. Quote messages that you respond to with the > prefix:

   (setq mail-yank-prefix ">")

2. Send yourself a blind copy of every message

   (setq mail-self-blind t)

3. Alternatively, archive all your outgoing messages to a separate file:

   (setq mail-archive-file-name "/home/username/mail/sent-mail")

4. To have Rmail insert your signature in every message that you send:

   (setq mail-signature t)

   and add the actual contents of your signature to `.signature' at your home directory.

Once Rmail is configured, to start downloading your email, run M-x rmail in Emacs. Emacs will load your mail, prompt you for your POP password if necessary, and download your email from the internet provider. Then, Emacs will display the first new message. You may quickly navigate by pressing n to go to the next message or p to go to the previous message. It is much better however to tell Emacs to compile a summary of your messages and let you to navigate your mailbox using the summary. To do that, press h. Emacs will split your frame to two windows: one window will display the current message, and the other window the summary. A highlighted bar in the summary indicates what the current message is. Emacs will also display any labels that you have associated with your messages. While the current buffer is the summary, you can navigate from message to message with the cursor keys (up and down in particular). You can also run any of the following commands:

h

display a summary of all the messages

s

save any changes made to the mail box

<

go to the first message in the summary

>

go to the last message in the summary

g

download any new email

r

reply to a message

f

forward a message

m

compose a new message

d

delete the current message

u

undelete the current message

x

expunge messages marked for deletion

a label <RET>

add the label label to the current message

k label <RET>

remove the label label from the current message

l label <RET>

display a summary only of the messages with label label

o folder <RET>

add the current message to another folder

w filename <RET>

write the body of the current message to a file

Other than browsing email, here is some things that you will want to do:

• Compose a message: To compose a message press m. Emacs will take you to a new buffer where you can write the actual contents of your message. Emacs separates this buffer with a line that says:

  --text follows this line--

  Before this line you may edit the message's headers. After this line, you edit the actual body of the of the message. When you are done composing the message, you can do one of the following:

  C-c C-w

  Insert the signature

  C-c C-y

  Quote (yank) the current message

  C-c C-c

  Send the message

  Mail → Cancel

  Cancel the message

  These commands are also available when you reply to or forward email messages.

• Reply to a message: To reply to a message press r. Emacs will do the same thing that it does when you ask it to compose a message. The only difference is that it writes the headers of the message for you automatically such that the response is sent to all the people that have received the original message. You may edit the headers to add or remove recipient email addresses. Emacs will not quote the message that you respond to by default. To quote it use C-c C-y.
• Forward a message: To forward a message press f. Emacs will write the headers for you and it will also quote the message that you are forwarding, however it will not prefix it with `>' (or whatever character you use to prefix messages that you reply to). The quoted message is clearly delimited with markers that indicate that it is the forwarded message. You can add commentary, preferably, before the markers so that the recipient can see it before seeing the forwarded message.

In every one of these three cases you may need to edit the message's headers. The most commonly used header entries that Emacs recognizes are:

`To:'

list address of the recipient to whom the message is directed

`Cc:'

list addresses of other recipients that need to receive courtesy copies of the message

`BCC:'

list addresses of other recipients to send a copy to, without showing their email address on the actual message

`FCC:'

list folders (filenames) where you would like the outgoing message to be appended to

`Subject:'

the subject field for the message

The fields `To:', `CC:', `BCC:' and `FCC:' can also have continuation lines: any subsequent lines that begin with a space are considered part of the field.

3.8 Handling patches

Believe it or not, I really don't know how to do that. I need a volunteer to explain this to me so I can explain it then in this section

3.9 Inserting copyright notices with Emacs

When you develop free software, you must place copyright notices at every file that invokes the General Public License. If you don't place any notice whatsoever, then the legal meaning is that you refuse to give any permissions whatsoever, and the software consequently is not free. For more details see Applying the GPL. Many hackers, who don't take the law seriously, complain that adding the copyright notices takes too much typing. Some of these people live in countries where copyright is not really enforced. Others simply ignore it.

There is an Emacs package, called `gpl', which is currently distributed with Autotoolset, that makes it possible to insert and maintain copyright notices with minimal work. To use this package, in your `.emacs' you must declare your identity by adding the following commands:

(setq user-mail-address "me@here.com")
(setq user-full-name "My Name")

Then you must require the packages to be loaded:

(require 'gpl)
(require 'gpl-copying)

This package introduces the following commands:

gpl

Insert the standard GPL copyright notice using appropriate commenting.

gpl-fsf

Toggle FSF mode. Causes the gpl command to insert a GPL notice for software that is assigned to the Free Software Foundation. The gpl command autodetects what type of file you are editing, from the filename, and uses the appropriate commenting.

gpl-personal

Toggle personal mode. Causes the gpl command to insert a GPL notice for software in which you keep the copyright.

If you are routinely assigning your software to an organization other than the Free Software Foundation, then insert:

(setq gpl-organization "name")

after the `require' statements in your `.emacs'.

3.10 Hacker sanity with Emacs

Every once in a while, after long heroic efforts in front of the computer monitor, a software developer will need to some counseling to feel better about herself. In RL (real life) counseling is very expensive and it also involves getting up from your computer and transporting yourself to another location, which decreases your productivity. Emacs can help you. Run M-x doctor, and you will talk to a psychiatrist for free.

Many people say that hackers work too hard and they should go out for a walk once in a while. In Emacs, it is possible to do that without getting up from your chair. To enter an alternate universe, run M-x dunnet. Aside from being a refreshing experience, it is also a very effective way to procrastinate away work that you don't want to do. Why do today, what you can postpone for tomorrow?

3.11 Further reading on Emacs

This chapter should be enough to get you going with GNU Emacs. This is really all you need to know to use Emacs to develop software. However, the more you learn about Emacs, the more effectively you will be able to use it, and there is still a lot to learn; a lot more than we can fit in this one chapter. In this section we refer to other manuals that you can read to learn more about Emacs. Unlike many proprietary manuals that you are likely to find in bookstores, these manuals are free (see section Why free software needs free documentation). Whenever possible, please contribute to the GNU project by ordering a bound copy of the free documentation from the Free Software Foundation, or by contributing a donation.

The Free Software Foundation publishes the following manuals on Emacs:

The Emacs Editor

This manual tells you all there is to know about all the spiffy things that Emacs can do, except for a few things here and there that are so spiffy that they get to have their own separate manual. The printed version, published by the Free Software Foundation, features our hero, Richard Stallman, riding a gnu. It also includes the GNU Manifesto. The machine readable source for the manual is distributed with GNU Emacs.

Programming in Emacs Lisp

A wonderful introduction to Emacs Lisp, written by Robert Chassell. If you want to learn programming in Emacs Lisp, start by reading this manual. You can order this manual as a bound book from the Free Software Foundation. You can also download a machine readable copy of the manual from any GNU ftp site. Look for `elisp-manual-20-2.5.tar.gz'.

The GNU Emacs Lisp Reference Manual

This is a comprehensive reference manual for the Emacs Lisp language. You can also order this manual as a bound book from the Free Software Foundation. You can also download a machine readable copy of the manual from any GNU ftp site. Look for `emacs-lisp-intro-1.05.tar.gz'.

The following manuals are also distributed with the GNU Emacs source code and they make for some very fun reading:

Gnus Manual

Gnus is the Emacs newsreader. You can also use it to sort out your email, especially if you are subscribed to twenty mailing lists and receive tons of email every day. This manual will tell you all you need to know about Gnus to use it effectively. (`gnus.dvi')

CC Mode

The Emacs C editing mode will help you write C code that is beautifully formatted and consistent with the GNU coding standards. If you develop software for an organization that follows different coding standards, you will have to customize Emacs to use their standards instead. If they are lame and haven't given you Elisp code for their standards, then this manual will show you how to roll your own. (`cc-mode.dvi')

Common Lisp Extensions

Emacs has a package that introduces many Common Lisp extensions to Emacs Lisp. This manual describes what extensions are available and how to use them. (`cl.dvi')

Writing Customization Definitions

Recent versions of Emacs have an elaborate user-friendly customization interface that will let users customize Emacs and update their `.emacs' files automatically for them. If you are writing large Emacs packages, it is very easy to add a customization interface to them. This manual explains how to do it. (`customize.dvi')

The Emacs Widget Library

It is possible to insert actual widgets in an Emacs buffer that are bound to Emacs Lisp functions. This feature of Emacs is used, for example, in the newly introduced customization interface. This manual documents the Elisp API for using these widgets in your own Elisp packages. (`widget.dvi')

RefTeX User Manual

If you are writing large documents with LaTeX that contain a lot of cross references, then the RefTeX package will make your life easier. (`reftex.dvi')

Ediff User's Manual

Ediff is a comprehensive package for dealing with patches under Emacs. If you receive a lot of patches to your software projects from contributors, you can use Ediff to apply them to your source code. (`ediff.dvi')

Supercite User's Manual

If you think that quoting your responses to email messages with `>' is for lamers and you want to be elite, then use Supercite. (`sc.dvi')

Viper Is a Package for Emacs Rebels

This manual has more than you will ever need to know about Viper, the Emacs vi emulation. Using vi emulation, actually describes all the features of Viper that you will ever really need. But still, it's a good reading for a long airplane trip. (`viper.dvi')

4. Compiling with Makefiles

In this chapter we describe how to use the compiler to compile simple software and libraries, and how to use makefiles.

4.1 Compiling simple programs

It is very easy to compile simple C programs on the GNU system. For example, consider the famous "Hello world" program:

`hello.c'

#include <stdio.h> int main () { printf ("Hello world\n"); }

The simplest way to compile this program is to type:

% gcc hello.c

on your shell. The resulting executable file is called `a.out' and you can run it from the shell like this:

% ./a.out
Hello world

To cause the executable to be stored under a different filename pass the `-o' flag to the compiler:

% gcc hello.c -o hello
% ./hello
Hello world

Even with simple one-file hacks like this, the GNU compiler can accept many options that modify its behaviour:

`-g'

The `-g' flag causes the compiler to output debugging information to the executable. This way, you can step your program through a debugger if it crashes. (FIXME: Crossreference)

`-O, -O2, -O3'

The `-O', `-O2', `-O3' flags activate optimization. The numbers are called optimization levels. When you compile your program with optimization enabled, the compiler applies certain algorithms to the machine code output to make it go faster. The cost is that your program compiles much more slowly and that although you can step it through a debugger if you used the `-g' flag, things will be a little strange. During development the programmer usually uses no optimization, and only activates it when person is about to run the program for a production run. A good advice: always test your code with optimization activated as well. If optimization breaks your code, then this is telling you that you have a memory bug. Good luck finding it.

`-Wall'

The `-Wall' flag tells the compiler to issue warnings when it sees bad programming style. Some of these warning catch actual bugs, but occasionally some of the warnings complain about something correct that you did on purpose. For this reason this flag is feature is not activated by default.

Here are some variations of the above example:

% gcc -g -O3 hello.c hello
% gcc -g -Wall hello.c -o hello
% gcc -g -Wall -O3 hello.c -o hello

To run a compiled executable in the current directory just type its name, prepended by `./'. In general, once you compile a useful program, you should install it so that it can be run from any current directory, simply by typing its name without prepending `./'. To install an executable, you need to move it to a standard directory such as `/usr/bin' or `/usr/local/bin'. If you don't have permissions to install files there, you can instead install them on your home directory at `/home/username/bin' where username is your username. When you write the name of an executable, the shell looks for the executable in a set of directories listed in the environment variable `PATH'. To add a nonstandard directory to your path do

% export PATH="$PATH:/home/username/bin"

if you are using the Bash shell, or

% setenv PATH "$PATH:/home/username/bin"

if you are using a different shell.

4.2 Programs with many source files

Now let's consider the case where you have a much larger program made of source files `foo1.c', `foo2.c', `foo3.c' and header files `header1.h' and `header2.h'. One way to compile the program is like this:

% gcc foo1.c foo2.c foo3.c -o foo

This is fine when you have only a few files to deal with. Eventually, when you have more than a few dozen files, it becomes wasteful to compile all of the files, all the time, every time you make a change in only one of the files. For this reason, the compiler allows you to compile every file separately into an intermediate file called object file, and link all the object files together at the end. This can be done with the following commands:

% gcc -c foo1.c
% gcc -c foo2.c
% gcc -c foo3.c
% gcc foo1.o foo2.o foo3.o -o foo

The first three commands generate the object files `foo1.o', `foo2.o', `foo3.o' and the last command links them together to the final executable file `foo'. The `*.o' suffix is reserved for use only by object files.

If you make a change only in `foo1.c', then you can rebuild `foo' like this:

% gcc -c foo1.c
% gcc foo1.o foo2.o foo3.o -o foo

The object files `foo2.o' and `foo3.o' do not need to be rebuilt since only `foo1.c' changed, so it is not necessary to recompile them.

Object files `*.o' contain definitions of variables and subroutines written out in assembly (machine language "pseudo code"). Most of these definitions will eventually be embedded in the final executable program at a specific address. At this stage however these memory addresses are not known so they are being referred to symbolically. These symbolic references are called symbols. It is possible to list the symbols defined in an object file with the `nm' command. For example:

% nm xmalloc.o
         U error
         U malloc
         U realloc
00000000 D xalloc_exit_failure
00000000 t xalloc_fail
00000004 D xalloc_fail_func
00000014 R xalloc_msg_memory_exhausted
00000030 T xmalloc
00000060 T xrealloc

The first column lists the symbol's address within the object file, when the symbol is actually defined in that object file. The second column lists the symbol type. The third column is the symbolic name of the symbol. In the final executable, these names become irrelevant. The following types commonly occur:

`T'

A function definition

`t'

A private function definition. Such functions are defined in C with the keyword static.

`D'

A global variable

`R'

A read-only (const) global variable

`U'

A symbol used but not defined in this object file.

For more details, see the Binutils manual.

The job of the compiler is to translate all the C source files to object files containing a corresponding set of symbolic definitions. It is the job of another program, the linker, to put the object files together, resolve and evaluate all the symbolic addresses, and build a complete machine language program that can actually be executed. When you ask `gcc' to link the object files into an executable, the compiler is actually running the linker to do the job.

During the process of linking, all the machine language instructions that refer to a specific memory address need to be modified to use the correct addresses within the executable, as opposed to the addresses within their object file. This becomes an issue when you want to your program to load and link compiled object files during run-time instead of compile-time. To make such dynamic linking possible, your symbols need to be relocatable. This means that your symbols definitions must be correct no matter where you place them in memory. There should be no memory addresses that need to be modified. One way to do this is by referring to memory addresses within the object file by giving an offset from the referring address. Memory addresses outside the object file must be treated as interlibrary dependencies and you must tell the compiler what you expect them to be when you attempt to build relocatable machine code. Unfortunately some flavours of Unix do not handle interlibrary dependencies correctly. Fortunately, all of this mess can be dealt with in a uniform way, to the extent that this is possible, by using GNU Libtool. See section Using Libtool, for more details.

On GNU and Unix, all compiled languages compile to object files, and it is possible, in principle, to link object files that have originated from source files written in different programming languages. For example it is possible to link source code written in Fortran together with source code written in C or C++. In such cases, you need to know how the compiler converts the names with which the program language calls its constructs (such as variable, subroutines, etc.) to symbol names. Such conversions, when they actually happen, are called name-mangling. Both C++ and Fortran do name-mangling. C however is a very nice language, because it does absolutely no name-mangling. This is why when you want to write code that you want to export to many programming languages, it is best to write it in C. See section Using Fortran effectively, for more details on how to deal with the name-mangling done by Fortran compilers.

4.3 Building libraries

In many cases a collection of object files form a logical unit that is used by more than one executable. On both GNU and Unix systems, it is possible to collect such object files and form a library. On the GNU system, to create a library, you use the `ar' command:

ar cru libfoo.a foo1.o foo2.o foo3.o

This will create a file `libfoo.a' from the object files `foo1.o', `foo2.o' and `foo3.o'. The suffix `*.a' is reserved for object file libraries. Before using the library, it needs to be "blessed" by a program called `ranlib':

% ranlib libfoo.a

The GNU system, and most Unix systems require that you run `ranlib', but there have been some Unix systems where doing so is not necessary. In fact there are Unix systems, like some versions of SGI's Irix, that don't even have the `ranlib' command!

The reason for this is historical. Originally ar was meant to be used merely for packaging files together. The more well known program tar is a descendant of ar that was designed to handle making such archives on a tape device. Now that tape devices are more or less obsolete, tar is playing the role that was originally meant for ar. As for ar, way back, some people thought to use it to package *.o files. However the linker wanted a symbol table to be passed along with the archive. So the ranlib program was written to generate that table and add it to the *.a file. Then some Unix vendors thought that if they incorporated ranlib to ar then users wouldn't have to worry about forgetting to call ranlib. So they provided ranlib but it did nothing. Some of the more evil ones dropped it all-together breaking many people's scripts.

Once you have a library, you can link it with other object files just as if it were an object file itself. For example

% gcc bar.o libfoo.a -o foo

using `libfoo.a' as defined above, is equivalent to writing

% gcc bar.o foo1.o foo2.o foo3.o -o foo

Libraries are particularly useful when they are installed. To install a library you need to move the file `libfoo.a' to a standard directory. The actual location of that directory depends on your compiler. The GNU compiler looks for installed libraries in `/usr/lib' and `/usr/local/lib'. Because many Unix systems also use the GNU compiler, it is safe to say that both of these directories are standard in these systems too. However there are some Unix compilers that don't look at `/usr/local/lib' by default. Once a library is installed, it can be used in any project from any current directory to compile an executable that uses the subroutines that that library provides. You can direct the compiler to link an installed library with a set of executable files to form an executable by using the `-l' flag like this:

% gcc -o foo bar.o -lfoo

Note that if the filename of the library is `libfoo.a', the corresponding argument to the `-l' flag must be only the substring `foo', hence `-lfoo'. Libraries must be named with names that have the form `lib*.a'. If you have installed the `libfoo.a' library in a non-standard directory, you can tell the linker to look for the library in that directory as well by using the `-L' flag. For example, if the library was installed in `/home/lf/lib' then we would have to invoke the linking like this:

gcc -o bar bar.o -L/home/lf/lib -lfoo

The `-L' flag must appear before the `-l' flag.

Some people like to pass `-L.' to the compiler so they can link uninstalled libraries in the current working directory using the `-l' flag instead of typing in their full filenames. The idea is that they think "it looks better" that way. Actually this is considered bad style. You should use the `-l' flag to link only libraries that have already been installed and use the full pathnames to link in uninstalled libraries. The reason why this is important is because, even though it makes no difference when dealing with ordinary libraries, it makes a lot of difference when you are working with shared libraries. (FIXME: Crossreference). It makes a difference whether or not you are linking to an uninstalled or installed shared library, and in that case the `-l' semantics mean that you are linking an installed shared library. Please stick to this rule, even if you are not using shared libraries, to make it possible to switch to using shared libraries without too much hassle.

Also, if you are linking in more than one library, please pay attention to the order with which you link your libraries. When the linker links a library, it does not embed into the executable code the entire library, but only the symbols that are needed from the library. In order for the linker to know what symbols are really needed from any given library, it must have already parsed all the other libraries and object files that depend on that library! This implies that you first link your object files, then you link the higher-level libraries, then the lower-level libraries. If you are the author of the libraries, you must write your libraries in such a manner, that the dependency graph of your libraries is a tree. If two libraries depend on each other bidirectionally, then you may have trouble linking them in. This suggests that they should be one library instead!

4.4 Dealing with header files

In general libraries are composed of many `*.c' files that compile to object files, and a few header files (`*.h'). The header files declare the resources that are defined by the library and need to be included by any source files that use the library's resources. In general a library comes with two types of header files: public and private. The public header files declare resources that you want to make accessible to other software. The private header files declare resources that are meant to be used only for developing the library itself. To make an installed library useful, it is also necessary to install the corresponding public header files. The standard directory for installing header files is `/usr/include'. The GNU compiler also understands `/usr/local/include' as an alternative directory. When the compiler encounters the directive

#include <foo.h>

it searches these standard directories for `foo.h'. If you have installed the header files in a non-standard directory, you can tell the compiler to search for them in that directory by using the `-I' flag. For example, to build a program `bar' from a source file `bar.c' that uses the libfoo library installed at `/home/username' you would need to do the following:

% gcc -c -I/home/lf/include bar.c
% gcc -o bar bar.o -L/home/lf/lib -lfoo

You can also do it in one step:

% gcc -I/home/lf/include -o bar bar.o -L/home/lf/lib -lfoo

For portability, it is better that the `-I' appear before the filenames of the source files that we want to compile.

A good coding standard is to distinguish private from public header files in your source code by including private header files like

#include "private.h"

and public header files like

#include <public.h>

in your implementation of the library, even when the public header files are not yet installed while building the library. This way source code can be moved in or out of the library without needing to change the header file inclusion semantics from `<..>' to `".."' back and forth. In order for this to work however, you must tell the compiler to search for "installed" header files in the current directory too. To do that you must pass the `-I' flag with the current directory `.' as argument (`-I.').

In many cases a header file needs to include other header files, and it is very easy for some header files to be included more than once. When this happens, the compiler will complain about multiple declarations of the same symbols and throw an error. To prevent this from happening, please surround the contents of your header files with C preprocessor conditional like this:

#ifndef __defined_foo_h
#define __defined_hoo_h
[...contents...]
#endif

The defined macro __defined_foo_h is used as a flag to indicate that the contents of this header file have been included. To make sure that each one of these macros is unique to only one header file, please combine the prefix __defined with the pathname of the header file when it gets installed. If your header file is meant to be installed as in `/usr/local/include/foo.h' or `/usr/include/foo.h' then use __defined_foo_h. If your header files is meant to be installed in a subdirectory like `/usr/include/dir/foo.h' then please use __defined_dir_foo_h instead.

In principle, every library can be implemented using only one public header file and perhaps only one private header file. There are problems with this approach however:

• This header file grows to be very large and slows down compilation by processing many symbols declarations that are not relevant to the specific source file that is being compiled.
• If you change the contents of the header file, it will be difficult to determine the minimum set of object files that need to be rebuilt since the change could reflect to all of them, in principle. So you will end up rebuilding the entire library unnecessarily.

For small libraries, these problems are not very serious. For large libraries however, you may need to split the one large header file to many smaller files. Sometimes a good approach is to have a matching header file for each source file, meaning that if there is a `foo.c' there should be a `foo.h'. Some other times it is better to distribute declarations among header files by splitting the library's provided resources to various logical categories and declaring each category on a separate header file. It is up to the developer to decide how to do this best.

Once this decision is made, a few issues still remain:

• We don't want to burden the users of the library that use the library's features extensively with including many header files. It should be possible to declare the entire library with only one inclusion.
• The more header files we use, the more likely it is that their filenames conflict with the filenames of header files from other installed libraries.

One way of preventing the filename conflicts is to install the library's header files in a subdirectory bellow the standard directory for installing header files. Then we install one header file in the standard directory itself that includes all the header files in the subdirectory.

For example, if the Foo library wants to install headers `foo1.h', `foo2.h' and `foo3.h', it can install them under `/usr/include/foo' and install in `/usr/include/' only a one header file `foo.h' containing only:

#include <foo/foo1.h>
#include <foo/foo2.h>
#include <foo/foo3.h>

Please name this "central" header and the directory for the subsidiary headers consistently after the corresponding library. So the `libfoo.a' library should install a central header named `foo.h' and all subsidiary headers under the subdirectory `foo'.

The subsidiary header files should be guarded with preprocessor conditionals, but it is not necessary to also guard the central header file that includes them. To make the flag macros used in these preprocessor conditionals unique, you should include the directory name in the flag macro's name. For example, `foo/foo1.h' should be guarded with

#ifndef __defined_foo_foo1_h
#define __defined_foo_foo1_h
[...contents...]
#endif

and similarly with `foo/foo2.h' and `foo/foo3.h'.

This approach creates yet another problem that needs to be addressed. If you recall, we suggested that you use the include "..." semantics for private header files and the include <...> semantics for public header files. This means that when you include the public header file `foo1.h' from one of the source files of the library itself, you should write:

#include <foo/foo1.h>

Unfortunately, if you place the `foo1.h' in the same directory as the file that attempts to include it, using these semantics, it will not work, because there is no subdirectory `foo' during compile time.

The simplest way to resolve this is by placing all of the source code for a given library under a directory and all such header files in a subdirectory named `foo'. The GNU build system in general requires that all the object files that build a specific library be under the same directory. This means that the C files must be in the same directory. It is okey however to place header files in a subdirectory.

This will also work if you have many directories, each containing the sources for a separate library, and a source file in directory `bar', for example, tries to include the header file `<foo/foo1.h>' from a directory `foo' bellow the directory containing the source code for the library libfoo. To make it work, just pass `-I' flags to the compiler for every directory of containing the source code of every library in the package. See section Libraries with Automake, for more details. It will also work even if there are already old versions of `foo/foo1.h' installed in a standard directory like `/usr/include', because the compiler will first search under the directories mentioned in the `-I' flags before trying the standard directories.

4.5 The GPL and libraries

A very common point of contention is whether or not using a software library in your program, makes your program derived work from that library. For example, suppose that your program uses the readline () function which is defined in the library `libreadline.a'. To do this, your program needs to link with this library. Whether or not this makes the program derived work makes a big difference. The readline library is free software published under the GNU General Public License, which requires that any derived work must also be free software and published under the same terms. So, if your program is derived work, you have to free it; if not, then you are not required to by the law.

When you link the library with your object files to create an executable, you are copying code from the library and combining it with code from your object files to create a new work. As a result, the executable is derived work. It doesn't matter if you create the executable by hand by running an assembler and putting it together manually, or if you automate the process by letting the compiler do it for you. Legally, you are doing the same thing.

Some people feel that linking to the library dynamically avoids making the executable derived work of the library. A dynamically linked executable does not embed a copy of the library. Instead, it contains code for loading the library from the disk during run-time. However, the executable is still derived work. The law makes no distinction between static linking and dynamic linking. So, when you compile an executable and you link it dynamically to a GPLed library, the executable must be distributed as free software with the library. This also means that you can not link dynamically both to a GPLed library and a proprietary library because the licenses of the two libraries conflict. The best way to resolve such conflicts is by replacing the proprietary library with a free one, or by convincing the owners of the proprietary library to license it as free software.

The law is actually pretty slimy about what is derived work. In the entertainment industry, if you write an original story that takes placed in the established universe of a Hollywood serial, like Star Trek, in which you use characters from that serial, like Captain Kirk, your story is actually derived work, according to the law, and Paramount can claim rights to it. Similarly, a dynamically linked executable does not contain a copy of the library itself, but it does contain code that refers to the library, and it is not self-contained without the library.

Note that there is no conflict when a GPLed utility is invoked by a proprietary program or vice versa via a system () call. There is a very specific reason why this is allowed: When you were given a copy of the invoked program, you were given permission to run it. As a technical matter, on Unix systems and the GNU system, using a program means forking some process that is already running to create a new process and loading up the program to take over the new process, until it exits. This is exactly what the system () call does, so permission to use a program implies that you have permission to call it from any other program via system (). This way, you can run GNU programs under a proprietary sh shell on Unix, and you can invoke proprietary programs from a GNU program. However, a free program that depends on a proprietary program for its operation can not be included in a free operating system, because the proprietary program would also have to be distributed with the system.

Because any program that uses a library becomes derived work of that library, the GNU project occasionally uses another license, the Lesser GPL, (often called LGPL) to copyleft libraries. The LGPL protects the freedom of the library, just like the GPL does, but allows proprietary executables to link and use LGPLed libraries. However, this permission should only be given when it benefits the free software community, and not to be nice to proprietary software developers. There's no moral reason why you should let them use your code if they don't let you use theirs. See section The LGPL vs the GPL, for a detailed discussion of this issue.

4.6 The language runtime libraries.

When you compile ordinary programs, like the hello world program the compiler will automatically link to your program a library called `libc.a'. So when you type

% gcc -c hello.c
% gcc -o hello hello.o

what is actually going on behind the scenes is:

% gcc -c hello.c
% gcc -o hello hello.c -lc

To see why this is necessary, try `nm' on `hello.o':

% nm hello.o
00000000 t gcc2_compiled.
00000000 T main
         U printf

The file `hello.o' defines the symbol `main', but it marks the symbol `printf' as undefined. The reason for this is that `printf' is not a built-in keyword of the C programming language, but a function call that is defined by the `libc.a' library. Most of the facilities of the C programming language are defined by this library. The include files `stdio.h', `stdlib.h', and so on are only header files that declare parts of the C library. You can read all about the C library in the Libc manual.

The catch is that there are many functions that you may consider standard features of C that are not included in the `libc.a' library itself. For example, all the math functions that are declared in `math.h' are defined in a library called `libm.a' which is not linked by default. So if your program is using math functions and including `math.h', then you need to explicitly link the math library by passing the `-lm' flag. The reason for this particular separation is that mathematicians are very picky about the way their math is being computed and they may want to use their own implementation of the math functions instead of the standard implementation. If the math functions were lumped into `libc.a' it wouldn't be possible to do that.

For example, consider the following program that prompts for a number and prints its square root:

`dude.c'

#include <stdio.h> #include <math.h> int main () { double a; printf ("a = "); scanf ("%f", &a); printf ("sqrt(a) = %f", sqrt(a)); }

To compile this program you will need to do:

% gcc -o dude dude.c -lm

otherwise you will get an error message from the linker about sqrt being an unresolved symbol.

On GNU, the `libc.a' library is very comprehensive. On many Unix systems however, when you use system-level features you may need to link additional system libraries such as `libbsd.a', `libsocket.a', `libnsl.a', etc. If you are linking C++ code, the C++ compiler will link both `libc.a' and the C++ standard library `libstdc++.a'. If you are also using GNU C++ features however, you will explicitly need to link `libg++.a' yourself. Also if you are linking Fortran and C code together you must also link the Fortran run-time libraries. These libraries have non-standard names and depend on the Fortran compiler that you use. (see section Using Fortran effectively) Finally, a very common problem is encountered when you are writing X applications. The X libraries and header files like to be placed in non-standard locations so you must provide system-dependent -I and -L flags so that the compiler can find them. Also the most recent version of X requires you to link in some additional libraries on top of libX11.a and some rare systems require you to link some additional system libraries to access networking features (recall that X is built on top of the sockets interface and it is essentially a communications protocol between the computer running the program and computer that controls the screen in which the X program is displayed.) FIXME: Cross references, if we explain all this in more details.

Because it is necessary to link system libraries to form an executable, under copyright law, the executable is derived work from the system libraries. This means that you must pay attention to the license terms of these libraries. The GNU `libc' library is under the LGPL license which allows you to link and distribute both free and proprietary executables. The `stdc++' library is also under terms that permit the distribution of proprietary executables. The `libg++' library however only permits you to build free executables. If you are on a GNU system, including Linux-based GNU systems, the legalese is pretty straightforward. If you are on a proprietary Unix system, you need to be more careful. The GNU GPL does not allow GPLed code to be linked against proprietary library. Because on Unix systems, the system libraries are proprietary, their terms also may not allow you to distribute executables derived from them. In practice, they do however, since proprietary Unix systems do want to attract proprietary applications. In the same spirit, the GNU GPL also makes an exception and explicitly permits the linking of GPL code with proprietary system libraries, provided that these libraries are a major component of the operating system (i.e. they are part of the compiler, or the kernel, and so on), unless the copy of the library itself accompanies the executable!

This includes proprietary `libc.a' libraries, the `libdxml.a' library in Digital Unix, proprietary Fortran system libraries like `libUfor.a', and the X11 libraries.

4.7 Basic Makefile concepts

To build a very large program, you need an extended set of invocations to the `gcc' compiler and utilities like `ar', `ranlib'. As we explained (see section Programs with many source files) if you make changes only to a few files in your source code, it is not necessary to rebuild everything; you only need to rebuild the object files that get to change because of your modifications and link those together with all the other object files to form an updated executable. The `make' utility was written mainly to automate rebuilding software by determining the minimum set of commands that need to be called to do this, and invoking them for you in the right order. It can also handle, many other tasks. For example, you can use `make' to install your program's files in the standard directories, and clean up the object files when you no longer need them.

To learn all about `make' and especially `GNU Make', please read the excellent GNU Make manual. In general, to use the GNU build system you don't need to know the most esoteric aspects of the GNU make, because makefiles will be automatically compiled for you from higher level descriptions. However it is important to understand the basic aspects of `make' to use the GNU build system effectively. In the following sections we will explain only these basic aspects.

The `make' utility reads its instructions from a file named `Makefile' in the current directory. `make' itself has no knowledge about the syntax of the files that it works with, and it relies on the instructions in `Makefile' to figure out what it needs to do. A makefile is essentially a list of rules. Each rule has the form:

TARGET: DEPENDENCIES
TAB COMMAND
TAB .....
TAB .....
[BLANK LINE]

The TABs are mandatory. The blank line at the end of the rule definition is not necessary when using GNU make but it is a good idea if you would like backwards compatibility with Unix.

• The target is either the name of a file that is generated by a program or the name of an action to carry out. Object files and executable files are examples of files that are generated by other programs. Cleaning up the object files is an example of an action that we might want to carry out. Targets that correspond to an action are sometimes called phony targets.
• In general a dependency is a file that is used as input to create a target. If a target has more than one dependencies, they must be separated by spaces, but they must remain on the same line. It is possible for a target to have no dependencies. In that case, the space after the semicolon must be left blank. It is also possible for a target, even one that represents an action, to be a dependency for another target.
• The commands following the target and the dependencies must be prepended with TAB. If the target is a file, then the commands explain how to create that file. If the target is an action, then the commands describe the action. These commands are your usual shell commands that you get to type in your prompt.

When you invoke `make' you must tell it which target you want to build. If you don't specify a target, then `make' will build the first target that is mentioned in the makefile.

When we talk about `make' building a target, we mean that we want `make' to do the following things:

1. Build the dependencies. If a dependency is a file written by you, this means do nothing. If a dependency is a target defined elsewhere in the makefile, this means build that target first, which recurses this two-step process.
2. If at least one of the dependencies is newer than the target, or the file with the name of the target does not exist, then invoke the commands that correspond to the target. If the target is a file, then the commands should create the file. If the target is an action, then the commands will not create any file, but they will carry out the action.

Assuming that both a dependency and the target are files, we say that the dependency is newer than the target, if the dependency was last modified more recently than the target. The target then should be rebuild to reflect the most recent modifications of the dependency.

If the requested target exists as a file, and there are no dependencies newer than the target, then `make' will do nothing except printing a message saying that it has nothing to do. If the requested target is an action, no file will ever exist having the same name as the name describing the action, so every time you ask `make' to build that target, it will always carry out your request. If one of the dependencies is a target corresponding to an action, `make' will always attempt to build it and consequently always carry out that action. These three observations are only corollaries of the general algorithm.

To see how all this comes together in practice let's write a `Makefile' for compiling the hello world program. The simplest way to do this is with the following makefile:

hello: hello.c
TAB gcc -o hello hello.c

This simply says that the target `hello' is being built from the file `hello.c' by invoking the `gcc' command

% gcc -o hello hello.c

A more complicated way of doing the same thing is by explicitly building the intermediate object file:

hello: hello.o
TAB gcc -o hello hello.o

hello.o: hello.c
TAB gcc -c hello.c

Note that the target that we really want to build, `hello' is listed first, to make sure that it is the default target. Finally, we can add two more phony targets install and clean to install the hello world program and clean up the build after installation. We get then the following:

hello: hello.o
TAB gcc -o hello hello.o

hello.o: hello.c
TAB gcc -c hello.c

clean:
TAB rm -f hello hello.o

install: hello
TAB mkdir -p /usr/local/bin
TAB rm -f /usr/local/bin/hello
TAB cp hello /usr/local/bin/hello

The clean needs no dependencies since it just does what it does. However, the install target needs to first make sure that the file `hello' exists before attempting to install it, so it is necessary to list `hello' as a dependency to install.

Please note that this simple `Makefile' is for illustration only, and it is far from ideal. For example, we use the `mkdir' command to make sure that the installation directory exists before attempting an install, but the `-p' flag is not portable in Unix. Also, we usually want to use a BSD compatible version of the `install' utility to install executables instead of `cp'. Fortunately, you will almost never have to worry about writing `clean' and `install' targets, because those will be generated for you automatically by Automake.

4.8 More about Makefiles

Now let's consider a more complicated example. Suppose that we want to build a program `foo' whose source code is four source files

foo1.c, foo2.c, foo3.c, foo4.c

and three header files:

gleep1.h, gleep2.h, gleep3.h

Suppose also, for the sake of argument, that

1. `foo1.c' includes `gleep2.h' and `gleep3.h'
2. `foo2.c' includes `gleep1.h'
3. `foo3.c' includes `gleep1.h' and `gleep2.h'
4. `foo4.c' includes `gleep3.h'

To build the executable file `foo', we need to build the object files `foo1.o', `foo2.o', `foo3.o' and `foo4.o' that correspond to the source files and link them together. If any of the `*.c' files is modified, then only the corresponding object file and the executable need to be updated. However, if one of the header files is modified, then all the object files whose corresponding `*.c' file includes the modified header file should be rebuilt. It follows that each of the object files depends on the corresponding `*.c' file and all the header files that that file includes. We get then the following `Makefile':

foo: foo1.o foo2.o foo3.o foo4.o
TAB gcc -o foo foo1.o foo2.o foo3.o foo4.o 
 
foo1.o: foo1.c gleep2.h gleep3.h
TAB gcc -c foo1.c

foo2.o: foo2.c gleep1.h
TAB gcc -c foo2.c

foo3.o: foo3.c gleep1.h gleep2.h
TAB gcc -c foo3.c

foo4.o: foo4.c gleep3.h
TAB gcc -c foo4.c

clean:
TAB rm -f foo foo1.o foo2.o foo3.o foo4.o

install: foo
TAB mkdir -p /usr/local/bin
TAB rm -f /usr/local/bin/foo
TAB cp foo /usr/local/bin/foo

This idea can be easily generalized for any program. If you would like to build more than one programs, then you should add a phony target in the beginning that depends on the programs that you want to build. The usual way we do this is by adding a line like

all: prog1 prog2 prog3

to the beginning of the `Makefile'.

Unfortunately, this `Makefile' has a lot of unnecessary redundancy:

• All object files get built the same way. It would be nice then, if we didn't have to write a rule for every one of them and instead describe how it's done in general.
• If we decide to change the compiler used, we would would need to edit the `Makefile' in 6 places. It should be easier than that.
• The list of object files foo1.o, ..., foo4.o appears in at least two places.
• The directory name `/usr/local/bin' appears in two places.

This redundancy can be eliminated by using makefile variables and abstract rules.

• Makefile variables are actually more like macro definitions. The syntax for defining a makefile variable is:

  variable = value

  Then, in every other rule or variable definition, the symbol $(variable) is substituted with value.

• An abstract rule is a definition that explains how to build a file `*.s2' from a file `*.s1', where s1 and s2 are suffixes. The general syntax for an abstract rule is:

  .s1.s2: TAB COMMAND TAB COMMAND TAB .....

  where s1 is the suffix of the source file, and s2 is the suffix of the corresponded generated file and COMMAND is the set of commands that generate `*.s2' from `*.s1'. Note that no dependencies are mentioned, because dependencies don't make sense in the general case. They must be explicitly provided for each individual case separately.

In the context of an abstract rule, the following punctuation marks have the following meanings:

`$<'

are the dependencies that changed causing the target to need to be rebuilt

`$@'

is the target

`$^'

are all the dependencies for the current rule

For example, the abstract rule for building an object file from a source file is:

.c.o:
TAB gcc -c $<

Similarly, the rule for building the executable file from a set of object files is:

.o:
TAB gcc $^ -o $@

Note that because executables don't have a suffix, we only mention the suffix of the object files. When only one suffix appears, it is assumed that it is suffix s1 and that suffix s2 is the empty string.

The suffixes involved in your abstract rules, need to be listed in a directory that takes the form:

.SUFFIXES: s1 s2 ... sn

where s1, s2, etc. are suffixes. Also, if you've written an abstract rule, you still need to write rules where you mention the specific targets and their dependencies, except that you can omit the command-part since they are covered by the abstract rule.

Putting all of this together, we can enhance our `Makefile' like this:

CC = gcc
CFLAGS = -Wall -g
OBJECTS = foo1.o foo2.o foo3.o foo4.o
PREFIX = /usr/local

.SUFFIXES: .c .o

.c.o:
TAB $(CC) $(CFLAGS) -c $<

.o:
TAB $(CC) $(CFLAGS) $^ -o $@

foo: $(OBJECTS)
foo1.o: foo1.c gleep2.h gleep3.h
foo2.o: foo2.c gleep1.h
foo3.o: foo3.c gleep1.h gleep2.h
foo4.o: foo4.c gleep3.h

clean:
TAB rm -f $(OBJECTS)

install: foo
TAB mkdir -p $(PREFIX)/bin
TAB rm -f $(PREFIX)/bin/foo
TAB cp foo $(PREFIX)/bin/foo

The only part of this Makefile that still requires some thinking on your part, is the part where you list the object files and their dependencies:

foo1.o: foo1.c gleep2.h gleep3.h
foo2.o: foo2.c gleep1.h
foo3.o: foo3.c gleep1.h gleep2.h
foo4.o: foo4.c gleep3.h

Note however, that in principle even that can be automatically generated. Even though the `make' utility does not understand C source code and can not determine the dependencies, the GNU C compiler can. If you use the `-MM' flag, then the compiler will compute and output the dependency lines that you need to include in your Makefile. For example:

% gcc -MM foo1.c
foo1.o: foo1.c gleep2.h gleep3.h
% gcc -MM foo2.c
foo2.o: foo2.c gleep1.h
% gcc -MM foo3.c
foo3.o: foo3.c gleep1.h gleep2.h
% gcc -MM foo4.c
foo4.o: foo4.c gleep3.h

Unfortunately, unlike all the other compiler features we have described up until now, this feature is not portable in Unix. If you have installed the GNU compiler on your Unix system however, then you can also do this.

Dealing with dependencies is one of the major drawbacks of writing makefiles by hand. Another drawback is that even though we have moved many of the parameters to makefile variables, these variables still need to be adjusted by somebody. There is something rude about asking the installer to edit `Makefile'. Developers that ask their users to edit `Makefile' make their user's life more difficult in an unacceptable way. Yet another annoyance is writing clean, install and such targets. Doing so every time you write a makefile gets to be tedious on the long run. Also, because these targets are, in a way, mission critical, it is really important not to make mistakes when you are writing them. Finally, if you want to use multiple directories for every one of your libraries and programs, you need to setup your makefiles to recursively call `make' on the subdirectories, write a whole lot of makefiles, and have a way of propagating configuration information to every one of these makefiles from a centralized source.

These problems are not impossible to deal with, but you need a lot of experience in makefile writing to overcome them. Most developers don't want to bother as much with all this, and would rather be debugging their source code. The GNU build system helps you set up your source code to make this possible. For the same example, the GNU developer only needs to write the following `Makefile.am' file:

bin_PROGRAMS = foo
foo_SOURCES = foo1.c foo2.c foo3.c foo4.c 
noinst_HEADERS = gleep1.h gleep2.h gleep3.h

and set a few more things up. This file is then compiled into an intermediate file, called `Makefile.in', by Automake, and during installation the final `Makefile' is generated from `Makefile.in' by a shell script called `configure'. This shell script is provided by the developer and it is also automatically generated with Autoconf. For more details see Hello world example with Autoconf and Automake.

In general you will not need to be writing makefiles by hand. It is useful however to understand how makefiles work and how to write abstract rules.

5. The GNU build system

The GNU build system has two goals. The first is to simplify the development of portable programs. The second is to simplify the building of programs that are distributed as source code. The first goal is achieved by the automatic generation of a `configure' shell script, which configures the source code to the installer platform. The second goal is achieved by the automatic generation of Makefiles and other shell scripts that are typically used in the building process. This way the developer can concentrate on debugging per source code, instead of per overly complex Makefiles. And the installer can compile and install the program directly from the source code distribution by a simple and automatic procedure.

5.1 Introducing the GNU tools

When we speak of the GNU build system we refer primarily to the following four packages:

1. Autoconf produces a configuration shell script, named `configure', which probes the installer platform for portability related information which is required to customize makefiles, configuration header files, and other application specific files. Then it proceeds to generate customized versions of these files from generic templates. This way, the user will not need to customize these files manually.
2. Automake produces makefile templates, `Makefile.in' to be used by Autoconf, from a very high level specification stored in a file called `Makefile.am'. Automake produces makefiles that conform to the GNU makefile standards, taking away the extraordinary effort required to produce them by hand. Automake requires Autoconf in order to be used properly.
3. Libtool makes it possible to compile position independent code and build shared libraries in a portable manner. It does not require either Autoconf, or Automake and can be used independently. Automake however supports libtool and interoperates with it in a seamless manner.
4. Autotoolset helps you develop portable source code that conforms to the GNU coding standards by generating various boilerplate files from which you can plunge into developing your software.

Some tasks that are simplified by the GNU build system include:

1. Building multidirectory software packages. It is much more difficult to use raw make recursively. Having simplified this step, the developer is encouraged to organize per source code in a deep directory tree rather than lump everything under the same directory. Developers that use raw make often can't justify the inconvenience of recursive make and prefer to disorganize their source code. With the GNU tools this is no longer necessary.
2. Automatic configuration. You will never have to tell your users that they need to edit your Makefile. You yourself will not have to edit your Makefiles as you move new versions of your code back and forth between different machines.
3. Automatic makefile generation. Writing makefiles involves a lot of repetition, and in large projects very error-prone. Also, certain portions of a good makefile, such as the `install' and `uninstall' targets are very critical because they are run by the superuser. They must be written without any bugs! The GNU build system automates makefile writing. You are only required to write `Makefile.am' files that are much more terse and easy to maintain.
4. Support for test suites. You can very easily write test suite code, and by adding one extra line in your `Makefile.am' make a check target available such that you can compile and run the entire test suite by running make check.
5. Automatic distribution building. The GNU build tools are meant to be used in the development of free software, therefore if you have a working build system in place for your programs, you can create a source code distribution out of it by running make distcheck.
6. Shared libraries. Building shared libraries becomes as easy as building static libraries.

The GNU build system needs to be installed only when you are developing programs that are meant to be distributed. To build a program from distributed source code, the installer only needs a working make utility, a compiler, a shell, and sometimes standard Unix utilities like sed, awk, yacc, lex. The objective is to make software installation as simple and as automatic as possible for the installer. Also, by setting up the GNU build system such that it creates programs that don't require the build system to be present during their installation, it becomes possible to use the build system to bootstrap itself.

5.2 Installing the GNU build system

If you are on a Unix system, don't be surprised if the GNU development tools are not installed. Some Unix sysadmins have never heard about them. If you do have them installed check to see whether you have the most recent versions. To do that, type:

% autoconf --version
% automake --version
% libtool --version

This manual is current with Autoconf 2.57, Automake 1.6.3 and Libtool 1.4.3.

If you don't have any of the above packages, you need to get a copy and install them on your computer. The distribution filenames for the GNU build tools, are:

autoconf-2.57.tar.gz
automake-1.6.3.tar.gz
libtool-1.4.3.tar.gz
autotoolset-0.11.6.tar.gz

Before installing these packages however, you will need to install the following needed packages from the FSF:

make-*.tar.gz
m4-*.tar.gz
texinfo-4.3.tar.gz
tar-*.shar.gz

The asterisks in the version numbers mean that the version for these programs is not critically important.

You will need the GNU versions of make, m4 and tar even if your system already has native versions of these utilities. To check whether you do have the GNU versions see whether they accept the --version flag. If you have proprietary versions of make or m4, rename them and then install the GNU ones. You will also need to install Perl, the GNU C compiler, and the TeX typesetting system. These programs are always installed on a typical Debian GNU/Linux system.

It is important to note that the end user will only need a decent shell and a working make to build a source code distribution. The developer however needs to gather all of these tools in order to create the distribution.

The installation process, for all of these tools that you obtain as `*.tar.gz' distributions is rather straightforward:

% ./configure
% make
% make check
% make install

Most of these tools include documentation which you can build with

% make dvi

5.3 Hello world example with Autoconf and Automake

To get started we will show you how to do the Hello world program using `autoconf' and `automake'. In the fine tradition of K&R, the C version of the hello world program is:

#include <stdio.h>
main()
{
 printf("Howdy world!\n");
}

Call this `hello.c' and place it under an empty directory. Simple programs like this can be compiled and ran directly with the following commands:

% gcc hello.c -o hello
% hello

If you are on a Unix system instead of a GNU system, your compiler might be called `cc' but the usage will be pretty much the same.

Now to do the same thing the `autoconf' and `automake' way create first the following files:

`Makefile.am'

bin_PROGRAMS = hello hello_SOURCES = hello.c

`configure.in'

AC_INIT(hello.c) AM_INIT_AUTOMAKE(hello,0.1) AC_PROG_CC AC_PROG_INSTALL AC_OUTPUT(Makefile)

Now run `autoconf':

% aclocal
% autoconf

This will create the shell script `configure'. Next, run `automake':

% automake -a
required file "./install-sh" not found; installing
required file "./mkinstalldirs" not found; installing
required file "./missing" not found; installing
required file "./INSTALL" not found; installing
required file "./NEWS" not found
required file "./README" not found
required file "./COPYING" not found; installing
required file "./AUTHORS" not found
required file "./ChangeLog" not found

The first time you do this, `automake' will complain a couple of things. First it notices that the files `install-sh', `mkinstalldirs' and `missing' are not present, and it installs copies. These files contain boiler-plate shell scripts that are needed by the makefiles that `automake' generates. It also complains that the following files are not around:

INSTALL, COPYING, NEWS, README, AUTHORS, ChangeLog

These files are required to be present by the GNU coding standards, and we discuss them in detail in Maintaining the documentation files. At this point, it is important to at least touch these files, otherwise if you attempt to do a `make distcheck' it will deliberately fail. To make these files exist, type:

% touch NEWS README AUTHORS ChangeLog

and to make Automake aware of the existence of these files, rerun it:

% automake -a

You can assume that the generated `Makefile.in' is correct, only when Automake completes without any error messages.

Now the package is exactly in the state that the end-user will find it when person unpacks it from a source code distribution. For future reference, we will call this state autoconfiscated. Being in an autoconfiscated state means that, you are ready to type:

% ./configure
% make
% ./hello

to compile and run the hello world program. If you really want to install it, go ahead and call the `install' target:

# make install

To undo installation, that is to uninstall the package, do:

# make uninstall

If you didn't use the `--prefix' argument to point to your home directory, or a directory in which you have permissions to write and execute, you may need to be superuser to invoke the install and uninstall commands. If you feel like cutting a source code distribution, type:

make distcheck

This will create a file called `hello-0.1.tar.gz' in the current working directory that contains the project's source code, and test it out to see whether all the files are actually included and whether the source code passes the regression test suite.

In order to do all of the above, you need to use the GNU `gcc' compiler. Automake depends on `gcc''s ability to compute dependencies. Also, the `distcheck' target requires GNU make and GNU tar.

The GNU build tools assume that there are two types of hats that people like to wear: the developer hat and the installer hat. Developers develop the source code and create the source code distribution. Installers just want to compile and install a source code distribution on their system. In the free software community, the same people get to wear either hat depending on what they want to do. If you are a developer, then you need to install the entire GNU build system, period (see section Installing the GNU build system). If you are an installer, then all you need to compile and install a GNU package is a minimal `make' utility and a minimal shell. Any native Unix shell and `make' will work.

Both Autoconf and Automake take special steps to ensure that packages generated through the `distcheck' target can be easily installed with minimal tools. Autoconf generates `configure' shell scripts that use only portable Bourne shell features. (see section Portable shell programming) Automake ensures that the source code is in an autoconfiscated state when it is unpacked. It also regenerates the makefiles before adding them to the distribution, such that the installer targets (`all', `install', `uninstall', `check', `clean', `distclean') do not depend on GNU make features. The regenerated makefiles also do not use the `gcc' cruft to compute dependencies. Instead, precomputed dependencies are included in the regenerated makefiles, and the dependencies generation mechanism is disabled. This will allow the end-user to compile the package using a native compiler, if the GNU compiler is not available. For future reference we will call this the installer state.

Now wear your installer hat, and install `hello-0.1.tar.gz':

% gunzip hello-0.1.tar.gz
% tar xf hello-0.1.tar
% cd hello-0.1
% configure
% make 
% ./hello

This is the full circle. The distribution compiles, and by typing `make install' it installs. If you need to switch back to the developer hat, then you should rerun `automake' to get regenerate the makefiles.

When you run the `distcheck' target, `make' will create the source code distribution `hello-0.1.tar.gz' and it will pretend that it is an installer and see if it the distribution can be unpacked, configured, compiled and installed. It will also run the test suite, if one is bundled. If you would like to skip these tests, then run the `dist' target instead:

% make dist

Nevertheless, running `distcheck' is extremely helpful in debugging your build cruft. Please never release a distribution without getting it through `distcheck'. If you make daily distributions for off-site backup, please do pass them through `distcheck'. If there are files missing from your distribution, the `distcheck' target will detect them. If you fail to notice such problems, then your backups will be incomplete leading you to a false sense of security.

5.4 Understanding the hello world example

When you made the `hello-0.1.tar.gz' distribution, most of the files were automatically generated. The only files that were actually written by your fingers were:

`hello.c'

#include <stdio.h> main() { printf("Howdy, world!\n"); }

`Makefile.am'

bin_PROGRAMS = hello hello_SOURCES = hello.c

`configure.in'

AC_INIT(hello.cc) AM_INIT_AUTOMAKE(hello,1.0) AC_PROG_CC AC_PROG_INSTALL AC_OUTPUT(Makefile)

In this section we explain briefly what the files `Makefile.am' and `configure.in' mean.

The language of `Makefile.am' is a logic language. There is no explicit statement of execution. Only a statement of relations from which execution is inferred. On the other hand, the language of `configure.in' is procedural. Each line of `configure.in' is a command that is executed.

Seen in this light, here's what the `configure.in' commands shown do:

• The AC_INIT command initializes the configure script. It must be passed as argument the name of one of the source files. Any source file will do.
• The AM_INIT_AUTOMAKE performs some further initializations that are related to the fact that we are using `automake'. If you are writing your `Makefile.in' by hand, then you don't need to call this command. The two comma-separated arguments are the name of the package and the version number.
• The AC_PROG_CC checks to see which C compiler you have.
• The AC_PROG_INSTALL checks to see whether your system has a BSD compatible install utility. If not then it uses `install-sh' which `automake' will install at the root of your package directory if it's not there yet.
• The AC_OUTPUT tells the configure script to generate `Makefile' from `Makefile.in'

The `Makefile.am' is more obvious. The first line specifies the name of the program we are building. The second line specifies the source files that compose the program.

For now, as far as `configure.in' is concerned you need to know the following additional facts:

• If you are building a library, then your configure script must determine how to handle `ranlib'. To do that, add the AC_PROG_RANLIB command.
• If your source code contains C++ files, you need to add the AC_PROG_CXX to your `configure.in'.
• If your source code contains `yacc' and `lex' files, then you need to add:

  AC_PROG_YACC AC_PROG_LEX

  to your `configure.in'.

• If your source code contains Fortran source code, you need to add `AC_PROG_FC' to your code. If you want to mix C and Fortran, then you need to do a lot more than just that. See section Using Fortran effectively, for more details.
• If you have any makefiles in subdirectories you must also put them in the AC_OUTPUT statement like this:

  AC_OUTPUT(Makefile \ dir1/Makefile \ dir2/Makefile \ )

  Note that the backslashes are not needed if you are using the bash shell. For portability reasons, however, it is a good idea to include them. Make sure that every subdirectory where building takes place, is mentioned!

Now consider the commands that are used to build the hello world distribution:

% aclocal
% autoconf
% touch README AUTHORS NEWS ChangeLog
% automake -a 
% ./configure
% make

The first three commands bring the package in autoconfiscated state. The remaining two commands do the actual configuration and building. More specifically:

• The `aclocal' command installs a file called `aclocal.m4'. Normally, in that file you are supposed to place the definitions of any `autoconf' macros that you've written that happen to be in use in `configure.in'. We will teach you how to write `autoconf' macros later. The `automake' utility uses the AM_INIT_AUTOMAKE macro which is not part of the standard `autoconf' macros. For this reason, it's definition needs to be placed in `aclocal.m4'. If you call `aclocal' with no arguments then it will generate the appropriate `aclocal.m4' file. Later we will show you how to use `aclocal' to also install your own `autoconf' macros.
• The `autoconf' command combines the `aclocal.m4' and `configure.in' files and produces the `configure' script. And now we are in business.
• The `touch' command makes the files `README' and friends exist. It is important that these files exist before calling Automake, because Automake decides whether to include them in a distribution by checking if they exist at the time that you invoke `automake'. Automake must decide to include these files, because when you type `make distcheck' the presence of these files will be required.
• The `automake' command compiles a `Makefile.in' file from `Makefile.am' and if absent it installs various files that are required either by the GNU coding standards or by the makefile that will be generated.

The `configure' script probes your platform and generates makefiles that are customized for building the source code on your platform. The specifics of how the probing should be done are programmed in `configure.in'. The generated makefiles are based on templates that appear in `Makefile.in' files. In order for these templates to cooperate with `configure' and produce makefiles that conform to the GNU coding standards they need to contain a tedious amount of boring stuff. This is where Automake comes in. Automakes generates the `Makefile.in' files from the more terse description in `Makefile.am'. As you have seen in the example, `Makefile.am' files can be very simple in simple cases. Once you have customized makefiles, your make utility takes over.

How does `configure' actually convert the template `Makefile.in' to the final makefile? The `configure' script really does two things:

1. It maintains a list of substitutions that it accumulates while probing the installer platform. Each one of these substitutions consists of a symbolic name, and the actual text that we want to substitute. When the `configure' script runs AC_OUTPUT it parses all of the files listed in AC_OUTPUT and every occurence of @FOO@ in these files is substituted with the text that corresponds to FOO. For example, if you add the following lines to `configure.in' you will cause @FOO@ to be substituted with `hello':

   FOO="hello" AC_SUBST(FOO)

   This is how `configure' exports compile-time decisions to the makefile, such as what compiler to use, what flags to pass the compilers and so on. Occasionally, you want to use `configure''s substitution capability directly on files that are not makefiles. This is why it is important to be aware of it. See section Scripts with Automake, for an example.

2. It maintains a list of C preprocessor macros with defined values that it also accumulates while probing the installer platforms. Before finishing off, `configure' will either generate a configuration file that defines these C preprocessor macros to the desired values, or set a flag in the generated makefile (through substitution) that will pass `-D' flags to the compiler. We discuss configuration headers in the following

See section Writing Autoconf macros, for more details on the internals of `configure' scripts.

5.5 Using configuration headers

If you inspect the output of `make' while compiling the hello world example, you will see that the generated Makefile is passing `-D' flags to the compiler that define the macros PACKAGE and VERSION. These macros are assigned the arguments that are passed to the AM_INIT_AUTOMAKE command in `configure.in'. One of the ways in which `configure' customizes your source code to a specific platform is by getting such C preprocessors defined. The definition is requested by appropriate commands in the `configure.in'. The AM_INIT_AUTOMAKE command is one such command.

The GNU build system by default implements C preprocessor macro definitions by passing `-D' flags to the compiler. When there is too many of these flags, we have two problems: the `make' output becomes hard to read, and more importantly we are running the risk of hitting the buffer limits of braindead Unix implementations of `make'. To work around this problem, you can ask Autoconf to use another approach in which all macros are defined in a special header file that is included in all the sources. This header file is called a configuration header.

A hello world program using this technique looks like this

`configure.in'

AC_INIT(hello.c) AM_CONFIG_HEADER(config.h) AM_INIT_AUTOMAKE(hello,0.1) AC_PROG_CC AC_PROG_INSTALL AC_OUTPUT(Makefile)

`Makefile.am'

bin_PROGRAMS = hello hello_SOURCES = hello.c

`hello.c'

#ifdef HAVE_CONFIG_H #include <config.h> #endif #include <stdio.h> main() { printf("Howdy, pardner!\n"); }

To request the use of a configuration header we use the AM_CONFIG_HEADER command. The configuration header must be installed conditionally with the following three lines:

#if HAVE_CONFIG_H
#include <config.h>
#endif

It is important that `config.h' is the first thing that gets included. Now autoconfiscate the source code by typing:

% aclocal
% autoconf
% touch NEWS README AUTHORS ChangeLog
% autoheader
% automake -a

It is important to type these commands in the order shown. The difference between this, and what we did in Hello world example with Autoconf and Automake, is that we had to run a new program: `autoheader'. This program scans `configure.in' and generates a template file `config.h.in' listing all the C preprocessor macros that might be defined and comments that should accompany the macros describing what they do. When you run `configure', it will load in `config.h.in' and use it to generate the final `config.h' that will be used by the source code during compilation.

Now you can go ahead and build the program:

% configure
% make
gcc -DHAVE_CONFIG_H -I. -I. -I.   -g -O2 -c hello.c
gcc -g -O2  -o hello  hello.o 

Note that now instead of multiple -D flags, there is only one such flag passed: -DHAVE_CONFIG_H. Also, appropriate -I flags are passed to make sure that `hello.c' can find and include `config.h'. To test the distribution, type:

% make distcheck
......
========================
hello-0.1.tar.gz is ready for distribution
========================

and it should all work out.

The `config.h' files go a long way back in history. In the past, there used to be packages where you would have to manually edit `config.h' files and adjust the macros you wanted defined by hand. This made these packages very difficult to install because they required intimate knowledge of your operating system. For example, it was not unusual to see a comment saying "if your system has a broken vfork, then define this macro". Many installers found this frustrating because they didn't really know how to configure the esoteric details of the `config.h' files. With autoconfiguring source code all of these details can be taken care of automatically, shifting this burden from the installer to the developer where it belongs.

5.6 Maintaining the documentation files

Every software project must have its own directory. A minimal "project" is the example that we described in Hello world example with Autoconf and Automake. In general, even a minimal project must have the files:

README, INSTALL, AUTHORS, THANKS, NEWS, ChangeLog

Before distributing your source code, it is important to write the real contents of these files. In this section we give a summary overview on how these files should be maintained. For more details, please see the GNU coding standards as published by the FSF.

• The README file: Every distribution must contain this file. This is the file that the installer must read fully after unpacking the distribution and before configuring it. You should briefly explain the purpose of the distribution, and reference all other documentation available. Instructions for installing the package normally belong in the `INSTALL' file. However if you have something that you feel the installer should know then mention it in this file. Do not make the configuration or installation process more complex, because you fear installers will not `README' files. Assume this file is being read.

  For pretest releases, only, you might also decide to distribute a file `README-alpha' containing special comments for your friendly pretesters. If you use the recommended version numbering scheme (see section Handling version numbers), you can automate it's distribution by adding the following code in your `configure.in':

  changequote(,)dnl case $VERSION in [0-9]*.[0-9]*[a-z]) DIST_ALPHA="README-alpha";; [0-9]*.[0-9]*.[0-9]*) DIST_ALPHA="README-alpha";; *) DIST_ALPHA=;; esac changequote([, ])dnl AC_SUBST(DIST_ALPHA)

  In your top-level `Makefile.am', add something like:

  EXTRA_DIST = $(DIST_ALPHA)

• The INSTALL file: Because the GNU installation procedure is streamlined, a standard `INSTALL' file will be created for you automatically by Automake. If you have something very important to say, it may be best to say it in the `README' file instead. the `INSTALL' file is mostly for the benefit of people who've never installed a GNU package before. However, if your package is very unusual, you may decide that it is best to modify the standard INSTALL file or write your own.
• The AUTHORS file: This file should collect a trace of all the legal paperwork that you have exchanged with contributors for your particular package. This information is very useful for registering the copyright of your package. The file might have an introductory blurb similar to this one:

  Authors of PACKAGE The following contributions warranted legal paper exchanges with [the Free Software Foundation | Your Name]. Also see files ChangeLog and THANKS

  Then, list who the contributors are and what files they have worked on. Indicate whether they created the file, or whether they modified it. For example:

  Random J. Hacker: entire files -> foo1.c , foo2.c , foo3.c modifications -> foo4.c , foo5.c

• The THANKS file: All distributions should contain a `THANKS' file containing a two column list of the contributors, one per line, alphabetically sorted. The left column gives the contributor's name, while the right column gives the last known good email address for this contributor. This list should be introduced with a wording similar to this one:

  PACKAGE THANKS file PACKAGE has originally been written by ORIGINAL AUTHOR. Many people further contributed to PACKAGE by reporting problems, suggesting various improvements or submitting actual code. Here is a list of these people. Help me keep it complete and exempt of errors.

  The easiest policy with this file is to thank everyone who contributes to the project, without judging the value of the contribution.

  Unlike `AUTHORS', the `THANKS' file is not maintained for legal reasons. It is maintained to thank all the contributors that helped you out in your project. The `AUTHORS' file can not be used for this purpose because certain contributions, like bug reports or ideas and suggestions do not require legal paper exchanges.

  You can also decide to send some kind of special greeting when you initially add a name to your `THANKS' file. The mere presence of a name in `THANKS' is then a flag to you that the initial greeting has been sent.

• The NEWS file: List the major new features of this distribution and identify the version that they pertain to. Don't discard items from previous versions. Leave them in the file after the newer items, so that a user upgrading from any previous version can see what is new.
• The ChangeLog file: Use this file to record all the changes that you make to your source code. If your source code is distributed among many subdirectories, and there is reason enough to think of the contents of the subdirectories as different subpackages,then please maintain a separate `ChangeLog' file for each subdirectory. For example, although there is usually no need to maintain a `ChangeLog' for your documentation, if you do decide to maintain one anyway, it should be separate from your sources `ChangeLog'.

  The GNU coding standards explain in a lot of detail how you should structure a `ChangeLog', so you should read about it there. The basic idea is to record semi-permanant modifications you make to your source code. It is not necessary to continuously record changes that you make while you are experimenting with something. But once you decide that you got a modification worked out, then you should record Please do record version releases on the central `ChangeLog' (see section Handling version numbers). This way, it will be possible to tell what changes happened between versions.

  You can automate `ChangeLog' maintenance with emacs. See section Navigating source code, for more details. Recently versions of Emacs use the ISO 8601 standard for dates which is: YYYY-MM-DD (year-month-date). A typical `ChangeLog' entry looks like this:

  1998-05-17 Eleftherios Gkioulekas <lf@amath.washington.edu> * src/acmkdir.sh: Now acmkdir will put better default content to the files README, NEWS, AUTHORS, THANKS

  Every entry contains all the changes you made within the period of a day. The most recent changes are listed at the top, the older changes slowly scroll to the bottom. Changes are sorted together in groups, per day of work.

• COPYING This file contains the copyright permissions for your distribution, in particular the General Public License (see section Licensing Free Software). This file is generated automatically for you by Automake. However, it requires that you insert copyright headers in your source code that refer to this file. See section Applying the GPL, for more details.

  Copyright is one of the many legal concerns that you need to be aware of if you develop free software. See section Legal issues with Free Software, for more details. The philosophy of the GNU project, that software should be free, is very important to the future of our community. See section Philosophical issues, to read Richard Stallman's essays on this topic.

5.7 Organizing your project in subdirectories

If your program is very small, you can place all your files in the top-level directory, like we did in the Hello World example (see section Hello world example with Autoconf and Automake). Such packages are called shallow.

In general, it is preferred to organize your package as a deep package. In a deep package, the documentation files

README, INSTALL, AUTHORS, THANKS, ChangeLog, COPYING

as well as the build cruft are placed at the top-level directory, and the rest of the files are placed in subdirectories. It is standard practice to use the following subdirectories:

`src'

The actual source code that gets compiled. Every library should have it's own subdirectory. Executables should get their own directory as well. If each executable corresponds only to one or two files then it is sensible to put them all under the same directory. If your executables need more source files, or they can be separated in distinct classes of functionalities you may like to regroup them under multiple directories. Feel free to use your judgment on how to do this best. It is easiest to place the library test suites on the same directory with the library source code. If that does not sit well with you however, you should put the test suite for each library in subdirectories under that library's directory. It is a massively bad idea to put the test suites for different libraries under the same directory.

`lib'

An optional directory where you put portability-related source code. This is mainly replacement implementation for system calls that are unavailable on some systems. You can also put tools here that you commonly use across many different packages, tools that are too simple to just make libraries out of every one of them. Common files encountered here are files that replace system calls to the GNU C library that are not available in proprietary C libraries.

`doc'

A directory containing the documentation for your package. You have the creative freedom to present the documentation in any way that is effective. However the preferred way to document software is by using Texinfo. Texinfo has the advantage that you can produce both on-line help as well as nice printed books from the same source. Documentation is discussed in more detail in See section Maintaining Documentation.

`m4'

A directory containing `m4' files that you package may need to install. These files define new `autoconf' macros that you should make available to other developers who want to use your libraries. This is discussed in more detail in FIXME: cross reference.

`intl'

A directory containing boilerplate portability source code that allows your program to speak in many human languages. The contents of this directory are automatically maintained by `gettext'. (FIXME: cross reference)

`po'

A directory containing message catalogs for your software package. This is where the maintainer places the translations of per software in multiple human languages. (FIXME: cross reference)

Automake makes it very easy to maintain multidirectory source code packages, so you shouldn't shy away from taking advantage of it. Multidirectory packages are more convenient for most projects.

5.8 Applying the GPL

The General Public License (GPL) is the legal implementation of the idea that the program, to which it is applied, belongs to the public. It means that the public is free to use it, free to modify it and redistribute it. And it also means that no-one can steal it from the public and use it to create a derived work that is not free. This is different from public domain, where anyone can take a work, make a few changes, slap a copyright notice on it, and forbid the public to use the resulting work without a proprietary license. The idea, that a work is owned by the public in this sense, is often called copyleft.

Unfortunately our legal system does not recognize this idea properly. Every work must have an owner, and that person or entity is the only one that can enforce per copyright. As a result, when a work is distributed under the GPL, with the spirit that it belongs to the public, only the nominal "owner" has the right to sue hoarders that use the work to create proprietary products. Unfortunately, the law does not extend that right to the public. Despite this shortcoming, the GPL has proven to be a very effective way to distribute free software. Almost all of the components of the GNU system are distributed under the GPL.

To apply the GPL to your programs you need to do the following things:

1. Attach a copy of the GNU general public license to the toplevel directory of your source code in a file called `COPYING'.
2. Include a legal notice to every file that you want covered by the GPL, saying that it is covered by the GPL. It is important that all files that constitute source code must include this notice, including `Makefile.am', `configure.in' files and shell scripts. The legal notice should look like this:

   Copyright (C) (years) (Your Name) <your@email.address> This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

   If you have assigned your copyright to an organization, like the Free Software Foundation, then you should probably fashion your copyright notice like this:

   Copyright (C) (years) Free Software Foundation (your name) <your@email.address> (initial year) etc...

   This legal notice works like a subroutine. By invoking it, you invoke the full text of the GNU General Public License which is too lengthy to include in every source file. Where you see `(years)' you need to list all the years in which you finished preparing a version that was actually released, and which was an ancestor to the current version. This list is not the list of years in which versions were released. It is a list of years in which versions, later released, were completed. If you finish a version on Dec 31, 1997 and release it on Jan 1, 1998, you need to include 1997, but you do not need to include 1998. This rule is complicated, but it is dictated by international copyright law.

   Some developers don't like inserting a proper legal notice to every file in their source code, because they don't want to do the typing. However, it is not sufficient to just say something like "this file is GPLed". You have to make an unambiguous and exact statement, and you have to include the entire boilerplate text to do that. Fortunately, you can save typing by having Emacs insert copyright notices for you. See section Inserting copyright notices with Emacs, for more details.

3. Use the `AUTHORS' file to keep records of who wrote what. See section Maintaining the documentation files, for details.
4. If you modify someone else's GPL covered file make sure to comply with section 2 of the GPL. To do that place notices stating that you changed the file and the date of the change. Also your program should advertise the fact that it is free software, that there is no warranty and that it can be redistributed under the condition of the GPL. A standard way of doing this is to make your program output this notice when it is passed the --version command-line flag.
5. Finally, help others to use and improve your program by writing documentation. A free software project is not truly complete without documentation. To make it possible for your users to update the documentation to reflect the changes that they make, it is necessary to make the documentation free too. However, the issues for writings are different from the issues for software. See section Why free software needs free documentation, for a discussion of these issues. See section Maintaining Documentation, for the technical details on how to write documentation for your programs.

5.9 Handling version numbers

The number `0.1' in the filename `hello-0.1.tar.gz' is called the version number of the source code distribution. The purpose of version numbers is to label the various releases of a source code distribution so that it's development can be tracked. If you use the GNU build system, then the name of the package and the version number are specified in the line that invokes the `AM_INIT_AUTOMAKE' macro. In the hello world example (see section Hello world example with Autoconf and Automake) we used the following line to set the version number equal to 0.1:

AM_INIT_AUTOMAKE(hello,0.1)

Whenever you publish a new version of your program, you must increase the version number. We also recommend that you note on `ChangeLog' the release of the new version. This way, when someone inspects your `ChangeLog', person will be able to determine what changes happened between any two specific versions.

To release the current version of your source code, run

% make distcheck

to build the distribution and apply the test suite to validate it. Once you get this to work, change your version number in `configure.in', record an entry in `ChangeLog' saying that you are cutting the new version, and update the `NEWS' file. Without making any other changes, do

% make dist

to rebuild the distribution without having to wait for the test suite to run all over again.

Most packages declare their version with two integers: a major number and a minor number that are separated by a dot in the middle. In our example above, the major number is 0 and the minor number is 1. The minor number should be increased when you release a version that contains new features and improvements over the old version that you want your users to upgrade to. The major number should be increased when the incremental improvements bring your program into a new level of maturity and stability. A major number of 0 indicates that your software is still experimental and not ready for prime time. When you release version 1.0, you are telling people that your software has developed to the point that you recommend it for general use. Releasing version 2.0 means that your software has significantly matured from user feedback.

Before releasing a new major version, it is a good idea to publish a prerelease to your beta-testers. In general, the prerelease for version 1.0 is labeled 0.90 regardless of what the previous minor number was. (8) When releasing a 0.90 version, development should freeze, and you should only be fixing bugs. If the prerelease turns out to be stable, it becomes the stable version. If not, you may need to release further bug-fixing prereleases: 0.91, 0.92, etc.

Many maintainers like to publish working versions of their code, so that contributors can donate code against the most recent version of the source code. These unofficial versions should only be used by people who are interested in contributing to the project, and not by end users. It is useful to use a third integer for writing the version numbers for these "unofficial" releases. Please use only two integers for official releases so that it is easy to distinguish them from unofficial releases. A possible succession of version numbers might look like this:

0.1, 0.1.1, 0.1.2, ... , 0.2, ..., 0.3, ..., 0.90, ..., 1.0

It is always a good idea to maintain an archive of at least all the official releases that you ever publish.

5.10 Hello world with acmkdir

Whenever you start out a new programming project, there is quite a bit of overhead setup that you need to do in order to use the GNU build system. You need to install the documentation files described in Maintaining the documentation files, and set up the directory structure described in Organizing your project in subdirectories. In the quest for never-ending automation, you can do these tasks automatically with the `acmkdir' utility.

Start by typing the following command on the shell:

% acmkdir hello

`acmkdir' will ask you to enter the name of your program, your name and your email address. When you are done, `acmkdir' will ask you if you really want to go for it. Say `y'. Then, `acmkdir' will do the following routine work for you:

• Create the `hello-0.1' directory and the `doc', `m4' and `src' subdirectories.
• Generate the following `configure.in':

  AC_INIT AM_CONFIG_HEADER(config.h) AM_INIT_AUTOMAKE(test,0.1) AC_PROG_CC AC_PROG_CXX AC_PROG_RANLIB AC_OUTPUT(Makefile doc/Makefile m4/Makefile src/Makefile)

  By default, both the C and C++ compilers are initialized, but you can take out `AC_PROG_CXX' if you don't plan to use C++. You can edit and customize this file to your needs. More specifically, you will need to update the version number in `AM_INIT_AUTOMAKE' every time you cut a new distribution (see section Handling version numbers). You should also make sure to list all the subdirectories that have a `Makefile.am' in `AC_OUTPUT'.

• Place boilerplate `Makefile.am' files on the toplevel directory as well as the `doc', `m4' and `src' subdirectories. The toplevel `Makefile.am' contains:

  EXTRA_DIST = reconf configure SUBDIRS = m4 doc src

  The ones in the src and doc subdirectories are empty. The one in `m4' contains a template `Makefile.am' which you should edit if you want to add new Autoconf macros. (FIXME: Crossreference)

• Create the files `COPYING', `INSTALL', `AUTHORS', `NEWS', `README', `THANKS' and `ChangeLog' and generate their default contents which you should edit further as you develop your package. (see section Maintaining the documentation files)
• Create a `reconf' script for reconfiguring your package every time you make a change in `configure.in'. Running `reconf' is equivalent to running the following commands on the shell from the top-level directory:

  % rm -f config.cache % rm -f acconfig.h % aclocal -I m4 % autoconf % acconfig % autoheader % automake -a

  Before `acmkdir' exits, it will call the `reconf' script for you once to set things up.

At this point, you can run

% ./configure
% make

but nothing interesting will happen because the package is still empty.

To add a simple hello world program, all you need to do is create the following two files:

`src/Makefile.am'

bin_PROGRAMS = hello hello_SOURCES = hello.c

`src/hello.c'

#if HAVE_CONFIG_H # include <config.h> #endif #include <stdio.h> int main () { printf ("Hello world\n"); }

and type the following commands from the toplevel directory:

% ./reconf
% ./configure
% make
% make distcheck

to compile the hello world program and build a distribution. It's that simple!

In general, to develop simple programs with the GNU build system you setup the project directory tree with `acmkdir', you write your source code, you put together the necessary `Makefile.am' and update `configure.in' and you are set to go. In fact, at this point you practically know all you need to know to develop source code distributions that compile and install simple C programs. All you need to do is write the source code and list the source files in `*_SOURCES'.

In the following chapters we will explain in more detail how to use the GNU build system to develop software that conforms to the GNU coding standards.

6. Using Automake and Autoconf

6.1 Hello World revisited

To begin, let's review the simplest example, the hello world program:

`hello.c'

#include <stdio.h> main() { printf("Howdy, world!\n"); }

`Makefile.am'

bin_PROGRAMS = hello hello_SOURCES = hello.c

`configure.ac'

AC_INIT([Hello Program], [1.0], [Author Of The Program <aotp@zxcv.com>], [hello]) AM_INIT_AUTOMAKE AC_PROG_CC AC_PROG_INSTALL AC_CONFIG_FILES([Makefile]) AC_OUTPUT

The language of `Makefile.am' is a logic language. There is no explicit statement of execution. Only a statement of relations from which execution is inferred. On the other hand, the language of `configure.ac' is procedural. Each line of `configure.ac' is a command that is executed.

Seen in this light, here's what the `configure.ac' commands shown do:

• The AC_INIT command initializes the configure script. Its arguments are the name of the package, the version number of the package, the name of the author of the program and his e-mail, and the name of the tar-file, if it is different from the first argument.
• The AM_INIT_AUTOMAKE performs some further initializations that are related to the fact that we are using `automake'. If you are writing your `Makefile.in' by hand, then you don't need to call this command.
• The AC_PROG_CC checks to see which C compiler you have.
• The AC_PROG_INSTALL checks to see whether your system has a BSD compatible install utility. If not then it uses `install-sh' which `automake' will install at the root of your package directory if it's not there yet.
• The AC_CONFIG_FILES tells configure which files must be generated from templates. In this case, `Makefile' will be generated from the template `Makefile.in'. Remember that if we use `automake', the file `Makefile.in' is generated from `Makefile.am'. But other files could have been specified in the parameter of this macro.
• The AC_OUTPUT tells the configure script to generate the files specified in the list of AC_CONFIG_FILES from their templates (the `*.in' files).

The `Makefile.am' is more obvious. The first line specifies the name of the program we are building. The second line specifies the source files that compose the program.

For now, as far as `configure.ac' is concerned you need to know the following additional facts:

• If you are building a library, then your configure script must determine how to handle `ranlib'. To do that, add the AC_PROG_RANLIB command.
• If you want to have your makefiles call recursively makefiles at subdirectories then the configure script needs to be told to find out how to do that. For this purpose you add the AC_PROG_MAKE_SET command.
• If you have any makefiles in subdirectories you must also put them in the AC_CONFIG_FILES statement like this:

  AC_CONFIG_FILES([ Makefile dir1/Makefile dir2/Makefile])

  Do not put the ending parenthesis in another line, separated from the ending bracket, this will cause a misbehaviour of the macro.

As we explained before to build this package you need to execute the following commands:

% aclocal
% autoconf
% touch README AUTHORS NEWS ChangeLog
% automake -a 
configure.ac: installing `./install-sh'
configure.ac: installing `./mkinstalldirs'
configure.ac: installing `./missing'
Makefile.am: installing `./INSTALL'
Makefile.am: installing `./COPYING'
Makefile.am: installing `./depcomp'
% ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking for C compiler default output... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for a BSD-compatible install... /usr/bin/install -c
configure: creating ./config.status
config.status: creating Makefile
config.status: executing depfiles commands
% make
source='hello.c' object='hello.o' libtool=no \
depfile='.deps/hello.Po' tmpdepfile='.deps/hello.TPo' \
depmode=gcc3 /bin/sh ./depcomp \
gcc -DPACKAGE_NAME=\"Hello\ Program\" -DPACKAGE_TARNAME=\"hello\" -DPACKAGE_VERSION=\"1.0\" -DPACKAGE_STRING=\"Hello\ Program\ 1.0\" -DPACKAGE_BUGREPORT=\"Author\ Of\ The\ Program\ \<aotp@zxcv.com\>\" -DPACKAGE=\"hello\" -DVERSION=\"1.0\"  -I. -I.     -g -O2 -c `test -f 'hello.c' || echo './'`hello.c
gcc  -g -O2   -o hello  hello.o

The first four commands, are for the maintainer only. When the user unpacks a distribution, he should be able to start from `configure' and move on.

• The `aclocal' command installs a file called `aclocal.m4'. Normally, in that file you are supposed to place the definitions of any `autoconf' macros that you've written that happen to be in use in `configure.ac'. We will teach you how to write `autoconf' macros later. The `automake' utility uses the AM_INIT_AUTOMAKE macro which is not part of the standard `autoconf' macros. For this reason, it's definition needs to be placed in `aclocal.m4'. If you call `aclocal' with no arguments then it will generate the appropriate `aclocal.m4' file. Later we will show you how to use `aclocal' to also install your own `autoconf' macros.
• The `autoconf' command combines the `aclocal.m4' and `configure.ac' files and produces the `configure' script. And now we are in business.
• The `touch' command makes the files `README' and friends exist. It is important that these files exist before calling Automake, because Automake decides whether to include them in a distribution by checking if they exist at the time that you invoke `automake'. Automake must decide to include these files, because when you type `make distcheck' the presence of these files will be required.
• The `automake' command compiles a `Makefile.in' file from `Makefile.am' and if absent it installs various files that are required either by the GNU coding standards or by the makefile that will be generated.

If you are curious you can take a look at the generated `Makefile'. It looks like gorilla spit but it will give you an idea of how one gets there from the `Makefile.am'.

The `configure' script is an information gatherer. It finds out things about your system. That information is given to you in two ways. One way is through defining C preprocessor macros that you can test for directly in your source code with preprocessor directives. This is done by passing -D flags to the compiler. The other way is by making certain variables defined at the `Makefile.am' level. This way you can, for example, have the configure script find out how a certain library is linked, export it as a `Makefile.am' variable and use that variable in your `Makefile.am'. Also, through certain special variables, `configure' can control how the compiler is invoked by the `Makefile'.

6.2 OLD Using configuration headers

Notice: This section is somewhat obsolete. The `acconfig' program distributed with Autotoolset is no longer needed, because the file `acconfig.h' is no longer used by Autoconf/Automake. This section is kept here only for historical reasons and may soon be removed or rewritten. Nevertheless, the description of the behaviour of `autoheader' has been updated and is correct. That said...

As you may have noticed, the `configure' script in the previous example defines two preprocessor macros that you can use in your code: PACKAGE and VERSION. As you become a power-user of `autoconf' you will get to define even more such macros. If you inspect the output of `make' during compilation, you will see that these macros get defined by passing `-D' flags to the compiler, one for each macro. When there is too many of these flags getting passed around, this can cause two problems: it can make the `make' output hard to read, and more importantly it can hit the buffer limits of various braindead implementations of `make'. To work around this problem, an alternative approach is to define all these macros in a special header file and include it in all the sources.

A hello world program using this technique looks like this

`configure.ac'

AC_INIT([Hello Program], [1.0], [Author Of The Program <aotp@zxcv.com>], [hello]) AM_CONFIG_HEADER(config.h) AM_INIT_AUTOMAKE AC_PROG_CC AC_PROG_INSTALL AC_CONFIG_FILES([Makefile]) AC_OUTPUT

`Makefile.am'

bin_PROGRAMS = hello hello_SOURCES = hello.c

`hello.c'

#ifdef HAVE_CONFIG_H #include <config.h> #endif #include <stdio.h> main() { printf("Howdy, partner!\n"); }

Note that we call a new macro in `configure.ac': AM_CONFIG_HEADER. Also we include the configuration file conditionally with the following three lines:

#ifdef HAVE_CONFIG_H
#include <config.h>
#endif

It is important to make sure that the `config.h' file is the first thing that gets included. Now do the usual routine:

% aclocal
% autoconf
% touch NEWS README AUTHORS ChangeLog
% automake -a
configure.ac: installing `./install-sh'
configure.ac: installing `./mkinstalldirs'
configure.ac: installing `./missing'
Makefile.am: installing `./INSTALL'
Makefile.am: installing `./COPYING'
configure.ac:5: required file `./config.h.in' not found
Makefile.am: installing `./depcomp'

Automake will give you an error message saying that it needs a file called `config.h.in'. You can generate such a file with the `autoheader' program. So run:

% autoheader
% configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking for C compiler default output... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for a BSD-compatible install... /usr/bin/install -c
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
config.status: executing depfiles commands
% make
make  all-am
make[1]: Entering directory `/home/mroberto/programs/autotoolset/hello5'
source='hello.c' object='hello.o' libtool=no \
depfile='.deps/hello.Po' tmpdepfile='.deps/hello.TPo' \
depmode=gcc3 /bin/sh ./depcomp \
gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c `test -f 'hello.c' || echo './'`hello.c
gcc  -g -O2   -o hello  hello.o
make[1]: Leaving directory `/home/mroberto/programs/autotoolset/hello5'

In older versions of the automake/autoconf package, you would get error messages. The problem was that autoheader was bundled with the autoconf distribution, not the automake distribution, and consequently didn't know how to deal with the PACKAGE and VERSION macros. This problem is now solved with the use of the new syntax of the macro AC_INIT. But we choose to keep this discussion here because (a) it is still useful and (b) someone may be using the old syntax that was kept for compatibility or (c) you have old versions of the automake/autoconf packages.

Of course, if `configure' defines a macro, there's nothing to know. On the other hand, when a macro is not defined then there are at least two possible defaults:

#undef PACKAGE
#define PACKAGE 0

The autoheader program here used to complain that it didn't know the defaults for the PACKAGE and VERSION macros. To provide the defaults, we would create a new file `acconfig.h':

`acconfig.h'

#undef PACKAGE #undef VERSION

and run `autoheader' again:

% autoheader

At this point you would run autoconf again, so that it took into account the presence of acconfig.h:

% aclocal
% autoconf

Now you would go ahead and build the program:

% configure
% make
Computing dependencies for hello.c...
echo > .deps/.P
gcc -DHAVE_CONFIG_H -I. -I. -I.   -g -O2 -c hello.c
gcc -g -O2  -o hello  hello.o  

Note that now instead of multiple -D flags, there is only one such flag passed: -DHAVE_CONFIG_H. Also, appropriate -I flags are passed to make sure that `hello.c' can find and include `config.h'. To test the distribution, type:

% make distcheck
......
========================
hello-1.0.tar.gz is ready for distribution
========================

and it should all work out.

The `config.h' files go a long way back in history. In the past, there used to be packages where you would have to manually edit `config.h' files and adjust the macros you wanted defined by hand. This made these packages very difficult to install because they required intimate knowledge of your operating system. For example, it was not unusual to see a comment saying "if your system has a broken vfork, then define this macro". How the hell are you supposed to know if your systems vfork is broken?? With auto-configuring packages all of these details are taken care of automatically, shifting the burden from the user to the developer where it belongs.

Note: the use of the file `acconfig.h' is deprecated, but the discussion is kept here for the reasons explained above.

Normally in the `acconfig.h' file you would put statements like

#undef MACRO
#define MACRO default

These values were copied over to `config.h.in' and are supplemented with additional defaults for C preprocessor macros that got defined by native autoconf macros like AC_CHECK_HEADERS, AC_CHECK_FUNCS, AC_CHECK_SIZEOF, AC_CHECK_LIB.

If the file `acconfig.h' contained the string @TOP@ then all the lines before the string would be included verbatim to `config.h' before the custom definitions. Also, if the file `acconfig.h' contained the string @BOTTOM@ then all the lines after the string would be included verbatim to `config.h' after the custom definitions. This allowed you to include further preprocessor directives that are related to configuration. Some of these directives may be using the custom definitions to conditionally issue further preprocessor directives. Due to a bug in some versions of autoheader if the strings @TOP@ and @BOTTOM@ do appear in your acconfig.h file, then you must make sure that there is at least one line appearing before @TOP@ and one line after @BOTTOM@, even if it has to be a comment. Otherwise, autoheader may not work correctly.

With `autotoolset' we distribute a utility called `acconfig' which will build `acconfig.h' automatically. By default it will always make sure that

#undef PACKAGE
#undef VERSION

are there. Additionally, if you install macros that are `acconfig' friendly then `acconfig' will also install entries for these macros. The acconfig program may be revised in the future and perhaps it might be eliminated (note: indeed...). There is an unofficial patch to Autoconf that will automate the maintenance of `acconfig.h', eliminating the need for a separate program. I am not yet certain if that patch will be part of the official next version of Autoconf, but I very much expect it to (note: I think it has been included). Until then, if you are interested, see: http://www.clark.net/pub/dickey/autoconf/autoconf.html This situation creates a bit of a dilemma about whether I should document and encourage acconfig in this tutorial or not. I believe that the Autoconf patch is a superior solution. However since I am not the one maintaining Autoconf, my hands are tied. For now let's say that if you confine yourself to using only the macros provided by autoconf, automake, and autotoolset then `acconfig.h' will be completely taken care for you by `acconfig'. In the future, I hope that acconfig.h will be generated by configure and be the sole responsibility of Autoconf.

You may be wondering whether it is worth using `config.h' files in the programs you develop if there aren't all that many macros being defined. My personal recommendation is yes. Use `config.h' files because perhaps in the future your `configure' might need to define even more macros. So get started on the right foot from the beginning. Also, it is nice to just have a config.h file lying around because you can have all your configuration specific C preprocessor directives in one place. In fact, if you are one of these people writing peculiar system software where you get to #include 20 header files on every single source file you write, you can just have them on all thrown into config.h once and for all. In the next chapter we will tell you about the LF macros that get distributed with autotoolset and this tutorial. These macros do require you to use the `config.h' file. The bottom line is: `config.h' is your friend; trust the config.h.

6.3 The building process

FIXME: write about VPATH builds and how to modify optimization

6.4 Some general advice

In software engineering, people start from a precise, well-designed specification and proceed to implementation. In research, the specification is fluid and immaterial and the goal is to be able to solve a slightly different problem every day. To have the flexibility to go from variation to variation with the least amount of fuss is the name of the game. By fuss, we refer to debugging, testing and validation. Once you have a code that you know gives the right answer to a specific set of problems, you want to be able to move on to a different set of similar problems with reinventing, debugging and testing as little as possible. These are the two distinct situations that computer programmers get to confront in their lives.

Software engineers can take good care of themselves in both situations. It's part of their training. However, people whose specialty is the scientific problem and not software engineering, must confront the hardest of the two cases, the second one, with very little training in software engineering. As a result they develop code that's clumsy in implementation, clumsy in usage, and with only redeeming quality the fact that it gives the right answer. This way, they do get the work of the day done, but they leave behind them no legacy to do the work of tomorrow. No general-purpose tools, no documentation, no reusable code.

The key to better software engineering is to focus away from developing monolithic applications that do only one job, and focus on developing libraries. One way to think of libraries is as a program with multiple entry points. Every library you write becomes a legacy that you can pass on to other developers. Just like in mathematics you develop little theorems and use the little theorems to hide the complexity in proving bigger theorems, in software engineering you develop libraries to take care of low-level details once and for all so that they are out of the way every time you make a different implementation for a variation of the problem.

On a higher level you still don't create just one application. You create many little applications that work together. The centralized all-in-one approach in my experience is far less flexible than the decentralized approach in which a set of applications work together as a team to accomplish the goal. In fact this is the fundamental principle behind the design of the Unix operating system. Of course, it is still important to glue together the various components to do the job. This you can do either with scripting or with actually building a suite of specialized monolithic applications derived from the underlying tools.

The name of the game is like this: Break down the program to parts. And the parts to smaller parts, until you get down to simple subproblems that can be easily tested, and from which you can construct variations of the original problem. Implement each one of these as a library, write test code for each library and make sure that the library works. It is very important for your library to have a complete test suite, a collection of programs that are supposed to run silently and return normally (exit(0);) if they execute successfully, and return abnormally (assert(false); exit(1);) if they fail. The purpose of the test suite is to detect bugs in the library, and to convince you, the developer, that the library works. The best time to write a test program is as soon as it is possible! Don't be lazy. Don't just keep throwing in code after code after code. The minute there is enough new code in there to put together some kind of test program, just do it! I can not emphasize that enough. When you write new code you have the illusion that you are producing work, only to find out tomorrow that you need an entire week to debug it. As a rule, internalize the reality that you know you have produced new work every time you write a working test program for the new features, and not a minute before. Another time when you should definitely write a test suite is when you find a bug while ordinarily using the library. Then, before you even fix the bug, write a test program that detects the bug. Then go fix it. This way, as you add new features to your libraries you have insurance that they won't reawaken old bugs.

Please keep documentation up to date as you go. The best time to write documentation is right after you get a few new test programs working. You might feel that you are too busy to write documentation, but the truth of the matter is that you will always be too busy. After long hours debugging these segfaults, think of it as a celebration of triumph to fire up the editor and document your brand-spanking new cool features.

Please make sure that computational code is completely separated from I/O code so that someone else can reuse your computational code without being forced to also follow your I/O model. Then write programs that invoke your collection of libraries to solve various problems. By dividing and conquering the problem library by library with a test suite for each step along the way, you can write good and robust code. Also, if you are developing numerical software, please don't expect that other users of your code will be getting a high while entering data for your input files. Instead write an interactive utility that will allow users to configure input files in a user friendly way. Granted, this is too much work in Fortran. Then again, you do know more powerful languages, don't you?

Examples of useful libraries are things like linear algebra libraries, general ODE solvers, interpolation algorithms, and so on. As a result you end up with two packages. A package of libraries complete with a test suite, and a package of applications that invoke the libraries. The package of libraries is well-tested code that can be passed down to future developers. It is code that won't have to be rewritten if it's treated with respect. The package of applications is something that each developer will probably rewrite since different people will probably want to solve different problems. The effect of having a package of libraries is that C++ is elevated to a Very High Level Language that's closer to the problems you are solving. In fact a good rule of thumb is to make the libraries sufficiently sophisticated so that each executable that you produce can be expressed in one source file. All this may sound like common sense, but you will be surprised at how many scientific developers maintain just one does-everything-program that they perpetually hack until it becomes impossible to maintain. And then you will be even more surprised when you find that some professors don't understand why a "simple mathematical modification" of someone else's code is taking you so long.

Every library must have its own directory and Makefile. So a library package will have many subdirectories, each directory being one library. And perhaps if you have too many of them, you might want to group them even further down. Then, there's the applications. If you've done everything right, there should be enough stuff in your libraries to enable you to have one source file per application. Which means that all the source files can probably go down under the same directory.

Very often you will come to a situation where there's something that your libraries to-date can't do, so you implement it and stick it along in your source file for the application. If you find yourself cut and pasting that implementation to other source files, then this means that you have to put this in a library somewhere. And if it doesn't belong to any library you've written so far, maybe to a new library. When you are in a deadline crunch, there's a tendency not to do this since it's easier to cut and paste. The problem is that if you don't take action right then, eventually your code will degenerate to a hard-to-use mess. Keeping the entropy down is something that must be done on a daily basis.

Finally, a word about the age-old issue of language-choice. The GNU coding standards encourage you to program in C and avoid using languages other than C, such as C++ or Fortran. The main advantage of C over C++ and Fortran is that it produces object files that can be linked by any C or C++ compiler. In contrast, C++ object files can only be linked by the compiler that produced them. As for Fortran, aside from the fact that Fortran 90 and 95 have no free compilers, it is not very trivial to mix Fortran 77 with C/C++, so it makes no sense to invite all that trouble without a compelling reason. Nevertheless, my suggestion is to code in C++. The main benefit you get with C++ is robustness. Having constructors and destructors and references can go a long way towards helping you to void memory errors, if you know how to make them work for you.

6.5 Standard organization with Automake

Now we get into the gory details of software organization. I'll tell you one way to do it. This is advice, not divine will. It's simply a way that works well in general, and a way that works well with autoconf and automake in particular.

The first principle is to maintain the package of libraries separate from the package of applications. This is not an iron-clad rule. In software engineering, where you have a crystal clear specification, it makes no sense to keep these two separate. I found from experience that it makes a lot more sense in research. Either of these two packages must have a toplevel directory under which live all of its guts. Now what do the guts look like?

First of all you have the traditional set of information files that we described in Chapter 1:

README, AUTHORS, NEWS, ChangeLog, INSTALL, COPYING

You also have the following subdirectories:

`m4'

Here, you install any new `m4' files that your package may want to install. These files define new `autoconf' commands that you may want to make available to other developers who want to use your libraries.

`doc'

Here you put the documentation for your code. You have the creative freedom to present the documentation in any way you desire. However, the preferred way to document software is to use Texinfo. Texinfo has the advantage that you can produce both on-line help as well as a nice printed book from the same source. We will say something about Texinfo later.

`src'

Here's the source code. You could put it at the toplevel directory as many developers do, but I find it more convenient to keep it away in a subdirectory. Automake makes it trivially easy to do recursive `make', so there is no reason not to take advantage of it to keep your files more organized.

`include'

This is an optional directory for distributions that use many libraries. You can have the configure script link all public header files in all the subdirectories under src to this directory. This way it will only be necessary to pass one -I flag to test suites that want to access the include files of other libraries in the distribution. We will discuss this later.

`lib'

This is an optional directory where you put portability-related source code. This is mainly replacement implementations for system calls that may not exist on some systems. You can also put tools here that you commonly use across many different packages, tools that are too simple to just make libraries out of every each one of them. It is suggested that you maintain these tools in a central place. We will discuss this much later.

Together with these subdirectories you need to put a `Makefile.am' and a `configure.ac' file. I also suggest that you put a shell script, which you can call `reconf', that contains the following:

#!/bin/sh
rm -f config.cache
aclocal -I m4
autoconf
autoheader
automake -a
exit

This will generate `configure' and `Makefile.in' and needs to be called whenever you change a `Makefile.am' or a `configure.ac' as well as when you change something under the `m4' directory. It will also call `autoheader' to make config.h.in.

At the toplevel directory, you need to put a `Makefile.am' that will tell the computer that all the source code is under the `src' directory. The way to do it is to put the following lines in `Makefile.am':

EXTRA_DIST = reconf
SUBDIRS = m4 doc src

• The first line tells automake that the `reconf' script is part of the distribution and must be included when you do make dist.
• The second line tells automake that the rest of the distribution is in the subdirectories `m4', `doc' and `src'. It instructs `make' to recursively call itself in these subdirectories. It is important to include the `doc' and `m4' subdirectories here and enhance them with `Makefile.am' so that make dist includes them into the distribution.

If you are also using a `lib' subdirectory, then it should be built before `src':

EXTRA_DIST = reconf
SUBDIRS = m4 doc lib src

The `lib' subdirectory should build a static library that is linked by your executables in `src'. There should be no need to install that library.

At the toplevel directory you also need to put the `configure.ac' file. That should look like this:

AC_INIT(packagename,versionnumber)
AM_INIT_AUTOMAKE
[...put your tests here...]
AC_CONFIG_FILES([Makefile
          doc/Makefile
          m4/Makefile
          src/Makefile
          src/dir1/Makefile
          src/dir2/Makefile
          src/dir3/Makefile
          ............
          src/dir1/foo1/Makefile])
AC_OUTPUT

You will not need another `configure.ac' file. However, every directory level on your tree must have a `Makefile.am'. When you call automake on the top-level directory, it looks at `AC_CONFIG_FILES' at your `configure.ac' to decide what other directories have a `Makefile.am' that needs parsing. As you can see from above, a `Makefile.am' file is needed even under the `doc' and `m4' directories. How to set that up is up to you. If you aren't building anything, but just have files and directories hanging around, you must declare these files and directories in the `Makefile.am' like this:

SUBDIRS = dir1 dir2 dir3
EXTRA_DIST = file1 file2 file3

Doing that will cause make dist to include these files and directories to the package distribution.

This tedious setup work needs to be done every time that you create a new package. If you create enough packages to get sick of it, then you want to look into the `acmkdir' utility that is distributed by Autotoolset. We will describe it at the next chapter.

6.6 Programs and Libraries with Automake

Next we explain how to develop `Makefile.am' files for the source code directory levels. A `Makefile.am' is a set of assignments. These assignments imply the Makefile, a set of targets, dependencies and rules, and the Makefile implies the execution of building.

The first set of assignments going at the beginning look like this:

INCLUDES = -I/dir1 -I/dir2 -I/dir3 ....
LDFLAGS = -L/dir1 -L/dir2 -L/dir3 .... 
LDADD = -llib1 -llib2 -llib3 ...

• The `INCLUDES' assignment is where you insert the -I flags that you need to pass to your compiler. If the stuff in this directory is dependent on a library in another directory of the same package, then the -I flag must point to that directory.
• The `LDFLAGS' assignment is where you insert the -L flags that are needed by the compiler when it links all the object files to an executable.
• The `LDADD' assignment is where you list a long set of installed libraries that you want to link in with all of your executables. Use the -l flag only for installed libraries. You can list libraries that have been built but not installed yet as well, but do this only be providing the full path to these libraries.

If your package contains subdirectories with libraries and you want to link these libraries in another subdirectory you need to put `-I' and `-L' flags in the two variables above. To express the path to these other subdirectories, use the `$(top_srcdir)' variable. For example if you want to access a library under `src/libfoo' you can put something like:

INCLUDES = ... -I$(top_srcdir)/src/libfoo ...
LDFLAGS  = ... -L$(top_srcdir)/src/libfoo ...

on the `Makefile.am' of every directory level that wants access to these libraries. Also, you must make sure that the libraries are built before the directory level is built. To guarantee that, list the library directories in `SUBDIRS' before the directory levels that depend on it. One way to do this is to put all the library directories under a `lib' directory and all the executable directories under a `bin' directory and on the `Makefile.am' for the directory level that contains `lib' and `bin' list them as:

SUBDIRS = lib bin

This will guarantee that all the libraries are available before building any executables. Alternatively, you can simply order your directories in such a way so that the library directories are built first.

Next we list the things that are to be built in this directory level:

bin_PROGRAMS    = prog1 prog2 prog3 ....
lib_LIBRARIES   = libfoo1.a libfoo2.a libfoo3.a ....
check_PROGRAMS  = test1 test2 test3 ....
TESTS           = $(check_PROGRAMS)
include_HEADERS = header1.h header2.h ....

• The `bin_PROGRAMS' line lists all the executable files that will be compiled with make and installed with make install under `/prefix/bin', where `prefix' is usually `/usr/local'.
• The `lib_LIBRARIES' line lists all the library files that will be compiled with make and installed with make install under `/prefix/lib'.
• The `check_PROGRAMS' line lists executable files that are not compiled with a simple make but only with a make check. These programs serve as tests that you, the user can use to test the library.
• The `TESTS' line lists executable files which are to be compiled and executed when you run make check. These programs constitute the test suite and they are indispensable when you develop a library. It is common to just set

  TESTS = $(check_PROGRAMS)

  This way by commenting the line in and out, you can modify the behaviour of make check. While debugging your test suite, you will want to comment out this line so that make check doesn't run it. However, in the end product, you will want to comment it back in.

• The `include_HEADERS' line lists public headers present in this directory that you want to install in /prefix/include. You must list a header file here if you want to cause it to be installed. You can also list it under libfoo_a_SOURCES for the library that it belongs to, but it is imperative to list public headers here so that they can be installed.

It is good programming practice to keep libraries and executables under separate directory levels. However, it is okey to keep the library and the check executables that test the library under the same directory level because that makes it easier for you to link them with the library.

For each of these types of targets, we must state information that will allow automake and make to infer the building process.

• For each Program: You need to declare the set of files that are sources of the program, the set of libraries that must be linked with the program and (optionally) a set of dependencies that need to be built before the program is built. These are declared in assignments that look like this:

  prog1_SOURCES = foo1.cc foo2.cc ... header1.h header2.h .... prog1_LDADD = -lbar1 -lbar2 -lbar3 prog1_LDFLAGS = -L/dir1 -L/dir2 -L/dir3 ... prog1_DEPENDENCIES = dep1 dep2 dep3 ...

  In each assignment substitute `prog1' with the name of the program that you are building as it appeared in `bin_PROGRAMS' or `check_PROGRAMS'.

  • - `prog1_SOURCES': Here you list all the `*.cc' and `*.h' files that compose the source code of the program. The presence of a header file here doesn't cause the file to be installed at `/prefix/include' but it does cause it to be added to the distribution when you do make dist. To cause header files to be installed you must also put them in `include_HEADERS'.
  • - `prog1_LDADD': Here you add primarily the -l flags for linking whatever libraries are needed by your code. You may also list object files, which have been compiled in an exotic way, as well as paths to uninstalled yet libraries.
  • - `prog_LDFLAGS': Here you add the -L flags that are needed to resolve the libraries you passed in `prog_LDADD'. Certain flags that need to be passed on every program can be expressed on a global basis by assigning them at `LDFLAGS'.
  • - `prog1_DEPENDENCIES': If for any reason you want certain other targets to be built before building this program, you can list them here.

  This is all you need to do. There is no need to write an extended Makefile with all the targets, dependencies and rules that are required to build the program. They are computed for you by this minimal information by `automake'. Moreover, the targets `dist', `install', `clean' and `distclean' are appropriately setup to handle the program. You don't need to take care of them by yourself.

• For each Library: There's a total of four assignments that are relevant to building libraries:

  lib_LIBRARIES = ... libfoo1.a ... libfoo1_a_SOURCES = foo1.cc foo2.cc private1.h private2.h ... libfoo1_a_LIBADD = obj1.o obj2.o obj3.o libfoo1_a_DEPENDENCIES = dep1 dep2 dep3 ...

  Note that if the name of the library is `libfoo1.a' the prefix that appears in the variables that are related with that library is `libfoo1_a_'.

  • - `libfoo1_a_SOURCES': Just like with programs, here you list all the `*.cc' files as well as all the private header files that compose the library. By private header file we mean a header file that is used internally by the library and the maintainers of the library, but is not exported to the end-user. You can list public header files also if you like, and perhaps you should for documentation purposes, but if you mention them in include_HEADERS it is not required to repeat them a second time here.
  • - `libfoo1_a_DEPENDENCIES': If there are any other targets that need to be built before this library is built, list them here.
  • - `libfoo1_a_LIBADD': If there are any other object files that you want to include in the library list them here. You might be tempted to list them as dependencies in `libfoo1_a_DEPENDENCIES', but that will not work. If you do that, the object files will be built before the library is built but they will not be included in the library! By listing an object file here, you are stating that you want it to be built and you want it to be included in the library.

6.7 General Automake principles

In the previous section we described how to use Automake to compile programs, libraries and test suites. To exploit the full power of Automake however, it is important to understand the fundamental ideas behind it.

The simplest way to look at a `Makefile.am' is as a collection of assignments which infer a set of Makefile rules, which in turn infer the building process. There are three types of such assignments:

• Global assignments modify the behaviour of the entire Makefile for the given subdirectory. Examples of such assignments are `INCLUDES', `LDADD', `LDFLAGS', `TESTS'. These assignments affect the behaviour of the Makefile in the given directory independent of what gets built. In order for an assignment to be global, the name of the variable to which you are assigning must have a special meaning to Automake. If it does not, then the assignment has no effect, but it may be used as a variable in other assignments.
• Primitive assignments declare the primitives that we want to build. Such assignments are `bin_PROGRAMS', `lib_LIBRARIES', and others. The general pattern of these assignments is two words separated by an underscore. The second word is always in all-caps, it is the type of the primitive being built, and it affects what Makefile rules are generated for building the primitive. The first word contains information about where to install the primitive once its built, so it affects the Makefile rules that handle the `install' and `uninstall' targets. The way this works is that for `bin' there corresponds a global assignment for `bindir' containing the installation directory. For example the symbols `bin', `lib', `include' have the following default assignments:

  bindir = $(prefix)/bin libdir = $(prefix)/lib includedir = $(prefix)/include

  These are the directories where you install executables, libraries and public header files. You can override the defaults by inserting different assignments in your `Makefile.am', but please don't do that. Instead define new assignments. For example, if you do

  foodir = $(prefix)/foo

  then you can use `foo_PROGRAMS', `foo_LIBRARIES', etc. to list programs and libraries that you want installed in `/prefix/foo'. The symbols `check' and `noinst' have special meanings and you should not ever try to assign to `checkdir' and `noinstdir'.

  • - The `check' symbol, suggests that the primitive should only be built when the user invokes `make check' and it should not be installed. It is only meant to be executed as part of a test suite and then get scrapped.
  • - The `noinst' symbol, suggests that the primitive should not be installed. It will be built however normally, when you invoke `make'. You could use this to build convenience libraries which you intend to link in statically to executables which you do plan to install. You could also use this to build executables which will generate source code that will subsequently be used to build something installable.

  Usually, you should install executables in `/prefix/bin', libraries in `/prefix/lib' and public header files in `/prefix/include'. In general however, the GNU coding standards suggest a dozen of different places on which you may want to install files. For more details See section Installation standard directories.

• Property assignments define the properties for every primitive that you declare. A property is also made of two words that are separated by an underscore. The first word names the primitive to which the property refers to. The second word names the name of the property itself. For example when you define

  bin_PROGRAMS = hello

  this means that you can then say:

  hello_SOURCES = ... hello_LDADD = ...

  and so on. The `SOURCES' and `LDADD' are properties of `hello' which is a `PROGRAMS' primitive.

In addition to assignments, it is also possible to include ordinary targets and abstract targets in a `Makefile.am' just as you would in an ordinary `Makefile.am'. This can be particularly useful in situations like the following:

• You may want to have some of your C, C++ or Fortran source code written by another program. See section Dealing with built sources.
• You may want to generate object files from an obscure kind of source file. For example, see Embedded text with Automake.
• You may want to write programs as shell scripts in Bash, Perl or in Guile. See section Scripts with Automake, and Guile with Automake.
• You may want to install data files that are generated during compile-time from a program distributed with your software package. See section Data files with Automake.

Ordinary rules simply build things. Abstract rules however have a special meaning to Automake. If you define an abstract rule that compiles files with an arbitrary suffix into `*.o' an object file, then files with such a suffix can appear in the `*_SOURCES' of programs and libraries. You must however write the abstract rule early enough in your `Makefile.am' for Automake to parse it before encountering a sources assignment in which such files appear. You must also mention all the additional suffixes by assigning the variable `SUFFIXES'. Automake will use the value of that variable to put together the .SUFFIXES construct (see section More about Makefiles).

If you need to write your own rules or abstract rules, then check at some point that your distribution builds properly with `make distcheck'. It is very important, when you define your own rules, to follow the following guidelines:

• Prepend all the files that you wrote, both in the dependencies and the rules, with `$(srcdir)'. This variable points to the directory where your source code is located during the current building. Note that this may not be necessarily the same directory as the one returned by ``pwd`' if you are doing a VPATH build (see section Doing a VPATH build). During a build, the current working directory is the directory in which files are written, not the directory from which files are read. If you mess this up, then you will know when make distcheck fails, which attempts to do a VPATH build.
• Files that in an ordinary build are written by a program in the same directory as the corresponding `Makefile.am', in general, are written in the current working directory during a VPATH build. Therefore, you can refer to such files in the same `Makefile.am' as `./filename'.
• If you need to refer to any files under the top-level directory of your project, use $(top_srcdir) for files which you write (and your compiler tools read) and $(top_builddir) for files which the compiler tools write.
• The symbols `$<', `$@', `$^' don't need to prepended with anything, unlike ordinary filenames. GNU make will handle these symbols correctly during a VPATH build. Also see More about Makefiles.
• For your rules use only the following commands directory:

  ar cat chmod cmp cp diff echo egrep expr false grep ls mkdir mv pwd rm rmdir sed sleep sort tar test touch true

  Any other programs that you want to use, you must use them through make variables. That includes programs such as these:

  awk bash bison cc flex install latex ld ldconfig lex ln make makeinfo perl ranlib shar texi2dvi yacc

  The make variables can be defined through Autoconf in your `configure.ac'. For special-purpose tools, use the AC_PATH_PROGS macro. For example:

  AC_PATH_PROGS(BASH, bash) AC_PATH_PROGS(PERL, perl perl5)

  Some special tools have their own autoconf macros:

  AC_PROG_MAKE_SET → $(MAKE) → make AC_PROG_RANLIB → $(RANLIB) → ranlib | (do-nothing) AC_PROG_AWK → $(AWK) → mawk | gawk | nawk | awk AC_PROG_LEX → $(LEX) → flex | lex AC_PROG_YACC → $(YACC) → 'bison -y' | byacc | yacc AC_PROG_LN_S → $(LN_S) → ln -s

  See the Autoconf manual for more information.

6.8 Simple Automake examples

A real life example of a `Makefile.am' for libraries is the one I use to build the Blas-1 library. It looks like this:

• `blas1/Makefile.am'

SUFFIXES = .f
.f.o:
       $(F77) $(FFLAGS) -c $<

lib_LIBRARIES = libblas1.a
libblas1_a_SOURCES = f2c.h caxpy.f ccopy.f cdotc.f cdotu.f crotg.f cscal.f \
 csrot.f csscal.f cswap.f dasum.f daxpy.f dcabs1.f dcopy.f ddot.f dnrm2.f \
 drot.f drotg.f drotm.f drotmg.f dscal.f dswap.f dzasum.f dznrm2.f icamax.f \
 idamax.f isamax.f izamax.f sasum.f saxpy.f scasum.f scnrm2.f scopy.f \ 
 sdot.f snrm2.f srot.f srotg.f srotm.f srotmg.f sscal.f sswap.f zaxpy.f \ 
 zcopy.f zdotc.f zdotu.f zdrot.f zdscal.f zrotg.f zscal.f zswap.f 

Because the Blas library is written in Fortran, I need to declare the Fortran suffix at the beginning of the `Makefile.am' with the `SUFFIXES' assignment and then insert an implicit rule for building object files from Fortran files. The variables `F77' and `FFLAGS' are defined by Autoconf, by using the Fortran support provided by Autotoolset. For C or C++ files there is no need to include implicit rules. We discuss Fortran support at a later chapter.

Another important thing to note is the use of the symbol `$<'. We introduced these symbols in Chapter 2, where we mentioned that `$<' is the dependencies that changed causing the target to need to be rebuilt. If you've been paying attention you may be wondering why we didn't say `$(srcdir)/$<' instead. The reason is because for VPATH builds, `make' is sufficiently intelligent to substitute `$<' with the Right Thing.

Now consider the `Makefile.am' for building a library for solving linear systems of equations in a nearby directory:

• `lin/Makefile.am'

SUFFIXES = .f
.f.o:
       $(F77) $(FFLAGS) -c $<
INCLUDES = -I../blas1 -I../mathutil

lib_LIBRARIES = liblin.a
include_HEADERS = lin.h
liblin_a_SOURCES = dgeco.f dgefa.f dgesl.f f2c.h f77-fcn.h lin.h lin.cc

check_PROGRAMS = test1 test2 test3
TESTS = $(check_PROGRAMS)
LDADD = liblin.a ../blas1/libblas1.a ../mathutil/libmathutil.a $(FLIBS) -lm

test1_SOURCES = test1.cc f2c-main.cc
test2_SOURCES = test2.cc f2c-main.cc  
test3_SOURCES = test3.cc f2c-main.cc

In this case, we have a library that contains mixed Fortran and C++ code. We also have an example of a test suite, which in this case contains three test programs. What's new here is that in order to link the test suite properly we need to link in libraries that have been built already in other directories but haven't been installed yet. Because every test program requires to be linked against the same libraries, we set these libraries globally with an `LDADD' assignment for all executables. Because the libraries have not been installed yet we specify them with their full path. This will allow Automake to track dependencies correctly; if `libblas1.a' is modified, it will cause the test suite to be rebuilt. Also the variable `INCLUDES' is globally assigned to make the header files of the other two libraries accessible to the source code in this directory. The variable `$(FLIBS)' is assigned by Autoconf to link the run-time Fortran libraries, and then we link the installed `libm.a' library. Because that library is installed, it must be linked with the `-l' flag. Another peculiarity in this example is the file `f2c-main.cc' which is shared by all three executables. As we will explain later, when you link executables that are derived from mixed Fortran and C or C++ code, then you need to link with the executable this kludge file.

The test-suite files for numerical code will usually invoke the library to perform a computation for which an exact result is known and then verify that the result is true. For non-numerical code, the library will need to be tested in different ways depending on what it does.

6.9 Libraries with Automake

A good example, and all about how libraries should be tested and documented. Needs thinking.

6.10 Dealing with built sources

In some complicated packages, you want to generate part of their source code by executing a program at compile time. For example, in one of the packages that I wrote for an assignment, I had to generate a file `incidence.out' that contained a lot of hairy matrix definitions that were too ugly to just compute and write by hand. That file was then included by `fem.cc' which was part of a library that I wrote to solve simple finite element problems, with a preprocessor statement:

#include "incidence.out"

All source code files that are to be generated during compile time should be listed in the global definition of `BUILT_SOURCES'. This will make sure that these files get compiled before anything else. In our example, the file `incidence.out' is computed by running a program called `incidence' which of course also needs to be compiled before it is run. So the `Makefile.am' that we used looked like this:

noinst_PROGRAMS = incidence
lib_LIBRARIES = libpmf.a

incidence_SOURCES = incidence.cc mathutil.h
incidence_LDADD = -lm

incidence.out: incidence
      ./incidence > incidence.out

BUILT_SOURCES = incidence.out
libpmf_a_SOURCES = laplace.cc laplace.h fem.cc fem.h mathutil.h

check_PROGRAMS = test1 test2
TESTS = $(check_PROGRAMS)

test1_SOURCES = test1.cc
test1_LDADD = libpmf.a -lm

test2_SOURCES = test2.cc
test2_LDADD = libpmf.a -lm

Note that because the executable `incidence' has been created at compile time, the correct path is `./incidence'. Always keep in mind, that the correct path to source files, such as `incidence.cc' is `$(srcdir)/incidence.cc'. Because the `incidence' program is used temporarily only for the purposes of building the `libpmf.a' library, there is no reason to install it. So, we use the `noinst' prefix to instruct Automake not to install it.

6.11 Embedded text with Automake

In some cases, we want to embed text to the executable file of an application. This may be on-line help pages, or it may be a script of some sort that we intend to execute by an interpreter library that we are linking with, like Guile or Tcl. Whatever the reason, if we want to compile the application as a stand-alone executable, it is necessary to embed the text in the source code. Autotoolset provides with the build tools necessary to do this painlessly.

As a tutorial example, we will write a simple program that prints the contents of the GNU General Public License. First create the directory tree for the program:

% acmkdir foo

Enter the directory and create a copy of the txtc compiler:

% cd foo-0.1
% mktxtc

Then edit the file `configure.ac' and add a call to the LF_PROG_TXTC macro. This macro depends on

AC_PROG_CC
AC_PROG_AWK

so make sure that these are invoked also. Finally add `txtc.sh' to your AC_OUTPUT. The end-result should look like this:

AC_INIT(reconf)
AM_CONFIG_HEADER(config.h)
AM_INIT_AUTOMAKE(foo,0.1)
AC_PROG_CC
AC_PROG_RANLIB
AC_PROG_AWK
LF_PROG_TXTC
AC_OUTPUT(Makefile txtc.sh doc/Makefile m4/Makefile src/Makefile)

In the `src' directory use Emacs to create a file `src/text.txt' containing some random text. The `text.txt' file is the text that we want to print. You can substitute it with any text you want. This file will be compiled into `text.o' during the build process. The `text.h' file is a header file that gives access to the symbols defined by `text.o'. The file `copyleft.cc' is where the main will be written.

Next, create the following files with Emacs:

text.h

extern int text_txt_length; extern char *text_txt[];

foo.c

#if HAVE_CONFIG_H # include <config.h> #endif #include <stdio.h> #include "text.h" main() { for (i = 1; i<= text_txt_length; i++) printf ("%s\n", text_txt[i]); }

Makefile.am

SUFFIXES = .txt .txt.o: $(TXTC) $< bin_PROGRAMS = foo foo_SOURCES = foo.c text.h text.txt

and now you're set to build. Go back to the toplevel directory and go for it:

$ cd ..
$ reconf
$ configure
$ make
$ src/foo | less

To verify that this works properly, do the following:

$ cd src
$ foo > foo.out 
$ diff text.txt foo.out

The two files should be identical. Finally, convince yourself that you can make a distribution:

$ make distcheck

and there you are.

Note that in general the text file, as encoded by the text compiler, will not be always identical to the original. There is one and only one modification being made: If any line has any blank spaces at the end, they are trimmed off. This feature was introduced to deal with a bug in the Tcl interpreter, and it is in general a good idea since it conserves a few bytes, it never hurts, and additional whitespace at the end of a line shouldn't really be there.

This magic is put together from many different directions. It begins with the LF_PROG_TXTC macro:

Macro: LF_PROG_TXTC

This macro will define the variable TXTC to point to a Text-to-C compiler. To create a copy of the compiler at the toplevel directory of your source code, use the mktxtc command:

% mktxtc

The compiler is implemented as a shell script, and it depends on sed, awk and the C compiler, so you should call the following two macros before invoking AC_PROG_TXTC:

AC_PROG_CC AC_PROG_AWK

The compiler is intended to be used as follows:

$(TXTC) text1.txt text2.txt text3.txt ...

such that given the files `text1.txt', `text2.txt', etc. object files `text1.o', `text2.o', etc, are generated that contains the text from these files.

From the Automake point of view, you need to add the following two lines to Automake:

SUFFIXES = .txt
.txt.o:
        $(TXTC) $<

assuming that your text files will end in the .txt suffix. The first line informs Automake that there exist source files using non-standard suffixes. Then we describe, in terms of an abstract Makefile rule, how to build an object file from these non-standard suffixes. Recall the use of the symbol $<. Also note that it is not necessary to use $(srcdir) on $< for VPATH builds. If you embed more than one type of files, then you may want to use more than one suffixes. For example, you may have `.hlp' files containing online help and `.scm' files containing Guile code. Then you want to write a rule for each suffix as follows:

SUFFIXES = .hlp .scm
.hlp.o:
        $(TXTC) $<
.scm.o:
        $(TXTC) $<

It is important to put these lines before mentioning any SOURCES assignments. Automake is smart enough to parse these abstract makefile rules and recognize that files ending in these suffixes are valid source code that can be built to object code. This allows you to simply list `gpl.txt' with the other source files in the SOURCES assignment:

foo_SOURCES = foo.c text.h text.txt

In order for this to work however, Automake must be able to see your abstract rules first.

When you "compile" a text file `foo.txt' this makes an object file that defines the following two symbols:

int foo_txt_length;
char *foo_txt[];

Note that the dot characters are converted into underscores. To make these symbols accessible, you need to define an appropriate header file with the following general form:

extern int foo_txt_length; 
extern char *foo_txt[];

When you include this header file into your other C or C++ files then:

• You can obtain the filename containing the original text from

  foo_txt[0];

  and use it to print diagnostic messages.

• You can obtain the text itself line by line:

  char *foo_txt[1]; → first line char *foo_txt[2]; → second line ...

• The last line is set to NULL and foo_txt_length is defined such that

  char *foo_txt[foo_txt_length+1] == NULL

  The last line of the text is:

  char *foo_txt[foo_txt_length];

  You can use a for loop (or the loop macro defined by LF_CPP_PORTABILITY) together with foo_txt_length to loop over the entire text, or you can exploit the fact that the last line points to NULL and do a while loop.

and that's all there is to it.

6.12 Installation standard directories

Previously, we mentioned that the symbols `bin', `lib' and `include' refer to installation locations that are defined respectively by the variables `bindir', `libdir' and `includedir'. For completeness, we will now list the installation locations available by default by Automake and describe their purpose.

All installation locations are placed under one of the following directories:

`prefix'

The default value of `$(prefix)' is `/usr/local' and it is used to construct installation locations for machine-indepedent files. The actual value is specified at configure-time with the `--prefix' argument. For example:

configure --prefix=/home/lf

`exec_prefix'

The default value of `$(exec_prefix)' is `$(prefix)' and it used to construct installation location for machine-dependent files. The actual value is specified at configure-time with the `--exec-prefix' argument. For example:

configure --prefix=/home/lf --exec-prefix=/home/lf/gnulinux

The purpose of using a separate location for machine-dependent files is because then it makes it possible to install the software on a networked file server and make that available to machines with different architectures. To do that there must be separate copies of all the machine-dependent files for each architecture in use.

The GNU coding standards describe in detail the standard directories in which you should install your files. All of these standard locations are supported by Automake. So, for example, you can write things like

sbin_PROGRAMS = prog ...
sharedstate_DATA = foo ...
....

without having to define the variables `sbindir', `sharedstatedir' and so on.

1. Program-related files are installed in one of the following locations:

   bindir = $(exec_prefix)/bin

   The directory for installing executable programs that users can run. The default value for this directory is `/usr/local/bin'.

   sbindir = $(exec_prefix)/sbin

   The directory for installing executable programs that can be run from the shell, but are only generally useful to system administrators. The default value for this directory is `/usr/local/sbin'.

   libexecdir = $(exec_prefix)/libexec

   The directory for installing executable programs to be run by other programs rather than by users. The default value for this directory is `/usr/local/libexec'.

   libdir = $(exec_prefix)/lib

   The directory for installing libraries to be linked by other programs. The default value for this directory is `/usr/local/lib'. Please don't use this directory to install data files.

   includedir = $(prefix)/include

   The directory for installing public header files that declare the symbols of installed libraries.

2. Data files should be installed in one of the following directories:

   datadir = $(prefix)/share

   The directory for installing read-only architecture independent data files. The default value for this directory is `/usr/local/share'. Usually, most data files that you would like to install will have to go under this directory. These files are part of the program implementation and should not be modified.

   sysconfdir = $(prefix)/etc

   The directory for installing read-only data files that pertain to a single machine's configuration. Even though applications should only read, and not modify, these files, the user may have to modify these files to configure the application. Examples of files that belong in this directory are mailer and network configuration files, password files and so on. Do not install files that are modified in the normal course of their use (programs whose purpose is to change the configuration of the system excluded). Those probably belong in `localstatedir'.

   sharedstatedir = $(prefix)/com

   The directory for installing architecture-independent data files which the programs modify while they run. The default value for this directory is `/usr/local/com'.

   localstatedir = $(prefix)/var

   The directory for installing data files which the programs modify while they run, and that pertain to one specific machine. Users should never have to modify the files in this directory to configure the package's operation. The default value for this directory is `/usr/local/var'. System logs and mail spools are examples of data files that belong in this directory.

3. Then there are some directories for developing various eccentric types of files:

   lispdir = $(datadir)/emacs/site-lisp

   The directory for installing Emacs Lisp files. The default value of this directory is

   `/usr/local/share/emacs/site-lisp'.

   This directory is not automatically defined by Automake. To define it, you must invoke

   AM_PATH_LISPDIR

   from Autoconf. See section Emacs Lisp with Automake.

   m4dir = $(datadir)/aclocal

   The directory for installing Autoconf macros. This directory is not automatically defined by Automake so you will have to add a line in `Makefile.am':

   m4dir = $(datadir)/aclocal

   to define it yourself. See section Writing Autoconf macros.

4. Documentation should be installed in the following directories:

   infodir = $(prefix)/info

   The directory for installing the Info files for this package. The default value for this directory is `/usr/local/info'. See section Introduction to Texinfo.

   mandir = $(prefix)/man

   The top-level directory for installing the man pages (if any) for this package. The default value for this directory is `/usr/local/man'. See section Writing man pages.

   man1dir = $(prefix)/man1

   The top-level directory for installing section 1 man pages.

   man2dir = $(prefix)/man2

   The top-level directory for installing section 2 man pages.

Automake also defines the following subdirectories for your convenience:

pkglibdir     = $(libdir)/@PACKAGE@
pkgincludedir = $(includedir)/@PACKAGE@
pkgdatadir    = $(datadir)/@PACKAGE@

These subdirectories are useful for separating the files of your package from other packages. Of these three, you are most likely to want to use pkgincludedir to segragate public header files, as we discussed in Dealing with header files. For similar reasons you might like to segregate your data files. The only reason for using pkglibdir is to install dynamic libraries that are meant to be loaded only at run-time while an application is running. You should not use a subdirectory for libraries that are linked to programs, even dynamically, while the programs are being compiled, because that will make it more difficult to compile your programs. However, things like plug-ins, widget themes and so on should have their own directory.

6.13 Scripts with Automake

Sometimes you may feel the need to implement some of your programs in a scripting language like Bash or Perl. For example, the `autotoolset' package is exclusively a collection of shell scripts. Theoretically, a script does not need to be compiled. However, there are still issues pertaining to scripts such as:

• You want scripts to be installed with make install, uninstalled with make uninstall and distributed with make dist.
• You want scripts to get the path in the #! right.

To let Automake deal with all this, you need to use the `SCRIPTS' primitive. By listing a file under a `SCRIPTS' primitive assignment, you are telling Automake that this file needs to be built, and must be allowed to be installed in a location where executable files are normally installed. Automake by default will not clean scripts when you invoke the `clean' target. To force Automake to clean all the scripts, you need to add the following line to your `Makefile.am':

CLEANFILES = $(bin_SCRIPTS)

You also need to write your own targets for building the script by hand.

For example:

`hello1.sh'

# -* bash *- echo "Howdy, world!" exit 0

`hello2.pl'

# -* perl *- print "Howdy, world!\n"; exit(0);

`Makefile.am'

bin_SCRIPTS = hello1 hello2 CLEANFILES = $(bin_SCRIPTS) EXTRA_DIST = hello1.sh hello2.pl hello1: $(srcdir)/hello1.sh rm -f hello1 echo "#! " $(BASH) > hello1 cat $(srcdir)/hello1.sh >> hello1 chmod ugo+x hello1 hello2: $(srcdir)/hello2.pl $(PERL) -c hello2.pl rm -f hello2 echo "#! " $(PERL) > hello2 cat $(srcdir)/hello2.pl >> hello2 chmod ugo+x hello2

`configure.ac'

AC_INIT AM_INIT_AUTOMAKE(hello,0.1) AC_PATH_PROGS(BASH, bash sh) AC_PATH_PROGS(PERL, perl perl5.004 perl5.003 perl5.002 perl5.001 perl5) AC_OUTPUT(Makefile)

Note that in the "source" files `hello1.sh' and `hello2.pl' we do not include a line like

#!/bin/bash
#!/usr/bin/perl

Instead we let Autoconf pick up the correct path, and then we insert it during make. Since we omit the #! line, we leave a comment instead that indicates what kind of file this is.

In the special case of perl we also invoke

perl -c hello2.pl

This checks the perl script for correct syntax. If your scripting language supports this feature I suggest that you use it to catch errors at "compile" time. The AC_PATH_PROGS macro looks for a specific utility and returns the full path.

If you wish to conform to the GNU coding standards, you may want your script to support the --help and --version flags, and you may want --version to pick up the version number from AM_INIT_AUTOMAKE.

Here's an enhanced hello world scripts:

• version.sh.in

  VERSION=@VERSION@

• version.pl.in

  $VERSION="@VERSION@";

• hello1.sh

  # -* bash *- function usage { cat << EOF Usage: % hello [OPTION] Options: --help Print this message --version Print version information Bug reports to: monica@whitehouse.gov EOF } function version { cat << EOF hello $VERSION - The friendly hello world program Copyright (C) 1997 Monica Lewinsky <monica@whitehouse.gov> This is free software, and you are welcome to redistribute it and modify it under certain conditions. There is ABSOLUTELY NO WARRANTY for this software. For legal details see the GNU General Public License. EOF } function invalid { echo "Invalid usage. For help:" echo "% hello --help" } # ------------------------- if test $# -ne 0 then case $1 in --help) usage exit ;; --version) version exit ;; *) invalid exit ;; fi # ------------------------ echo "Howdy world" exit

• hello2.pl

  # -* perl *- sub usage { print <<"EOF"; Usage: % hello [OPTION] Options: --help Print this message --version Print version information Bug reports to: monica@whitehouse.gov EOF exit(1); } sub version { print <<"EOF"; hello $VERSION - The friendly hello world program Copyright (C) 1997 Monica Lewinsky <monica@whitehouse.gov> This is free software, and you are welcome to redistribute it and modify it under certain conditions. There is ABSOLUTELY NO WARRANTY for this software. For legal details see the GNU General Public License. EOF exit(1); } sub invalid { print "Invalid usage. For help:\n"; print "% hello --help\n"; exit(1); } # -------------------------- if ($#ARGV == 0) { do version() if ($ARGV[0] eq "--version"); do usage() if ($ARGV[0] eq "--help"); do invalid(); } # -------------------------- print "Howdy world\n"; exit(0);

• Makefile.am

  bin_SCRIPTS = hello1 hello2 CLEANFILES = $(bin_SCRIPTS) EXTRA_DIST = hello1.sh hello2.pl hello1: $(srcdir)/hello1.sh $(srcdir)/version.sh rm -f hello1 echo "#! " $(BASH) > hello1 cat $(srcdir)/version.sh $(srcdir)/hello1.sh >> hello1 chmod ugo+x hello1 hello2: $(srcdir)/hello2.pl $(srcdir)/version.pl $(PERL) -c hello2.pl rm -f hello2 echo "#! " $(PERL) > hello2 cat $(srcdir)/version.pl $(srcdir)/hello2.pl >> hello2 chmod ugo+x hello2

• configure.ac

  AC_INIT(hello,0.1) AM_INIT_AUTOMAKE AC_PATH_PROGS(BASH, bash sh) AC_PATH_PROGS(PERL, perl perl5.004 perl5.003 perl5.002 perl5.001 perl5) AC_CONFIG_FILES([Makefile version.sh version.pl])

Basically the idea with this approach is that when configure calls AC_OUTPUT it will substitute the files version.sh and version.pl with the correct version information. Then, during building, the version files are merged with the scripts. The scripts themselves need some standard boilerplate code to handle the options. I've included that code here as a sample implementation, which I hereby place in the public domain.

This approach can be easily generalized with other scripting languages as well, like Python and Guile.

6.14 Emacs Lisp with Automake

If your package requires you to edit a certain type of files, you might want to write an Emacs editing mode for it. Emacs modes are written in Emacs LISP, and Emacs LISP source code is written in files that are suffixed with `*.el'. Automake can byte-compile and install Emacs LISP files using Emacs for you.

To handle Emacs LISP, you need to invoke the

AM_PATH_LISPDIR

macro in your `configure.ac'. In the directory containing the Emacs LISP files, you must add the following line in your `Makefile.am':

lisp_LISP = file1.el file2.el ...

where `$(lispdir)' is initialized by `AM_PATH_LISPDIR'. The `LISP' primitive also accepts the `noinst' location.

Most Emacs LISP files are meant to be simply compiled and installed. Then the user is supposed to add certain invocations in per `.emacs' to use the features that they provide. However, because Emacs LISP is a full programming language you might like to write full programs in Emacs LISP, just like you would in any other language, and have these programs be accessible from the shell. If the installed file is called `foo.el' and it defines a function main as an entry point, then you can run it with:

% emacs --batch -l foo -f main

In that case, it may be useful to install a wrapper shell script containing

#!/bin/sh
emacs --batch -l foo -f main

so that the user has a more natural interface to the program. For more details on handling shell scripts See section Scripts with Automake. Note that it's not necessary for the wrapper program to be a shell script. You can have it be a C program, if it should be written in C for some reason.

Here's a tutorial example of how that's done. Start by creating a directory:

% mkdir hello-0.1
% cd hello-0.1

Then create the following files:

`configure.ac'

AC_INIT AM_INIT_AUTOMAKE(hello,0.1) AM_PATH_LISPDIR AC_OUTPUT(Makefile)

`hello.el'

(defun main () "Hello world program" (princ "Hello world\n"))

`hello.sh'

#!/bin/sh emacs --batch -l hello.el -f main exit

`Makefile.am'

lisp_LISP = hello.el EXTRA_DIST = hello.el hello.sh bin_SCRIPTS = hello CLEANFILES = $(bin_SCRIPTS) hello: $(srcdir)/hello.sh TAB rm -f hello TAB cp $(srcdir)/hello.sh hello TAB chmod ugo+x hello

Then run the following commands:

% touch NEWS README AUTHORS ChangeLog
% aclocal
% autoconf
% automake -a
% ./configure
% make
% make distcheck
# make install

FIXME: Discussion

6.15 Guile with Automake

FIXME: Do you want to volunteer for this section?

6.16 Data files with Automake

To install data files, you should use the `DATA' primitive instead of `SCRIPTS'. The main difference is that `DATA' will allow you to install files in data installation locations, whereas `SCRIPTS' will only allow you to install files in executable installation locations.

Normally it is assumed that the files listed in `DATA' are written by you and are not generated by a program, therefore they are not cleaned by default. If you want your data to be generated by a program, you must provide a target for building the data, and you must also mention the data file in `CLEANFILES' so that it's cleaned when you type `make clean'. You should of course include the source for the program and the appropriate lines in `Makefile.am' for building the program. For example: