18.1.2 Autotest Logs

When run, the test suite creates a log file named after itself, e.g., a test suite named testsuite creates testsuite.log. It contains a lot of information, usually more than maintainers actually need, but therefore most of the time it contains all that is needed:

command line arguments

A bad but unfortunately widespread habit consists of setting environment variables before the command, such as in ‘CC=my-home-grown-cc ./testsuite’. The test suite does not know this change, hence (i) it cannot report it to you, and (ii) it cannot preserve the value of CC for subsequent runs. Autoconf faced exactly the same problem, and solved it by asking users to pass the variable definitions as command line arguments. Autotest requires this rule, too, but has no means to enforce it; the log then contains a trace of the variables that were changed by the user.

ChangeLog excerpts

The topmost lines of all the ChangeLog files found in the source hierarchy. This is especially useful when bugs are reported against development versions of the package, since the version string does not provide sufficient information to know the exact state of the sources the user compiled. Of course, this relies on the use of a ChangeLog.

build machine

Running a test suite in a cross-compile environment is not an easy task, since it would mean having the test suite run on a machine build, while running programs on a machine host. It is much simpler to run both the test suite and the programs on host, but then, from the point of view of the test suite, there remains a single environment, host = build. The log contains relevant information on the state of the build machine, including some important environment variables.

tested programs

The absolute file name and answers to --version of the tested programs (see Writing testsuite.at, AT_TESTED).

configuration log

The contents of config.log, as created by configure, are appended. It contains the configuration flags and a detailed report on the configuration itself.