18.4 Making testsuite Scripts

For putting Autotest into movement, you need some configuration and makefile machinery. We recommend, at least if your package uses deep or shallow hierarchies, that you use tests/ as the name of the directory holding all your tests and their makefile. Here is a check list of things to do.

• Make sure to create the file package.m4, which defines the identity of the package. It must define AT_PACKAGE_STRING, the full signature of the package, and AT_PACKAGE_BUGREPORT, the address to which bug reports should be sent. For sake of completeness, we suggest that you also define AT_PACKAGE_NAME, AT_PACKAGE_TARNAME, and AT_PACKAGE_VERSION. See Initializing configure, for a description of these variables. We suggest the following makefile excerpt:

            $(srcdir)/package.m4: $(top_srcdir)/configure.ac
                    {                                      \
                      echo '# Signature of the current package.'; \
                      echo 'm4_define([AT_PACKAGE_NAME],      [@PACKAGE_NAME@])'; \
                      echo 'm4_define([AT_PACKAGE_TARNAME],   [@PACKAGE_TARNAME@])'; \
                      echo 'm4_define([AT_PACKAGE_VERSION],   [@PACKAGE_VERSION@])'; \
                      echo 'm4_define([AT_PACKAGE_STRING],    [@PACKAGE_STRING@])'; \
                      echo 'm4_define([AT_PACKAGE_BUGREPORT], [@PACKAGE_BUGREPORT@])'; \
                    } >'$(srcdir)/package.m4'
       

  Be sure to distribute package.m4 and to put it into the source hierarchy: the test suite ought to be shipped!

• Invoke AC_CONFIG_TESTDIR.
  — Macro: AC_CONFIG_TESTDIR (directory, [test-path = ‘directory’])
  

  An Autotest test suite is to be configured in directory. This macro requires the instantiation of directory/atconfig from directory/atconfig.in, and sets the default AUTOTEST_PATH to test-path (see testsuite Invocation).

• Still within configure.ac, as appropriate, ensure that some AC_CONFIG_FILES command includes substitution for tests/atlocal.
• The tests/Makefile.in should be modified so the validation in your package is triggered by ‘make check’. An example is provided below.

With Automake, here is a minimal example about how to link ‘make check’ with a validation suite.

     EXTRA_DIST = testsuite.at $(TESTSUITE) atlocal.in
     TESTSUITE = $(srcdir)/testsuite
     
     check-local: atconfig atlocal $(TESTSUITE)
             $(SHELL) '$(TESTSUITE)' $(TESTSUITEFLAGS)
     
     installcheck-local: atconfig atlocal $(TESTSUITE)
             $(SHELL) '$(TESTSUITE)' AUTOTEST_PATH='$(bindir)' \
               $(TESTSUITEFLAGS)
     
     clean-local:
             test ! -f '$(TESTSUITE)' || \
              $(SHELL) '$(TESTSUITE)' --clean
     
     AUTOTEST = $(AUTOM4TE) --language=autotest
     $(TESTSUITE): $(srcdir)/testsuite.at
             $(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
             mv $@.tmp $@

You might want to list explicitly the dependencies, i.e., the list of the files testsuite.at includes.

With strict Autoconf, you might need to add lines inspired from the following:

     subdir = tests
     
     atconfig: $(top_builddir)/config.status
             cd $(top_builddir) && \
                $(SHELL) ./config.status $(subdir)/$@
     
     atlocal: $(srcdir)/atlocal.in $(top_builddir)/config.status
             cd $(top_builddir) && \
                $(SHELL) ./config.status $(subdir)/$@

and manage to have atconfig.in and $(EXTRA_DIST) distributed.

With all this in place, and if you have not initialized ‘TESTSUITEFLAGS’ within your makefile, you can fine-tune test suite execution with this variable, for example:

     make check TESTSUITEFLAGS='-v -d -x 75 -k AC_PROG_CC CFLAGS=-g'