Customizing DejaGnu

Local Config File

It is usually more convenient to keep these manual overrides in the site.exp local to each test directory, rather than in the global site.exp in the installed DejaGnu library. This file is mostly for supplying tool specific info that is required by the testsuite.

All local site.exp files have two sections, separated by comment text. The first section is the part that is generated by make. It is essentially a collection of Tcl variable definitions based on Makefile environment variables. Since they are generated by make, they contain the values as specified by configure. (You can also customize these values by using the --site option to configure.) In particular, this section contains the Makefile variables for host and target configuration data. Do not edit this first section; if you do, your changes are replaced next time you run make.

	## these variables are automatically generated by make ##
	# Do not edit here. If you wish to override these values
	# add them to the last section
	

In the second section, you can override any default values (locally to DejaGnu) for all the variables. The second section can also contain your preferred defaults for all the command line options to runtest. This allows you to easily customize runtest for your preferences in each configured test-suite tree, so that you need not type options repeatedly on the command line. (The second section may also be empty, if you do not wish to override any defaults.)

	## All variables above are generated by configure. Do Not Edit ##
	

You can make any changes under this line. If you wish to redefine a variable in the top section, then just put a duplicate value in this second section. Usually the values defined in this config file are related to the configuration of the test run. This is the ideal place to set the variables host_triplet, build_triplet, target_triplet. All other variables are tool dependant, i.e., for testing a compiler, the value for CC might be set to a freshly built binary, as opposed to one in the user's path.

Here's an example local site.exp file, as used for GCC/G++ testing.

      ## these variables are automatically generated by make ##
      # Do not edit here. If you wish to override these values
      # add them to the last section
      set rootme "/build/devo-builds/i586-pc-linux-gnulibc1/gcc"
      set host_triplet i586-pc-linux-gnulibc1
      set build_triplet i586-pc-linux-gnulibc1
      set target_triplet i586-pc-linux-gnulibc1
      set target_alias i586-pc-linux-gnulibc1
      set CFLAGS ""
      set CXXFLAGS "-isystem /build/devo-builds/i586-pc-linux-gnulibc1/gcc/../libio -isystem $srcdir/../libg++/src -isystem $srcdir/../libio -isystem $srcdir/../libstdc++ -isystem $srcdir/../libstdc++/stl -L/build/devo-builds/i586-pc-linux-gnulibc1/gcc/../libg++ -L/build/devo-builds/i586-pc-linux-gnulibc1/gcc/../libstdc++"
      append LDFLAGS " -L/build/devo-builds/i586-pc-linux-gnulibc1/gcc/../ld"
      set tmpdir /build/devo-builds/i586-pc-linux-gnulibc1/gcc/testsuite
      set srcdir "${srcdir}/testsuite"
      ## All variables above are generated by configure. Do Not Edit ##

      

This file defines the required fields for a local config file, namely the three config triplets, and the srcdir. It also defines several other Tcl variables that are used exclusivly by the GCC testsuite. For most test cases, the CXXFLAGS and LDFLAGS are supplied by DejaGnu itself for cross testing, but to test a compiler, GCC needs to manipulate these itself.

Global Config File

The master config file is where all the target specific config variables for a whole site get set. The idea is that for a centralized testing lab where people have to share a target between multiple developers. There are settings for both remote targets and remote hosts. Here's an example of a Master Config File (also called the Global config file) for a canadian cross. A canadian cross is when you build and test a cross compiler on a machine other than the one it's to be hosted on.

Here we have the config settings for our California office. Note that all config values are site dependant. Here we have two sets of values that we use for testing m68k-aout cross compilers. As both of these target boards has a different debugging protocol, we test on both of them in sequence.

      # Make sure we look in the right place for the board description files.
      if ![info exists boards_dir] {
          set boards_dir {}
      }
      lappend boards_dir "/nfs/cygint/s1/cygnus/dejagnu/boards"

      verbose "Global Config File: target_triplet is $target_triplet" 2
      global target_list

      case "$target_triplet" in {
          { "native" } {
              set target_list "unix"
          }
          { "sparc64-*elf" } {
              set target_list "sparc64-sim"
          }
          { "mips-*elf" } {
              set target_list "mips-sim wilma barney"
          }
          { "mips-lsi-elf" } {
              set target_list "mips-lsi-sim{,soft-float,el}"
          }
          { "sh-*hms" } {
              set target_list { "sh-hms-sim" "bloozy" }
          }
      }
      

In this case, we have support for several cross compilers, that all run on this host. For testing on operating systems that don't support Expect, DejaGnu can be run on the local build machine, and it can connect to the remote host and run all the tests for this cross compiler on that host. All the remote OS requires is a working telnetd.

As you can see, all one does is set the variable target_list to the list of targets and options to test. The simple settings, like for sparc64-elf only require setting the name of the single board config file. The mips-elf target is more complicated. Here it sets the list to three target boards. One is the default mips target, and both wilma barney are symbolic names for other mips boards. Symbolic names are covered in the Adding A New Board chapter. The more complicated example is the one for mips-lsi-elf. This one runs the tests with multiple iterations using all possible combinations of the --soft-float and the --el (little endian) option. Needless to say, this last feature is mostly compiler specific.

Board Config File

The board config file is where board specfic config data is stored. A board config file contains all the higher-level configuration settings. There is a rough inheritance scheme, where it is possible to base a new board description file on an existing one. There are also collections of custom procedures for common environments. For more information on adding a new board config file, go to the Adding A New Board chapter.

An example board config file for a GNU simulator is as follows. set_board_info is a procedure that sets the field name to the specified value. The procedures in square brackets [] are helper procedures. Thes are used to find parts of a tool chain required to build an executable image that may reside in various locations. This is mostly of use for when the startup code, the standard C lobraries, or the tool chain itself is part of your build tree.

      # This is a list of toolchains that are supported on this board.
      set_board_info target_install {sparc64-elf}

      # Load the generic configuration for this board. This will define any
      # routines needed by the tool to communicate with the board.
      load_generic_config "sim"

      # We need this for find_gcc and *_include_flags/*_link_flags.
      load_base_board_description "basic-sim"

      # Use long64 by default.
      process_multilib_options "long64"

      setup_sim sparc64

      # We only support newlib on this target. We assume that all multilib
      # options have been specified before we get here.
      set_board_info compiler  "[find_gcc]"
      set_board_info cflags  "[libgloss_include_flags] [newlib_include_flags]"
      set_board_info ldflags  "[libgloss_link_flags] [newlib_link_flags]"
      # No linker script.
      set_board_info ldscript "";

      # Used by a few gcc.c-torture testcases to delimit how large the
      # stack can be.
      set_board_info gcc,stack_size 16384
      # The simulator doesn't return exit statuses and we need to indicate this
      # the standard GCC wrapper will work with this target.
      set_board_info needs_status_wrapper 1
      # We can't pass arguments to programs.
      set_board_info noargs 1
      

There are five helper procedures used in this example. The first one, find gcc looks for a copy of the GNU compiler in your build tree, or it uses the one in your path. This will also return the proper transformed name for a cross compiler if you whole build tree is configured for one. The next helper procedures are libgloss_include_flags & libgloss_link_flags. These return the proper flags to compiler and link an executable image using Libgloss, the GNU BSP (Board Support Package). The final procedures are newlib_include_flag & newlib_include_flag. These find the Newlib C library, which is a reentrant standard C library for embedded systems comprising of non GPL'd code.

Remote Host Testing

Thanks to Dj Delorie for the original paper that this section is based on.

DejaGnu also supports running the tests on a remote host. To set this up, the remote host needs an ftp server, and a telnet server. Currently foreign operating systems used as remote hosts are VxWorks, VRTX, DOS/Windows 3.1, MacOS and Windows.

The recommended source for a Windows-based FTP server is to get IIS (either IIS 1 or Personal Web Server) from http://www.microsoft.com. When you install it, make sure you install the FTP server - it's not selected by default. Go into the IIS manager and change the FTP server so that it does not allow anonymous FTP. Set the home directory to the root directory (i.e. c:\) of a suitable drive. Allow writing via FTP.

It will create an account like IUSR_FOOBAR where foobar is the name of your machine. Go into the user editor and give that account a password that you don't mind hanging around in the clear (i.e. not the same as your admin or personal passwords). Also, add it to all the various permission groups.

You'll also need a telnet server. For Windows, go to the Ataman web site, pick up the Ataman Remote Logon Services for Windows, and install it. You can get started on the eval period anyway. Add IUSR_FOOBAR to the list of allowed users, set the HOME directory to be the same as the FTP default directory. Change the Mode prompt to simple.

Ok, now you need to pick a directory name to do all the testing in. For the sake of this example, we'll call it piggy (i.e. c:\piggy). Create this directory.

You'll need a unix machine. Create a directory for the scripts you'll need. For this example, we'll use /usr/local/swamp/testing. You'll need to have a source tree somewhere, say /usr/src/devo. Now, copy some files from releng's area in SV to your machine:

      cd /usr/local/swamp/testing
      mkdir boards
      scp darkstar.welcomehome.org:/dejagnu/cst/bin/MkTestDir .
      scp darkstar.welcomehome.org:/dejagnu/site.exp .
      scp darkstar.welcomehome.org:/dejagnu/boards/useless98r2.exp boards/foobar.exp
      export DEJAGNU=/usr/local/swamp/testing/site.exp

      

You must edit the boards/foobar.exp file to reflect your machine; change the hostname (foobar.com), username (iusr_foobar), password, and ftp_directory (c:/piggy) to match what you selected.

Edit the global site.exp to reflect your boards directory:

	lappend boards_dir "/usr/local/swamp/testing/boards"
	

Now run MkTestDir, which is in the contrib directory. The first parameter is the toolchain prefix, the second is the location of your devo tree. If you are testing a cross compiler (ex: you have sh-hms-gcc.exe in your PATH on the PC), do something like this:

	./MkTestDir sh-hms /usr/dejagnu/src/devo
	

If you are testing a native PC compiler (ex: you have gcc.exe in your PATH on the PC), do this:

	./MkTestDir '' /usr/dejagnu/src/devo
	

To test the setup, ftp to your PC using the username (iusr_foobar) and password you selected. CD to the test directory. Upload a file to the PC. Now telnet to your PC using the same username and password. CD to the test directory. Make sure the file is there. Type "set" and/or "gcc -v" (or sh-hms-gcc -v) and make sure the default PATH contains the installation you want to test.

	cd /usr/local/swamp/testing
	make  -k -w check RUNTESTFLAGS="--host_board foobar --target_board foobar -v -v" > check.out 2>&1
	

To run a specific test, use a command like this (for this example, you'd run this from the gcc directory that MkTestDir created):

	make check RUNTESTFLAGS="--host_board sloth --target_board sloth -v compile.exp=921202-1.c"
	

Note: if you are testing a cross-compiler, put in the correct target board. You'll also have to download more .exp files and modify them for your local configuration. The -v's are optional.

Config File Values

DejaGnu uses a named array in Tcl to hold all the info for each machine. In the case of a canadian cross, this means host information as well as target information. The named array is called target_info, and it has two indices. The following fields are part of the array.

	set all_flag 1
	set RLOGIN /usr/ucb/rlogin
	set RSH /usr/local/sbin/ssh