CVS and generated files

Background: distributed generated files

Packages made with Autoconf and Automake ship with some generated files like configure or Makefile.in. These files were generated on the developer's host and are distributed so that end-users do not have to install the maintainer tools required to rebuild them. Other generated files like Lex scanners, Yacc parsers, or Info documentation, are usually distributed on similar grounds.

Automake outputs rules in Makefiles to rebuild these files. For instance make will run autoconf to rebuild configure whenever configure.in is changed. This makes development safer by ensuring a configure is never out-of-date with respect to configure.in.

As generated files shipped in packages are up-to-date, and because tar preserves timestamps, these rebuild rules are not triggered when a user unpacks and builds a package.

Background: CVS and timestamps

Unless you use CVS keywords (in which case files must be updated at commit time), CVS preserves timestamps during cvs commit and cvs import -d operations.

When you check out a file using cvs checkout its timestamp is set to that of the revision which is being checked out.

However, during cvs update, files will have the date of the update, not the original timestamp of this revision. This is meant to make sure that make notices sources files have been updated.

This timestamp shift is troublesome when both sources and generated files are kept under CVS. Because CVS processes files in alphabetical order, configure.in will appear older than configure after a cvs update that updates both files, even if configure was newer than configure.in when it was checked in. Calling make will then trigger a spurious rebuild of configure.

Living with CVS in Autoconfiscated projects

There are basically two clans amongst maintainers: those who keep all distributed files under CVS, including generated files, and those who keep generated files out of CVS.

All files in CVS

• The CVS repository contains all distributed files so you know exactly what is distributed, and you can checkout any prior version entirely.
• Maintainers can see how generated files evolve (for instance you can see what happens to your Makefile.ins when you upgrade Automake and make sure they look OK).
• Users do not need the autotools to build a checkout of the project, it works just like a released tarball.
• If users use cvs update to update their copy, instead of cvs checkout to fetch a fresh one, timestamps will be inaccurate. Some rebuild rules will be triggered and attempt to run developer tools such as autoconf or automake.

  Actually, calls to such tools are all wrapped into a call to the missing script discussed later (see maintainer-mode). missing will take care of fixing the timestamps when these tools are not installed, so that the build can continue.

• In distributed development, developers are likely to have different version of the maintainer tools installed. In this case rebuilds triggered by timestamp lossage will lead to spurious changes to generated files. There are several solutions to this:
  • All developers should use the same versions, so that the rebuilt files are identical to files in CVS. (This starts to be difficult when each project you work on uses different versions.)
  • Or people use a script to fix the timestamp after a checkout (the GCC folks have such a script).
  • Or configure.in uses AM_MAINTAINER_MODE, which will disable all these rebuild rules by default. This is further discussed in maintainer-mode.
• Although we focused on spurious rebuilds, the converse can also happen. CVS's timestamp handling can also let you think an out-of-date file is up-to-date.

  For instance, suppose a developer has modified Makefile.am and rebuilt Makefile.in, and then decide to do a last-minute change to Makefile.am right before checking in both files (without rebuilding Makefile.in to account for the change).

  This last change to Makefile.am make the copy of Makefile.in out-of-date. Since CVS processes files alphabetically, when another developer cvs update his or her tree, Makefile.in will happen to be newer than Makefile.am. This other developer will not see Makefile.in is out-of-date.

Generated files out of CVS

One way to get CVS and make working peacefully is to never store generated files in CVS, i.e., do not CVS-control files which are Makefile targets (or derived files in Make terminology).

This way developers are not annoyed by changes to generated files. It does not matter if they all have different versions (assuming they are compatible, of course). And finally, timestamps are not lost, changes to sources files can't be missed as in the Makefile.am/Makefile.in example discussed earlier.

The drawback is that the CVS repository is not an exact copy of what is distributed and that users now need to install various development tools (maybe even specific versions) before they can build a checkout. But, after all, CVS's job is versioning, not distribution.

Allowing developers to use different versions of their tools can also hide bugs during distributed development. Indeed, developers will be using (hence testing) their own generated files, instead of the generated files that will be released actually. The developer who prepares the tarball might be using a version of the tool that produces bogus output (for instance a non-portable C file), something other developers could have noticed if they weren't using their own versions of this tool.

Third-party files

Another class of files not discussed here (because they do not cause timestamp issues) are files which are shipped with a package, but maintained elsewhere. For instance tools like gettextize and autopoint (from Gettext) or libtoolize (from Libtool), will install or update files in your package.

These files, whether they are kept under CVS or not, raise similar concerns about version mismatch between developers' tools. The Gettext manual has a section about this, see CVS Issues.