GNU Source-highlight, given a source file, produces a document with syntax highlighting.
This is Edition 2.5 of the Source-highlight manual.
This file documents GNU Source-highlight version 2.5.
This manual is for GNU Source-highlight (version 2.5, 5 October 2006), which given a source file, produces a document with syntax highlighting.
Copyright © 2005, 2006 Lorenzo Bettini.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover Texts being “A GNU Manual,” and with the Back-Cover Texts as in (a) below. A copy of the license is included in the section entitled “GNU Free Documentation License.”(a) The FSF's Back-Cover Text is: “You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development.”
GNU Source-highlight, given a source file, produces a document with syntax highlighting. The colors and the styles can be specified (bold, italics, underline) by means of a configuration file, and some other options can be specified at the command line.
The program already recognizes many programming languages (e.g., C++, Java, Perl, etc.) and file formats (e.g., log files, ChangeLog, etc.), and some output formats (e.g., HTML, ANSI color escape sequences, LaTeX, etc.). Since version 2.0, it allows you to specify your own input source language via a simple syntax described later in this manual (Language Definitions). Since version 2.1, it allows you to specify your own output format language via a simple syntax described later in this manual (Output Language Definitions). Since version 2.2, it is able to generate cross references (e.g., to variable names, field names, etc.) by relying on the program ctags, http://ctags.sourceforge.net (Generating References).
The complete list of languages (indeed, file extensions) natively
supported by this version of Source-highlight (2.5), as
reported by --lang-list
, is the following:
Supported languages (file extensions) and associated language definition files C = cpp.lang H = cpp.lang bib = bib.lang bison = bison.lang c = cpp.lang caml = caml.lang cc = cpp.lang changelog = changelog.lang cls = latex.lang cpp = cpp.lang cs = csharp.lang csharp = csharp.lang diff = diff.lang docbook = xml.lang dtx = latex.lang eps = postscript.lang flex = flex.lang fortran = fortran.lang h = cpp.lang hh = cpp.lang hpp = cpp.lang htm = html.lang html = html.lang java = java.lang javascript = javascript.lang js = javascript.lang l = flex.lang lang = langdef.lang langdef = langdef.lang latex = latex.lang lex = flex.lang lgt = logtalk.lang ll = flex.lang log = syslog.lang logtalk = logtalk.lang lua = lua.lang ml = caml.lang mli = caml.lang outlang = outlang.lang pas = pascal.lang pascal = pascal.lang patch = diff.lang perl = perl.lang php = php.lang php3 = php.lang pl = prolog.lang pm = perl.lang postscript = postscript.lang prolog = prolog.lang ps = postscript.lang py = python.lang python = python.lang rb = ruby.lang ruby = ruby.lang sh = sh.lang shell = sh.lang sig = sml.lang sml = sml.lang sql = sql.lang sty = latex.lang style = style.lang syslog = syslog.lang tcl = tcl.lang tex = latex.lang tk = tcl.lang txt = nohilite.lang xhtml = xml.lang xml = xml.lang y = bison.lang yacc = bison.lang yy = bison.lang
The complete list of output formats natively supported by this version
of Source-highlight (2.5), as reported by
--outlang-list
, is the following:
Supported output languages and associated language definition files docbook = docbook.outlang esc = esc.outlang esc-doc = esc.outlang html = html.outlang html-css = css_common.outlang html-css-doc = cssdoc.outlang html-doc = htmldoc.outlang javadoc = javadoc.outlang latex = latex.outlang latex-doc = latexdoc.outlang latexcolor = latexcolor.outlang latexcolor-doc = latexcolordoc.outlang texinfo = texinfo.outlang xhtml = xhtml.outlang xhtml-css = xhtmlcss.outlang xhtml-css-doc = xhtmldoc.outlang xhtml-doc = xhtmldoc.outlang
The meaning of the suffixes -doc
, -css
and -css-doc
is explained in Output Language map.
Please, keep in mind, that I haven't tested personally all these
language definitions: I actually checked that the definition file is
correct (with the command line option --check-lang
, Invoking source-highlight), but I'm not sure their definition actually respects
that language syntax (e.g., I've put up together some language
definitions by searching for information in the Internet, but I've never
programmed in that language). So, if you find that a language
definition is not precise, please let me know. Moreover, if you have a
program example in a language that's not included in the tests
directory, please send it to me so that I can include it in the test
suite.
You can also use source-highlight as a simple formatter of input file, i.e., without performing any highlighting1.
You can achieve this by using, as the language definition file for input
sources the file nohilite.lang, using the command line option
--lang-def
(Invoking source-highlight). Since that
language definition is empty, no highlighting will be performed;
however, source-highlight will transform the input file in the output
format. Notice, in the input language associations in Supported languages, that nohilite.lang is also associated to txt files.
This, for instance, makes source-highlight useful in cases you want to transform a text file into HTML or LaTeX. During the output, in fact, source-highlight will correctly generate characters that have a specific meanings in the output format.
For instance, in this Texinfo manual,
if I want to insert a @ or a {
I have to “escape” them to make them appear literally
since they have a special meaning in Texinfo.
The same holds, e.g.,
for <
, >
or &
in HTML.
If you use source-highlight,
it will take care of this, automatically for you.
This is the Texinfo source of the above sentence:
For instance, in this Texinfo manual, if I want to insert a @@ or a @{ I have to ``escape'' them to make them appear literally since they have a special meaning in Texinfo. The same holds, e.g., for @code{<}, @code{>} or @code{&} in HTML. If you use source-highlight, it will take care of this, automatically for you.
This was processed by source-highlight as a simple text file, without no highlighting; however since it was formatted in Texinfo, all the necessary escaping was automatically performed. This way, it is very easy to insert, in the same document, a code, and its result (as in this example).
This is actually the formatting performed by source-highlight; except for the comment, this is basically what you should have written yourself to do all the escaping stuff manually:
@c Generator: GNU source-highlight, by Lorenzo Bettini, http://www.gnu.org/software/src-highlite @example For instance, in this Texinfo manual, if I want to insert a @@@@ or a @@@{ I have to ``escape'' them to make them appear literally since they have a special meaning in Texinfo. The same holds, e.g., for @@code@{<@}, @@code@{>@} or @@code@{&@} in HTML. If you use source-highlight, it will take care of this, automatically for you. @end example
In case source-highlight does not handle a specific input language, you
can still use the option --failsafe
(Invoking source-highlight) and also in that case no highlighting will be
performed, but source-highlight will transform the input file in the
output format.
Notice, however, that if the input language cannot be established, the default.lang will be used: an empty language definition file which you might want to customize.
See the file INSTALL for detailed building and installation instructions; anyway if you're used to compiling Linux software that comes with sources you may simply follow the usual procedure, i.e., untar the file you downloaded in a directory and then:
cd <source code main directory> ./configure make make install
However, before you do this, please check that you have everything that is needed to build source-highlight, What you need to build source-highlight.
Note: unless you specify a different install directory by
--prefix
option of
configure (e.g. ./configure --prefix=<your home>
),
you must be root to run make install
.
Files will be installed in the following directories:
Executables
/prefix/bin
docs and samples
/prefix/share/doc/source-highlight
conf files
/prefix/share/source-highlight
Default value for prefix is /usr/local
but you may change it with --prefix
option to configure.
NOTICE: Originally, instead of Source-highlight, there were two separate programs, namely GNU java2html and GNU cpp2html. There are two shell scripts with the same name that will be installed together with Source-highlight in order to facilitate the migration (however their use is not advised and it is deprecated).
You can download it from GNU's ftp site: ftp://ftp.gnu.org/gnu/src-highlite or from one of its mirrors (see http://www.gnu.org/prep/ftp.html).
I do not distribute Windows binaries anymore; since, they can be easily built by using Cygnus C/C++ compiler, available at http://www.cygwin.com. However, if you don't feel like downloading such compiler, you can request such binaries directly to me, by e-mail (find my e-mail at my home page) and I can send them to you. An MS-Windows port of Source-highlight is available from http://gnuwin32.sourceforge.net.
Archives are digitally signed by me (Lorenzo Bettini) with GNU gpg (http://www.gnupg.org). My GPG public key can be found at my home page (http://www.lorenzobettini.it).
You can also get the patches, if they are available for a particular release (see below for patching from a previous version).
This project's CVS repository can be checked out through anonymous (pserver) CVS with the following instruction:
cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/src-highlite co src-highlite
Further instructions can be found at the address:
http://savannah.gnu.org/projects/src-highlite.
Please notice that this way you will get the latest development sources of Source-highlight, which may also be unstable. This solution is the best if you intend to correct/extend this program: you should send me patches against the latest cvs repository sources.
If, on the contrary, you want to get the sources of a given release,
through cvs, say, e.g., version X.Y.Z, you must specify the tag
rel_X_Y_Z
when you run the cvs command or the cvs update
command.
When you compile the sources that you get through the cvs repository,
before running the configure
and make
commands, you
should, at least the first time, run the command:
sh reconf
This will run the autotools commands in the correct order, and also copy
possibly missing files. You should have installed recent versions of
automake
and autoconf
in order for this to succeed.
You will also need flex
and bison
.
NOTICE: This convention holds since release 2.1.
Since version 2.0 Source-highlight relies on regular expressions as provided by boost (http://www.boost.org), so you need to install at least the regex library from boost.
Most GNU/Linux distributions provide this library already in a compiled form. If you use your distribution packages, please be sure to install also the development package of the boost libraries.
If you experience problems in installing Boost Regex library, or in compiling source-highlight because of this library, please take a look at Tips on installing Boost Regex library.
If you want to use a specific version of the Boost regex library
(because you have many versions of it), you can use the configure option
--with-boost-regex
to specify a particular suffix. For instance,
./configure --with-boost-regex=boost_regex-gcc-1_31
Source-highlight has been developed under GNU/Linux, using gcc (C++), and bison (yacc) and flex (lex), and ported under Win32 with Cygnus C/C++compiler, available at http://www.cygwin.com. I used the excellent GNU Autoconf and GNU Automake. I also used Autotools (ftp://ftp.ugcs.caltech.edu/pub/elef/autotools) which creates a starting source tree (according to GNU standards) with autoconf, automake starting files. Finally I used GNU gengetopt (http://www.gnu.org/software/gengetopt), for command line parsing.
I started to use also doublecpp (http://www.lorenzobettini.it/software/doublecpp) that permits achieving dynamic overloading.
Actually, apart from the boost regex library, you don't need the other tools above to build source-highlight (indeed I provide the output sources generated by the above mentioned tools), unless you want to develop source-highlight.
However, if you obtained sources through CVS, you need some other tools, see Anonymous CVS Access.
I created this section because many users reported some problems after installing Boost Regex library from sources; other users had problems in compiling source-highlight even if this library was already correctly installed (especially windows users, using cygwin). I hope this section sheds some light in installing/using the Boost Regex library. Please, notice that this section does not explain how to compile the Boost libraries (the documentation you'll find on http://www.boost.org is well done); it explains how to tweak things if you have problems in compiling source-highlight even after a successful installation of Boost libraries.
If you experience no problem in compiling source-highlight, you can happily skip this section :-)
First of all, if your distribution provides packages for the Boost regex
library, please be sure to install also the development package of the
boost libraries, i.e., those providing also the header files needed to
compile a program using these libraries. For instance, on my Debian
system I had to install the package libboost-regex-dev
, besides
the package libboost-regex
.
If your distribution does not provide these packages then you have to download the sources of Boost libraries from http://www.boost.org and follow the instructions for compilation and installation. However, I suggest you specify /usr as prefix for installation, instead of relying on the default prefix /usr/local (unless /usr/local/include is already in the inclusion path of your C++ compiler), since this will make things easier when compiling source-highlight. I suggest this, since /usr/include is usually the place where C++ searches for header files during compilation.
If you successfully compiled and installed the Boost Regex library, or you installed the package from your distribution, but you STILL experience problems in compiling source-highlight, then you simply have to adjust some things as described in the following.
If the ./configure
command of source-highlight reports this
error:
ERROR! Boost::regex library not installed.
then, the compiler cannot find the header files for this library. In this case, check that the directory /usr/include/boost actually exists; if it does not, then probably you'll find a similar directory, e.g., /usr/include/boost-1_33/boost, depending on the version of the library you have installed. Then, all you have to do is to create a symbolic link as follows:
ln -s /usr/include/boost-1_33/boost /usr/include/boost
Alternatively, you might run source-highlight's configure as follows:
./configure CXXFLAGS=-I/usr/include/boost-1_33/
If then ./configure
command of source-highlight reports this
other error:
ERROR! Boost::regex library is installed, but you must specify the suffix with --with-boost-regex at configure for instance, --with-boost-regex=boost_regex-gcc-1_31
then, there's still another thing to fix: you must find out the exact names of the files of your installed Boost Regex libraries; you can do this by using the command:
$ ls -l /usr/lib/libboost_regex*
that, for instance, on one of my cygwin installation reports:
-rwxr-x---+ Nov 9 23:29 /usr/lib/libboost_regex-gcc-mt-s-1_33.a -rwxr-x---+ Nov 22 09:22 /usr/lib/libboost_regex-gcc-mt-s.a -rwxr-x---+ Nov 9 23:29 /usr/lib/libboost_regex-gcc-mt-s-1_33.so -rwxr-x---+ Nov 22 09:22 /usr/lib/libboost_regex-gcc-mt-s.so
Now, you have all the information to correctly run the source-highlight's configure command:
./configure --with-boost-regex=boost_regex-gcc-mt-s-1_33
or, if you solved the first problem in the second way2,
./configure CXXFLAGS=-I/usr/include/boost-1_33/ \ --with-boost-regex=boost_regex-gcc-mt-s-1_33
Of course, you have to modify this command according to the names of your Boost Regex library installed files.
These instructions managed to let many users, who were experiencing problems, to compile source-highlight If you still have problems, please send me an e-mail.
If you downloaded a patch, say source-highlight-1.3-1.3.1-patch.gz (i.e., the patch to go from version 1.3 to version 1.3.1), cd to the directory with sources from the previous version (source-highlight-1.3) and type:
gunzip -cd ../source-highlight-1.3-1.3.1.patch.gz | patch -p1
and restart the compilation process (if you had already run configure a simple make should do).
This was suggested by Konstantine Serebriany. The script src-hilite-lesspipe.sh will be installed together with source-highlight. You can use the following environment variables:
export LESSOPEN="| /path/to/src-hilite-lesspipe.sh %s" export LESS=' -R '
This way, when you use less to browse a file, if it is a source file handled by source-highlight, it will be automatically highlighted.
Christian W. Zuckschwerdt added support for building an .rpm and an .rpm.src. You can issue the following command
rpm -tb source-highlight-2.5.tar.gz
for building an .rpm with binaries and
rpm -ts source-highlight-2.5.tar.gz
for building an .rpm.src with sources.
Martin Gebert is also implementing a KDE interface to source-highlight programs (and he did a wonderful job!), and it is called ksrc2html; if you want to test it: http://www.mgebert.de.
CGI support was enabled thanks to Robert Wetzel; I haven't tested it personally yet, so you may ask him directly. Moreover he set up some examples at the page http://www.inf.tu-dresden.de/~rw8/java2.html. If you want to use source-highlight as a CGI program, you have to use the executable source-highlight-cgi. You can build such executable by issuing
make source-highlight-cgi
in the src directory.
Moreover there's also a Java version of java2html, you can find it at http://www.generationjava.com/projects/Java2Html.shtml.
GNU Source-highlight is free software; you are free to use, share and modify it under the terms of the GNU General Public License that accompanies this software (see COPYING).
GNU source-highlight was written and maintained by Lorenzo Bettini http://www.lorenzobettini.it.
Here are some realistic examples of running source-highlight3.
Source-highlight only does a lexical analysis of the source code, so the program source is assumed to be correct!
Here's how to run source-highlight (for this example we will use C/C++ input files, but this is valid also for other source-highlight input languages):
source-highlight --src-lang cpp --out-format html \ --input <C++ file> \ --output <html file> \ --style-file <style file> \ options
For input files, apart from the -i (--input)
option and the
standard input redirection, you can simply specify some files at the
command line and also use regular expressions (for instance
*.java). In this case the name for the output files will be
formed using the name of the source file with a .<ext> appended, where
<ext> is the extension chosen according to the output format specified
(in this example it would be .html). The style file
(Output format style)
contains information on how to format specific language parts
(e.g., keywords in blue and boldface, etc.).
If STDOUT
string is passed as -o (--output)
option, then
the output is forced to the standard output anyway.
If -s (--src-lang)
is not specified, the source language is
inferred by the extension of the input file (this, of course, does not
work with standard input redirection). For further details, see
How the input language is discovered.
If -f (--out-format)
is not specified, the output will be
produced in HTML.
If --style-file
is not specified, the default.style, which
is included in the distribution, will be used (see Output format style
for further information).
The default output format for HTML uses fixed width fonts by inserting
all the formatted output between <tt>
and </tt>
. Thus,
for instance, specification for fixed width and not fixed width (see
Output format style) will have no effect: every character will
have fixed width. If you don't like this default behavior and would
like to have not fixed fonts by default (as it happens, e.g., with
LaTeX output) you can use the file html_notfixed.outlang
with the command line argument --outlang-def
.
When using LaTeX output format you can choose between monochromatic
output (by using -f latex
) or colored output (by using -f
latexcolor
). When using colored output, you need the
color
package (again this should be present in your system).
Of course, you are free to define your own LaTeX output format,
see Output Language Definitions.
When using the Texinfo output format, you may want to use a dedicated
style file, texinfo.style, which comes with the source-highlight
distribution, with the option --style-file
. For instance, the
example in Examples is formatted with this style file.
If you're using this output format, for instance together with
less
(see Using source-highlight with less), you may
want to use the esc.style, which comes with the source-highlight
distribution, with the option --style-file
. This should
result in a more pleasant coloring output.
During execution, source-highlight needs some files where it finds
directives on how to recognize the source language (if not specified
explicitly with --src-lang
or --lang-def
), on which output
format to use (if not specified explicitly with --out-format
or
--outlang-def
), on how to format specific source elements (e.g.,
keywords, comments, etc.), and source and output language definitions.
These files will be explained in the next sections.
If the directory for such files is not explicitly specified with the
command line option --data-dir
, these files are searched for in
the following order:
If you want to be sure about which file is used during the
execution, you can use the command line option --verbose
.
You must specify your options for syntax highlighting in the file
default.style4.
You can specify formatting options for each element defined
by a language definition file (you can get the list of such elements,
by using --show-lang-elements
, see Listing Language Elements). Here's the one that comes with this distribution (this is
formatted by using the style.lang that is shown in
Tutorials on Language Definitions):
keyword blue b ; // for language keywords type darkgreen ; // for basic types string red f ; // for strings and chars specialchar pink f ; // for special chars, e.g., \n, \t, \\ comment brown i, noref; // for comments number purple ; // for literal numbers preproc darkblue b ; // for preproc directives (e.g. #include, import) symbol darkred ; // for simbols (e.g. <, >, +) function black b; // for function calls and declarations cbracket red; // for block brackets (e.g. {, }) todo b; // for TODO and FIXME // line numbers linenum black f; // Internet related url blue u, f; // other elements for ChangeLog and Log files date blue b ; time darkblue b ; ip darkgreen ; file darkblue b ; name darkgreen ; // for Prolog, Perl... variable darkgreen ; // explicit for Latex italics darkgreen i; bold darkgreen b; underline darkgreen u; fixed green f; argument darkgreen; optionalargument purple; math orange; bibtex blue; // for diffs oldfile orange; newfile darkgreen; difflines blue;
This files tries to define a style for most elements defined in the language definition files that comes with Source-highlight distribution.
You can specify your own file (it doesn't have to be named
default.style) with the command line option
--style-file
5, see
Invoking source-highlight.
You can also specify the color of normal text by adding this line
normal darkblue ;
As you might see the syntax of this file is quite straightforward:
b = bold i = italics u = underline f = fixed nf = not fixed noref = no reference information is generated for these elements
Since version 2.2, the color specification is not required. For instance, the texinfo.style is as follows (we avoid colors for Texinfo outputs):
keyword b ; type b ; variable f, i ; string f ; comment nf, i, noref ; preproc b ; // line numbers linenum f; // Internet related url f; // for diffs oldfile i; newfile i; difflines b;
You may also specify more than on of these options separated by commas, e.g.
keyword blue u, b ;
Please keep in mind that in this case the order of these specified
options is kept during the generation of the output; for instance,
depending on the specific output format, the sequences u, b
and
b, u
may lead to different results. In particular, the style
that comes first is used after the ones that follow. For instance, in
the case of HTML, the sequence u, b
will lead to the following
formatting: <u><b>...</b></u>
.
The noref
option specifies that for this element reference
information are not generated (see Generating References). For
instance, this is used for the comment
element, since we do not
want that elements in a comment are searched for cross-references.
These are all possible color logical names handled by source-highlight6:
black red darkred brown yellow cyan blue pink purple orange brightorange green brightgreen darkgreen teal gray darkblue
You can also use the direct color scheme for the specific output format,
by using double quotes, such as, e.g., "#00FF00"
in
HTML7. Of course, the double quotes will be discarded during the
generation.
This configuration file associates a file extension to a specific
language definition file. You can also use such file extension to
specify the --src-lang
option (see Simple Usage).
Source-highlight comes with such a file, called lang.map.
Of course, you can override the settings of this file by
writing your own language map file and specify such file
with the command line option --lang-map
).
Moreover, as explained above, if a file lang.map
is present in the current directory, such version will be used.
The format of such file is quite simple:
extension = language definition file
The default language definition file is shown in Introduction.
These files are crucial for source-highlight since they specify the source elements that have to be highlighted. These files also allow to specify your own language definitions in order to deal with a language that is not handled by source-highlight8. The syntax for these files is explained in Language Definitions.
This configuration file associates an output format to a specific output
language definition file. You can use the name of that output format to
specify the --out-format
option (see Simple Usage).
Source-highlight comes with such a file, called outlang.map.
Of course, you can override the settings of this file by
writing your own output language map file and specify such file
with the command line option --outlang-map
).
Moreover, as explained above, if a file outlang.map
is present in the current directory, such version will be used.
The format of such file is quite simple:
output format name = language definition file
The default language definition file is shown in Introduction.
In particular, there is a convention for the output format name
in the output language map, according to the suffix of the name
with a dash -
:
-doc
--doc
command line option is given
-css-doc
--css
command line option is given
-css
--css
and --no-doc
command line options
are given
If a combination of the above mentioned command line options is given for a specific output format, and a corresponding definition file is not specified in the map file, then an error is raised.
For instance, if you specified the definition file for your language
mylang
and also one for dealing with --doc
option, i.e., a
definition file for mylang-doc
, and you run source-highlight as
follows:
source-highlight -f mylang --css mycss.css
You will get the following error:
source-highlight: output language mylang-css-doc not handled
These files are crucial for source-highlight since they specify how the source elements are highlighted. These files also allow to specify your own output format definitions in order to deal with an output format that is not handled by source-highlight9. The syntax for these files is explained in Output Language Definitions.
I encourage those who write new language definitions or correct/modify existing language definitions to send them to me so that they can be added to the source-highlight distribution!
Since these files require more explanations (that, however, are not necessary to the standard usage of source-highlight), they are carefully explained in separate parts: Language Definitions and Output Language Definitions.
The format for running the source-highlight program is:
source-highlight option ...
source-highlight
supports the following options, shown by
the output of source-highlight --help
:
source-highlight Highlight the syntax of a source file (e.g. Java) into a specific format (e.g. HTML) Usage: source-highlight [OPTIONS]... -h, --help Print help and exit -V, --version Print version and exit -i, --input=filename input file. default std input -o, --output=filename output file. default std output. If STDOUT is specified, the output is directed to standard output -s, --src-lang=STRING source language (use --lang-list to get the complete list). If not specified, the source language will be guessed from the file extension. --lang-list list all the supported language and associated language definition file --outlang-list list all the supported output language and associated language definition file -f, --out-format=STRING output format (use --outlang-list to get the complete list) (default=`html') -d, --doc create an output file that can be used as a stand alone document (e.g., not to be included in another one) --no-doc cancel the --doc option even if it is implied (e.g., when css is given) -c, --css=filename the external style sheet filename. Implies --doc -T, --title=STRING give a title to the output document. Implies --doc -t, --tab=INT specify tab length. (default=`8') -H, --header=filename file to insert as header -F, --footer=filename file to insert as footer --style-file=filename specify the file containing format options (default=`default.style') --outlang-def=filename output language definition file --outlang-map=filename output language map file (default=`outlang.map') --data-dir=path directory where language definition files and language maps are searched for. If not specified these files are searched for in the current directory and in the data dir installation directory --output-dir=path output directory --lang-def=filename language definition file --lang-map=filename language map file (default=`lang.map') --show-lang-elements=filename prints the language elements that are defined in the language definition file --infer-lang force to infer source script language (overriding given language specification) reference generation: -n, --line-number number all output lines --line-number-ref[=prefix] number all output lines and generate an anchor, made of the specified prefix + the line number (default=`line') --gen-references=STRING generate references (possible values="inline", "postline", "postdoc" default=`inline') --ctags-file=filename specify the file generated by ctags that will be used to generate references (default=`tags') --ctags=cmd how to run the ctags command. If this option is not specified, ctags will be executed with the default value. If it is specified with an empty string, ctags will not be executed at all (default=`ctags --excmd=n --tag-relative=yes') testing: -v, --verbose verbose mode on --statistics print some statistics (i.e., elapsed time) --gen-version put source-highlight version in the generated file (default=on) --check-lang=filename only check the correctness of a language definition file --check-outlang=filename only check the correctness of an output language definition file --failsafe if no language definition is found for the input, it is simply copied to the output -g, --debug-langdef[=type] debug a language definition. In dump mode just dumps all the steps; in interactive, at each step, waits for some input (press ENTER to step) (possible values="interactive", "dump" default=`dump') --show-regex=filename show the regular expression automaton corresponding to a language definition file
Let us explain some options in details (apart from those that should be
clear from the --help
output itself, and those already explained
in Simple Usage).
--doc
-d
--title
, the your source file name will be used as the title.
--no-doc
--doc
option above is actually implied by other command line
options (e.g., --css
). If you do not want this (e.g., you want
to include the output in an existing document containing the global
style sheet), you can disable this by using --no-doc
.
--css
-c
--tab
-t
--output-dir
--infer-lang
--line-number-ref
--line-number
, this option numbers all the output lines, and,
additionally, generates an anchor for each line. The anchor consists of
the specified prefix (default is line
) and the line number (e.g.,
line25
). For instance, as prefix, if you deal with many files,
you can use the file name. Notice that some output languages might not
support this feature (e.g., esc
, since it makes no sense in such
case). See Anchors and References for defining how to generate an
anchor in a specific output language.
--failsafe
When using --failsafe
, if no input language can be established,
source-highlight will use the input language definition file
default.lang, which is an empty file. You might want to
customize such file, though.
--debug-lang
--show-regex
The other command line options dealing with references are explained in more details in Generating References.
As already explained, Simple Usage, source-highlight uses a
language definition file according the language specified with the
option --src-lang
, or --lang-def
, or by using the input
file extension.
Since version 2.5, source-highlight can use an inference mechanism to deduce the input language. For the moment, it can detect script languages based on the “sha-bang” mechanism, i.e., when the first line of a script contains a line such as, e.g.,
#!/bin/sh
It also recognizes the Emacs convention, of declaring the Emacs major
mode using the format -*- lang -*-
.
For instance, a script starting as the following one:
#!/bin/bash # -*- Tcl -*-
will be interpreted as a Tcl script, and not as bash script.
This inference mechanism is performed, by default, in case the input language is neither explicitly specified nor found in the language map file by using the input file extension (the input file may also have no extension at all).
Furthermore, this mechanism can be given priority with the command line
option --infer-lang
. For instance, this is used in the script
src-hilite-lesspipe.sh (Using source-highlight with less)
when running source-highlight, in order to avoid the problem of
formatting a Perl script as a Prolog program (since the extension
.pl is associated to Prolog programs in the language map file).
Since version 2.0 source-highlight uses a specific syntax to specify source language elements (e.g., keywords, strings, comments, etc.). Before version 2.0, language elements were scanned through Flex. This had the drawback of writing a new flex file to deal with a new language; even worse, a new language could not be added “dynamically”: you had to recompile the whole source-highlight program.
Instead, now, language elements are specified in a file, loaded dynamically, through a (hopefully) simple syntax. Then, these definitions are used internally to create, on-the-fly, regular expressions that are used to highlight the elements. In particular, we use the regular expressions provided by the Boost library (see Installation). Thus, when writing a language definition file you will surely have to deal with regular expressions. Of course, we use the Boost regex library regular expression syntax. We refer to Boost documentation for such syntax, http://www.boost.org/libs/regex/doc/syntax.html.
Here, we see such syntax in details, by relying on many examples. This allows a user to easily modify an existing language definition and create a new one. These files have, typically, extension .lang.
Each definition basically associates a regular expression to a language
element and defines a name for the language element. Such name will be
used to associate a particular style (e.g., bold face, color, etc.) to
the highlighting of such elements. You cannot use names that are the
same of keywords used in the language definition syntax (e.g.,
start
, as shown later, is a reserved word).
Comments can be given by using #
; the rest of the line is
considered as a comment.
Source-highlight will scan each line of the input file separately. So a regular expression that tries to match new line characters is destined to fail. However, the language definition syntax provides means to deal with multiple lines (see Delimited definitions and State/Environment Definitions).
The simpler way of specify language elements is to list the possible alternatives. This is the case, for instance, for keywords. For instance, in java.lang you have:
keyword = "abstract|assert|break|case|catch|class|const", "continue|default|do|else|extends|false|final", "finally|for|goto|if|implements|instanceof|interface" keyword = "native|new|null|private|protected|public|return", "static|strictfp|super|switch|synchronized|throw", "throws|true|this|transient|try|volatile|while"
The elements must be specified in double quotes. You can separate
quoted definitions with commas. Alternatively, within a quoted
definition, alternatives can be separated with the pipe symbol |
.
The above definition defines the language element keyword
. Each
time an element is found in the source file, it is highlighted with the
style for the element with the same name in the output format style file
(notice that all elements shown in the example are take from the
language definition files that come with source-highlight and there is a
style for each of such elements, see Configuration files). If
such an element is not specified in the output format style file, it is
simply not highlighted (so pay attention to typos :-).
From the above example you may have noticed that language element
definitions are cumulative, so the second keyword
definition does
not replace the first one. (Indeed, in some case you may want to
actually redefine a language element; this is possible as explained in
the following sections.)
Notice that words specified in double quotes have to match exactly in a
source file, and they must be isolated (not surrounded by anything but
spaces). Thus for instance class
is matched as a keyword, but in
my_class
the substring class
is not matched as keyword.
From the point of view of regular expressions a string such as
class
in a double quote simple definition is intended as
\<(class)\>
.
Special characters have to be escaped with the character \
. So
for instance if you want to specify the character |
, which is
normally used to separate alternatives in double quoted strings, you
have to specify \|
.
Definitions in double quotes are interpreted literally (thus, e.g., a
dot .
is interpreted as the character .
not as the regular
expression wild card). If you want to enjoy the full power of regular
expressions to specify a language alternative, you have to use single
quoted strings ('
), instead of double quoted strings.
For instance, the following is the definition for a preprocessor directive in C/C++:
preproc = '^[[:blank:]]*#([[:blank:]]*[[:word:]]*)'
Notice that the definition 'class'
is different from
"class"
, as explained above. Thus, for instance 'class'
matches also the sub-expression class
inside my_class
.
Finally, at the end of a list of definitions, one can specify the
keyword nonsensitive
; in that case, the specified strings will be
interpreted in a non case sensitive way. For instance, we use this
feature in Pascal language definition, pascal.lang where keywords
are parsed in a non sensitive way:
keyword = "alfa|and|array|begin|case|const|div", "do|downto|else|end|false|file|for|function|get|goto|if|in", "label|mod|new|not|of|or|pack|packed|page|program", "put|procedure|read|readln|record|repeat|reset|rewrite|set", "text|then|to|true|type|unpack|until|var|while|with|writeln|write" nonsensitive
It is often useful to define a language element that affects all the
remaining characters up to the end of the line. For such definitions,
instead of the =
you must use the keyword start
. For
instance, the following is the definition of a single line comment in
C++:
comment start "//"
This says that when the two characters //
are encountered in the
source file, everything from these characters, include, up to the end of
the line, will be highlighted according to the style comment
.
It is important to observe that the order of language definitions is important since it will be used during regular expression matching. You then have to make sure that, if there are definitions that start with same characters, the longest expression is specified first in the file. For instance if you write
symbol = "/" comment start "//"
The first expression will always be matched first, and the second expression will never be matched. The right order is
comment start "//" symbol = "/"
Many elements are delimited by specific character sequences. For instance, strings and multiline comments. The syntax for such an element definition is
<name> delim <left delimited> <right delimiter> \ {escape <escape character>} \ {multiline} {nested}
The escape
specification allows to specify the escape
character that may precede one of the delimiters inside the
element. This is optional.
For instance, this is the definition of C-like strings:
string delim "\"" "\"" escape "\\"
Notice that \
is a special characters in definitions so it has to
be escaped. If the escape
specification was omitted, the C
string "write \"hello\" string"
would have been highlight
incorrectly (it would have been highlighted as the string
"write \"
, the normal character sequence hello\
and
the string " string"
).
The option multiline
specifies that the element can spawn
multiple lines. For instance, PHP strings are defined as follows:
string delim "\"" "\"" escape "\\" multiline
The option nested
instructs to count possible multiple
occurrences of delimited characters and to match relative
multiple occurrences. For instance, C-like multiline comments
are specified as follows:
comment delim "/*" "*/" multiline nested
If nested
was not used the following nested comment
would have not been highlighted correctly:
/* This is a /* nested comment */ */
As said above, definitions are cumulative, and they are also cumulative even when using different syntactic forms. Thus, for instance, the complete definition for C++-style comments are the following:
comment start "//" comment delim "/*" "*/" multiline nested
It is possible to define variables to be re-used in many parts in a language definition file. A variable is defined by using
vardef
<name of the variable> =
<list of definitions>
Once defined, a variable can be used by prepending the
symbol $
to its name. For instance,
vardef FUNCTION = '(?:[[:alpha:]]|_)[[:word:]]*[[:blank:]]*(?=\()' function = $FUNCTION
The capital letters are used only for readability.
It is also possible to concatenate variables and expressions, and reuse variables inside further variable definitions:
vardef basic_time = '[[:digit:]]{2}:[[:digit:]]{2}:[[:digit:]]{2}' vardef time = '\<' + $basic_time + '\>'
It is possible to include other language definition files into another
file. This is inclusion actually physically includes the contents of
the included file into the current file during parsing, at the exact
point of inclusion (just like the #include
in C/C++). This is
useful for re-using definitions in many files. For instance, C++
comment definitions are given in a file c_comment.lang, and this
file is included in the Java and C++ definition files. The same happens
for number and functions. For instance, the file java.lang
contains the following include instructions:
include "c_comment.lang" include "number.lang" keywords ... include "function.lang"
Notice that the order of inclusion is crucial since the order of
definition is crucial. If function definition was included before
keyword definitions, then the sentence if (exp)
would be
highlighted as a function invocation.
Sometimes you want some source element to be highlighted only if they are surrounded by other elements. Source-highlight language definitions provides also this feature.
state|environment <standard definition> begin <other definitions> end
This structure is recursive (so other state/environment definitions can
be given within a state/environment). The meaning of a
state/environment is that the definitions within the begin
... end
are matched only if the definitions that define the
state/environment have been matched. When entering a state/environment,
however, the definitions given outside the state/environment are not
matched. The difference between state
and environment
is
that in the latter, normal parts of the source language (i.e., those
that do not match any definition) are highlighted according to the style
of the definition that defines the environment.
As an example, the following defines the multiline nested C comment, and highlights URL and e-mail addresses only when they appear inside a comment (notice that this uses file inclusion):
environment comment delim "/*" "*/" multiline nested begin include "url.lang" end
Notice that we used environment
because everything else inside a
comment has to be formatted according to the comment style.
While for programming language definitions states/environments can be avoided (although they allow to highlight some parts only if inside a specific environment, e.g., URLs inside comments, or documentation tags in Javadoc comments), they are pretty important for highlighting files such as logs and ChangeLog files, since elements have to be highlighted when they appear in a specific position. For instance, for ChangeLog (see changelog.lang), we use a state for highlighting the date, name, e-mail or URL (taken from url.lang):
state date start '[[:digit:]]{2,4}-?[[:digit:]]{2}-?[[:digit:]]{2}' begin include "url.lang" name = '([[:word:]]|[[:punct:]])+' end
Notice that definitions that appear inside a state/environment have the
same scope of the expressions that define the environment. While this
makes sense for start
and delim
definitions, it may makes
less sense for simple definitions (i.e., those that simply lists all
possible expressions): in fact, in this case, such expressions do not
define a scope. For such definitions, the semantics of
state/environment is that the state/environment starts after matching one
of the alternatives. And where will it end? In this case you must
explicitly exit the environment. For instance, you can say that, when
inside a state/environment, a specific language definition, when
encountered also exits the environment (with the keyword exit
).
You can even exit all the environments with exitall
. For
instance, the following definition, highlights a non empty string
following a web method:
vardef non_empty = '[^[:blank:]]+' state webmethod = "OPTIONS|GET|HEAD|POST|PUT|DELETE", "TRACE|CONNECT|PROPFIND|MKCOL|COPY|MOVE|LOCK|UNLOCK" begin string = $non_empty exit end
If you ever need such advanced features, you may want to take a look at the log.lang definition file that defines highlighting for several log files (access logs, Apache logs, etc.).
These two features are useful when you want to define
a language by re-using an existing language definition
with some changes. Typically you include
another
language definition file and you redefine/substitute some
elements.
When you use redef
you erase all the previous
definitions of that language elements with the new one.
The new language element definition will be placed exactly
in the point of the new definition.
We use this feature, for instance, when we define the
sml
language by re-using the caml
one:
they differ only for the keywords11. In fact, the contents of
sml.lang is summarized as follows:
include "caml.lang" redef keyword = "abstraction|abstype|and|andalso..." redef type = "int|byte|boolean|char|long|float|double|short|void"
Since the new language element definition appears in the
exact point of the redefinition, this means that
such a regular expression will be matched only if all
the previous ones (the ones of the included file) cannot
be matched. This may lead to unwanted results in some
cases (not in the sml
case though).
In other words the following code
keyword = "foo" keyword = "bar" type = "int" redef keyword = "myfoo"
is equivalent to the following one
type = "int" keyword = "myfoo"
If this is not what you want, you can use subst
,
which is similar to redef
apart from that it
replaces the previous first definition of that language
element in the exact point of that first definition
(all other possible definitions are simply erased).
That is to say that the following code
keyword = "foo" keyword = "bar" type = "int" subst keyword = "myfoo"
is equivalent to the following one
keyword = "myfoo" type = "int"
It is up to you to decide which one fits best your needs.
We use this feature to define javascript
in terms
of java
:
include "java.lang" subst keyword = "abstract|break|case|catch|class|if..."
Here using redef
would have led to the unwanted behavior that
if (exp)
would have been highlighted as a function call, since
the function element definition would have come first (and then
matched first) than the redefinition of if
as a keyword.
Another example is the language definition for C# by reusing the one
for C/C++, Highlighting C/C++ and C#.
Although we refer to Boost documentation for such syntax12, we want to provide here some explanations of some forms of regular expressions that might be unknown but that are pretty useful in language definitions.
Typically, when you need to group sub-expressions with parenthesis, but
you don't want the parenthesis to spit out another marked
sub-expression, you can use a non-marking parenthesis
(?:expression)
. This is not necessary in the language definition
syntax: even though you use standard parenthesis, source-highlight will
transform it into a non-marking parenthesis.
A useful regular expression form is the Forward Lookahead Asserts that come in two forms, one for positive forward lookahead asserts, and one for negative lookahead asserts:
(?=abc)
(?!abc)
For instance, in the definition of a function we use the following regular expression:
([[:alpha:]]|_)[[:word:]]*[[:blank:]]*(?=\()
Thus after the name of a function we test, with the regular expression
(?=\()
whether an open parenthesis (
can be matched. If
it can be matched, however, we leave that part in the input (so that the
parenthesis will not be formatted the same way of a function name).
Please, be careful when using such regular expression forms: since part of the input is not actually removed you may end up always scanning the same input part (thus looping) if you do not write the regular expressions well. For instance, consider this language definition
state foo = '(?=foo)' begin foo = '(?=foo)' end
and the following input file:
hello foo bar
As soon as we match the word foo
we leave it in the input and we
enter a state where we try to match the word foo
still leaving it
in the input. As you might have guess this will make source-highlight
loop forever. Probably one might have wanted to write this
language definition:
state foo = '(?=foo)' begin foo = 'foo' end
but a cut-and-paste error had its way ;-)
You can also use Lookbehind Asserts:
(?<=pattern)
(?<!pattern)
In order for language definitions to be really useful they must be
used in proper combination with formatting styles (see Output format style). However, these different files might not be developed
by the same person, or simply some one may want to customize one of
these. In order to define good output formatting style files you should
be aware of each language element defined by a language definition file.
Instead of having to look inside the language definition file itself
(and recursively in each included file) you can use the command line
option --show-lang-elements
13, that
simply prints to the standard output all the language elements that
can be highlighted with a specific language definition file.
For instance, for cpp.lang
you get:
cbracket comment function keyword number preproc specialchar string symbol todo type url
while for log.lang
you get:
cbracket comment date function ip normal number port string symbol time twonumbers webmethod
By mixing all these features you can unleash your imagination and define
highlighting for complex source languages such as Flex and Bison by
writing few lines of code and re-use existing ones. For instance, Flex
and Bison have their own syntax and lets you write C/C++ code in
specific parts of the source language, e.g., the code between the
outmost brackets, in the following example, is C++ code, and should be
highlighted following C++ language definitions (apart from variables
that are prefixed with $
):
globaltags : options { if (...) { setTags( $1 ); } }
This is easy to do (taken from flex.lang):
state cbracket delim "{" "}" multiline nested begin variable = '\$.' include "cpp.lang" end
Notice that, since we used nested
we can be sure
that the C++ language definitions are not considered
anymore when we matched the last closing }
.
When writing a language definition file, it is quite useful to be able
to debug it (by using complex regular expressions one may experience
unwanted behaviors). Since version 2.1 the command line option
--debug-lang
is available. When using this option, some
additional information are printed to the standard output.
Since version 2.5 this option also accepts the a sub specification (see
Invoking source-highlight). When using dump
(the default)
all the additional information explained below will be dumped without
interaction with the user. When using interactive
, for each
formatted string the program will stop waiting for a command from the
user. In this very primordial version of interactive debug, the user
will only have to press ENTER
to make the program continue until
the next formatted string. This way, the programmer will have the
chance to step the highlighting of each part of the input file.
Moreover, when debugging is enabled, no buffering will be performed by
the program, thus each formatted element will be immediately available
in the output. For instance, you can use the command tail -f
to see the modifications on the output file on-the-fly.
When using this command line option the additional information produced has the following format:
<.lang filename>:<line number>: <matched subexpression> formatting: <source file string to be formatted> entering: <next state's regular expression> exiting: exitingall:
The lines starting with entering
, exiting
and
exitingall
are related to entering a new state/environment and
exiting one and all states/environments. The first line shows a link to
the .lang definition file and the line number, i.e., and the
sub-expression that matched and the line starting with formatting
shows the source file string that matched with that expression. If a
line starting with formatting
is not preceded by a line with the
link to the sub-expression, it means that no particular regular
expression has matched, and thus the style normal
will be used to
format that string.
Consider the following (simplified) Java source file:
01: /* 02: This is to demonstrate --debug-lang 03: http://www.lorenzobettini.it 04: */ 05: 06: package hello; 07: 08: public class Hello { 09: // just some greetings ;-) /* 10: int i = 10; 11: System.out.println("Hello World!"); 12: }
Now you can debug the java.lang file by using the
--debug-lang
command line option. And the output is as follows:
c_comment.lang:15: (/\*) formatting: "/*" as comment entering: (\*/)|(/\*)|... formatting: "" as comment formatting: " This is to demonstrate --debug-lang" as comment formatting: " " as comment url.lang:2: ((?:(?:[[:word:]]+://(?:[[:word:]]+[\./\-_]?)+))) formatting: "http://www.lorenzobettini.it" as url formatting: "" as comment c_comment.lang:15: (\*/) formatting: "*/" as comment exiting 1 level(s): (\<(?:import|package)\>)|(//)|... formatting: "" as normal formatting: "" as normal java.lang:1: (\<(?:import|package)\>) formatting: "package" as preproc formatting: " hello" as normal symbols.lang:1: ((?:~|!|%|\^|\*|\(|\)... formatting: ";" as symbol ... omissis ... c_comment.lang:2: (//) formatting: "//" as comment entering: (\z) formatting: " just some greetings ;-) /*" as comment c_comment.lang:2: (\z) formatting: "" as comment exiting 1 level(s): (\<(?:import|package)\>)|(//)|...
This should provide enough information to understand how the regular
expressions are used and how the states/environments are entered and
exited. Please notice that the sub-expressions that are shown may
differ from the original ones specified in the .lang file. This
is due to the preprocessing that is performed by Source-highlight.
Moreover, some sub-expressions are not defined at all in the
.lang file: for instance, this is the case for line wide
definitions, i.e., those that are defined with the keyword start
,
Line wide definitions. The last lines above, showing
entering: (\z)
, mean that we wait to reach the end of a line.
Another useful feature in debugging is the option --show-regex
that shows, on the standard output, the regular expression automaton
that source-highlight creates.
For instance, consider this language definition (comment-show.lang):
vardef TODO = '(TODO|FIXME)([:]?)' environment comment delim "/**" "*/" multiline nested begin type = '@[[:alpha:]]+' todo = $TODO end string delim "<" ">" string2 delim "<<" ">>" multiline
If you now execute the following command:
source-highlight --show-regex=comment-show.lang
you will get, on the standard output, the following output:
STATE 1 0: normal (exit level: 0, exit_all: 0, next: none) 1: comment (/\*\*) (exit level: 0, exit_all: 0, next: 2) STATE 2 0: comment (exit level: 0, exit_all: 0, next: none) 1: comment (\*/) (exit level: 1, exit_all: 0, next: none) 2: comment (/\*\*) (exit level: 0, exit_all: 0, next: 2) 3: type ((?:@[[:alpha:]]+)) (exit level: 0, exit_all: 0, next: none) 4: todo ((?:(?:TODO|FIXME)(?:[:]?))) (exit level: 0, exit_all: 0, next: none) 2: string (<(?:[^<>])*>) (exit level: 0, exit_all: 0, next: none) 3: string2 (<<) (exit level: 0, exit_all: 0, next: 3) STATE 3 0: string2 (exit level: 0, exit_all: 0, next: none) 1: string2 (>>) (exit level: 1, exit_all: 0, next: none)
This shows the states of the regular expression automaton that source-highlight creates and will use to format an input source.
Each state is associated a unique number in order to identify them.
Then for each state it shows the regular expressions associated to each
element. The first element (the one numbered with 0) of each state is
always the default style for that state, i.e., the style applied if no
regular expression is matched (in fact it does not have an associated
regular expression). For instance, in the initial state the default
style is normal. Then, we can see that if we match a /**
(it is
shown as a string with escaped special characters, /\*\*
) we
enter a new state, in this case the state 2 (next: 2
). This
corresponds to the delimited element defining a new environment. The
fact that it is actually and environment and not a state14 can be seen by the fact that the default
style is the same of the environment itself. If we match a */
,
i.e., the end of the delimited element, we exit one level (exit
level: 1
) meaning that we go back to state 1. Since the delimited
element is defined as nested, we can notice that in the state 2 we have
that if we match /**
we simply enter a new instance of state 2
itself.
The string
and string2
show the difference implied by the
multiline
option: since source-highlight handles a line of input
separately, the first delimited definition can be handled with a single
regular expression while the multiline version cannot.
Now we provide some examples of language definitions. In the previous sections we have already provided some code snippets, while here we provide complete examples of language definitions that are included in the source-highlight distribution itself.
In particular we will first show the language definition for the
language definition syntax itself (file langdef.lang). This will
be used to highlight the examples of language definitions that we will
show in this section (the highlighting will not be visible if you are
viewing this manual with the info
command). Of course, this
example is highlighted itself.
# this is the language definition for the # language definition syntax itself comment start "#" preproc = "include" string delim "\"" "\"" escape "\\" multiline string delim "'" "'" escape "\\" multiline keyword = "state|environment|begin|end|delim|escape|start", "multiline|nested|vardef|exitall|exit", "redef|subst|nonsensitive" symbol = "=|+|," vardef ID = '[[:word:]]+' variable = '\$' + $ID variable = $ID
The style that is used to highlight these examples in Texinfo is texinfo.style that is shown in Output format style. The language definition for the style syntax (file style.lang) is even simpler:
# this is the language definition for the # style definition syntax comment start "//" string delim "\"" "\"" escape "\\" keyword = "purple|orange|brightorange|brightgreen|darkgreen", "green|darkred|red|brown|pink|yellow|cyan", "black|teal|gray|darkblue|blue", "normal|linenum", "noref|nf|f|u|i|b" symbol = ",|;" variable = '[[:word:]]+'
Notice that this definition is pretty simple since the language definition syntax is simple. In the next examples we will see how to use more complex features to highlight more complex language syntaxes.
This is the language definition for C/C++, included in the file cpp.lang:
# definitions for C/C++ include "c_comment.lang" state preproc start '^[[:blank:]]*#(?:[[:blank:]]*include)' begin string delim "<" ">" string delim "\"" "\"" escape "\\" include "c_comment.lang" end preproc = '^[[:blank:]]*#([[:blank:]]*[[:word:]]*)' include "number.lang" include "c_string.lang" keyword = "__asm|__cdecl|__declspec|__export|__far16", "__fastcall|__fortran|__import", "__pascal|__rtti|__stdcall|_asm|_cdecl", "__except|_export|_far16|_fastcall", "__finally|_fortran|_import|_pascal|_stdcall|__thread|__try|asm|auto", "break|case|catch|cdecl|class|const|const_cast|continue|default|delete", "do|dynamic_cast|else|enum|explicit|extern|false|for|friend|goto", "if|inline|mutable|namespace|new|operator|pascal|private|protected", "public|register|reinterpret_cast|return|sizeof|static|static_cast", "struct|switch|template|this|throw|true", "try|typedef|typeid|typename|union", "using|virtual|volatile|while" type = "bool|char|double|float|int|long", "short|signed|unsigned|void|wchar_t" include "symbols.lang" cbracket = "{|}" include "function.lang"
Notice that this makes use of lots of include
s since these parts
are reused in other language definitions (e.g., Java has lots of parts
that are in common with C/C++ so we wrote these parts in separate
files). In particular the comments definitions:
# c_comment.lang vardef TODO = '(TODO|FIXME)([:]?)' # comments with documentation tags environment comment start "///" begin include "url.lang" include "html.lang" type = '@[[:alpha:]]+' todo = $TODO end comment start "//" # comments with documentation tags environment comment delim "/**" "*/" multiline nested begin include "url.lang" include "html.lang" type = '@[[:alpha:]]+' todo = $TODO end # standard comments environment comment delim "/*" "*/" multiline nested begin include "url.lang" todo = $TODO end
Here we have the definitions for line-wide comments (//
) and
for multi line comments where we highlight also URL addresses and
e-mail addresses (defined in the file url.lang not shown here).
Moreover, for comments that are used in automatic documentation
generation tools (such as Doxygen or Javadoc), i.e., those that start
with /**
or ///
) we also highlight the complete HTML
syntax (defined in the file html.lang not shown here).
Going back to cpp.lang we see that for preprocessor directives
#include
we use a state definition since in this case the file
included with the <file>
syntax must be formatted as strings (and
only in this context the <>
must be considered as strings,
anywhere else they are operators). Since a state erases definitions
defined outside the state we must include c_comment.lang again in
order to highlight comments also in this context15.
Then we have a definition of preproc
that catches all the
other preprocessor directives.
The included file number.lang defines the regular expression that catches number constants (not shown here), then we include the file c_string.lang that define strings (again shared by Java):
vardef SPECIALCHAR = '\\.' environment string delim "\"" "\"" begin specialchar = $SPECIALCHAR end environment string delim "'" "'" begin specialchar = $SPECIALCHAR end
inside a string we want to highlight in a different way the special
characters (such as, e.g., \n
, \t
, etc.) and in general
escaped characters, matched by the regular expression `\\.
'.
The included file symbols.lang defines all the symbols (shared also by other languages):
symbol = "~","!","%","^","*","(",")","-","+","=","[", "]","\\",":",";",",",".","/","?","&","<",">","\|"
This has nothing interesting but the fact that it shows that
the character \
and |
have to be escaped.
The included file function.lang defines the regular expression to match a function definition or invocation:
vardef FUNCTION = '([[:alpha:]]|_)[[:word:]]*[[:blank:]]*(?=\()' function = $FUNCTION
that shows an example of forward lookahead assert for the opening parenthesis (see Notes on regular expressions). As noted in File inclusion, it is crucial that this file is included after the keyword definition.
Now that we wrote the language definition for C/C++, writing the one
for C# is straightforward, since we only need to add the keyword
using
as a preprocessor element, and redefine (or better,
“substitute”, Redefinitions and Substitutions) the keywords
and types:
# definitions for C-sharp # by S. HEMMI, updated by L. Bettini. preproc = "using" number = '\<[+-]?((0x[[:xdigit:]]+)|(([[:digit:]]*\.)?[[:digit:]]+([eE][+-]?[[:digit:]]+)?))([FfDdMmUulL]+)?\>' include "cpp.lang" subst keyword = "abstract|event|new|struct ", "as|explicit|null|switch", "base|extern|this", "false|operator|throw", "break|finally|out|true", "fixed|override|try", "case|params|typeof", "catch|for|private", "foreach|protected", "checked|goto|public|unchecked", "class|if|readonly|unsafe", "const|implicit|ref", "continue|in|return", "virtual", "default|interface|sealed|volatile", "delegate|internal", "do|is|sizeof|while", "lock|stackalloc", "else|static", "enum|namespace", "get|partial|set", "value|where|yield" subst type = "bool|byte|sbyte|char|decimal|double", "float|int|uint|long|ulong|object", "short|ushort|string|void"
Now we want to highlight files that are generated by diff
(typically used to create patches). This program can generate outputs
in three different formats (at least at best of my knowledge).
With the option -u|--unified
the differences among files
are shown in the same context, for instance (the examples of the
diff files shown here are manually modified so that they can
fit in the page width):
diff -ruP source-highlight-2.1.1/source-highlight.spec ... --- source-highlight-2.1.1/source-highlight.spec ... +++ source-highlight-2.1.2/source-highlight.spec ... @@ -6,8 +6,8 @@ Summary: syntax highlighting for source documents Name: source-highlight -Version: 2.1.1 -Release: 2.1.1 +Version: 2.1.2 +Release: 2.1.2 License: GPL Group: Utilities/Console Source: ftp://ftp.gnu.org/gnu/source-highlight/%{name}-%{version}.tar.gz
With the option -c--context
the differences are shown into
two different parts:
diff -rc2P source-highlight-2.1.1/source-highlight.spec ... *** source-highlight-2.1.1/source-highlight.spec ... --- source-highlight-2.1.2/source-highlight.spec ... *************** *** 7,12 **** Summary: syntax highlighting for source documents Name: source-highlight ! Version: 2.1.1 ! Release: 2.1.1 License: GPL Group: Utilities/Console --- 7,12 ---- Summary: syntax highlighting for source documents Name: source-highlight ! Version: 2.1.2 ! Release: 2.1.2 License: GPL Group: Utilities/Console diff -rc2P source-highlight-2.1.1/src/latex.outlang ... *** source-highlight-2.1.1/src/latex.outlang ... --- source-highlight-2.1.2/src/latex.outlang ... *************** *** 35,37 **** --- 35,38 ---- "--" "-\\/-" "---" "-\\/-\\/-" + "\"" "\"{}" # avoids problems with some inputenc end
Without options it generates only the essential difference information without any addition context lines:
diff -rP source-highlight-2.1.1/source-highlight.spec ... 9,10c9,10 < Version: 2.1.1 < Release: 2.1.1 --- > Version: 2.1.2 > Release: 2.1.2
Summarizing, we would like to be able to handle all these three
different syntaxes; notice that the first format and the second format
have something conflicting: the first one uses the ---
to
indicate the new version of a file while the second format uses it to
indicate the old version of a file. Since we want to highlight
differently the old parts and the new parts (this is not visible in the
Texinfo highlighting due to the lack of enhanced formatting features,
but it is visible for instance in HTML output where we use two
different colors), this behavior adds some difficulties. Of course, we
could define three different language definitions, one for each diff
output format. However, we prefer to handle them all in the same
file!
This is the language definition for diff files:
# language definition for files created with 'diff' # diff created with -u option state oldfile = '(?=^[-]{3})' begin oldfile start '^[-]{3}' oldfile start '^[-]' newfile start '^[+]' difflines start '^@@' end # diff created with -c option state oldfile = '(?=^[*]{3})' begin environment oldfile = '^[*]{3}[[:blank:]]+[[:digit:]]' begin normal start '^[[:space:]]' newfile = '(?=^[-]{3})' exit end oldfile start '^[*]{3}' environment newfile = '^[-]{3}[[:blank:]]+[[:digit:]]' begin normal start '^[[:space:]]' newfile = '(?=^[*]{3})' exit normal start '^diff' exit end newfile start '^[-]{3}' end # otherwise, created without options state difflines = '(?=^[[:digit:]])' begin difflines start '^[[:digit:]]' oldfile start '^[<]' newfile start '^[>]' end
Since we can safely assume that when we process a diff file it contains
only information created with the same diff command line switch, we
define three different states that correspond to the three diff output
formats. Notice that these states are entered with a simple definition;
as noted in State/Environment Definitions, this means that no
automatic exit means are provided, and since no explicit exit condition
is specified, this means that once one of this state is entered it will
never be exited. This is consistent with our goal. Of course, the
expression that makes us enter a state must be defined correctly, and in
particular we first search for an initial ---
sequence since this
is used as the first difference specification by the -u|--unified
option, so this is a distinguishing feature to be used to
infer which diff format file we are processing.
Another interesting thing, is that we use the forward lookahead assert for the opening parenthesis (see Notes on regular expressions), since we only want to see which file format we are processing. Once we entered the right state we can define the regular expressions for the elements of the specific diff file format.
For the files created with the option -c|--context
we define two
inner environments, one for the new file part and one for the old file
part (these are delimited by a ---
or ***
and line number
information). Notice that these are environments, so anything that is
not matched by any expression is formatted according to the style of the
element that defines the environment. Thus, we provide an expression
for text that must be formatted as normal. For diff files this
corresponds to a line that start with a space or with diff
(take
a look at the examples above). In particular the latter case can take
place only during the new file part. In both environments we must
define the exit conditions. In both cases these correspond to the
beginning of the complementary part; also in this case we use forward
lookahead assertions, since we use it only to exit the environment. The
outer definitions for oldfile
and newfile
are used to
match the lines with source file information information.
The third state, corresponding to the normal diff output format, should be straightforward by now.
Source-highlight, by means of regular expressions can only perform lexical analysis of the input source. In particular, it is based on the assumption that the input source is syntactically correct with respect to the input language. However, by using the language definition syntax and by writing the right regular expression it is possible to simulate some sort of semantic analysis of the input source.
For instance, consider the following C (or C++) source file:
// test special #if 0 treatment int main() { #if 0 // equivalent to a comment int i = 10; printf("this should never be executed\n"); return 1; #else printf("Hello world!\n"); return 0; #endif printf("never reach here!\n"); }
It is easy to verify that the code between #if 0
and
#else
will be never executed (indeed it will not even
be compiled). Thus, we might want to format it as a comment.
We then write another language definition file, based on the file cpp.lang:
environment comment start '^[[:blank:]]*#if[[:blank:]]+0' begin comment start '^[[:blank:]]*#(else|endif)' exit end include "cpp.lang"
We intentionally included an error in this first version:
we used the start
element to start the environment,
but such element has the scope of a single line, thus,
it does not have the desired behavior:
// test special #if 0 treatment int main() { #if 0 // equivalent to a comment int i = 10; printf("this should never be executed\n"); return 1; #else printf("Hello world!\n"); return 0; #endif printf("never reach here!\n"); }
A better solution is the following one:
environment comment = '^[[:blank:]]*#[[:blank:]]*if[[:blank:]]+0' begin comment start '^[[:blank:]]*#[[:blank:]]*(else|endif)' exit end include "cpp.lang"
here we enter the comment
environment by not using a delimited
element, but simply the regular expression to match #ifdef 0
.
Then we exit the environment either when we match an #else
or a
#endif
. This seems to work:
// test special #if 0 treatment int main() { #if 0 // equivalent to a comment int i = 10; printf("this should never be executed\n"); return 1; #else printf("Hello world!\n"); return 0; #endif printf("never reach here!\n"); }
However, it does not work if we consider nested #if...#else
; for
instance consider the following code, formatted with the previous
language definition:
// test special #if 0 treatment int main() { #if 0 // equivalent to a comment int i = 10; printf("this should never be executed\n"); # ifdef FOO printf("foo\n"); # ifndef BAR printf("no bar\n"); # else # endif # else printf("no foo\n"); # endif // FOO return 1; #else printf("Hello world!\n"); return 0; #endif printf("never reach here!\n"); }
The problem is that the previous language definition does not consider
nested #if
and thus, the first time it matches a #else
or
an #endif
it exits the comment
environment.
We must then take into account possible nested occurrences. This can be
done by using a delimited element with the nested
option
(Delimited definitions):
# treat the preprocess statement # #if 0 # ... # #else # as a comment environment comment = '^[[:blank:]]*#[[:blank:]]*if[[:blank:]]+0' begin comment start '^[[:blank:]]*#[[:blank:]]*else' exit comment delim '^[[:blank:]]*#[[:blank:]]*if' '^[[:blank:]]*#[[:blank:]]*endif' multiline nested end include "cpp.lang"
This time the right block of code is correctly formatted as a comment:
// test special #if 0 treatment int main() { #if 0 // equivalent to a comment int i = 10; printf("this should never be executed\n"); # ifdef FOO printf("foo\n"); # ifndef BAR printf("no bar\n"); # else # endif # else printf("no foo\n"); # endif // FOO return 1; #else printf("Hello world!\n"); return 0; #endif printf("never reach here!\n"); }
Notice that it is crucial to exit the environment even when we match an
#else
(not only an #endif
, since, this way, we can match
again another #ifdef 0
; consider, for instance, the following
code:
// test special #if 0 treatment int main() { #if 0 // equivalent to a comment int i = 10; printf("this should never be executed\n"); return 1; #else printf("Hello world!\n"); # if 0 // another one return 1; # else return 0; # endif #endif printf("never reach here!\n"); }
Since version 2.1 source-highlight uses a specific syntax to specify output formats (e.g., how to format in HTML, LaTeX, etc.). Before version 2.1, in order to add a new output format, many C++ classes had to be written. This had the drawback that a new output format could not be added “dynamically”: you had to recompile the whole source-highlight program.
Instead, now, an output format is specified in a file, loaded dynamically, through a (hopefully) simple syntax. Then, these definitions are used internally to create, on-the-fly, text formatters.
Here, we see such syntax in details, by relying on many examples. This allows a user to easily modify an existing output format definition and create a new one. These files have, typically, extension .outlang.
Each definition basically associates a text style (such as, e.g., bold,
italics, colors, etc.) to the representation of that style into the
output format (such as, e.g., <b>$text</b>
in HTML). The
representation is given in "
and you can use the classic escape
character \
to use the "
inside the definition. If you
want to specify the ASCII code for a character you can do so by
specifying the numeric code in hexadecimal notation preceded by
\x
, for an example, see Style template.
If no definition is given for a specific style, e.g., bold, then when that style is requested during formatting, the text will be formatted as it is, i.e., the style without the definition is simply ignored.
Comments can be given by using #
; the rest of the line is
considered as a comment.
Files can be included in the same way as for language definitions, File inclusion.
In any case, if a definition for a style is given more than once, the last definition replaces all the others.
With the line:
extension "<file extension>"
you define the default file extension (without the .
) used to
generate files formatted according to this output format. This is used
when no output file name is specified; if the file extension is not
included in the .outlang
is not defined, and no output file name
is specified, an error will occur.
For instance, this is used in html_common.outlang:
extension "html"
These are the text styles that one can define:
bold italics underline notfixed fixed
These, of course, correspond to the ones used to specify the output format style, Output format style.
These definitions, for instance, are from the HTML format definition:
bold "<b>$text</b>" italics "<i>$text</i>" underline "<u>$text</u>"
Inside a definition you use the special variable $text
to specify
where the actual text to be formatted has to be inserted. For instance,
the definition of bold
above says that if you need to format the
keyword class
in bold in HTML, the following text will be
generated: <b>class</b>
. This variable is used also when mixing
more than one styles recursively, in particular if you want to format in
bold and italics (i.e, first bold and then italics, or, in other words,
the sequence i, b
is used in the the output format style file, see
Output format style), then first the text
class
is substituted for $text
into <b>$text</b>
and then the text <b>class</b>
will be substituted for
$text
into <i>$text</i>
, thus obtaining
<i><b>class</b></i>
.
The definition for using colors during formatting requires the definition for the color style:
color "..."
For instance, for HTML we have:
color "<font color=\"$style\">$text</font>"
Apart from the variable $text
that we already saw, we
have also the variable $style
, that will be replaced
with the actual color.
Source-highlight recognizes a number of color constants, see Output format style.
You then must associate a color constant to the color definition in the
output format, through the colormap
definition:
colormap "color constant" "color representation" "color constant" "color representation" ... default "default color representation" end
The default
row (notice the absence of "
) defines the
color to be used in case a color constant is used during formatting, but
it is not defined in the output format.
For instance, for HTML we have:
colormap "green" "#33CC00" "red" "#FF0000" "darkred" "#990000" "blue" "#0000FF" "brown" "#9A1900" "pink" "#CC33CC" "yellow" "#FFCC00" "cyan" "#66FFFF" "purple" "#993399" "orange" "#FF6600" "brightorange" "#FF9900" "brightgreen" "#33FF33" "darkgreen" "#009900" "black" "#000000" "teal" "#008080" "gray" "#808080" "darkblue" "#000080" default "#000000" end
If your output format does not handle colors you can simply avoid the
definitions of color
and colormap
and Source-highlight
will simply ignore colors.
The color is applied after applying the other styles, e.g., bold, italics, etc.
Thus, by continuing the example of the previous section, suppose you defined the following output style for keywords:
keyword blue i, b;
then the class
text will be replaced to $text
variable and
the value #0000FF
to $style
inside the color definition
<font color="$style">$text</font>
obtaining <font
color="#0000FF">class</font>
which will then be replaced to
$text
in <b>$text</b>
and so on for italics, finally
obtaining
<i><b><font color="#0000FF">class</font></b></i>
.
When using the command line option --line-number-ref
(Invoking source-highlight) an anchor is generated in the output
file for each line numbering. The style of the anchor is defined by the
definition anchor
. If this is not defined, the option
--line-number-ref
has no effect. The $linenum
variable will
be replaced with the line number, and the $text
variable
with the actual text.
For instance, for HTML we have
anchor "<a name=\"$linenum\">$text</a>"
Since version 2.2 source-highlight can also generate references to
several elements (e.g., variables, class definitions, etc.),
Generating References. Also in this case the definition
anchor
is used; furthermore, the definition of reference
is required. In the definition of anchor
and reference
,
apart from the variable $linenum
, we also have the variables
$infile
(the name of the original input file) and
$infilename
(the name of the original input file without the
path) and in the definition of reference
we also have the
variable $outfile
(the name of the file where the anchor is).
One can decide how to define an anchor and a reference by using these
two variables. For instance, for HTML we have
reference "<a href=\"$outfile#$linenum\">$text</a>"
Notice, that in this case we use the $outfile
since we actually
generate a link to another (or possibly the same) output file.
On the contrary, for LaTeX, since we do not generate a “clickable”
reference, we refer to the original input file (we use both
$infilename
and $linenum
in both definitions of anchor
and reference
):
anchor "\label{$infilename:$linenum}$text" reference "{\hfill $text $\rightarrow$ $infile:$linenum, \ page~\pageref{$infilename:$linenum}}"
In particular, we use $infilename
for generating the
\label
and not $infile
because the path symbol would
“disturb” LaTeX (while we use the complete file path in the textual
information of the reference).
This will generate a right aligned reference. Notice that it is assumed
that when generating references in LaTeX one uses
--gen-references=postline
or --gen-references=postdoc
and
not --gen-references=inline
(Generating References), since
it makes no sense to generate an inline reference (or at least I would
not know how to generate a nice looking one :-).
Furthermore, for Texinfo:
anchor "@anchor{$infilename:$linenum}$text" reference "@flushright @xref{$infilename:$linenum,$text,$text $infile:$linenum}. @end flushright"
Notice that using both $infilename
(and not $infile
for
the same reasons) and $linenum
also in the definition of
anchor
somehow ensures that there are no duplicate anchors; this
is done for LaTeX and Texinfo but not for HTML because it is assumed
that the generated .tex and .texinfo file is included
directly in a master file, as it is done in this manual (while, for
instance, it is assumed that a separate HTML file is generated for each
source and kept separate). If this is not your case you can change the
definitions of anchor
and reference
as you see fit. Some
examples of outputs with references in Texinfo are shown in
Examples.
Indeed, one can use three more definitions for reference
that
corresponds to the three arguments that can be passed to
--gen-references
command line option (Generating References): inline_reference
, postline_reference
and
postdoc_reference
. If one of this not defined, then the same
definition of reference
is used. Having the possibility of
specifying different definitions is useful for instance in the case of
HTML: the same style for an inline reference is pretty ugly when used
also for a postline or postdoc reference:
postline_reference "<a href=\"$outfile#$linenum\">$text -> $infile:$linenum</a>" postdoc_reference "<a href=\"$outfile#$linenum\">$text -> $infile:$linenum</a>" reference "<a href=\"$outfile#$linenum\">$text</a>"
If the output format you are defining does not have a specific style
for bold, italics, ... and for colors you can simply use the definition
onestyle
, where you can use both $style
and $text
.
This will be used for any style (indeed any other definition such as
bold, italics, color will be ignored). Indeed, in this case, it is
assumed that the style of each source element is defined in a file with
its own syntax, i.e., not with a syntax defined by Source-highlight.
(This is the case, for instance, of HTML using CSS style sheets.)
Moreover, since the output format style is not used, during formatting
the variable $style
will be replaced with the name of the element
to highlight (e.g., keyword
, comment
, etc.).
For instance, for HTML CSS, we simply have:
onestyle "<span class=\"$style\">$text</span>"
In fact, HTML CSS relies on style definitions provided in a separate
file (the .css file indeed). Thus, when formatting a
keyword
, e.g., abstract
, we will obtain:
<span class="keyword">abstract</span>
Of course, the style for keyword
must be defined in the
.css file.
Some output formats are based on a unique template that where the other styles are composed; during composition the styles can be separated with a specific separator:
styletemplate "..." styleseparator "..."
This is used, for instance, for the ANSI color escape sequence output format (esc.outlang):
styletemplate "\x1b[$stylem$text\x1b[m" styleseparator ";" bold "01$style" underline "04$style" italics "$style" color "$style"
Notice that, since more than one style can be mixed into the style
template, bold
, underline
, ... explicitly use the variable
$style
.
This feature allows you to generate a string as the prefix of each generated line that corresponds to an input line (i.e., this prefix is not generated for other generated output elements, e.g., the lines in the header, footer, etc.).
We use this feature in the LaTeX output (LaTeX output):
lineprefix "\mbox{}"
This way each line in the LaTeX output is prefixed with
\mbox{}
16.
Another interesting example that uses lineprefix
is the javadoc
output, see Generating HTML output.
Some character sequences that are in the source file may have a special meaning in an output format, so they need some preprocessing (e.g., escaping them). You can specify the translation table with:
translations "original sequence" "transformed sequence" 'regex' "transformed sequence" ... end
The difference between "original sequence"
and
'regex'
17 is that with the former
you specify a character sequence that will be matched literally, apart
from special characters such as \
(which, if needed to be
inserted, must be escaped), \n
(new line) and \t
(tab
character). Instead, with the latter, you can specify a regular
expression (this is basically the same difference between "
and
'
in language definitions, see Simple definitions).
For instance, for HTML, we have the following translation table:
translations "&" "&" "<" "<" ">" ">" end
For LaTeX, the translation table is a little bit bigger; here we
show only a little part, that shows how to escape special characters
(such as \
), to translate a new line character and tab
character:
translations "<" "$<$" ">" "$>$" "&" "\\&" "\\" "\\textbackslash{}" "\n" " \\\\\n" " " "\\ " "\t" "\\ \\ \\ \\ \\ \\ \\ \\ " end
Notice that, since a new character must be translated in LaTeX with
\\
, we have to escape two \
(i.e., \\\\
) and then
we want to actually insert a new line in the output file \n
.
For HTML with not fixed font by default, html_notfixed.outlang
(see HTML output), we need two translate two space sequence
(i.e., two adjacent spaces, since in HTML more adjacent spaces are
rendered as only one space18, while we want them as they are), and we also
need to translate a space starting a new line in the source (thus we
use the regular expression ^
, enclosed in '
); thus we
have:
translations "\n" "<br>\n" " " " " '^ ' " " # a space at the beginning of a line "\t" " " end
You can define the document template, i.e., the beginning and the end of an output file, with
doctemplate "...beginning..." "...end..." end
For instance, for HTML we have
doctemplate "<!-- Generator: $additional --> $header<pre><tt>" "</tt></pre>$footer " end
Notice that in the end part there is an explicit new line.
In the definition of the doctemplate
the following variables can
be used and will be replaced during the output generation:
$title
--title
command line option;
$header
--header
;
$footer
--footer
;
$css
--css
;
$additional
For instance, for an HTML document with css, (file cssdoc.outlang) we have:
doctemplate "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0//EN\" \"http://www.w3.org/TR/REC-html40/strict.dtd\"> <html> <head> <meta http-equiv=\"Content-Type\" content=\"text/html; charset=iso-8859-1\"> <meta name=\"GENERATOR\" content=\"$additional\"> <title>$title</title> <link rel=\"stylesheet\" href=\"$css\" type=\"text/css\"> </head> <body> $header<pre><tt>" "</tt></pre> $footer</body> </html> " end
As a complete example we show the file html_common.outlang which contains the common definitions for the various HTML output formats (html.outlang, htmldoc.outlang, etc.):
extension "html" bold "<b>$text</b>" italics "<i>$text</i>" underline "<u>$text</u>" color "<font color=\"$style\">$text</font>" anchor "<a name=\"$linenum\">$text</a>" postline_reference "<a href=\"$outfile#$linenum\">$text -> $infile:$linenum</a>" postdoc_reference "<a href=\"$outfile#$linenum\">$text -> $infile:$linenum</a>" reference "<a href=\"$outfile#$linenum\">$text</a>" colormap "green" "#33CC00" "red" "#FF0000" "darkred" "#990000" "blue" "#0000FF" "brown" "#9A1900" "pink" "#CC33CC" "yellow" "#FFCC00" "cyan" "#66FFFF" "purple" "#993399" "orange" "#FF6600" "brightorange" "#FF9900" "brightgreen" "#33FF33" "darkgreen" "#009900" "black" "#000000" "teal" "#008080" "gray" "#808080" "darkblue" "#000080" default "#000000" end translations "&" "&" "<" "<" ">" ">" end
Moreover, this file is also used for generating javadoc output:
include "html_common.outlang" doctemplate " * <!-- Generated by Source-highlight --> * <pre><tt> " " * </tt></pre> " end lineprefix " * " translations "*/" "*/" # this avoids the */ to be interpreted as # the end of a comment inside a javadoc comment end
The javadoc output format is useful to format code snippets that have to
be included inside a javadoc comment of another Java
file19. Apart from being formatted nicely in the
generated HTML documentation, this also releases the programmer from
escaping specific characters in the code snippet (i.e., &
,
<
and >
). Notice also that it also avoids the
sequence */
to be interpreted as the closing of the
(javadoc) comment. For instance, if you write this code:
/** * This is an example of usage * * <pre><tt> * System.out.println("*/"); * </tt></pre> */
The resulting Java code contains a syntax error. If you use source-highlight to format the code to insert in a javadoc comment you will avoid these problems.
An example of a javadoc generated HTML page containing a code snippet formatted with source-highlight can be found in the file SimpleClass-doc.html in the documentation directory.
Since version 2.2 Source-highlight also produces references to fields, variables, etc. In order to do this it relies on the program Exuberant Ctags, by Darren Hiebert, available at http://ctags.sourceforge.net. Thus, you must install this program if you want Source-highlight to provide this feature.
The ctags
program generates an index (or “tag”) file for a
variety of language objects found in file(s). This allows these items
to be quickly and easily located by a text editor or other utility (as
in this case for Source-highlight). A “tag” signifies a language
object for which an index entry is available (or, alternatively, the
index entry created for that object)20.
This means that Source-highlight is able to generate references for a
specific source language if and only if ctags
handles such
language. We refer to the command line options of ctags
:
--list-maps
and --list-languages
to find out the
associations of file extensions and supported languages.
Reference generation is enable by using the command line option
--gen-references
(Invoking source-highlight). This option
takes an argument that rules how references will be generated:
inline
postline
postdoc
There is an exception: when an element has more than one reference
(because a variable is defined in many sources or because a method is
overloaded) then if inline
is specified, the generation switches
to postline
for that occurrence.
When --gen-references
is specified, Source-highlight first
invokes ctags
. The use can customize this call by using the
command line option --ctags
(Invoking source-highlight).
In particular, if one does not want ctags
to be invoked by
Source-highlight (e.g., because the tags file has already been
generated) then --ctags
must be passed an empty string,
""
. In this case or when the specified ctags
command line
generates an alternative output tag file (the default generated file is
tags), one can specify the exact tag file with the command line
option --ctags-file
.
Once the tag file is generated, Source-highlight relies on the library
readtags provided by the ctags
distribution, and included
in the Source-highlight sources.
Notice that if a program element is formatted according to a style that
has the option noref
(see Output format style) then this
element is not considered a tag, and no reference is generated. This is
the case, for instance, for a comment
element: each string that
is generated with the comment
style, since this is declared with
the option noref
, it is not considered a tag (see Examples).
Here we provide some examples of sources formatted with
Source-highlight using the -f texinfo
command line option. Please keep in mind that the highlighting
will not be visible in the Info file, but only in the
printed manual and in the HTML output (well, at least line
numbers are visible everywhere :-).
The first example is produced by using the command:
source-highlight -f texinfo -i test.java -o test.java.texinfo -n
and here's the result
01: /* 02: This is a classical Hello program 03: to test source-highlight with Java programs. 04: 05: to have an html translation type 06: 07: source-highlight -s java -f html --input Hello.java --output Hello.html 08: source-highlight -s java -f html < Hello.java > Hello.html 09: 10: or type source-highlight --help for the list of options 11: 12: written by 13: Lorenzo Bettini 14: http://www.lorenzobettini.it 15: http://www.gnu.org/software/src-highlite 16: */ 17: 18: package hello; 19: 20: import java.io.* ; 21: 22: /** 23: * <p> 24: * A simple Hello World class, used to demonstrate some 25: * features of Java source highlighting. 26: * </p> 27: * TODO: nothing, just to show an highlighted TODO or FIXME 28: * 29: * @author Lorenzo Bettini 30: * @version 2.0 31: */ 32: public class Hello { 33: int foo = 1998 ; 34: int hex_foo = 0xCAFEBABE; 35: boolean b = false; 36: Integer i = null ; 37: char c = '\'', d = 'n', e = '\\' ; 38: String xml = "<tag attr=\"value\">ä</tag>", foo2 = "\\" ; 39: 40: public static void main( String args[] ) { 41: // just some greetings ;-) /* 42: System.out.println( "Hello from java2html :-)" ) ; 43: System.out.println( "\tby Lorenzo Bettini" ) ; 44: System.out.println( "\thttp://www.lorenzobettini.it" ) ; 45: if (argc > 0) 46: String param = argc[0]; 47: //System.out.println( "bye bye... :-D" ) ; // see you soon 48: } 49: }
The second example shows the use of --gen-references
functionality. In particular, the following output is generated with
the command:
source-highlight -f texinfo -i test.h -o test_ref.h.texinfo -n \ --gen-references=postline
and here's the result (notice how the comment line containing the string
mysum
does not contain references, since it is a comment
element, and this element has the option noref
in the
texinfo.style, see Output format style. The same holds for
the _TEXTGEN_H
comment in the last comment line).
01: /* 02: ** Copyright (C) 1999, 2000, 2001 Lorenzo Bettini 03: ** 04: ** This program is free software; you can redistribute it and/or modify 05: ** it under the terms of the GNU General Public License as published by 06: ** the Free Software Foundation; either version 2 of the License, or 07: ** (at your option) any later version. 08: ** 09: ** This program is distributed in the hope that it will be useful, 10: ** but WITHOUT ANY WARRANTY; without even the implied warranty of 11: ** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 12: ** GNU General Public License for more details. 13: ** 14: ** You should have received a copy of the GNU General Public License 15: ** along with this program; if not, write to the Free Software 16: ** Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. 17: ** 18: */ 19: 20: // this file also contains the definition of mysum as a #define 21: 22: // textgenerator.h : Text Generator class && 23: 24: #ifndef _TEXTGEN_HSee _TEXTGEN_H.25: #define _TEXTGEN_H 26: 27: #define foo(x) (x + 1) 28: 29: #define mysum myfunbody 30: 31: #include <iostream.h> // for cerr 32: 33: #include "genfun.h" /* for generating functions */ 34: 35: class TextGenerator { 36: public : 37: virtual void generate( const char *s ) const { (*sout) << s ; } 38: virtual void generate( const char *s, int start, int end ) const 39: { 40: for ( int i = start ; i <= end ; ++i ) 41: (*sout) << s[i] ; 42: return a<p->b ? a : 3; 43: } 44: virtual void generateln( const char *s ) const 45: { 46: generate( s ) ;See generate.See generate.47: (*sout) << endl ; 48: } 49: virtual void generateEntire( const char *s ) const 50: { 51: startTextGeneration() ;See startTextGeneration.See startTextGeneration.52: generate(s) ;See generate.See generate.53: endTextGeneration() ;See endTextGeneration.See endTextGeneration.54: } 55: virtual void startTextGeneration() const {} 56: virtual void endTextGeneration() const {} 57: virtual void beginText( const char *s ) const 58: { 59: startTextGeneration() ;See startTextGeneration.See startTextGeneration.60: if ( s ) 61: generate( s ) ;See generate.See generate.62: } 63: virtual void endText( const char *s ) const 64: { 65: if ( s ) 66: generate( s ) ;See generate.See generate.67: endTextGeneration() ;See endTextGeneration.See endTextGeneration.68: } 69: } ; 70: 71: // Decorator 72: class TextDecorator : public TextGenerator {See TextGenerator.73: protected : 74: TextGenerator *decorated ;See TextGenerator.75: 76: public : 77: TextDecorator( TextGenerator *t ) : decorated( t ) {}See TextGenerator.See decorated.78: 79: virtual void startTextGeneration() const 80: { 81: startDecorate() ; 82: if ( decorated )See decorated.83: decorated->startTextGeneration() ;See startTextGeneration.See decorated.See startTextGeneration.84: } 85: virtual void endTextGeneration() const 86: { 87: if ( decorated )See decorated.88: decorated->endTextGeneration() ;See endTextGeneration.See decorated.See endTextGeneration.89: endDecorate() ; 90: mysum;See mysum.91: } 92: 93: // pure virtual functions 94: virtual void startDecorate() const = 0 ; 95: virtual void endDecorate() const = 0 ; 96: } ; 97: 98: #endif // _TEXTGEN_H
If you find a bug in source-highlight, please send electronic mail to
bug-source-highlight at gnu dot org
Include the version number, which you can find by running `source-highlight --version'. Also include in your message the output that the program produced and the output you expected.
If you have other questions, comments or suggestions about source-highlight, contact the author via electronic mail (find the address at http://www.lorenzobettini.it). The author will try to help you out, although he may not have time to fix your problems.
The following mailing lists are available:
help-source-highlight at gnu dot org
for generic discussions about the program and for asking for help about it (open mailing list), http://mail.gnu.org/mailman/listinfo/help-source-highlight
info-source-highlight at gnu dot org
for receiving information about new releases and features (read-only mailing list), http://mail.gnu.org/mailman/listinfo/info-source-highlight.
If you want to subscribe to a mailing list just go to the URL and follow the instructions, or send me an e-mail and I'll subscribe you.
--show-lang-elements
: Listing Language Elements--show-lang-elements
: Output format style[1] Although this might have been achieved with previous version, it is an official supported feature since version 2.5.
[2] Command
lines that are too long are split into multiple indented lines separated
by a \
. Of course these commands are to be given in one line
only, anyway.
[3] Command lines that are too long are
split into multiple indented lines separated by a \
. Of course
these commands are to be given in one line only, anyway.
[4] Before version 2.1, this file was called tags.j2h which used to be a very obscure name. I hope this name convention is a better one :-).
[5] Before version 2.1, this command line
option was called --tags-file
which used to be a very obscure
name. I hope this name convention is a better one :-).
[6] You can see these colors in HTML in the file colors.html.
[7] Notice that, since version 2.2, you must use double quotes.
[8] This is the main difference introduced in version 2.0 with respect the the previous version.
[9] This is the main difference introduced in version 2.1 with respect the the previous version.
[10] As explained before, originally Source-highlight was thought mainly for generating HTML output, this is why the term css is used for style sheets.
[11] At least, to the best of my knowledge :-)
[12] http://www.boost.org/libs/regex/doc/syntax.html
[13] Since version 2.4.
[14] Please notice that this concept of state is different from the concept of “state” of an automaton.
[15] As a future extension we might think of providing a way, in the language definition syntax, to define a state/environment that extends the outer contexts instead of overriding them.
[16] This is a sort of trick to insert spaces at
the beginning of a line without using a tabular environment; without the
leading \mbox{}
these spaces would be ignored. This is the
only way I found to achieve this, if you have suggestions, please let me
know!
[17] Since version 2.4.
[18] Unless they are inside a
<tt>...</tt>
.
[19] Although I haven't tested it, I think this will work also for Doxygen comments.
[20] This description is taken from the ctags man page