Next: Past and Future, Previous: fnid invocation, Up: Top
xtokid accepts the names of files and/or directories on the command line, then extracts and prints a stream of tokens from those files for which it has a valid, enabled scanner. This is useful primarily for debugging new mkid scanners (see Defining scanners).
xtokid extracts tokens from source files, therefore it accepts the `--lang-map', `--include', `--exclude', and `--lang-option' options, as well as the language-specific scanner options, all of which are described in Extraction options. xtokid walks file trees, therefore it handles file and directory names on its command line and the `--prune' option as described in Walker options.
The name `xtokid' indicates that it is the “eXtract TOKens ID utility”.