The GNU Awk User's Guide

General Introduction

This file documents awk, a program that you can use to select particular records in a file and perform operations upon them.

Copyright © 1989, 1991, 1992, 1993, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003 Free Software Foundation, Inc.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with the Invariant Sections being “GNU General Public License”, the Front-Cover texts being (a) (see below), and with the Back-Cover Texts being (b) (see below). A copy of the license is included in the section entitled “GNU Free Documentation License”.

1. “A GNU Manual”
2. “You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development.”

• Foreword: Some nice words about this Web page.
• Preface: What this Web page is about; brief history and acknowledgments.
• Getting Started: A basic introduction to using awk. How to run an awk program. Command-line syntax.
• Regexp: All about matching things using regular expressions.
• Reading Files: How to read files and manipulate fields.
• Printing: How to print using awk. Describes the print and printf statements. Also describes redirection of output.
• Expressions: Expressions are the basic building blocks of statements.
• Patterns and Actions: Overviews of patterns and actions.
• Arrays: The description and use of arrays. Also includes array-oriented control statements.
• Functions: Built-in and user-defined functions.
• Internationalization: Getting gawk to speak your language.
• Advanced Features: Stuff for advanced users, specific to gawk.
• Invoking Gawk: How to run gawk.
• Library Functions: A Library of awk Functions.
• Sample Programs: Many awk programs with complete explanations.
• Language History: The evolution of the awk language.
• Installation: Installing gawk under various operating systems.
• Notes: Notes about gawk extensions and possible future work.
• Basic Concepts: A very quick intoduction to programming concepts.
• Glossary: An explanation of some unfamiliar terms.
• Copying: Your right to copy and distribute gawk.
• GNU Free Documentation License: The license for this Web page.
• Index: Concept and Variable Index.
• History: The history of gawk and awk.
• Names: What name to use to find awk.
• This Manual: Using this Web page. Includes sample input files that you can use.
• Conventions: Typographical Conventions.
• Manual History: Brief history of the GNU project and this Web page.
• How To Contribute: Helping to save the world.
• Acknowledgments: Acknowledgments.
• Running gawk: How to run gawk programs; includes command-line syntax.
• One-shot: Running a short throwaway awk program.
• Read Terminal: Using no input files (input from terminal instead).
• Long: Putting permanent awk programs in files.
• Executable Scripts: Making self-contained awk programs.
• Comments: Adding documentation to gawk programs.
• Quoting: More discussion of shell quoting issues.
• Sample Data Files: Sample data files for use in the awk programs illustrated in this Web page.
• Very Simple: A very simple example.
• Two Rules: A less simple one-line example using two rules.
• More Complex: A more complex example.
• Statements/Lines: Subdividing or combining statements into lines.
• Other Features: Other Features of awk.
• When: When to use gawk and when to use other things.
• Regexp Usage: How to Use Regular Expressions.
• Escape Sequences: How to write nonprinting characters.
• Regexp Operators: Regular Expression Operators.
• Character Lists: What can go between `[...]'.
• GNU Regexp Operators: Operators specific to GNU software.
• Case-sensitivity: How to do case-insensitive matching.
• Leftmost Longest: How much text matches.
• Computed Regexps: Using Dynamic Regexps.
• Locales: How the locale affects things.
• Records: Controlling how data is split into records.
• Fields: An introduction to fields.
• Nonconstant Fields: Nonconstant Field Numbers.
• Changing Fields: Changing the Contents of a Field.
• Field Separators: The field separator and how to change it.
• Regexp Field Splitting: Using regexps as the field separator.
• Single Character Fields: Making each character a separate field.
• Command Line Field Separator: Setting FS from the command-line.
• Field Splitting Summary: Some final points and a summary table.
• Constant Size: Reading constant width data.
• Multiple Line: Reading multi-line records.
• Getline: Reading files under explicit program control using the getline function.
• Plain Getline: Using getline with no arguments.
• Getline/Variable: Using getline into a variable.
• Getline/File: Using getline from a file.
• Getline/Variable/File: Using getline into a variable from a file.
• Getline/Pipe: Using getline from a pipe.
• Getline/Variable/Pipe: Using getline into a variable from a pipe.
• Getline/Coprocess: Using getline from a coprocess.
• Getline/Variable/Coprocess: Using getline into a variable from a coprocess.
• Getline Notes: Important things to know about getline.
• Getline Summary: Summary of getline Variants.
• Print: The print statement.
• Print Examples: Simple examples of print statements.
• Output Separators: The output separators and how to change them.
• OFMT: Controlling Numeric Output With print.
• Printf: The printf statement.
• Basic Printf: Syntax of the printf statement.
• Control Letters: Format-control letters.
• Format Modifiers: Format-specification modifiers.
• Printf Examples: Several examples.
• Redirection: How to redirect output to multiple files and pipes.
• Special Files: File name interpretation in gawk. gawk allows access to inherited file descriptors.
• Special FD: Special files for I/O.
• Special Process: Special files for process information.
• Special Network: Special files for network communications.
• Special Caveats: Things to watch out for.
• Close Files And Pipes: Closing Input and Output Files and Pipes.
• Constants: String, numeric and regexp constants.
• Scalar Constants: Numeric and string constants.
• Nondecimal-numbers: What are octal and hex numbers.
• Regexp Constants: Regular Expression constants.
• Using Constant Regexps: When and how to use a regexp constant.
• Variables: Variables give names to values for later use.
• Using Variables: Using variables in your programs.
• Assignment Options: Setting variables on the command-line and a summary of command-line syntax. This is an advanced method of input.
• Conversion: The conversion of strings to numbers and vice versa.
• Arithmetic Ops: Arithmetic operations (`+', `-', etc.)
• Concatenation: Concatenating strings.
• Assignment Ops: Changing the value of a variable or a field.
• Increment Ops: Incrementing the numeric value of a variable.
• Truth Values: What is ``true'' and what is ``false''.
• Typing and Comparison: How variables acquire types and how this affects comparison of numbers and strings with `<', etc.
• Boolean Ops: Combining comparison expressions using boolean operators `||' (``or''), `&&' (``and'') and `!' (``not'').
• Conditional Exp: Conditional expressions select between two subexpressions under control of a third subexpression.
• Function Calls: A function call is an expression.
• Precedence: How various operators nest.
• Pattern Overview: What goes into a pattern.
• Regexp Patterns: Using regexps as patterns.
• Expression Patterns: Any expression can be used as a pattern.
• Ranges: Pairs of patterns specify record ranges.
• BEGIN/END: Specifying initialization and cleanup rules.
• Using BEGIN/END: How and why to use BEGIN/END rules.
• I/O And BEGIN/END: I/O issues in BEGIN/END rules.
• Empty: The empty pattern, which matches every record.
• Using Shell Variables: How to use shell variables with awk.
• Action Overview: What goes into an action.
• Statements: Describes the various control statements in detail.
• If Statement: Conditionally execute some awk statements.
• While Statement: Loop until some condition is satisfied.
• Do Statement: Do specified action while looping until some condition is satisfied.
• For Statement: Another looping statement, that provides initialization and increment clauses.
• Switch Statement: Switch/case evaluation for conditional execution of statements based on a value.
• Break Statement: Immediately exit the innermost enclosing loop.
• Continue Statement: Skip to the end of the innermost enclosing loop.
• Next Statement: Stop processing the current input record.
• Nextfile Statement: Stop processing the current file.
• Exit Statement: Stop execution of awk.
• Built-in Variables: Summarizes the built-in variables.
• User-modified: Built-in variables that you change to control awk.
• Auto-set: Built-in variables where awk gives you information.
• ARGC and ARGV: Ways to use ARGC and ARGV.
• Array Intro: Introduction to Arrays
• Reference to Elements: How to examine one element of an array.
• Assigning Elements: How to change an element of an array.
• Array Example: Basic Example of an Array
• Scanning an Array: A variation of the for statement. It loops through the indices of an array's existing elements.
• Delete: The delete statement removes an element from an array.
• Numeric Array Subscripts: How to use numbers as subscripts in awk.
• Uninitialized Subscripts: Using Uninitialized variables as subscripts.
• Multi-dimensional: Emulating multidimensional arrays in awk.
• Multi-scanning: Scanning multidimensional arrays.
• Array Sorting: Sorting array values and indices.
• Built-in: Summarizes the built-in functions.
• Calling Built-in: How to call built-in functions.
• Numeric Functions: Functions that work with numbers, including int, sin and rand.
• String Functions: Functions for string manipulation, such as split, match and sprintf.
• Gory Details: More than you want to know about `\' and `&' with sub, gsub, and gensub.
• I/O Functions: Functions for files and shell commands.
• Time Functions: Functions for dealing with timestamps.
• Bitwise Functions: Functions for bitwise operations.
• I18N Functions: Functions for string translation.
• User-defined: Describes User-defined functions in detail.
• Definition Syntax: How to write definitions and what they mean.
• Function Example: An example function definition and what it does.
• Function Caveats: Things to watch out for.
• Return Statement: Specifying the value a function returns.
• Dynamic Typing: How variable types can change at runtime.
• I18N and L10N: Internationalization and Localization.
• Explaining gettext: How GNU gettext works.
• Programmer i18n: Features for the programmer.
• Translator i18n: Features for the translator.
• String Extraction: Extracting marked strings.
• Printf Ordering: Rearranging printf arguments.
• I18N Portability: awk-level portability issues.
• I18N Example: A simple i18n example.
• Gawk I18N: gawk is also internationalized.
• Nondecimal Data: Allowing nondecimal input data.
• Two-way I/O: Two-way communications with another process.
• TCP/IP Networking: Using gawk for network programming.
• Portal Files: Using gawk with BSD portals.
• Profiling: Profiling your awk programs.
• Command Line: How to run awk.
• Options: Command-line options and their meanings.
• Other Arguments: Input file names and variable assignments.
• AWKPATH Variable: Searching directories for awk programs.
• Obsolete: Obsolete Options and/or features.
• Undocumented: Undocumented Options and Features.
• Known Bugs: Known Bugs in gawk.
• Library Names: How to best name private global variables in library functions.
• General Functions: Functions that are of general use.
• Nextfile Function: Two implementations of a nextfile function.
• Assert Function: A function for assertions in awk programs.
• Round Function: A function for rounding if sprintf does not do it correctly.
• Cliff Random Function: The Cliff Random Number Generator.
• Ordinal Functions: Functions for using characters as numbers and vice versa.
• Join Function: A function to join an array into a string.
• Gettimeofday Function: A function to get formatted times.
• Data File Management: Functions for managing command-line data files.
• Filetrans Function: A function for handling data file transitions.
• Rewind Function: A function for rereading the current file.
• File Checking: Checking that data files are readable.
• Empty Files: Checking for zero-length files.
• Ignoring Assigns: Treating assignments as file names.
• Getopt Function: A function for processing command-line arguments.
• Passwd Functions: Functions for getting user information.
• Group Functions: Functions for getting group information.
• Running Examples: How to run these examples.
• Clones: Clones of common utilities.
• Cut Program: The cut utility.
• Egrep Program: The egrep utility.
• Id Program: The id utility.
• Split Program: The split utility.
• Tee Program: The tee utility.
• Uniq Program: The uniq utility.
• Wc Program: The wc utility.
• Miscellaneous Programs: Some interesting awk programs.
• Dupword Program: Finding duplicated words in a document.
• Alarm Program: An alarm clock.
• Translate Program: A program similar to the tr utility.
• Labels Program: Printing mailing labels.
• Word Sorting: A program to produce a word usage count.
• History Sorting: Eliminating duplicate entries from a history file.
• Extract Program: Pulling out programs from Texinfo source files.
• Simple Sed: A Simple Stream Editor.
• Igawk Program: A wrapper for awk that includes files.
• V7/SVR3.1: The major changes between V7 and System V Release 3.1.
• SVR4: Minor changes between System V Releases 3.1 and 4.
• POSIX: New features from the POSIX standard.
• BTL: New features from the Bell Laboratories version of awk.
• POSIX/GNU: The extensions in gawk not in POSIX awk.
• Contributors: The major contributors to gawk.
• Gawk Distribution: What is in the gawk distribution.
• Getting: How to get the distribution.
• Extracting: How to extract the distribution.
• Distribution contents: What is in the distribution.
• Unix Installation: Installing gawk under various versions of Unix.
• Quick Installation: Compiling gawk under Unix.
• Additional Configuration Options: Other compile-time options.
• Configuration Philosophy: How it's all supposed to work.
• Non-Unix Installation: Installation on Other Operating Systems.
• Amiga Installation: Installing gawk on an Amiga.
• BeOS Installation: Installing gawk on BeOS.
• PC Installation: Installing and Compiling gawk on MS-DOS and OS/2.
• PC Binary Installation: Installing a prepared distribution.
• PC Compiling: Compiling gawk for MS-DOS, Windows32, and OS/2.
• PC Using: Running gawk on MS-DOS, Windows32 and OS/2.
• PC Dynamic: Compiling gawk for dynamic libraries.
• Cygwin: Building and running gawk for Cygwin.
• VMS Installation: Installing gawk on VMS.
• VMS Compilation: How to compile gawk under VMS.
• VMS Installation Details: How to install gawk under VMS.
• VMS Running: How to run gawk under VMS.
• VMS POSIX: Alternate instructions for VMS POSIX.
• Unsupported: Systems whose ports are no longer supported.
• Atari Installation: Installing gawk on the Atari ST.
• Atari Compiling: Compiling gawk on Atari.
• Atari Using: Running gawk on Atari.
• Tandem Installation: Installing gawk on a Tandem.
• Bugs: Reporting Problems and Bugs.
• Other Versions: Other freely available awk implementations.
• Compatibility Mode: How to disable certain gawk extensions.
• Additions: Making Additions To gawk.
• Adding Code: Adding code to the main body of gawk.
• New Ports: Porting gawk to a new operating system.
• Dynamic Extensions: Adding new built-in functions to gawk.
• Internals: A brief look at some gawk internals.
• Sample Library: A example of new functions.
• Internal File Description: What the new functions will do.
• Internal File Ops: The code for internal file operations.
• Using Internal File Ops: How to use an external extension.
• Future Extensions: New features that may be implemented one day.
• Basic High Level: The high level view.
• Basic Data Typing: A very quick intro to data types.
• Floating Point Issues: Stuff to know about floating-point numbers.