Introduction to awk and collection of study notes Page 1/3_PHP Tutorial

Copyright © 2004 This article complies with the GPL agreement. Reprinting, modification, and distribution are welcome.

First release date: August 6, 2004


----------------------- -------------------------------------------------- -------

Table of Contents

1. Introduction to awk
2. awk command format and options
2.1. There are two forms of awk syntax
2.2. Command options
3. Modes and operations
3.1. Modes
3.2. Operations
4. awk environment variables
5. awk operators
6. Records and fields
6.1. Records
6.2. Domains
6.3. Domain separators
7. gawk-specific regular expression metacharacters
8. POSIX character set
9. Match operator (~ )
10. Comparison expressions
11. Range templates
12. An example of verifying the validity of the passwd file
13. Several examples
14. awk programming
14.1. Variables
14.2. BEGIN module
14.3. END module
14.4. Redirection and pipeline
14.5. Conditional statement
14.6. Loop
14.7. Array
14.8. Internals of awk Creating functions
15. How-to
1. Introduction to awk
Awk is a programming language used to process text and data under linux/unix. Data can come from standard input, one or more files, or the output of other commands. It supports advanced functions such as user-defined functions and dynamic regular expressions, and is a powerful programming tool under Linux/Unix. It is used from the command line, but more commonly as a script. The way awk processes text and data is that it scans the file line by line, from the first line to the last line, looking for lines that match a specific pattern, and performs the operations you want on these lines. If no processing action is specified, matching lines are displayed to the standard output (screen). If no mode is specified, all lines specified by the operation are processed. awk respectively represents the first letter of its author's last name. Because its authors are three people, namely Alfred Aho, Brian Kernighan, and Peter Weinberger. gawk is the GNU version of awk, which provides some extensions from Bell Labs and GNU. The awk introduced below takes GUN's gawk as an example. In the Linux system, awk has been linked to gawk, so the following is all introduced using awk.

2. awk command format and options
2.1. There are two forms of awk syntax
awk [options] 'script' var=value file(s)

awk [ options] -f scriptfile var=value file(s)

2.2. Command options
-F fs or --field-separator fs
Specifies the input file separator, fs is a string Or a regular expression, such as -F:.

-v var=value or --asign var=value
Assign a user-defined variable.

-f scripfile or --file scriptfile
Read the awk command from the script file.

-mf nnn and -mr nnn
Set intrinsic limits on the nnn value. The -mf option limits the maximum number of blocks allocated to nnn; the -mr option limits the maximum number of records. These two functions are extended functions of the Bell Labs version of awk and are not applicable in standard awk.

-W compact or --compat, -W traditional or --traditional
Run awk in compatibility mode. So gawk behaves exactly like standard awk, and all awk extensions are ignored.

-W copyleft or --copyleft, -W copyright or --copyright
Print a brief copyright information.

-W help or --help, -W usage or --usage
Print all awk options and a brief description of each option.

-W lint or --lint
Print warnings about structures that are not portable to traditional unix platforms.

-W lint-old or --lint-old
Print warnings about structures that are not portable to legacy unix platforms.

-W posix
Turn on compatibility mode. However, it has the following restrictions and is not recognized: x, function keywords, func, escape sequences, and when fs is a space, the new line is used as a field separator; the operators ** and **= cannot replace ^ and ^= ;ffflush is invalid.

-W re-interval or --re-inerval
Allows the use of interval regular expressions, refer to (Posix character class in grep), such as bracket expression [[:alpha:]].

-W source program-text or --source program-text
Use program-text as the source code, which can be mixed with the -f command.

-W version or --version
Print the version of the bug report information.

3. Modes and operations
Awk script is composed of modes and operations:
pattern {action} such as $awk '/root/' test, or $awk '$3 < 100' test.

Both are optional. If there is no pattern, the action is applied to all records. If there is no action, the output matches all records. By default, each input line is a record, but the user can specify different delimiters through the RS variable.

3.1. Pattern
The pattern can be any of the following:

/regular expression/: an expanded set using wildcards.

Relational expression: You can use the relational operators in the operator table below to perform operations. It can be a comparison of strings or numbers. For example, $2>%1 selects the second field to be longer than the first field. OK.

Pattern matching expression: use operators ~ (match) and ~! (not match).

Mode, mode: Specify a range of lines. This syntax cannot include BEGIN and END patterns.

BEGIN: Allows the user to specify actions that occur before the first input record is processed. Global variables can usually be set here.

END: Let the user take actions after the last input record is read.

3.2. Operation
An operation consists of one or more commands, functions, and expressions, separated by newlines or semicolons, and located within curly brackets. There are four main parts:

Variable or array assignment

Output command

Built-in function

Control flow command

4. awk Environment variables of awk
Table 1. Environment variables of awk

Variable Description
$n The nth field of the current record, the fields are separated by FS.
$0 Complete input record.
ARGC The number of command line parameters.
ARGIND The position of the current file in the command line (counting from 0).
ARGV An array containing command line arguments.
CONVFMT digital conversion format (default value is %.6g)
ENVIRON environment variable associative array.
ERRNO Description of the last system error.
FIELDWIDTHS field width list (separated by space bar).
FILENAME Current file name.
FNR Same as NR, but relative to the current file.
FS field separator (default is any space).
IGNORECASE If true, perform a case-ignoring match.
NF The number of fields in the current record.
NR Current record number.
OFMT digital output format (default value is %.6g).
OFS output field delimiter (default value is a space).
ORS output record delimiter (default value is a newline character).
RLENGTH The length of the string matched by the match function.
RS record separator (default is a newline character).
RSTART The first position of the string matched by the match function.
SUBSEP array subscript separator (default value is