awk.md

AWK

---

find matching line

awk '/findme/' <>

not matching (or with numbered matching lines)

awk '!/shitty/' <> awk '/foo/{print FNR,$0}' <>

Find all the lines longer than 80 characters

awk 'length($0)>80{print FNR,$0}' <>

less than 80

awk 'length < 80

add empty line between each line

awk '1; { print "" }' <>

print line numbers (2nd way is fancy)

awk '{ print FNR "\t" $0 }' awk '{ printf("%5d : %s\n", NR, $0) }'

Print line numbers for only non-blank lines

awk 'NF { $0=++a " :" $0 }; { print }'

Print the line and the next two (i=5) lines after the line matching regexp

awk '/foo/{i=5+1;}{if(i){i--; print;}}'

Print the lines starting at the line matching 'server {' until the line matching '}'

awk '/server {/,/}/'

Print multiple columns with separators

awk -F' ' '{print "ip:\t" $2 "\n port:\t" $3'

Remove empty lines

awk 'NF > 0'

alternative:

awk NF

Delete trailing white space (spaces, tabs)

awk '{sub(/[ \t]*$/, "");print}'

Delete leading white space

awk '{sub(/^[ \t]+/, ""); print}'

Remove duplicate consecutive lines

uniq

awk 'a !~ $0{print}; {a=$0}'

Remove duplicate entries in a file without sorting

awk '!x[$0]++'

Exclude multiple columns

awk '{$1=$3=""}1'

Substitute foo for bar on lines matching regexp

awk '/regexp/{gsub(/foo/, "bar")};{print}'

Add some characters at the beginning of matching lines

awk '/regexp/{sub(/^/, "++++"); print;next;}{print}'

Get the last hour of Apache logs

awk '/'$(date -d "1 hours ago" "+%d\/%b\/%Y:%H:%M")'/,/'$(date "+%d\/%b\/%Y:%H:%M")'/ { print $0 }'
/var/log/httpd/access_log

awk

Select lines in a csv file based on text comparison

awk -F ',' '$5 ~ /baddomain.com/' FILE

Print specific fields of a csv file

awf -F '|' '{print $3" "$4} FILE

Tab separated files:

awk -F $'\t' ...