The problem
You need to search through logs, extract specific data, or transform text files.
The solution
# Grep patterns
grep "pattern" file.txt
grep -r "error" /var/log/ # Recursive
grep -i "warning" file.txt # Case insensitive
grep -v "exclude" file.txt # Invert match
grep -E "pattern1|pattern2" # Extended regex
grep -A 2 -B 2 "context" # Show 2 lines before/after
grep -c "count" file.txt # Count matches
# Sed stream editing
sed 's/old/new/g' file.txt # Replace all occurrences
sed 's/old/new/2' file.txt # Replace 2nd occurrence per line
sed '5,10d' file.txt # Delete lines 5-10
sed -n '10,20p' file.txt # Print lines 10-20 only
sed '/pattern/d' file.txt # Delete lines matching pattern
sed -i.bak 's/old/new/' file # In-place with backup
# Awk text processing
awk '{print $1}' file.txt # Print first column
awk -F: '{print $1}' /etc/passwd # Custom delimiter
awk '/pattern/ {print $0}' # Print matching lines
awk 'NR > 5 && NR < 10' # Lines 6-9
awk '{sum+=$3} END {print sum}' # Sum column 3
awk '!seen[$0]++' file.txt # Remove duplicates
# Combine commands
grep "error" log.txt | awk '{print $2, $5}' | sort | uniq -c
cat access.log | awk '{print $1}' | sort | uniq -c | sort -nr
# Cut columns
cut -d',' -f1,3 file.csv # Comma delimited, fields 1&3
cut -f1-5 file.tsv # Tab delimited
# Sort and unique
sort file.txt | uniq
sort -u file.txt # Unique sort
sort -nr -k2 data.txt # Numeric reverse by column 2
# Word count
wc -l file.txt # Line count
wc -w file.txt # Word count
wc -m file.txt # Character count
# Head and tail
head -20 file.txt
tail -f log.txt # Follow (live view)
tail -n +100 file.txt # Start from line 100Put it to work
# Extract IP addresses from logs
grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' access.log | sort | uniq -c | sort -nr
# Replace all tabs with commas
sed 's/\t/,/g' data.tsv > data.csv
# Sum values in second column
awk '{sum+=$2} END {print "Total:", sum}' sales.txt
# Find top 10 largest files in list
ls -la | awk '{print $5, $9}' | sort -nr | head -10
# Monitor error logs in real-time
tail -f /var/log/nginx/error.log | grep -i "error\|warning"Worth knowing
Awk is a full programming language. Sed uses regex patterns. Use -E for extended regex in grep. Always test on copies before in-place editing.