Introduction
This tutorial covers five essential Linux text processing tools: grep, sed, awk, cut, and tr.
1. grep (Global Regular Expression Print)
grep is used for searching text using patterns.
Example 1: Basic text search
$ echo -e "This is an error\nThis is a warning\nThis is info" | grep "error" This is an error Example 2: Case-insensitive search
$ echo -e "Warning: System overload\nwarning: Low memory" | grep -i "warning" Warning: System overload warning: Low memory Example 3: Display line numbers
$ echo -e "Line 1\nLine 2\nCritical error\nLine 4" | grep -n "Critical" 3:Critical error Example 4: Recursive search in directories
Assume we have a directory structure:
project/ ├── file1.txt (contains "TODO: Fix this") └── subdir/ └── file2.txt (contains "TODO: Implement feature") $ grep -r "TODO" project/ project/file1.txt:TODO: Fix this project/subdir/file2.txt:TODO: Implement feature Example 5: Invert match
$ echo -e "Success: Task 1\nError: Task 2\nSuccess: Task 3" | grep -v "Success" Error: Task 2 Example 6: Use regular expressions
$ echo -e "Phone: 123-456-7890\nInvalid: 123-45-6789" | grep -E "[0-9]{3}-[0-9]{3}-[0-9]{4}" Phone: 123-456-7890 Example 7: Print only matched parts (-o option)
$ echo "The IP address is 192.168.1.1 and 10.0.0.1" | grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" 192.168.1.1 10.0.0.1 2. sed (Stream Editor)
sed is used for text transformation.
Example 1: Basic substitution
$ echo "The color is red" | sed 's/red/blue/' The color is blue Example 2: Global substitution
$ echo "one two one two" | sed 's/one/three/g' three two three two Example 3: In-place editing
Assume we have a file named "colors.txt" with content "The color is red"
$ sed -i 's/red/green/' colors.txt $ cat colors.txt The color is green Example 4: Delete lines
$ echo -e "Keep this\nDelete me\nKeep this too" | sed '/Delete/d' Keep this Keep this too Example 5: Print specific lines
$ seq 10 | sed -n '3,7p' 3 4 5 6 7 Example 6: Multiple operations
$ echo -e "foo bar\nbaz foo" | sed -e 's/foo/qux/g' -e '/baz/d' qux bar 3. awk (Aho, Weinberger, and Kernighan)
awk is a powerful text-processing tool for working with structured data.
Example 1: Print specific fields
$ echo -e "John Doe 25\nJane Smith 30" | awk '{print $1, $3}' John 25 Jane 30 Example 2: Use custom field separator
$ echo "name:John:age:30" | awk -F':' '{print $2}' John Example 3: Sum values in a column
$ echo -e "10\n20\n30" | awk '{sum += $1} END {print sum}' 60 Example 4: Conditional printing
$ echo -e "John 25\nJane 35\nMike 40" | awk '$2 > 30 {print $0}' Jane 35 Mike 40 Example 5: Calculate average
$ echo -e "10\n20\n30\n40\n50" | awk '{sum += $1; count++} END {print sum/count}' 30 Example 6: Use regular expressions
$ echo -e "apple\nbanana\ncherry" | awk '/^b/ {print $0}' banana 4. cut
cut is used for extracting sections from each line of files.
Example 1: Extract specific characters
$ echo "Hello, World!" | cut -c 1-5 Hello Example 2: Extract specific fields with delimiter
$ echo "John,Doe,30,New York" | cut -d',' -f 2,4 Doe,New York Example 3: Extract range of fields
$ echo "field1:field2:field3:field4:field5" | cut -d':' -f 1-3 field1:field2:field3 Example 4: Extract to end of line
$ echo "Name: John Doe, Age: 30, City: New York" | cut -d':' -f 2- John Doe, Age: 30, City: New York Example 5: Exclude specific fields
$ echo "1,2,3,4,5" | cut -d',' --complement -f 2,4 1,3,5 5. tr (Translate)
tr is used for translating or deleting characters.
Example 1: Convert lowercase to uppercase
$ echo "hello world" | tr 'a-z' 'A-Z' HELLO WORLD Example 2: Delete specific characters
$ echo "hello 123 world" | tr -d '0-9' hello world Example 3: Squeeze repeating characters
$ echo "hello world" | tr -s ' ' hello world Example 4: Translate characters
$ echo "hello world" | tr 'hw' 'HW' Hello World Example 5: Delete all except specified characters
$ echo "hello 123 world" | tr -cd '0-9\n' 123 Conclusion
These five tools - grep, sed, awk, cut, and tr - are powerful utilities for text processing in Linux. The examples provided demonstrate their basic usage and some advanced features. Practice with these examples and explore their man pages for more advanced usage using man command.
Top comments (0)