AWK, SED and GREP

Introduction

Text processing is a fundamental aspect of working with data in a Unix/Linux environment. Three powerful command-line tools—Awk, Sed, and Grep—are indispensable for manipulating and extracting information from text. In this blog post, we will delve into the functionalities of Awk, Sed, and Grep.

Awk: A Versatile Text Processing Tool

Overview: Awk is a versatile programming language designed for pattern scanning and text processing. It operates on a line-by-line basis and excels at data extraction and reporting.

Example:

Let's consider a log file access.log with the following entries:

192.168.0.1 - - [01/Jan/2023:12:30:45] "GET /page1 HTTP/1.1" 200
192.168.0.2 - - [01/Jan/2023:12:35:20] "POST /submit HTTP/1.1" 404

To extract and print IP addresses, use the following Awk command:

awk '{print $1}' access.log

Use Case: Awk is particularly useful for generating reports, extracting specific columns, and performing calculations on data.

Advanced Awk Examples:

1. Print selected columns from a structured log file:

    awk '{print $1,$2,$7,$8}' samplelogfile.txt
   

2. Count occurrences of a specific pattern ("11") in a log file:

    awk '/11/ {count++} END {print "the count is", count}' samplelogfile.txt
   

3. Apply conditional printing based on column values:

    awk '$3 >= "15:21:26" && $2 <= "15:22:01" {print $3,$7,$8}' samplelogfile.txt
   

4. Print rows 2 to 10 from a log file:

    awk 'NR >= 2 && NR<=10 {print}' samplelogfile.txt
   

5. Print the row numbers from 2 to 10:

    awk 'NR >= 2 && NR<=10 {print NR}' samplelogfile.txt
   

---

Sed: Stream Editor for Text Transformation

Overview: Sed is a stream editor for filtering and transforming text. It operates on a line-by-line basis and is commonly used for search and replace operations.

Example:

Suppose we have a file file.txt with the content:

Hello, World!
This is a sample text.

To replace "sample" with "example," use the following Sed command:

sed 's/sample/example/' file.txt

Use Case: Sed is handy for batch editing files, substituting text patterns, and applying transformations across multiple lines.

Advanced Sed Examples:

1. Search for a pattern ("MAIL") and print matching lines:

    sed -n '/MAIL/p' samplelogfile.txt
   

2. Replace occurrences of a string ("INFO") with "KP" globally:

    sed 's/INFO/KP/g' samplelogfile.txt
   

3. Print lines containing a specific pattern ("KPdata") and display line numbers:

    sed -n -e '/KPdata/=' samplelogfile.txt
   

4. Search for a pattern ("KP") and print the matching line along with the next line:

   e means expression, n for number, –to search that kp comes in which line in file and print the line also. Thats why -e used two times

    sed -n -e '/KP/=' -e '/KP/p' samplelogfile.txt
   

5. Change occurrences of "INFO" to "kkpp" in lines 1 to 10:

    sed '1,10 s/INFO/kkpp/g' samplelogfile.txt
   

• -i means modify the actual file whatever is replaced by this command . Remember that this command will only display the modified content on the console. If you want to modify the file in place, you can use the -i option:

Grep: Search Patterns in Text

Overview: Grep is a powerful search tool that scans input for a specified pattern and prints lines that match.

Example:

Consider a file data.txt with various lines:

apple
banana
orange
grape

To search for lines containing "orange," use the following Grep command:

grep 'orange' data.txt

Use Case: Grep is essential for searching logs, filtering lines based on patterns, and isolating relevant information.

Advanced Grep Examples:

1. Search for a pattern ("hope") in a specific file:

    grep 'hope' test.txt
   

2. Perform a case-insensitive search for "HOPE" in a log file:

    grep -i 'HOPE' logsdekh
   

3. Count occurrences of "HOPE" in a file:

    grep -i -c 'HOPE' logsdekh
   

---

Awk vs. Sed vs. Grep: Understanding the Differences

Thanks for reading :) happy learning.