How can I grep for two values from the same input being piped to it?

Question

CentOS 5.x

I'm trying to build a shell script that searches data provided via stdin. Here's an example of the input stream:

Date: 1/1/11 Time: 12:00 AM Foo: 12345 Foo1: dskjflsdkjflksdjlfds Foo2: 123456789 Foo3: kdsjflskdjflkjsdlkfjsdlkjflksdjflkjsdklfjlksdjflk 

This information doesn't exist in a file, it would only be sent to the script in real-time as stdout from another app.

I want the sript to look at the data and parse out the values for Foo: and Foo2: storing those as variables for use later in the script.

My revised script attempt is this:

#!/bin/bash while read data; do SearchCriteria1=$(echo "$data" | grep "Foo: " | cut -c 5-) SearchCriteria2=$(echo "$data" | grep "Foo2: " | cut -c 6-) echo $SearchCriteria1 >> test.1 echo $SearchCriteria2 >> test.2 done 

The finished script won't actually use test.1 or test 2 files. I just have them listed here for easier examples.

In this example, I would expect test.1 to have:

12345 

In this example, I would expect test.2 to have:

123456789 

When I test this though, both test.1 and test.2 are blank and I know the data has the valid info in it.

I'm missing something obvious. Can someone please clarify?

Answer 1

Think about where your greps are getting their input from. You've given them no input filename, so they're reading from standard input. What is their standard input? There is no redirection for it, and there is no pipe going into the greps, so they are running with the inherited standard input of the loop.

The read is executing only once. It reads the first line into $data, which you never use anywhere (that should be a clue that something is wrong). Then the first grep runs, it consumes all of the input, and the second grep is attached to the same input so it gets immediate EOF.

This is probably closer to what you wanted:

SearchCriteria1=$(echo "$data" | grep "Foo " | cut -c 10-) SearchCriteria2=$(echo "$data" | grep "Foo2 " | cut -c 13-) 

The semicolons on the end were useless so I took them off.

Also you probably want to append to test.1 and test.2 otherwise each iteration through the loop overwrites what the previous one wrote.

Looking at the big picture, it appears that you want a "multi output grep". Your read loop with individual greps for each line is one way to implement it, but not very efficient. Some other ways were discussed recently at https://stackoverflow.com/questions/11676350/grepping-a-20g-file-in-bash/11676853

Answer 2

input stream producer | egrep 'Foo:|Foo2:' | while read foo data do case $foo in Foo:) echo "do something with $foo -- $data" ;; Foo2:) echo "do something else with $foo -- $data" ;; esac done 

The egrep line is not strictly necessary but cuts out unnecessary input for the loop. Then read into two variables (foo and data) and use case to choose between them and act accordingly (you could at this stage assign data to another variable if you wish).

Answer 3

How about this:

$ ./another.app | awk '/Foo: / { print $2 }' > test.1 $ ./another.app | awk '/Foo2: / { print $2 }' > test.2