0

I run this script as quanta suggest me 

#!/bin/bash EDATE=$(tail -1 a | awk '{ print substr($4, 2, length($4)) }') EEPOCH=$(date -d "$(echo "$EDATE" | awk 'BEGIN { FS = "[/:]"; } \ { print $1" "$2" "$3" "$4":"$5":"$6 }')" +%s) time=$(expr 60 \* 60 \* 24 \* 5) SEPOCH=$(expr $EEPOCH - $time) while read line do DATE=$(echo $line | awk '{ print substr($4, 2, length($4)-1) }' | \ awk 'BEGIN { FS = "[/:]"; } { print $1" "$2" "$3" "$4":"$5":"$6 }') DEPOCH=$(date -d "$DATE" +%s) [[ $DEPOCH -ge $SEPOCH && $DEPOCH -le $EEPOCH ]] && echo $line | \ awk '{ print substr($4, 2, length($4)) }' >> as1 done < a

I checked that and it seems that your script want to check log file line by line. Since it has more than 14000 items it takes a lot. So when I run it the cursor goes to next line and next line and I should press it 14000 times so that the whole log file be checked! It's impossible! It just work for short log file I think. Is the problem because of while?

Improve this question
7
  • It seems like this could be a lot more efficient, you're right. Could you add a few lines of the original source file and few lines of the intended output for those lines? Commented Sep 23, 2011 at 13:35
  • So for a given apache log file, all you want is a file as1 containing one unix epoch time (5 days prior to the log date) per line? Or do you want the file to contain all of the intermediate output, too? Commented Sep 23, 2011 at 14:01
  • yeah exactly file as1 contains one date and that is 5 days prior to the end date. I just need that file not all of those echo commands ! Commented Sep 23, 2011 at 14:11
  • No no. The aa.log should be the Apache access log. Commented Sep 24, 2011 at 16:36
  • I checked that and it seems that your script want to check log file line by line . since it has more than 14000 items it takes alot . so when I run it the cursor goes to next line and next line and I should press it 14000 times so that the whole log file be checked ! it's impossible ! it just work for short log file I think . is the problem because of while ? Commented Sep 24, 2011 at 17:05

2 Answers 2

Reset to default
2

This one liner (I've split it for clarity) should give you the same result. You can optionally add > as1 to the end to redirect the output to a file. Put the path to the apache log file where I've put <logfile>

for d in \ $(sed -nre 's/.*\[(..)\/(...)\/(....):(..:..:..) .*/\1 \2 \3 \4/p' <logfile> | date +%s -f-); do echo $[ $d - 86400 * 5]; done

The date command doesn't need an explicity UTC formatted date for the -dargument, although it doesn't understand dates as the apache logs output them, so I've done some substitution to swap the slashes and the colon separating the date and time with spaces.

Improve this answer
7
  • I run that but unsuccessful ! I could not believe that just one line code could do what I wrote ! ./8: command substitution: line 4: syntax error near unexpected token |' ./8: command substitution: line 4: sed -nre 's/.*[(..)\/(...)\/(....):(..:..:..) .*/\1 \2 \3 \4/p' <aa.log> | date +%s -f-)' Commented Sep 23, 2011 at 15:04
  • Drop the < and > from the aa.log bit. You just need the filename there. Commented Sep 23, 2011 at 15:14
  • I dont know what your magic code suppose to do but it gives me out put like this 1041022520 so your code convert the time to epoch time ? because I recognize that you also subtract 5 days , I dont understand Commented Sep 23, 2011 at 15:18
  • That looks about right, given that the log file sample you gave is from 2002. Use epochconverter.com to convert those numbers back to dates and you should see that they're 5 * 24 hours priror to the date in your log file. Commented Sep 23, 2011 at 15:27
  • It's a little bit different, my code get the end date and subtract 5 days from it and then print all of those 5 last days ! your's subtract 5 days from all of the log file ! Commented Sep 23, 2011 at 15:37
0

Here for you:

#!/bin/bash EDATE=$(tail -1 aa.log | awk '{ print substr($4, 2, length($4)) }') EEPOCH=$(date -d "$(echo "$EDATE" | awk 'BEGIN { FS = "[/:]"; } \ { print $1" "$2" "$3" "$4":"$5":"$6 }')" +%s) time=$(expr 60 \* 60 \* 24 \* 5) SEPOCH=$(expr $EEPOCH - $time) while read line do DATE=$(echo $line | awk '{ print substr($4, 2, length($4)-1) }' | \ awk 'BEGIN { FS = "[/:]"; } { print $1" "$2" "$3" "$4":"$5":"$6 }') DEPOCH=$(date -d "$DATE" +%s) [[ $DEPOCH -ge $SEPOCH && $DEPOCH -le $EEPOCH ]] && echo $line | \ awk '{ print substr($4, 2, length($4)) }' >> as1 done < aa.log

UPDATE

The input:

213.46.27.204 - - [21/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [22/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [23/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [24/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [25/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [26/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [27/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [28/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [29/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [30/Dec/2002:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" 213.46.27.204 - - [01/Jan/2003:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" ""

The output:

$ ./five_last_days.sh 27/Dec/2002:12:55:21 28/Dec/2002:12:55:21 29/Dec/2002:12:55:21 30/Dec/2002:12:55:21 01/Jan/2003:12:55:21
Improve this answer
5
  • it does not work ! Commented Sep 23, 2011 at 17:33
  • Show us the output? Commented Sep 23, 2011 at 17:36
  • the output is so long ! it print from 8th of December till the end ! while it should print from 27th ! the last item in log file is 213.46.27.204 - - [01/Jan/2003:12:55:21 +0100] "GET /scripts/..%252f../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 - "" "" Commented Sep 23, 2011 at 17:39
  • Cut 30 last lines from your input, run my script, and edit your first post (to keep formatting) to show me the output. Commented Sep 24, 2011 at 0:51
  • I checked the script with your data and the output was the same as yours , but in original logfile it does nothing ! actually when I run the cursor goes to next line and next line and after a long time I press control+C , maybe it try to check the whole logfile and because it has 14000 line it takes so much time ! I could not press enter for 2 hours so that the whole log file be checked ,how could I upload the original logfile here? Commented Sep 24, 2011 at 16:28

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.