Hi All..
I am looking for some help with a script. Currently I am doing the
following search for "error" in a log file:

grep -i error /logs/test.log | grep -v "101017" | grep -v "Broken pipe"
> /tmp/test.log.

I run the above command every hour thru cron to search for the word
"ERROR". But the word may appear more than once a day so if at 8am i
get the error at 9am it is going to give me the same error. At 9am I
dont want to see the 8am error. So I guess I want it to seach by time
as well.

Here is how the date is formatted in the log file:
####<Sep 30, 2005 9:33:23 AM EDT> Error

Thank you in advance for all the help

John <mkhan65@hotmail.com> wrote:
> I am looking for some help with a script. Currently I am doing the
> following search for "error" in a log file:
>
> grep -i error /logs/test.log | grep -v "101017" | grep -v "Broken pipe"
>> /tmp/test.log.
>
> I run the above command every hour thru cron to search for the word
> "ERROR". But the word may appear more than once a day so if at 8am i
> get the error at 9am it is going to give me the same error. At 9am I
> dont want to see the 8am error. So I guess I want it to seach by time
> as well.
>
> Here is how the date is formatted in the log file:
> ####<Sep 30, 2005 9:33:23 AM EDT> Error

You need to save the state between successive invocations of the
script, most likely in a small file somewhere.

You could for example store the number of lines (man wc) you encountered
last time you ran and only search lines after that (man tail).

This of course assumes that the log files doesn't get truncated.

You can also save the timestamp of the last run of the script.
Then you could compare the dates in the log file lines.
This is difficult, and you probably need GNU date to do it.

Yours,
Laurenz Albe

John,
Are you able to name the test.log whatever you want? If so, then you
could save the log with a built in date/time stamp. For example:

# This produces one log per day containing only taht day's messages
MYLOG1=/logs/test_$(date +"%Y%m%d).log

# This produces 24 logs that gets re-used every day.
MYLOG2=/logs/test_$(date +"%H").log

If you use the MYLOG2 example, then the grep commands you used will
only show error messages for the selected hour.

If you use the MYLOG1 example, then you will have one log per day that
contains all the logs for that day. This would mean that your grep
command would need to change as follows (btw- this would still work if
your log was just called "test.log", but you might have information
from previous days, and it might take a long time to search):

# Start of code #
CURR_HR=$(date +"%I")
if test ${CURR_HR} -eq 1
then
PREV_HR=12
else
cat ${MYLOG1} | grep "Error" |
egrep "$(date +"%Y") ${CURR_HR}:|$(date +"%Y") ${PREV_HR} |
grep " $(date +"%p EDT") |
pg -p "Press ENTER for next page: "
fi
# End of code #

By doing this logic, you are asking the grep to show you all errors in
the log, then use that to only show errors from the current or previous
hour...that is just in case you are trying to check for 9AM errors at
10:01AM.

You could also reduce the logic further if you used military time
(00-23) for the hours so you don't need the grep on "%p EDT".

A lot depends on what you plan to do with these error messages when you
find any. If you are issuing emails or warnings about the error, you
may want to have the program that issued the error to the log simply
call a script that does that immediately (ie. send an email warning and
log the error at same time).

If you are just tracking errors for the day, you could produce a report
using awk to format a nice text file that can be printed out once a
day.

HTH

Steve