Have a shell script check for a file to exist before processing another file

#1 (**permalink**) 01-05-2008, 11:47 AM

I have a shell script that runs all the time looking for a certain type
of file and then it processes the file through a series of other
scripts. The script is watching a directory that has files uploaded to
it via SFTP. It already checks the size of the file to make sure that
it is not still uploading before it starts processing. I would like to
place another check in the script that looks for the existance of
another file before processing begins. The script looks like:

Code:

#!/bin/ksh
PATH=/gers/nurev/menu/pub/sbin:/gers/nurev/menu/pub/bin:/gers/nurev/menu/pub/mac
:/gers/nurev/menu/adm/sbin:/gers/nurev/menu/adm/bin:/gers/nurev/menu/adm/mac:/ge
rs/nurev/custom:/gers/nurev/fix:/gers/nurev/src_rev/fix:/gers/nurev/opt/path:/ge
rs/nurev/bin:/g/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/sbin:.
ORACLE_HOME=/gers/nurev
ORACLE_SID=nurev
export PATH
export ORACLE_HOME
export ORACLE_SID

#
# Function : is_file_arrived file
# Arg(s)   : file = file to verify
# Output   : None
# Status   : 0 = yes file arrived, 1 = no
# Env.     : IFA_WAIT : interval (secs) for file size check (def=5)
#

is_file_arrived() {
[ -z "$1" ] && return 1
local file=$1
local arrived=1
local size1 size2
if [ -f "$file" -a -z "$(fuser $file 2> /dev/null)" ] ; then
size1=$(ls -l $file 2>/dev/null | awk '{print $5}')
sleep ${IFA_WAIT:-15}
size2=$(ls -l $file 2>/dev/null | awk '{print $5}')
[ ${size1:-1} -eq ${size2:-2} ] && arrived=0
fi
return $arrived
}


processFile ()
{
local fileName=$1
local fileExtension=$2
local fileNewName="/gers/nurev/datafiles/str${fileExtension}.asc"
local filePrintPath="/gers/nurev/print"
local fileTmpPath="/gers/nurev/tmp"
local fileODIName="str${fileExtension}.pos"
mv -Eignore $fileName $fileNewName
prepup $fileNewName $fileExtension
mv -Eignore  $filePrintPath/$fileODIName $fileTmpPath/$fileODIName
save2tmp $fileExtension
call_siu $fileExtension
}

# Main Processing

nsec=1
while [[ "$(date +%H%M)" -lt 2329 ]]
do
for fileName in
/gers/nurev/datafiles/[Uu][Pp][Ll][Oo][Aa][Dd].[0-9][0-9][0-9
][0-9]
do
fileExtension=${fileName#*.}
is_file_arrived "$fileName" && nsec=1 && processFile $fileName
$fileExtension
done
sleep $nsec
case $nsec in
1)   nsec=15;;
15)  nsec=45;;
45)  nsec=90;;
90)  nsec=300;;
300) nsec=600;;
600) nsec=900;;
*)   nsec=1800;;
esac
done

....I'd like to check for a file in /gers/genret/tmp/poll_####.txt
(where "####" is the $fileExtension variable. I think the file check
should be in the "Main Processing" section before it sends to the file
to the "is_file_arrived" function? I'f there is a "poll_####.txt" file
present I would just like the script to move onto the next file and
look at it again later. This would prevent two files of the same type
processing at the same time. I'm thinking something like a "if-then"?

#2 (**permalink**) 01-05-2008, 11:47 AM

heprox wrote:
> I have a shell script that runs all the time looking for a certain type
> of file and then it processes the file through a series of other
> scripts. The script is watching a directory that has files uploaded to
> it via SFTP. It already checks the size of the file to make sure that
> it is not still uploading before it starts processing. I would like to
> place another check in the script that looks for the existance of
> another file before processing begins. The script looks like:
>
>

Code:

> #!/bin/ksh
> PATH=/gers/nurev/menu/pub/sbin:/gers/nurev/menu/pub/bin:/gers/nurev/menu/pub/mac
> :/gers/nurev/menu/adm/sbin:/gers/nurev/menu/adm/bin:/gers/nurev/menu/adm/mac:/ge
> rs/nurev/custom:/gers/nurev/fix:/gers/nurev/src_rev/fix:/gers/nurev/opt/path:/ge
> rs/nurev/bin:/g/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/sbin:.
> ORACLE_HOME=/gers/nurev
> ORACLE_SID=nurev
> export PATH
> export ORACLE_HOME
> export ORACLE_SID
>
> #
> # Function : is_file_arrived file
> # Arg(s)   : file = file to verify
> # Output   : None
> # Status   : 0 = yes file arrived, 1 = no
> # Env.     : IFA_WAIT : interval (secs) for file size check (def=5)
> #
>
> is_file_arrived() {
>    [ -z "$1" ] && return 1
>    local file=$1
>    local arrived=1
>    local size1 size2
>    if [ -f "$file" -a -z "$(fuser $file 2> /dev/null)" ] ; then
>       size1=$(ls -l $file 2>/dev/null | awk '{print $5}')
>       sleep ${IFA_WAIT:-15}
>       size2=$(ls -l $file 2>/dev/null | awk '{print $5}')
>       [ ${size1:-1} -eq ${size2:-2} ] && arrived=0
>    fi
>    return $arrived
> }
>
>
> processFile ()
> {
>    local fileName=$1
>    local fileExtension=$2
>    local fileNewName="/gers/nurev/datafiles/str${fileExtension}.asc"
>    local filePrintPath="/gers/nurev/print"
>    local fileTmpPath="/gers/nurev/tmp"
>    local fileODIName="str${fileExtension}.pos"
>    mv -Eignore $fileName $fileNewName
>    prepup $fileNewName $fileExtension
>    mv -Eignore  $filePrintPath/$fileODIName $fileTmpPath/$fileODIName
>    save2tmp $fileExtension
>    call_siu $fileExtension
> }
>
> # Main Processing
>
> nsec=1
> while [[ "$(date +%H%M)" -lt 2329 ]]
> do
>    for fileName in
> /gers/nurev/datafiles/[Uu][Pp][Ll][Oo][Aa][Dd].[0-9][0-9][0-9
> ][0-9]
>    do
>       fileExtension=${fileName#*.}
>       is_file_arrived "$fileName" && nsec=1 && processFile $fileName
> $fileExtension
>    done
>    sleep $nsec
>    case $nsec in
>       1)   nsec=15;;
>       15)  nsec=45;;
>       45)  nsec=90;;
>       90)  nsec=300;;
>       300) nsec=600;;
>       600) nsec=900;;
>       *)   nsec=1800;;
>    esac
> done
>

>
> ...I'd like to check for a file in /gers/genret/tmp/poll_####.txt
> (where "####" is the $fileExtension variable. I think the file check
> should be in the "Main Processing" section before it sends to the file
> to the "is_file_arrived" function? I'f there is a "poll_####.txt" file
> present I would just like the script to move onto the next file and
> look at it again later. This would prevent two files of the same type
> processing at the same time. I'm thinking something like a "if-then"?

Hey,
We have a similar situation. Here is how we do it.
the program that creates the file should create it with an extension of
let's say tmp.
In your case: /gers/genret/tmp/poll_####.txt.tmp.
when the file is totally created the same program would rename the file
to: /gers/genret/tmp/poll_####.txt

Your script would always check for such
files:/gers/genret/tmp/poll_####.txt NOT the *.tmp

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode