Example Stream Setup

Example Stream Setup

• 1.
  Step 1 (CreateStream Directory): I recommend this be kept in a central location for all your jobs. It will make recovery and trouble shooting much easier. For this Stream I created SHDRAD in /opt/scripts/JobStream Step 2 (Create Sub-Directories): bin = Where your launcher scripts are kept (required) log = Where stdout and stderr are redirected (jobStream.sh can create this on the fly if necessary) run = Where the lock file for jobStream.sh is stored during its run this prevents the same job from running multiple times if the first stream has not finished. tmp = Where stdout and stderr are redirected during the run then moved to log Only bin is requied to be created up front because you have to have a place to store your launcher scripts, the others can be created by jobStream.sh if necessary, but is recommended to create them up front to avoid failures. james@localhost:/opt/scripts/JobStream> cd SHDRAD james@localhost:/opt/scripts/JobStream/SHDRAD> ll total 4 -rw-r--r-- 1 james users 231 Oct 11 15:49 SHDRAD.lst drwxrwxr-x 2 james users 96 Oct 11 15:42 bin -rw-r--r-- 1 james users 21 Oct 11 15:50 contacts.lst drwxrwxr-x 2 james users 96 Oct 11 15:39 log drwxrwxr-x 2 james users 96 Oct 11 15:39 run drwxrwxrwx 2 james users 96 Oct 11 15:39 tmp Step 3 (Create the Launcher scripts): Launcher scripts can be simple commands or complex shell scripts. Here are the ones I created for SHDRAD. james@localhost:/opt/scripts/JobStream/SHDRAD> cd bin james@localhost:/opt/scripts/JobStream/SHDRAD/bin> ll total 4 -rwxrwx--- 1 james users 147 Oct 11 15:46 cleanLog.sh -rwxrwx--- 1 james users 64 Oct 11 15:41 launchSHDRAD.sh The obvious one is launchSHDRAD.sh, which launches the rc.startjob for executing the Baan job SHDRAD. james@localhost:/opt/scripts/JobStream/SHDRAD/bin> cat launchSHDRAD.sh #!/bin/sh BSE=/msl4/baan; /opt/etc/rc.startjob SHDRAD; exit $?; Since jobStream.sh stores all log files from all runs I created a simple cleanup program for SHDRAD, which will run if SHDRAD completes successfully, otherwise it will not run at all (I'll explain that in a bit). james@localhost:/opt/scripts/JobStream/SHDRAD/bin> cat cleanLog.sh #!/bin/sh MYLOG=/opt/script/JobStream/SHDRAD/log; cd $MYLOG; find . ( -f -ctime +30 ) -exec rm -f {} ; ERR=$?; cd -; exit $ERR;
• 2.
  Step 4 (CreateJob List): I created the job list for SHDRAD directly under the main directory for the stream. james@localhost:/opt/scripts/JobStream> cd SHDRAD james@localhost:/opt/scripts/JobStream/SHDRAD> ll total 4 -rw-r--r-- 1 james users 231 Oct 11 15:49 SHDRAD.lst drwxrwxr-x 2 james users 96 Oct 11 15:42 bin -rw-r--r-- 1 james users 21 Oct 11 15:50 contacts.lst drwxrwxr-x 2 james users 96 Oct 11 15:39 log drwxrwxr-x 2 james users 96 Oct 11 15:39 run drwxrwxrwx 2 james users 96 Oct 11 15:39 tmp It looks similar to a crontab file. It is a simple space delimited file with the following format: Step#, Success#, Fail#, Script Where: Step# is a unique integer from 1-? that identifies each step (note two different steps can launch the same program if desired) Success# is the Step# to execute on a success condition. The number 0 means do nothing, and whatever the last step is in the stream this value is ignored. Fail# is just like Success# except it is for Failures. Failures are also handled differently in that a 0 in its Failure# will result in notification being sent out, whether via the log or via email is your choice. And finally Script is the full path to the launcher that you want to run. For SHDRAD it looks like this: james@localhost:/opt/scripts/JobStream/SHDRAD> cat SHDRAD.lst # Step 1 executes a wrapper for rc.startjob SHDRAD 1 2 0 /opt/scripts/JobStream/SHDRAD/bin/launchSHDRAD.sh # If successful execute logfile cleanup 2 0 0 /opt/scripts/JobStream/SHDRAD/bin/cleanLog.sh What this says is that the launchSHDRAD.sh script will launch as Step 1, if it succeds it then runs cleanLog.sh, otherwise it will send notification of its failure (coming up). Since CleanLog.sh is the last step the success and failure conditions do not matter. If you wanted to ignore cleanLog's failures you could implement a dummy script that always exited with a good status and place it in this file as CleanLog's failure step. Step 5 (Set up Contacts - optional): The contact list is nothing more than a list of email addresses. For SHDRAD I created the following: james@localhost:/opt/scripts/JobStream/SHDRAD> cat contacts.lst gjames@celestica.com By default jobStream.sh ignores comments (lines beginning with #). This also applies the the Job list file as well. This allows you to comment out lines without having to delete them, in case you need them in the future. Step 6 (Schedule the JobStream in cron): To accomplish this step and to keep your crontab file as clean as possible I recommend wrapping the execution of jobStream.sh and its parameters. Part 1 (Create file): In order to do this I created a directory in james's home directory called JobStreams james@localhost:/export/home/testuser> mkdir JobStreams
• 3.
  Then I createdthe file. james@localhost:/export/home/testuser> vi JobStreams/shdrad.sh "JobStreams/shdrad.sh" [New file] #!/bin/sh SDIR=/opt/scripts/JobStream/SHDRAD; EXEC=/opt/scripts/JobStream/jobStream.sh; lfil="$SDIR/SHDRAD.lst"; cfil="$SDIR/contact.lst"; $EXEC -l $lfil -n $cfil -p $SDIR -N SHDRAD_Stream; I used variables in the command line to keep things clean on the screen. Part 2 (Modify Cron): james@localhost:/export/home/testuser> crontab -l > crontab james@localhost:/export/home/testuser> vi crontab "crontab" 3 lines, 148 characters * 20 * * * /opt/scripts/compressadmn.sh # to compress and overwrite the previous days maxi log. #* 1 * * * /opt/log/compress crontmp7 #0 7 * * * /export/home/testuser/JobStreams/shdrad.sh james@localhost:/export/home/testuser> crontab crontab I always follow these steps to insure that I don't mess up the crontab file. You'll not that my entry for shdrad.sh is commented out till we are ready to start using it. Misc. The script jobStream.sh resides in /opt/scripts/JobStream: james@localhost:/export/home/testuser> cd /opt/scripts/JobStream james@localhost:/opt/scripts/JobStream> ll total 48 drwxr-xr-x 6 james users 1024 Oct 11 15:50 SHDRAD -rw-r----- 1 james users 4974 Oct 11 11:24 README drwxr-xr-x 6 james users 1024 Oct 11 11:40 StreamTest -rwxr-x--- 1 james users 16435 Oct 11 15:56 jobStream.sh The README has some documentation in it as well.