Advanced UNIX (BASH)


Objectives
 explain how to write Bourne and Bash Sh
ell scripts
Overview
1. Making a File Executable
2. Combining Commands
3. Redirecting Output
4. Executing Scripts
5. Variables
6. Control Flow

continued
7. Functions
8. Other Commands
9. Here Documents
10. Debugging
11. More Information
1. Making a File Executable

$ cat whoson
date
echo Users currently logged on
who


Wrong:
$ whoson
whoson: Permission denied
Right:

$ ls -lg whoson
-rw-r--r-- 1 ad pubs 42 Jun 17 10:55 whoson
$ chmod u+x whoson
$ ls -lg whoson
-rwxr--r-- 1 ad pubs 42 Jun 17 10:55 whoson


$ whoson
Tue Nov 7 13:21:34 ICT 2000
Users currently logged in
ad consol Nov 7 08:26
jenny tty02 Nov 7 10:04
Possible PATH Problem

$ whoson
whoson: Command not found


Due to PATH shell variable (see later)


Quick fixes:
$ ./whoson
or $ sh whoson
2. Combining Commands

Sequencing:
$ a ; b ; c


same as:
$ a
$ b
$ c
Process Creation (&)

$ a & b & c
14271 (PID for a)
14272 (PID for b)


$ a & b & c &
14290
14291
14292
$


$ a | b | c &
14302 (PID for piped commands)
Processes in Action
$ cat a
echo -n “aaaaaaaaaaaaaaaaaaaaa”
echo -n “aaaaaaaaaaaaaaaaaaaaa”
sleep 2
echo -n “aaaaaaaaaaaaaaaaaaaaa”
echo -n “aaaaaaaaaaaaaaaaaaaaa”


Similarly for b and c On the lab
machines there

Try the following a few times: isn't much
$ a & b & c & variation unless
the machine is
loaded.
3. Redirecting Output

1> redirect standard output (stdout)
2> redirect standard error (stderr)

$ cat a b 1> out 2> err

Files
stdout out

cat a b
stderr err
>&

redirect one stream into another:
 2>&1
redirect stderr into stdout

$ cat a b 1> theLot 2>&1

stdout theLot

cat a b
stderr
4. Executing Scripts

Make sure that a script is executed by the Bourn
e Shell:
$ sh whoson (no need for chmod)


or:
$ cat boss
#!/bin/sh
echo Definitely Bourne Shell Script

continued

On Linux machines (e.g. calvin), the Bourne s
hell has been replaced by Bash
 sh means the Bash shell
5. Variables
5.1. User-defined Variables

5.2. Environment Variables

5.3. Readonly Shell Variables

5.1. User-defined Variables

$ person=alex
No spaces
$ echo person around the =
person
$ echo $person
alex


$var returns the value stored in var
 called substitution
5.1.1. Switch off Substitution

Swich off substitution with 'var' or
\var


$ echo '$person'
$person
$ echo \$person
$person
5.1.2. Switch off Special Chars (")

"" switches off the special meaning of characters us
ed in filename generation (e.g. *, ?)


$ ls // directory contents
ad.report
ad.summary

$ memo=ad*
$ echo "$memo"
ad*
* means only *
$ echo $memo
ad.report ad.summary

* means any number

of characters
5.1.3. Exporting Variables

Normally a variable is local to the running scri
pt (the process).


It is sometimes useful if running scripts (proce
sses) can access another script’s variables.


e.g.
we want to
calls use cheese
extest subtest
in subtest
cheese=english
No Exporting: extest1 & s
ubtest

$ cat extest1 Using " is
cheese=english a good habit,
echo "extest1 1: $cheese"
see later
subtest
echo "extest1 2: $cheese"

$cat subtest
echo "subtest 1: $cheese"
cheese=swiss
echo "subtest 2: $cheese"

continued

$ extest1 subtest does not
extest1 1: english see extest1's
subtest 1: cheese value
subtest 2: swiss
extest1 2: english

extest1 is not affected

by subtest's setting of
cheese
Exporting: extest2 & subtest

$ cat extest2
export cheese
cheese=english
echo “extest2 1: $cheese”
subtest
echo “extest2 2: $cheese”


$ extest2
extest2 1: english
subtest 1: english
subtest 2: swiss cheese value passed in
extest2 2: english
change not exported
from subtest
5.1.4. Reading

$ cat readln read inputs everything
echo -n “Type: ” up to the newline
read ln
echo “You entered: $ln”


$ readln
Type: The Wizard of Oz
You entered: The Wizard of Oz
No Quotes

$ cat readlnnq
echo -n “Type: ”
read ln
echo You entered: $ln


$ ls
ad.report summary1 directory
$ readlnnq contents
Type: *
You entered: ad.report summary1
5.1.5. Executing Commands
A very simple shell

$ cat proc_cmd
echo -n “Enter a command: ”
read command
$command
echo Thanks


$ proc_cmd
Enter a command: echo Display this
Display this
Thanks
5.1.6. Splitting Input
Text is split

$ cat split3 based on white
echo -n “Enter something: ” space.
read word1 word2 word3
echo “Word 1 is: $word1”
echo “Word 2 is: $word2”
echo “Word 3 is: $word3”


$ split3
Enter something: this is something
Word1 is: this
Word2 is: is
Word3 is: something

$ split3
Enter something: this is something else, x
Word1 is: this
Word2 is: is
Word3 is: something else, x

The last variable gets

everything that is left
in the input.
5.1.7. Command Subsituti
on
Must use ‘

$ cat mydir
this_dir=‘pwd‘
echo “Using the $this_dir directory.”
this_date=$(date)
echo "Today's date: $this_date"
A Bash
addition

$ mydir
Using /home/ad/teach/adv-unix/bourne directory
Today's date: Tue Nov 7 13:52:46 ICT 2000
$
5.2. Environment Variables

Most environment variables get their values from
the shell at login.


The values of some of the variables can be set b
y editing the .profile file in your home directory
 Bash uses .bash_profile and .bashrc
5.2.1. Examples

HOME
pathname of your home directory


$ pwd
/home/ad/planning
$ echo $HOME
/home/ad cd uses HOME
$ cd
to return to your
$ pwd
/home/ad home directory

continued

PATH
 directories where executable can be found
 represented as a string of pathnames separated by ‘:’
s


$ echo $PATH
/usr/local/bin:/usr/bin:/bin: Extend the
$ PATH=SPATH":/home/ad/bin:." default PATH
$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/home/ad/bin:.
Note for SysAdmins

If you are the system administrator
(superuser, root) for your machine,
do not extend your path with "."
 it opens you to potential attack by
hackers
 e.g. 'fake' UNIX utilities placed in th
e current directory
5.2.2. Typical .profile

$ cat .profile
TERM=vt100
PATH=$PATH":/home/ad/bin:."
PS1=“ad: “
CDPATH=:$HOME
export TERM PATH PS1 CDPATH

stty kill ^u


$ . .profile
export needed
in the Bourne
shell
5.2.3. Typical .bash_profile

On calvin, .bash_profile simply invokes your .ba
shrc (if it exists):

umask 002
if [ -f ~/.bashrc -a PS1="\$ " ]; then
. ~/.bashrc
fi

These shell commands

will be explained later
continued
Typical .bashrc

PS1="\u@\h$ "
# PS1="\w[\#]$ " No export needed
PATH=$PATH":."

alias ls='/bin/ls -F'

alias dir='ls -ba'
alias cls="clear"
: These features
:
will be explained
psgrep() later.
{
ps aux | grep $1 | grep -v grep
}
5.2.4. set

The current settings for the environment varia
bles can be listed with set:

$ set | more
BASH=/bin/bash
:
PATH=/usr/local/bin:/usr/bin:/bin:.
:
PS1='\u@\h$ '
:
5.3. Readonly Shell Variables

These are environment variables t
hat cannot have their values chan
ged.


Most of them relate to the argume
nts supplied to a script when it is c
alled.
5.3.1. Script Name ($0)

$ cat abc
echo The name of this script is $0


$ abc
The name of this script is abc
5.3.2. Script Arguments ($1, $2,..., $9)


$ cat display_5args
echo The first five command line
echo arguments are $1 $2 $3 $4 $5


$ display_5args jenny alex helen
The first five command line
arguments are jenny alex helen

If the variable has no value,

then nothing is printed.
5.3.3. All Arguments ($*)

$ cat display_all
echo $*


$ display_all a b c de fg hi jk mno pqr
stu w x y z
a b c de fg hi jk mno pqr stu w x y z


$@is like $* but puts “...” around each printed a
rgument
5.3.4. Number of Arguments ($#)


$ cat num_args
echo “This script has $# arguments.”


num_args helen alex jenny
This script has 3 arguments
5.3.5. The shift Commnd


shift moves argument values on the
command line one $<number> to the le
ft.


Overcomes limit of 9 argument variabl
es
($1, $2, ..., $9)

continued

$ cat demo_shift
echo “arg1= $1 arg2= $2 arg3= $3”
shift
echo “arg1= $1 arg2= $2 arg3= $3”
shift
echo “arg1= $1 arg2= $2 arg3= $3”
shift


$ demo_shift alice helen jenny june
arg1= alice arg2= helen arg3= jenny
arg1= helen arg2= jenny arg3= june
arg1= jenny arg2= june arg3=

jenny "moves"
to the left.
5.3.6. The set Command (Again)


set ‘cmd‘ (must use ‘)

 evaluates cmd and assigns its valu

es to the script command line argu
ments ($1, $2, ..., $9, $*)
 the values in cmd output are separ
ated by whitespace

continued

$ date
Fri Jun 17 23:04:09 GMT+7 1996
$ cat dateset
set ‘date‘
echo $*
echo The date values
echo “Argument 1: $1” are assigned to $1,
echo “Argument 2: $2” $2, etc.
echo “Argument 3: $3”
echo $2 $3, $6

continued

$ dataset
Fri Jun 17 23:04:13 GMT+7 1996

Argument 1: Fri
Argument 2: Jun
Argument 3: 17
Jun 17, 1996
6. Command Flow
6.1. Branching
6.2. Test Forms
6.3. Looping
6.4. break, continue, :
6.5. trap
6.1. Branching

$ cat same_word
echo -n “word 1: ” Use " to stop
read word1 filename
echo -n “word 2: ” expansion
read word2
if test "$word1" = "$word2"
then
echo Match
fi
echo End of Program

continued

$ same_word
word1: peach
word2: peach
Match
End of Program
6.1.1. Second Format
Redirect

$ cat chkargs
stdout to stderr
if [ $# = 0 ]
then
echo Usage: chkargs argument... 1>&2
exit 1
fi
echo Program running
exit 0


$ chkargs
Usage: chkargs argument...
$ chkargs abc
Program running
6.1.2. if-then-else C programmers
prefer this format

$ cat show
if [ $# = 0 ]; then
echo Usage: show [-v] filenames 1>&2
exit 1
fi
if [ “$1” != “-v” ]
then
cat “$@”
else
shift
more “$@”
fi

Use:
$ show -v f1.txt f2.txt
6.1.3. if-then-elif For multiway branches


$ cat same3
echo -n “word 1: ”
read word1
echo -n “word 2: ” -a means "and"
read word2 \ means "continued
echo -n “word 3: ” on next line"
read word3
if [ “$word1” = “$word2” -a \
“$word2” = “$word3” ]
then
echo “Match: words 1, 2, and 3”

continued
elif [ “$word1” = “$word2” ]
then
echo “Match: words 1 and 2”
elif [ “$word1” = “$word3” ]
then
echo “Match: words 1 and 3”
elif [ “$word2” = “$word3” ]
then
echo “Match: words 2 and 3”
else
echo “No match”
fi
6.1.4. case Better style: specify
the shell, and comment
the code.

$cat command_menu
#!/bin/sh
# menu interface to simple commands

echo “\n COMMAND MENU\n”

echo “ a. Current date and time”
echo “ b. Users currently logged in”
echo “ c. Name of working directory”
echo “ d. Contents of working directory”
echo -n “Enter a, b, c, or d: ”
read answer
echo

continued
case “$answer” in
a) date
;;
b) who
;;
c) pwd
;;
d) ls -C * is the default;
;; always include it
*) echo “$answer not legal” at the end
;;
esac
echo

$ command_menu
COMMAND MENU

a. Current date and time

b. Users currently logged in
c. Name of working directory
d. Contents of working directory
Enter a, b, c, or d: a

Fri Jun 17 14:11:57 GMT 1996

Other case Patterns

? matches a single character


[...] any character in the brackets.
Use - to specify a range (e.g. a-z)


| or (e.g. a|A)


* it can be used to match "any
number of characters"

$cat timeDay
#!/bin/sh
echo Is it morning? Answer yes or no
read timeofday

case "$timeofday" in
"yes" | "y" | "Yes" | "YES" )
echo "Good Morning"
;;
[nN]* )
echo "Good Afternoon"
;;
*) echo "Uhh??"
;;
esac
6.2. test Forms
Used in the conditions of if's and loops.


Format:
test expr
[ expr ]

e.g.
test “$word1” = “$word2”
[ “$1” != “-v” ]
[ “$word2” = “$word3” ]
[ “$word1” = “$word2” -a \
“$word2” = “$word3” ]
6.2.1. String Expressions

string1 = string2


string1 != string2


-n string (true if string is not ““)


-z string (true if string is ““)
6.2.2. Numerical Expressi
ons

number1 -eq number2 (equality)

number1 -ne number2
(inequality)


number1 -lt number2 (<)

number1 -le number2 (<=)


number1 -gt number2 (>)

number1 -ge number2 (>=)
6.2.4. File Test Expression
s

-f file (file exists)

-d file (file exists but is a directory)


-r file (file is readable)

-w file (file is writable)

-x file (file is executable)
6.2.5. Combining Expressi
ons

! expr (not expr)


expr1 -a expr2 (and)
 Bash allows && as well


expr1 -o expr2 (or)
 Bash allows || as well


( expr )
6.3. Looping (for-in)

Format:

for loop-index in argument-lis

t
do
commands
done

$ cat fruit
for fruit in apples oranges pears
do
echo $fruit
done
echo Task completed


$ fruit
apples
oranges
pears
Task completed
Looking for “for” in files

First version of script:

$ cat file-fors1
for f in ‘ls‘
do
echo $f
done

$ cat file-fors2
for f in ‘ls‘ Ignore output
do
grep -i “for” $f > /dev/null
if [ $? == 0 ]
then
echo $f
fi
done
6.3.1. for

$ cat whos
# Give user details from /etc/passwd
if [ $# = 0 ]
then
echo Usage: whos id... 1>&2
exit 1
fi

for i # read as for i in “$@”

do
awk -F: ’{print $1, $5}’ /etc/passwd |
grep -i “$i”
done awk variables
with ’
6.3.2. while

$ cat count
number=0
while [ $number -lt 10 ]
do
echo -n "$number"
number=‘expr $number + 1‘
done
echo


$ count
0123456789
$
Watching for Mary to log in

while sleep 60
do
who | grep mary
done


Disadvantages:
 if Mary is already logged in, then we
must
wait 60 secs to find out
 we keep being told that Mary is logg
ed in
6.3.3. until

Format:
until command
do
loop body until
the command evaluates to true
done

Watching (v.2):
until who | grep mary
do
sleep 60
done
loop until grep ret
urns ‘true’ (0)
6.3.4. Extending Watching

Generalise so can watch for anyon
e:
$ watchfor ad


Watch for everyone logging in/out:
$ watchwho
watchfor
#!/bin/sh
# watchfor
# watch for person supplied as argument

if [ $# = 0 ]
then
echo “Usage: watchfor person”
exit 1
fi
until who | grep “$1”
do
sleep 60
done
watchwho

Once a minute, run who and compare its outp
ut to that from a minute ago. Report any differ
ences.


Keep the who output in files in /tmp


Give the files unique names by adding the sh
ell variable $$.
 $$ is the PID of the user’s shell
#!/bin/sh
# watchwho: watch who logs in and out

new=/tmp/wwho1.$$
old=/tmp/wwho2.$$
> $old # create an empty file

while true
do
who > $new An advanced
diff $old $new Shell programming
mv $new $old
sleep 60
technique: |
done | awk ’/>/ {$1 = “in: “; print}
/</ {$1 = “out: “; print}’
Use

$ watchwho
initial
in: root tty1 Nov 6 09:32 users
in: ad pts/3 Nov 8 08:49 (myrrh.coe.psu.ac.th)
in: s4010441 pts/5 Nov 8 10:11 (192.168.0.134)
in: ad pts/4 Nov 8 10:12 (myrrh.coe.psu.ac.th)
in: s4010143 pts/17 Nov 7 23:57 (192.168.0.204)
out: ad pts/4 Nov 8 10:12 (myrrh.coe.psu.ac.th)
in: ad pts/4 Nov 8 10:16 (myrrh.coe.psu.ac.th)
out: ad pts/4 Nov 8 10:18 (myrrh.coe.psu.ac.th)
:
Notes

diff uses ‘<‘ and ‘>‘ to distinguish data from $
old and $new


$ diff old new
< ad
> mary
old new
john john
ad mary

continued
while awk
loop


The output from the while loop is piped into awk
 only one call to awk is required
 the "pipe" programming style


A calvin problem:
 awk had to be called with the -W interactive option so that its output was not buffer
ed
 otherwise nothing would appear
6.3.5. Checking mail

Have a script watch your mailbox p
eriodically, and report whenever the
mailbox changes by printing “You hav
e mail”.


Usage:
 $ checkmail # checks every 60 secs
 $ checkmail 120
# checks every 120 secs
checkmail
#!/bin/sh
# checkmail: watch mailbox for growth

time=${1-60}

oldls="‘ls -l $MAIL‘"
while true
do
newls="‘ls -l $MAIL‘"
echo $oldls $newls
oldls="$newls"
sleep $time
done | awk ’$5 < $14 {print “You have mail”}’

uses the pipe technique

Notes

$MAIL is a builtin shell variable, with the
value /var/spool/mail/$USER

t=${1-60}sets t to $1 or, if no argument i
s provided, to 60
 General form: ${var-thing}
returns value of var if defined; otherwis
e thing

continued

Use awk to print a message only when the mailbo
x gets bigger:
 awk $5 $14 compares the size fields of the two ls -l ca
lls output at the end of each iteration

e.g.
$ ls -l foo 5th value
-rw-r--r-- 1 ad ad 34512 Nov 13 1996 foo
$ ls -l foo
-rw-r--r-- 1 ad ad 34512 Nov 13 1996 foo

14th value
6.4. break, continue, :

break and continue are used as in C:
 break escapes from a loop

for file in fred*

do
if [ -d "$file" ]; then # deleted?
break # finish loop
fi
# do something
#
done

continued

continue goes to the top of a loop:

for file in fred*

do
if [ -d "$file" ]; then # deleted?
continue # go back to loop top
fi
# do something
#
done

continued

The ':' command is the same as t
rue, but runs a tiny bit faster
 often used to simplify control logic

if [ -f fred ]; then # is fred

a file?
: # do nothing
else
# do something
#
fi
6.5. trap

Capture user interrupts or system call failures.


Format (with ’):
trap ’commands’ signal-numbers-or-name


Some signal numbers (names):
 2 (INT) press delete or control-C
 3 (QUIT) press control-| or control-\
 15 (TERM) kill command signal

continued

A complete list of available signals is given
by typing trap -l at the command line.


To ignore a signal, set the trap command t
o be empty (’’)


To reset signal processing, set the comma
nd to -

continued

$ cat inter
trap ’echo PROGRAM INTERRUPTED; exit 1’ INT
while true
do
echo Program running.
sleep 2
done
7. Functions

Bash allows shell scripts to use fun
ctions:
function_name() {
statements
}


Functions must be defined textuall
y before they are used.

continued

Functions can return integer values


Function parameters are passed by modifying $*,
$@, $#, $1 -- $9 while the function is executing.


Local variables are defined with the local keywor
d
 the other variables in a script are global by default
my_name

#!/bin/sh

yes_or_no() { # a function
echo "Is your name $* ?"
while true
do
echo -n "Enter yes or no: "
read x
case "$x" in
y | yes ) return 0;;
n | no ) return 1;;
* ) echo "Answer yes or no"
esac
done
} # end of yes_or_no()

continued
# the main part of the script

echo "Original parameters are $*"

if yes_or_no "$1"
then
echo "Hi $1, nice name"
else
echo "Never mind"
fi

exit 0
Use

$ my_name andrew davison
Original parameters are andrew davison
Is your name andrew ?
Enter yes or no: y
Hi andrew, nice name
$
8. Useful Scripting Comm
ands

expr evaluates its argument as an expression:
ans=‘expr $x + 1‘


The usually operators are available:
 + - * / (integer divison) % (modulo)
 > >= < <=
 = (equality) !=
 | (or) & (and)

continued
printf

Bash supports printf, as a more flexible echo
 printf "format string" parameter1 ...


Very similar to printf() in C
 main restriction is no support for floats
 only integers are supported in the shell


$ printf "%s %d\t%s\n" "hi there" 15 students
hi there 15 students
$
9. Here Documents

A here document allows input to be
passed into a command from within
a script
 the command thinks the input is co
ming from a file or input stream


A here document begins with <<LABE
L, and ends with LABEL
 LABEL can be anything

$ cat here1
#!/bin/sh

cat <<!FUNKY!
hello
this is a here the here
document document
!FUNKY!


$ here1
hello
this is a here
document
$
10. Debugging

Various options can be set when inv
oking a shell script to help with deb
ugging.


There are two ways to set the optio
ns:
 from the command line when calling
the script
 from within the script by using set

continued
Command Line Set

sh -n script set -n
 do not execute the script, only parse it


sh -v script set -v
 echoes commands after executing them


sh -x script set -u
 warns when an undefined variable is used
11. More Information

The Bourne Shell:
 Ch. 10, Sobell


Bourne/Bash:
 Beginning Linux Programming
Neil Matthew and Rick Stones
Chapter 2