Unit 4 scripting and the shell

Scripting and the Shell Prepared By Prof. Bhushan Pawar www.bhushanpawar.com

SHELL BASICS • All major shells in the sh(including Bourne shell(Bash),ksh(KornShell). • Command for editing :- – emacs commands – <Control-E> goes to the end of the line – <Control-A> to the beginning. – <Control-P> steps backward through recently executed commands and recalls them for editing. – <Control-R> searches incrementally through your history to find old commands. – http://www.computerhope.com/unix/uemacs.htm Prof.Bhushan Pawar www.bhushanpawar.com

SHELL BASICS (Continue…) • If you like vi, put your shell’s command-line editing into vi mode like this: $ set -o vi $ set -o emacs Prof.Bhushan Pawar www.bhushanpawar.com

Pipes and redirection • Every process has at least three communication channels available to it: – “standard input” (STDIN) – “standard output”(STDOUT) – “standard error” (STDERR). Prof.Bhushan Pawar www.bhushanpawar.com

Pipes and redirection(Continue…) • UNIX has a unified I/O model in which each channel is named with a small integer called a (File Descriptor). • The exact number assigned to a channel is not usually significant, but STDIN, STDOUT, and STDERR are guaranteed to correspond to file descriptors 0, 1, and 2, so it’s safe to refer to these channels by number. In the context of an interactive terminal window, STDIN normally reads from the keyboard and both STDOUT and STDERR write their output to the screen. Prof.Bhushan Pawar www.bhushanpawar.com

Pipes and redirection(Continue…) • The shell interprets the symbols <, >, and >> as instructions to reroute a command’s input or output to or from a file. A < symbol connects the command’s STDIN to the contents of an existing file. The > and >> symbols redirect STDOUT. • > replaces the file’s existing contents, and >> appends to them. Prof.Bhushan Pawar www.bhushanpawar.com

Pipes and redirection(Continue…) • $ echo "This is a test message." > /tmp/mymessage -it stores a single line in the file /tmp/mymessage, creating the file if necessary. • To connect the STDOUT of one command to the STDIN of another, use the | symbol, commonly known as a pipe. – E.g. $ ps -ef | grep httpd (example runs ps to generate a list of processes and pipes it through the grep command to select lines that contain the word httpd.) Prof.Bhushan Pawar www.bhushanpawar.com

Variables and quoting • Variable names are unmarked in assignments but prefixed with a dollar sign ($)when their values are referenced. – E.g. $ etcdir='/etc‘ $ echo $etcdir Prof.Bhushan Pawar www.bhushanpawar.com

Common filter commands • cut: separate lines into fields – Syntax:- cut -c4 file.txt – Explanation : The above cut command prints the fourth character in each line of the file – ( http://www.folkstalk.com/2012/02/cut- command-in-unix-linux-examples.html ) Prof.Bhushan Pawar www.bhushanpawar.com

sort command • Sort: Sort command in unix or linux system is used to order the elements or text. Sort command has the capability of sorting numerical values and strings. – Syntax :- sort [options] filename – (http://www.folkstalk.com/2012/08/sort- command-examples-in-unix-linux.html ) Prof.Bhushan Pawar www.bhushanpawar.com

sort options • -b : Ignores leading spaces in each line • -d : Uses dictionary sort order. Conisders only spaces and alphanumeric characters in sorting • -f : Uses case insensitive sorting. • -M : Sorts based on months. Considers only first 3 letters as month. Eg: JAN, FEB • -n : Uses numeric sorting • -R : Sorts the input file randomly. • -r : Reverse order sorting • -k : Sorts file based on the data in the specified field positions. • -u : Suppresses duplicate lines • -t : input field separator Prof.Bhushan Pawar www.bhushanpawar.com

Uniq command • Uniq command in unix or linux system is used to suppress the duplicate lines from a file • Syntax:- uniq [option] filename – (http://www.folkstalk.com/search?q=uniq+comm and+in+%3Blinux ) Prof.Bhushan Pawar www.bhushanpawar.com

uniq options • c : Count of occurrence of each line. • d : Prints only duplicate lines. • D : Print all duplicate lines • f : Avoid comparing first N fields. • i : Ignore case when comparing. • s : Avoid comparing first N characters. • u : Prints only unique lines. • w : Compare no more than N characters in lines Prof.Bhushan Pawar www.bhushanpawar.com

wc command • wc command in unix or linux is used to find the number of lines, words and characters in a file. • Syntax :- wc [options] filenames • ( http://www.folkstalk.com/2012/07/wc- command-examples-in-unix-linux.html#more ) Prof.Bhushan Pawar www.bhushanpawar.com

wc options • -l : Prints the number of lines in a file. • -w : prints the number of words in a file. • -c : Displays the count of bytes in a file. • -m : prints the count of characters from a file. • -L : prints only the length of the longest line in a file. Prof.Bhushan Pawar www.bhushanpawar.com

other commands • tee • head • tail • grep Prof.Bhushan Pawar www.bhushanpawar.com

tee command • tee :- tee command writes to the STDOUT, and to a file at a time . • The following command writes the output only to the file and not to the screen. – $ ls > filename • The following command (with the help of tee command) writes the output both to the screen (stdout) and to the file. – $ ls | tee filename (http://linux.101hacks.com/unix/tee-command- examples/ ) Prof.Bhushan Pawar www.bhushanpawar.com

head command • The head command reads the first few lines of any text given to it as an input and writes them to standard output (which, by default, is the display screen). • head's basic syntax is: – head [options] [file(s)] ( http://www.linfo.org/head.html ) Prof.Bhushan Pawar www.bhushanpawar.com

Options of head command • -c, --bytes=[-]N – print the first N bytes of each file; with the leading '-', print all but the last N bytes of each file • -n, --lines=[-]N – print the first N lines instead of the first 10; with the leading '-', print all but the last N lines of each file • -q, --quiet, --silent – never print headers giving file names • -v, --verbose – always print headers giving file names Prof.Bhushan Pawar www.bhushanpawar.com

tail command • tail :-output the last part of files • Syntax :- – tail [OPTION]... [FILE]... (http://linux.about.com/library/cmd/blcmdl1_tail.ht m ) Prof.Bhushan Pawar www.bhushanpawar.com

Options of tail command • -c, --bytes=N – output the last N bytes • -f, --follow[={name|descriptor}] – output appended data as the file grows; -f, -- follow, and --follow=descriptor are equivalent • -F – same as --follow=name --retry • -n, --lines=N – output the last N lines, instead of the last 10 Prof.Bhushan Pawar www.bhushanpawar.com

grep command • grep :- The grep command allows you to search one file or multiple files for lines that contain a pattern. Exit status is 0 if matches were found, 1 if no matches were found, and 2 if errors occurred. • Syntax – grep [options] pattern [files] (http://www.techonthenet.com/unix/basic/grep.ph p ) Prof.Bhushan Pawar www.bhushanpawar.com

Options of grep command Prof.Bhushan Pawar www.bhushanpawar.com

Bash Scripting • Comment start with hash mark (#) and continue to the end of line. #!/bin/bash echo "Hello, world!“ – The first line is known as the “shebang” statement and declares the text file to be a script for interpretation by /bin/bash. Prof.Bhushan Pawar www.bhushanpawar.com

Bash Scripting (Continue…) • The kernel looks for this syntax when deciding how to execute the file. From the perspective of the shell spawned to execute the script, the shebang line is just a comment. • If bash were in a different location, we need to adjust this line Prof.Bhushan Pawar www.bhushanpawar.com

Bash Scripting (Continue…) • To prepare the file for running, just turn on its execute bit $ chmod +x helloworld $ ./helloworld – Hello, world! • You can also invoke the shell as an interpreter directly: $ bash helloworld – Hello, world! $ source helloworld – Hello, world! • The first command runs helloworld in a new instance of bash, and the second makes your existing login shell read and execute the contents of the file Prof.Bhushan Pawar www.bhushanpawar.com

Regular Expression • Regular expressions are powerful, but they cannot recognize all possible grammars. • They are so common that the name is usually shortened to “regex.” Prof.Bhushan Pawar www.bhushanpawar.com

The matching process • Code that evaluates a regular expression attempts to match a single given text string to a single given pattern. The “text string” to match can be very long and can contain embedded newlines. • For the matcher to declare success, the entire search pattern must match a contiguous section of the search text. However, the pattern can match at any position. • After a successful match, the evaluator returns the text of the match along with a list of matches for any specially delimited subsections of the pattern. Prof.Bhushan Pawar www.bhushanpawar.com

Literal characters • In general, characters in a regular expression match themselves. So the pattern I am the walrus • Matches the string “I am the walrus” and that string only. Since it can match anywhere in the search text, the pattern can be successfully matched to the string “I am the egg man. I am the walrus. Koo koo ka-choo!” However, the actual match is limited to the “I am the walrus” portion. Matching is case sensitive. Prof.Bhushan Pawar www.bhushanpawar.com

Special characters http://perldoc.perl.org/perlrecharclass.htmlProf.Bhushan Pawar www.bhushanpawar.com

Example I am the (walrus|egg man). matches either “I am the walrus.” or “I am the egg man.”. This example also demonstrates escaping of special characters (here, the dot). The pattern (I am the (walrus|egg man). ?){1,2} matches any of the following: • I am the walrus. • I am the egg man. • I am the walrus. I am the egg man. • I am the egg man. I am the walrus. Prof.Bhushan Pawar www.bhushanpawar.com

PERL PROGRAMMING • Perl created by Larry Wall • More power than bash • Perl does not impose much stylistic discipline on developers, so Perl code written without regard for readability can be cryptic. Perl has been accused of being a write-only language • Perl’s catch phrase is that “there’s more than one way to do it.” Prof.Bhushan Pawar www.bhushanpawar.com

PERL PROGRAMMING(Continue…) • Perl statements are separated by semicolons. Comments start with a hash mark (#) and continue to the end of the line. Blocks of statements are enclosed in curly braces. Prof.Bhushan Pawar www.bhushanpawar.com

Program #!/usr/bin/perl print "Hello, world!n"; • As with bash programs, you must either chmod +x the executable file or invoke the Perl interpreter directly. $ chmod +x helloworld $ ./helloworld Hello, world! Prof.Bhushan Pawar www.bhushanpawar.com

Variables and arrays • three fundamental data types – Scalars (that is, unitary values such as numbers and strings) – Arrays – Hashes (also known as associative Arrays) • Scalar variables start with $, Array variables start with @, and Hash variables start with % Prof.Bhushan Pawar www.bhushanpawar.com

Variables and arrays(Continue…) • In Perl, the terms “list” and “array” are often used interchangeably, but it’s more accurate to say that a list is a series of values and an array is a variable that can hold such a list. • The individual elements of an array are scalars, so like ordinary scalar variables, their names begin with $. Array subscripting begins at zero, and the index of the highest element in array @a is $#a. • The array @ARGV contains the script’s command line arguments. Prof.Bhushan Pawar www.bhushanpawar.com

Program #!/usr/bin/perl @items = ("socks", "shoes", "shorts"); printf "There are %d articles of clothing.n",$#items + 1; print "Put on ${items[2]} first, then ", join(" and ", @items[0,1]), ".n"; • The output: $ perl clothes There are 3 articles of clothing. Put on shorts first, then socks and shoes. Prof.Bhushan Pawar www.bhushanpawar.com

Explanation of the program • Array and string literals – Perl does not strictly require that all strings be quoted so @items = (socks, shoes, shorts); is also same • Function calls – Both print and printf accept an arbitrary number of arguments, and the arguments are separated by commas Prof.Bhushan Pawar www.bhushanpawar.com

Explanation of the program • join receives three string arguments: “ and ”, “socks”, and “shoes”. It concatenates its second and subsequent arguments, inserting a copy of the first argument between each pair. The result is “socks and shoes”. Prof.Bhushan Pawar www.bhushanpawar.com

Explanation of the program • Type conversions in expressions The magic is in the + operator, which always implies arithmetic. It converts its arguments to numbers and produces a numeric result. Similarly, the dot operator (.), which concatenates strings, converts its operands as needed: "2" . (12 ** 2) yields “2144”. Prof.Bhushan Pawar www.bhushanpawar.com

Regular expressions in Perl • Regular expressions in Perl by “binding” strings to regex operations with the =~ operator. e.g if ($text =~ m/ab+c/) { } • checks to see whether the string stored in $text matches the regular expression ab+c. • To operate on the default string, $_ , you can simply omit the variable name and binding operator. • In fact, you can omit the m, too, since the operation defaults to matching Prof.Bhushan Pawar www.bhushanpawar.com

Program #!/usr/bin/perl $names = "huey dewey louie"; $regex = '(w+)s+(w+)s+(w+)'; if ($names =~ m/$regex/) { print "1st name is $1.n2nd name is $2.n3rd name is$3.n"; $names =~ s/$regex/2 1/; print "New names are "${names}".n"; } else { print qq{"$names" did not match "$regex".n}; } (http://www.tutorialspoint.com/perl/perl_qq.htm) Prof.Bhushan Pawar www.bhushanpawar.com

Output The output: $ perl testregex 1st name is huey. 2nd name is dewey. 3rd name is louie. New names are "dewey huey". Prof.Bhushan Pawar www.bhushanpawar.com

Input and output • When you open a file for reading or writing, you define a “filehandle” to identify the channel. • INFILE & OUTFILE are the filehandlers. • The while loop condition is <INFILE> Prof.Bhushan Pawar www.bhushanpawar.com

Program #!/usr/bin/perl open(INFILE, "</etc/passwd") or die "Couldn’t open /etc/passwd"; open(OUTFILE, ">/tmp/passwd") or die "Couldn’t open /tmp/passwd"; while (<INFILE>) { ($name, $pw, $uid, $gid, $gecos, $path, $sh) = split /:/; print OUTFILE "$uidt$namen"; } Prof.Bhushan Pawar www.bhushanpawar.com

Continue… Prof.Bhushan Pawar www.bhushanpawar.com

PYTHON SCRIPTING • Python was created by Guido van Rossum • It’s easier to code and more readable than Perl. Python offers a simple-to-understand syntax that is easy to follow even if you didn’t develop the code. If you’re tired of remembering which comparison operators to use, you’ll appreciate Python’s unified approach. • Python also offers additional data types that some system administrators find useful. Prof.Bhushan Pawar www.bhushanpawar.com

Simple Program #!/usr/bin/python print "Hello, world!“ Invoke the python interpreter directly: $ chmod +x helloworld $ ./helloworld Hello, world! Prof.Bhushan Pawar www.bhushanpawar.com

sys — (System-specific parameters and functions) • import sys (https://docs.python.org/2/library/sys.html) Prof.Bhushan Pawar www.bhushanpawar.com

Program #!/usr/bin/python name = 'Gwen' rating = 10 characters = [ 'SpongeBob', 'Patrick', 'Squidward' ] elements = ( 'lithium', 'carbon', 'boron' ) print "name:t%snrating:t%d" % (name, rating) print "characters:t%s" % characters print "elements:t%s" % (elements, ) This example produces the following output: $ python objects name: Gwen rating: 10 characters: ['SpongeBob', 'Patrick', 'Squidward'] elements: ('lithium', 'carbon', 'boron') Prof.Bhushan Pawar www.bhushanpawar.com

Loops • The fragment below uses a for…in construct to iterate through the range 1 to 10. for counter in range(1, 10): print counter, Output: 1 2 3 4 5 6 7 8 9 Prof.Bhushan Pawar www.bhushanpawar.com

Continue… • Both for and while loops can have else clauses at the end. The else clause is executed only if the loop terminates normally, as opposed to exiting through a break statement. • This feature may initially seem counterintuitive, but it handles certain use cases quite elegantly. Prof.Bhushan Pawar www.bhushanpawar.com

Any Question??? • If you having any doubt then you can freely ask me question on bhushan.pawar@mescoepune.org Or contact me on (+91)-7588318728 Prof.Bhushan Pawar www.bhushanpawar.com

Unit 4 scripting and the shell

More Related Content

What's hot

Viewers also liked

Similar to Unit 4 scripting and the shell

Recently uploaded

Unit 4 scripting and the shell