1. Basic Operations
   1.1. File Operations
   1.2. Text Operations
   1.3. Directory Operations
   1.4. SSH, System Info & Network Operations
   1.5. Process Monitoring Operations (TODO)
2. Basic Shell Programming
   2.1. Variables
   2.3. String Substitution
   2.4. Functions
   2.5. Conditionals
   2.6. Loops
3. Tricks
4. Debugging

Displays all environment variables and if you want to get detail of specific variable then use echo $VARIABLE_NAME

export

Example:

$ export SHELL=/bin/zsh AWS_HOME=/Users/adnanadnan/.aws LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 LESS=-R $ echo $SHELL /usr/bin/zsh

Whereis search for executables, source files, and manual pages using a database built by system automatically.

whereis name

Example:

$ whereis php /usr/bin/php

which search for executables in the directories specified by the environment variable PATH. This command will prints full path of the executable(s).

which program_name 

Example:

$ which php /c/xampp/php/php

Clears content on window

Lists your files. It has a lot of options like -l lists files in 'long format', which contains the exact size of the file, who owns the file and who has the right to look at it, and when it was last modified. -a lists all files, including hidden files. For more information on this command check this link

ls option

Example:

$ ls -al rwxr-xr-x 33 adnan staff 1122 Mar 27 18:44 . drwxrwxrwx 60 adnan staff 2040 Mar 21 15:06 .. -rw-r--r--@ 1 adnan staff 14340 Mar 23 15:05 .DS_Store -rw-r--r-- 1 adnan staff 157 Mar 25 18:08 .bumpversion.cfg -rw-r--r-- 1 adnan staff 6515 Mar 25 18:08 .config.ini -rw-r--r-- 1 adnan staff 5805 Mar 27 18:44 .config.override.ini drwxr-xr-x 17 adnan staff 578 Mar 27 23:36 .git -rwxr-xr-x 1 adnan staff 2702 Mar 25 18:08 .gitignore 

Creates or updates your file

touch filename

Example:

$ touch trick.md

It can be used for the following purposes under UNIX or Linux

• Display text files on screen
• Copy text files
• Combine text files
• Create new text files

cat filename cat file1 file2 cat file1 file2 > newcombinedfile

Shows the first part of a file (move with space and type q to quit)

more filename

Outputs the first 10 lines of file

head filename

Outputs the last 10 lines of file. Use -f to output appended data as the file grows

tail filename

Moves a file from one location to other

mv filename1 filename2

Where filename1 is the source path to the file and filename2 is the destination path to the file.

Copies a file from one location to other

cp filename1 filename2

Where filename1 is the source path to the file and filename2 is the destination path to the file.

Removes a file. But if you will apply this command on a directory directory, it will gives you an error rm: directory: is a directory So in order to remove directory you have to pass -rf to remove all the content of the directory recursively

rm filename

Compares files, and shows where they differ

diff filename1 filename2

Lets you change the read, write, and execute permissions on your files

chmod -options filename

Compresses files

gzip filename

Un-compresses files compressed by gzip

gunzip filename

Lets you look at gzipped file without actually having to gunzip it

gzcat filename

Print the file

lpr filename

Check out the printer queue

lpq

Example:

$ lpq Rank Owner Job File(s) Total Size active adnanad 59 demo 399360 bytes 1st adnanad 60 (stdin) 0 bytes

Remove something from the printer queue

lprm jobnumber

Awk is most usefull command for handling text files. It operates on entire file line by line. By default it uses whitespace to separate the fields. The most common syntax for awk command is

awk '/search_pattern/ { action_to_take_if_pattern_matches; }' file_to_parse

Lets take following file /etc/passwd. Here's the sample data that this file contains.

root:x:0:0:root:/root:/usr/bin/zsh daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x:2:2:bin:/bin:/usr/sbin/nologin sys:x:3:3:sys:/dev:/usr/sbin/nologin sync:x:4:65534:sync:/bin:/bin/sync 

So now lets get only username from this file. Where -F specfies that on which base we are going to separate the fields. In our case it's :. { print $1 } means print out the first matching field.

awk -F':' '{ print $1 }' /etc/passwd

After running the above command you will get following output.

root daemon bin sys sync 

For more detail on how to use awk, check following link.

Looks for text inside files. You can use grep to search for lines of text that match one or many regular expressions, and outputs only the matching lines.

grep pattern filename

Example:

$ grep admin /etc/passwd _kadmin_admin:*:218:-2:Kerberos Admin Service:/var/empty:/usr/bin/false _kadmin_changepw:*:219:-2:Kerberos Change Password Service:/var/empty:/usr/bin/false _krb_kadmin:*:231:-2:Open Directory Kerberos Admin Service:/var/empty:/usr/bin/false

You can also force grep to ignore word case by using -i option. Also -r can be used to search all files under the specified directory like

$ grep -r admin /etc/

And -w to search for words only. For more detail on grep, check following link.

Tells you how many lines, words and characters there are in a file

wc filename

Example:

$ wc demo.txt 7459 15915 398400 demo.txt

Where 7459 is lines, 15915 is words and 398400 is characters.

stream editor for filtering and transforming text

example.txt

Hello This is a Test 1 2 3 4

replace all spaces with hyphens

sed 's/ /-/g' example.txt

Hello-This-is-a-Test-1-2-3-4

replace all digits with "d"

sed 's/[0-9]/d/g' example.txt

Hello This is a Test d d d d

sort lines of text files

example.txt

f b c g a e d

sort example.txt

sort example.txt

a b c d e f g

randomize a sorted example.txt

sort example.txt | sort -R

b f a c d g e

report or omit repeated lines

example.txt

a a b a b c d c

show only unique lines of example.txt (first you need to sort it, otherwise it won't see the overlap)

sort example.txt | uniq

a b c d

show the unique items for each line, and tell me how many instances it found

sort example.txt | uniq -c

 3 a 2 b 2 c 1 d

remove sections from each line of files

example.txt

red riding hood went to the park to play

show me columns 2 , 7 , and 9 with a space as a separator

cut -d " " -f2,7,9 example.txt

riding park play

display a line of text

display "Hello World"

echo Hello World

Hello World

display "Hello World" with newlines between words

echo -ne "Hello\nWorld\n"

Hello World

simple optimal text formatter

example: example.txt (1 line)

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

output the lines of example.txt to 20 character width

cat example.txt | fmt -w 20

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

translate or delete characters

example.txt

Hello World Foo Bar Baz!

take all lower case letters and make them upper case

cat example.txt | tr 'a-z' 'A-Z' 

HELLO WORLD FOO BAR BAZ!

take all spaces and make them into newlines

cat example.txt | tr ' ' '\n'

Hello World Foo Bar Baz!

number lines of files

example.txt

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

show example.txt with line numbers

nl -s". " example.txt 

 1. Lorem ipsum 2. dolor sit amet, 3. consetetur 4. sadipscing elitr, 5. sed diam nonumy 6. eirmod tempor 7. invidunt ut labore 8. et dolore magna 9. aliquyam erat, sed 10. diam voluptua. At 11. vero eos et 12. accusam et justo 13. duo dolores et ea 14. rebum. Stet clita 15. kasd gubergren, 16. no sea takimata 17. sanctus est Lorem 18. ipsum dolor sit 19. amet.

print lines matching a pattern - Extended Expression (alias for: 'grep -E')

example.txt

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

display lines that have either "Lorem" or "dolor" in them.

egrep '(Lorem|dolor)' example.txt or grep -E '(Lorem|dolor)' example.txt

Lorem ipsum dolor sit amet, et dolore magna duo dolores et ea sanctus est Lorem ipsum dolor sit

print lines matching a pattern - FIXED pattern matching (alias for: 'grep -F')

example.txt

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor foo (Lorem|dolor) invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

Find the exact string '(Lorem|doloar)' in example.txt

fgrep '(Lorem|dolor)' example.txt or grep -F '(Lorem|dolor)' example.txt

foo (Lorem|dolor) 

Makes a new directory

mkdir dirname

Moves you from one directory to other. If you just run

$ cd

Then it will moves you to home. Also this command accepts an optional dirname, which if provided will moves you to that directory.

cd dirname

Tells you in which directory you currently are

pwd

ssh (SSH client) is a program for logging into a remote machine and for executing commands on a remote machine.

ssh user@host

This command also accepts an option -p that can to used to connect to specific port.

ssh -p port user@host

Return current logged in username

Allows the current logged user to change his password

Shows what your disk quota is

quota -v

Shows the current date and time

Shows the month's calendar

Shows current uptime

Displays who is online

Displays information about user

finger username

Shows kernel information

uname -a

Shows the manual for specified command

man command

Shows disk usage

Shows the disk usage of the files and directories in filename (du -s give only a total)

du filename

Lists your last logins of specified user

last yourUsername

Lists your processes

ps -u yourusername

Kills (ends) the processes with the ID you gave

kill PID

Kill all processes with the name

killall processname

Displays your currently active processes

Lists stopped or background jobs ; resume a stopped job in the background

Brings the most recent job in the foreground.

Pings host and outputs results

ping host

Gets whois information for domain

whois domain

Gets DNS information for domain

dig domain

Downloads file

wget file

Transfer files between a local host and a remote host or between two remote hosts.

copy from local host to remote host

scp source_file user@host:directory/target_file

copy from remote host to local host

scp user@host:directory/source_file target_file scp -r user@host:directory/source_folder farget_folder

This command also accepts an option -P that can to used to connect to specific port.

scp -P port user@host:directory/source_file target_file

This is first line that you will in bash script files called shebang. The shebang line in any script determines the script's ability to be executed like an standalone executable without typing sh, bash, python, php etc beforehand in the terminal.

#!/bin/bash

Creating variable in bash is similar to other language. There are no data types. A variable in bash can contain a number, a character, a string of characters. You have no need to declare a variable, just assigning a value to its reference will create it.

Example:

str="hello world"

The above line creates a variable str and assigns "hello world" to it. Then the value of variable is retrieved by putting the $ in the beginning of variable name.

Example:

echo $str # hello world

Also like other languages bash has also arrays. An array is variable containing multiple values. There's no maximum limit on the size of array. Array in bash are zero based. The first element is indexed with element 0. There are several ways for creating arrays in bash. Which are given below.

Examples:

array[0] = val array[1] = val array[2] = val array=([2]=val [0]=val [1]=val) array(val val val)

To display a value at specific index use following syntax

${array[i]} # where i is the index

One thing to note that if no index is supplied, array element 0 is assumed. To find out how many values there are in the array check following syntax

${#array[@]}

Bash has also support for the ternary conditions. Check some examples below.

${varname:-word} # if varname exists and isn't null, return its value; otherwise return word ${varname:=word} # if varname exists and isn't null, return its value; otherwise set it word and then return its value ${varname:+word} # if varname exists and isn't null, return word; otherwise return null ${varname:offset:length} # performs substring expansion. It returns the substring of $varname starting at offset and up to length characters

Check some of the syntax on how to manipulate strings

${variable#pattern} # if the pattern matches the beginning of the variable's value, delete the shortest part that matches and return the rest ${variable##pattern} # if the pattern matches the beginning of the variable's value, delete the longest part that matches and return the rest ${variable%pattern} # if the pattern matches the end of the variable's value, delete the shortest part that matches and return the rest ${variable%%pattern} # if the pattern matches the end of the variable's value, delete the longest part that matches and return the rest ${variable/pattern/string} # the longest match to pattern in variable is replaced by string. Only the first match is replaced ${variable//pattern/string} # the longest match to pattern in variable is replaced by string. All matches are replaced ${#varname} # returns the length of the value of the variable as a character string

As in almost any programming language, you can use functions to group pieces of code in a more logical way or practice the divine art of recursion. Declaring a function is just a matter of writing function my_func { my_code }. Calling a function is just like calling another program, you just write its name.

functname() { shell commands }

Example:

#!/bin/bash function hello { echo world! } hello function say { echo $1 } say "hello world!"

When you will run above example the hello function will output "world!". The above two functions hello and say are identical. The main difference is function say. This function, prints the first argument it receives. Arguments, within funtions, are treated in the same manner as arguments given to the script.

The conditional statement in bash is similar to other programming languages. Conditions have many form like the most basic form is if expression then statement where statement is only executed if expression is true.

if [expression]; then will execute only if expression is true else will execute if expression is false fi

Sometime if conditions becoming confusing so you can write the same condition using the case statements.

case expression in pattern1 ) statements ;; pattern2 ) statements ;; ... esac

Expression Examples:

statement1 && statement2 # both statements are true statement1 || statement2 # one of the statement is true str1=str2 # str1 matches str2 str1!=str2 # str1 does not match str2 str1<str2 # str1 is less than str2 str1>str2 # str1 is greater than str2 -n str1 # str1 is not null (has length greater than 0) -z str1 # str1 is null (has length 0) -a file # file exists -d file # file exists and is a directory -e file # file exists; same -a -f file # file exists and is a regular file (i.e., not a directory or other special type of file) -r file # you have read permission -r file # file exists and is not empty -w file # your have write permission -x file # you have execute permission on file, or directory search permission if it is a directory -N file # file was modified since it was last read -O file # you own file -G file # file's group ID matches yours (or one of yours, if you are in multiple groups) file1 -nt file2 # file1 is newer than file2 file1 -ot file2 # file1 is older than file2 -lt # less than -le # less than or equal -eq # equal -ge # greater than or equal -gt # greater than -ne # not equal

There are three types of loops in bash. for, while and until.

Different for Syntax:

for x := 1 to 10 do begin statements end for name [in list] do statements that can use $name done for (( initialisation ; ending condition ; update )) do statements... done

while Syntax:

while condition; do statements done

until Syntax:

until condition; do statements done

Open bash_profile by running following command nano ~/.bash_profile

alias dockerlogin='ssh www-data@adnan.local -p2222' # add your alias in .bash_profile

nano ~/.bashrc

export hotellogs="/workspace/hotel-api/storage/logs"

source ~/.bashrc cd hotellogs

You can easily debug the bash script by passing different options to bash command. For example -n will not run commands and check for syntax errors only. -v echo commands before running them. -x echo commands after command-line processing.

bash -n scriptname bash -v scriptname bash -x scriptname

• Report issues
• Open pull request with improvements
• Spread the word