Lesson 3 Working with Files in Linux

Linux System Fundamentals Lesson 3: Linux File Management Basics Sadia Bashir

Contents  Permissions  File permission  Directory permission  Essential File Management Tasks  Create/Move/Copy/Rename/Delete  Using Text File Editors  Nano, Vim, Sublime, Emacs  Working with Nano and Vim  More File operations  stat, touch, file, wc, du, df, diff  Linux Wildcards & Pattern matching  Finding Files  locate, find, whereis, which, whatis  I/O Redirection, Pipes and Filters  Using Common Text Manipulation Tools (tail, head, tr, tee)  Working with grep, egrep

Permissions  Permissions are the rights to act on a file/directory  Owners assign permission on every file/directory  Permissions are important to create a secure environment  What they are :  r : only open and view file/directory contents  w : modify file/directory contents  x : execute or run the file if it is a program or script  - : no read, write or execute permissions set on file/directory  = : make it the only permission  Permissions are specified for three sets of people[identities]:  owner [u] - a single person who owns the file. (typically the person who created the file but ownership may be granted to some one else by certain users)  group [g] - every file belongs to a single group  others/all [o] [a] - everyone else who is not in the group or the owner Note: Linux/UNIX OS has ability to multi-task and to have multiple users, who can have access to system simultaneously, a method to protect users from each other is required. This is where permissions come to play.

Format of setting Permissions Octal Decimal Permission Representation 000 0(0+0+0) No Permission - - - 001 1(0+0+1) Execute - - x 010 2(0+2+0) Write - w - 011 3(0+2+1) Write + Execute - w x 100 4(4+0+0) Read r - - 101 5(4+0+1) Read + Execute r - x 110 6(4+2+0) Read + Write r w - 111 7(4+2+1) Read + Write + Execute r w x  drwxr-xr-x -rw-r--r--  Difference between a Directory and File Permissions:  The first character identifies the file type : dash ( - ) indicates a normal file and d denotes it is a directory.

Working with Permissions  View Permissions – ls -l [path/to/directory/file]  ls -l /home/juno/linuxtutorial/test.txt  ls -ld /home/juno/linuxtutorial  Change permissions – chmod [permissions] [path]  chmod : change file mode  chmod has permission arguments that are made up of 3 components:  changing the permission for [ugoa] - user (or owner), group, others, all  granting or revoking the permission - indicated with either a plus ( + ) or minus ( - )  Which permission are we setting? - read ( r ), write ( w ) or execute ( x )  Examples:  chmod g+x test.txt  chmod u-w test.txt  chmod ugo+wx test.txt  chmod 777 test.txt  chmd 240 test.txt

Permissions for Directories  The same series of permissions may be used for directories but they have a slightly different behavior:  r - user has ability to read directory contents (i.e., do an ls)  w - user has ability to write into the directory (i.e., create files and directories)  x - user has ability to enter that directory (i.e., cd)

Essential File Management Tasks Creating a new Directory using CLI  mkdir : command to creates directories as a superuser  Syntax:  mkdir [option] [Directory_Name]  sudo mkdir mydir (creates a new directory whose parent is the current directory) OR  mkdir [option] [/home/path/to/Directory_Name]  sudo mkdir /home/juno/mydir  Options:  -m : Creates the mydir directory, and set its permissions  sudo –m a=rwx mydir  -p : mkdir will first create the parent directory /home/juno/a/b if it does not already exists  mkdir -p /home/juno/a/b/c  rmdir : Removes empty directories – sudo rmdir /home/juno/a

Creating a new File using CLI  touch : touch sample.txt  > sample.txt  cat > sample.txt

Naming [File/Directory] Conventions  A directory may have the same name as one of the items it contains  Different directories may contain files with same name  Some characters have special meanings to Unix hence it is best to avoid using these characters in file names:  / " ' * | ! ? ~ $ < >  Unix is case-sensitive : so each of these is a unique file: myfile, Myfile, myFile, and MYFILE

Rename/copy/move/delete a File/Directory  RENAME : mv FILE_NAME.txt/DIRECTORY [oldname] [newname]  FILE : mv sample.txt newsample.txt  DIRECTORY : mv sample newsample  COPY : cp FILE_NAME.txt/DIRECTORY [source] [destination]  FILE : cp sample.txt /home/juno/sample /home/juno/practice  DIRECTORY : cp testd/home/juno/sample /home/juno/practice  MOVE : mv FILE_NAME.txt/DIRECTORY [source] [destination]  FILE : mv sample.txt /home/juno/sample /home/juno/practice  DIRECTORY : mv testd/home/juno/sample /home/juno/practice  DELETE : rm FILE_NAME.txt - rm -rf DIRECTORY  FILE : rm sample.txt  DIRECTORY : rm -rf test - -r : recursively  Options : [-f: force, -i: iteratively ask before removing each sub-directory]

1. Using Text File Editors - Nano  Nano:  -c : open file showing number of lines : nano -c test.txt  File Controls  nano FILE.txt : Open or create the file  Ctrl-o Y : Enter Save changes  Ctrl-r Alt-f : Open a new file with a new buffer  Alt-> : Switch to the next file buffer  Alt-< : Switch to the previous file buffer  Ctrl-x : Quit nano  Navigating through file contents  Ctrl-a : Move to the beginning of the current line  Ctrl-e : Move to the end of the current line  Ctrl-v : Move down one page  Ctrl-y : Move up one page  Alt- : Go to the beginning of the file  Alt-/ : Go to the end of the file  Alt-g : Go to a target line number  Alt-a Alt-} : Select a block and indent the block  Alt-a Alt-{ : Select a block and outdent the block

1. Using Text File Editors - Nano (contd…)  Copy and Paste  Alt-a : To select a block for copy or cut operation, do Alt-a again to unselect  Alt-a Alt-^ : Copy a highlighted block to the clipboard  Alt-a Ctrl-k : Cut a highlighted block to the clipboard  Ctrl-k : Cut from the current cursor position to the end of the current line  Ctrl-u : Paste the contents from the clipboard at the current cursor position  Search and Replace  Ctrl-w : Search for a target string  Alt-w : Repeat the last search  Alt-r : Search and replace

2. Using Text File Editors - Vim  Vim:  CLI Options : vim -R text.txt  -R : Read-only mode (like "view")  -M : Modifications in text not allowed  -p and -o : open new tabs or windows at run time  + : start at end of file  +<lnum> : (+5) start at line <lnum>  File Control  vim test.txt : Open the file specified with vim in command window  vim File1.txt File2.txt : Open two files simultaneously  ESC :next : go to the next file  ESC :previous : go to the previous file  ESC :tabe : file3.txt - Open a new file while vim is already opened with one file, if the file doesn’t exist, it will create it  ESC :tabn : or ESC :tabp : go to the next file or previous file accordingly  ESC :wqa : to save works in all tabs and quit vim  ESC :bd : to close the file without quitting the vim  ESC :w backup.txt : save the current file to a new file with the file name “backup.txt”  ESC :w : Save changes  ESC :wq: Save the changes and exit  ESC :q! : Quit without save  Edit mode or Insert mode  ESC i : Enter insert mode and start editing

2. Using Text File Editors - Vim (contd…)  Line Control:  ESC 0 or ESC Home : move the cursor to the beginning of the current line  ESC $ or ESC End : move the cursor to the end of the current line  ESC :34 : go to line 34  ESC gg : go to the beginning of file  ESC G : go to the end of file  ESC ( : jump backward one sentence  ESC ) : jump forward one sentence  ESC dd : delete the current line  Cursor Control  h : Move Left  l : Move Right  j : Move Down  k : Move Up  Screen Control  ESC zt, zz, zb : move cursor to the Top, Middle, and Bottom of the screen respectively  ESC ctrl-f : Scroll down one screen page  ESC ctrl-b : Scroll up one screen page  Vim Undo and Redo  ESC u : Undo the changes  ESC Ctrl-r : Redo the changes

2. Using Text File Editors - Vim (contd…)  Search and Replace  ESC /sometext and enter : Search for the word specified.  n : To find the next occurrence  N : the previous occurrence  ESC :s/foo/bar/g : search and replace “foo” with “bar” in the current line  ESC :%s/foo/bar/gc : search and prompt to replace “foo” with “bar”  ESC :%s/foo/bar/g : search and replaces all “foo” with bar in the file  ESC :%s/[0-9]/a/g : find all digits and replaces it with “a”.  Copy and Paste  ESC v : move the cursor to select the text to cut or copy  Press d : to cut OR  Press y : to copy  Press p : Move cursor to the point to paste the text and Press p to paste it  Reveal hidden Characters  :set list  :set nolist  Help  ESC :help : open help page  ESC :q : exit help page

More File Operations  Viewing file timestamps : using touch command  Different types of timestamps are associated with files e.g., access timestamp, modification timestamp etc.  state test.txt  touch : command is used to change timestamps associated with a file :  touch -d "2013-01-10 10:00:07" -a test.txt :  -d : use specified time instead of current time  -a : change only the access time  touch -d "next saturday" -m test.txt - -m : changes file modification time  touch -d “5 hours ago" -a test.txt  Determine file types : using file command  file test.txt – view type of a specified file  file * : view type of all files in current directory  Print the number of new lines, words, and bytes in files: using wc command  wc test.txt OR wc -l -w -c test.txt where -l, -w, and -c represent line words and byte counts respectively

More File Operations (contd….)  Display disk usage of files and directories: using du command  du -h test.txt : -h : CLI option produces output in human readable format  du -s test.txt : -s : display the combined size of a directory and its subdirectories  du -S somedirectory : -S : display separate sizes  df -h test.txt : display the amount of disk space available on the file system containing a specific file or directory  df -h : shows disk usage for all the file systems  Compare two files: using diff command  diff : The command requires two filenames as arguments  diff FILE1 FILE2  %< lines from FILE1  %> lines from FILE2  %= lines common to FILE1 and FILE2  c : changed, d : delete, a : add  The command examines both files and produces output in a particular format to let the user know what changes are required to be made for the files to match.

Linux Wildcards & Pattern matching : *?[ ]  A wildcard is a character that can be used as a substitute for any class of characters to create a “pattern” to be used in a search in order to greatly increase the flexibility and efficiency of search  Star/asterisk (*) Wildcard : It represents zero characters, all single characters or any string, in other words, it represents everything  ls *.txt : list down all files having .txt extension  ls n* : list all files starting with lower case letter n  rm *enw* : delete all files in a current directory having e n or w as part of their name  file * : lists type of all file system objects in current directory  wildcards can be used at any point in the path  ls -lh /home/*/.bash_history : list .bash_history file in every users home directory  Question-mark (?) Wildcard : represents exactly a single character  ls *.???  file ?.*  ls ?i*  mv public_html/*.??g public_html/images/ : move all files of type either jpg/png into another directory

Linux Wildcards & Pattern matching (contd...)  Square Brackets Wildcard [] : represents a range of characters  file *[xyz]* : lists all objects in the current directory that have an x, y and/or z in them  ls *.[xyz]* : list all files that have an extension that begins with x, y or z  file [a-f]* : hyphen indicates a range inclusive of these two characters - this command would provide information about all of the objects in the current directory that begin with any letter from a through f  file *[0-9]* : provide information about every object in the current directory whose name includes at least one numeral  file [a-cx-z]* : specifies multiple sets of ranges - this command would return information about all objects whose name begin with the first three or the final three lower case letters  ls jones[0-9][0-9][0-9] : succession of square bracket wildcards – this command would display all filenames in the current directory that consist of jones followed by a three-digit number  ls [^a-k]* : caret ( ^ ) reverses the range, which means look for any character which is not one of the range

Finding Files - Locate  Locate : the quickest way to find the location of files/directories  locate [options] name(s)  When used without any options, locate displays every absolute pathname for which the user has access permission that contains any of the names of files and/or directories that are provided to it as arguments  locate sysctl.conf, locate updatedb.conf locate mlocate.db  Options :  -o : display all the output in one line  -c : get the count of number of matching entries  -i : perform case insensitive search  -e : check if the file is physically present in the system  -l <num> : restrict search results Note: The ‘locate’ command is fast as it searches for file paths in a database named “mlocate.db” rather than searching from the entire hard drive. This database is automatically updated on a regular basis by cron, which runs in background and updates this database file at a pre-configured frequency. Hence “updatedb” command is used to manually update mlocate.db file in order to get latest and reliable results.

Finding Files - Find  Find : The find command searches out live file-system for files that match the search criteria  The find command syntax is:  find /where/to/look/up criteria action OR  find /dir/path/look/up criteria action OR  find /dir/path/look/up -name "dir-name-here“ OR  find /dir/path/look/up -name "pattern“ OR  find /dir/path/look/up -name "dir-name-here" -print OR  find /dir/path/look/up -name "dir-name-here“ OR  find / -name "dir-name-here“  Searching through the root file system: sudo find / -name "apt“  Getting a detailed list of files/directories: sudo find / -name "apt" -ls  Listing directories only: find / -type d -name "apt" -ls  Performing a case insensitive search: find / -type d -iname "apt“  Finding a specified directory: find / -type d -name "project.images"  Finding with wildcards: find . -name pupp*  find / -maxdepth 2 -name passwd: Find file under root and one level down  find / -mindepth 3 -maxdepth 5 -name passwd: Find file between sub-directory level 2 and 4

Finding Files - Find (contd…)  Find file based on file permissions irrespective of other permissions  find . -perm -g=r -type f -exec ls -l {} ; : Find files which have read permission to group  find . -perm 040 -type f -exec ls -l {} ; : Find files which have read permission only to group [ search by octal ]  find . -perm g=r -type f -exec ls -l {} ; : Find files which have read permission only to group  Find empty files  find ~ -empty : find all empty files in home directory  find . -maxdepth 1 –empty : List all the empty files only in home directory  find . -maxdepth 1 -empty -not -name “.*“ : List only the non-hidden empty files only in the current directory  Find Files Based on file-type  find . -type d : Find all directories  find . -type f : Find normal files only  find . -type f -name ‘.*’ : Find hidden files  find -type d -name ‘.*’ : Find hidden directories  Find Files by Size : - means less than the give size, + means more than the given size, and no symbol means exact given size  find ~ -size +100M  find ~ -size -100M  find ~ -size 100M

Finding Files - whereis, which, whatis  whereis : command to locate the binary, source, and manual page files for a command  Usage: whereis [-options]  whereis -b whereis : Locate binaries for a command  whereis -m whereis : Locate man pages for a command  whereis -B /bin -f chmod : Locate binaries in a specified path  whereis -M /usr/share/man/man1 -f open : Locate man pages with limited scope  which : simple command to locate executables in the system.  It allows user to pass several command names as arguments to get their paths in the system,  searches the path of executable in system paths set in $PATH environment variable  Usage: which [-option]  which ls open chmod grep  which -a echo : display all paths  Whatis : displays brief information for command that matches the argument passed to it  whatis [-options] : whatis ls, whatis cat, whatis cd etc.

I/O Redirection >, <, >>, <<  Redirecting the source of an input to a command and the destination of the output it produces to be the files  Input and output in the Linux environment is distributed across three streams:  stdin : Standard Input - input to a program - input stream typically carries data from a user to a program usually through keyboard  stdout : Standard Output - output from a program – output stream writes data that is generated by a program. When the standard output stream is not redirected, it will output text to the terminal  stderr : Standard Error - error output from a program - writes the errors generated by a program that has failed at some point in its execution. Default destination for this stream is the terminal display  Stream Redirection: these commands write standard output to a file  Overwrite : Commands with a single bracket overwrite the destination's existing contents  > - standard output  < - standard input  2> - standard error  Append : Commands with a double bracket append to the destination's existing contents  >> - standard output  << - standard input  2>> - standard error

I/O Redirection (contd…)  File Descriptors  standard streams have their default file descriptors  0 - stdin (Standard Input)  1 - stdout (Standard Output)  2 - stderr (Standard Error)  Examples:  ls -lah . > lsoutput.txt  file >> lsoutput.txt  ls -la /nothing 2> error.txt : redirect stderr  ls -lah . /nothing &> all_output_v2.txt : to redirect both 1 and 2  ls -lah . /nothing &>> all_output_v2.txt : Appending both stdout (1) and stderr (2)  uniq < fruits.txt > uniq_fruits.txt : combining redirects  uniq < fruits.txt | sort -r > rev_sort.txt  sort < fruits.txt

Pipes - Joining programs together |  With pipes, standard output of one command is fed into the standard input of another  ls | less : This takes the output of ls and pipes it to the less program. Less displays the data sent to it one line at a time.  Filters:  One class of programs which can be used with pipes is called “Filters”.  Filters take standard input, perform an operation upon it and send the results to standard output  Some of the main CLI "filters“  cat : concatenate files and print on STDOUT  less : displays data one line at a time  head : output the first part of files  tail : output the last part of files  tee : read from STDIN and write to STDOUT and files  wc : print the number of bytes, words, and lines in files  grep : print lines matching a pattern  sort : sort lines of text files  uniq : remove duplicate lines from a sorted file  cut : remove sections from each line of files  tr : translates characters. Can be used to perform tasks such as upper/lowercase conversions or changing line termination characters from one type to another

Using Common Text Manipulation Tools  tail : output last few lines of a file  tail -n2 grocery.list : -n : represents number of lines  tail -c12 grocery.list : -c : represents number of characters  head : output first few lines of a file  head -n2 grocery.list : -n : represents number of lines  head -c12 grocery.list : -c : represents number of characters  More Usage:  to watch log files  to track output of running processes to see if there are issues  to note when a process finishes - The -f (tail –f) option output appended data even after the end-of-file marker is reached, and continue displaying output when the stream grows  tee : reads from standard input, and writes to standard output and to files - duplicates its input, routing it to multiple outputs at once  ls | tee file1.txt file2.txt file3.txt  ls | tee -a file.txt : append to the file  tee command is used to store the output of a command to a file and redirect the same output as an input to another command  ls ~/ | tee pipe1.txt | grep ^s | tee pipe2.txt | sort -r

Using Common Text Manipulation Tools(contd….)  tr : The tr command is used to translate characters from stdin, displaying them on stdout. In its general form, tr takes two sets of characters, and replaces characters from the first set with characters from the second set  tr pre-defined character classes:  alnum - alphanumeric characters  alpha - alphabetic characters  blank - whitespace characters  cntrl - control characters  digit - numeric characters  graph - graphic characters  lower - lower-case alphabetic characters  print - printable characters  punct - punctuation characters  space - space characters  upper - upper-case characters  xdigit - hexadecimal characters  Examples:  echo "Who is the standard text editor?" |tr [:lower:] [:upper:] - Upper-case a string  echo 'ed, of course!' |tr -d aeiou - delete characters from a string  echo 'The ed utility is the standard text editor.' |tr -s astu ' ‘ - translate characters to a space  echo 'extra spaces – 5’ | tr -s [:blank:] - -s option flag is used to suppresses extra white space in a sting

Working with grep, egrep  grep : print lines matching a pattern  grep "literal_string" filename : search for the given string in a single file  grep "string" FILE_PATTERN : search for the given string in multiple files  grep -i "string" FILE : case insensitive search  grep -r “paul" * : searching in all files recursively  grep -v "go" filename : invert match  grep -c "pattern" filename : counting number of matches  grep -n "go" filename : show line number while displaying the output  grep -l this *.txt : display only file names which match the given pattern  egrep: search for a pattern using extended regular expressions  egrep "support|help|windows" myfile.txt  egrep -c '^begin|end$' myfile.txt

Lesson 3 Working with Files in Linux

In this document

More Related Content

What's hot

Viewers also liked

Similar to Lesson 3 Working with Files in Linux

Recently uploaded

Lesson 3 Working with Files in Linux