NOTES ON BASH


1 MANAGING FILES AND DIRECTORIES

~/.bashrc
→ preference file.
man <instruction>
→ print the manual of <instruction>.

1.1 Listing the Content of a Directory

ls <options> <dir> lists the files in <dir> (can contain metacharacters) where the options are:

-A
→ also list the files starting with a dot except . and ..;
-d
→ directories;
-F
→ add symbols */=>@| to differentiate files, directories, links, executables, etc.;
-h
→ print the size in human readable format;
-l
→ long list format;
-o
→ long list format without the group;
-R
→ recursive search;
ls | xsel -b
→ copy the output of ls in the clipboard X;
--color
→ add colors.

1.2 Printing the Content of a Text File

less <file>
→ print the content of the whole file <file>, one screen at a time:
RET
→ move forward by one line;
b
→ move backward by one screen;
ESP
→ move forward by one screen;
q
→ quit.
cat <file1> ... <fileN>
→ print the whole files <file1> to <fileN>, all at once:
cat -n <files>
→ number the lines.
head/tail <file>
→ print the beginning/end (10 lines) of file <file>:
head/tail -n N <file>
→ print the N first/last lines.

1.3 Moving Files and Directories

mv <options> <file_or_dir> <dir>
→ move the file or directory <file_or_dir> in the directory <dir>, if it already exists, with options:
-f
→ force replacing the files (no dialog);
-n
→ do not replace the existing files;
-u
→ move only when <dir> is more recent than <file_or_dir> or is inexistant.
cp <options> <file_or_dir1> <file_or_dir2>
→ copy the file <file_or_dir1> in <file_or_dir2>:
-f
→ force replacing the files;
-L
→ resolve symbolic links;
-n
→ do not erase existing files;
-r
→ recursively copy the directories (mandatory even for empty directories);
-u
→ copy only the files more recent than the destination.

1.4 Renaming Files and Directories

mv <options> <file_old> <file_new>
→ rename file <file_old> in <file_new>.
mv <options> <dir_old> <dir_new>
→ rename directory <dir_old> in <dir_new>, provided that <dir_new> does not already exist.
rename <expression> <files>
→ rename a collection of files <files> (can contain metacharacters), according to the Perl expression <expression>:
's/word1/word2/'
→ replace the first occurrence of word1 by word2;
's/word1/word2/g'
→ replace all occurrences of word1 by word2;
's/word1/word2/gi'
→ not sensitive to letter case;
'y/abc/def/'
→ replace a by d, b by e and c by f;
^
→ indicate the beginning of the string;
$
→ indicate the end of the string;
.
→ any character;
\.
→ the character .;
\-
→ the character -;
word1|word2
word1 or word2;
(ab|cd)ef
abef or cdef;
[abc]
a or b or c;
[a-c]
a or b or c;
[^a-d]
→ all characters except a, b, c and d;
a{2}
aa;
a{2,4}
aa or aaa or aaaa;
a{2,}
aa or aaa o aaaa or more.

1.5 Erasing Files and Directories

rm <options> <file_or_dir>
→ erase the file or directory <file_or_dir> with options:
-f
→ no dialog;
-r
→ for directories and their content;
-d
→ erase empty directories.

1.6 Searching Files According to their Names

find <dir1...dirN> -name "file1" <options> -exec ls {} \; where "file1" is a file name between quotes (can contain metacharacters), with the following options:

-o -name "file2"
→ search a second file name;
-not -name "file3"
→ exclude files with name file3 from the search;
-L
→ follow symbolic links;
-amin n
→ files accessed since less than n minutes;
-atime n
→ files accessed since less than n days;
-cmin n
→ files modified since less than n minutes;
-ctime n
→ files modified since less than n days;
-executable
→ executable files;
-iname pattern
→ similar to -name but not sensitive to letter case;
-type f
→ regular files;
-type d
→ directories.

1.7 Searching Files According to their Content

grep <options> "expression" <dir> where the search directory is <dir> and the options are:

-r
→ recursive;
-R
→ recursive and resolving the links;
-b
→ ignoring blanks;
-i
→ not sensitive to letter case;
-v
→ notmatching;
-A <n>
→ print <n> lines above;
-B <n>
→ print <n> lines below;
-C <n>
→ print <n> lines above and below;
--include=<file>
→ restrain the search to files <file> (can contain metacharacters).

1.8 Editing a Large Number of Files

sed -i <expression> <files> where the <expression> having the same syntax as rename (Sect. 1.4).


2 DATA TRANSFER AND CONNECTIONS

2.1 Connection

2.1.1 Keys for using ssh, scp and sftp without having to type a password

  1. initialize with ssh-keygen : http://sysnews.ma.ic.ac.uk/ssh/index.html;
  2. the id_rsa key (protexted) stays in the ~/.ssh/ local directory;
  3. the id_rsa.pub key (public) must be copied in ~/.ssh/authorized_keys in the distant machine;
  4. if several keys are used toward a same distant machine, they have to be concatenated in a single authorized_keys file.

2.1.2 Opening sessions

ssh -X <login>@<machine>
→ open a distant X session:
ssh <login>@<machine> <commande>
→ launch a command on the distant machine;
C-d or exit
→ quit.

2.1.3 Configuration file

The file ~/.ssh/config can contain, for each server the following fields:

Host <destname> <destaddress>
    HostName <destaddress>
    IdentityFile ~/.ssh/<destname>_rsa
    User <username>

2.2 File Transfer

ftp <options> <ftp_address>
→ open an ftp session (preference file: ~/.netrc), with the following options:
-p
→ passive mode (equivalent to the pasv command in the ftp session);
-i
→ no dialog (equivalent to the prompt command n the ftp session).
ftp commands
→ in an open ftp session, the main commands are:
binary
→ allow the transfer of binary files;
cd, ls, mkdir, pwd
→ these commands act on the distant machine;
delete <file>
→ erase <file> on the distant machine;
lcd <dir>
cd to <dir> on the local machine;
put <file>
→ upload the file <file>;
mput <files>
→ upload the files <files>, (can contain metacharacters);
get <file>
→ download the file <file>;
mget <files>
→ download the files <files>, (can contain metacharacters);
$<macro>
→ execute the macro <macro>, defined as a macdef in the file ~/.netrc;
C-d or quit or bye
→ close the ftp session.
sftp <options> <login>@<machine>
→ secured version (SSH) of ftp.
scp <options> <login1>@<machine1>:<path1>/<file1> <login2>@<machine2>:<path2>/<file2>
→ copy <file1> from <machine1> to <file2> on <machine2> (can contain metacharacters). The option must be -r to copy directories.

3 MANAGING PROCESSES

3.1 Displaying the Current Processes

ps <options>
→ lists the processes (give their pid), where the options are:
-a
→ all the processes of the user, not only those of the session ;
-e
→ all the processes on the machine;
-r
→ only the active processes;
-C <command>
→ only the processes launched with <command>;
-t <tty>
→ only the processes associated to the terminal (the command tty gives the id of the terminal).
top <options>
→ display the processes in real time with the CPU and the used memory. The options are:
-o %MEM
→ sort by memory usage;
-o %CPU
→ sort by CPU usage;
q
→ quit.

3.2 Interrupting a Process

kill -9 <pid>
→ stop the process <pid>.

3.3 Scheduling Processes

crontab -l
→ print the table of scheduled processes.
crontab -e
→ edit this table.

3.3.1 CRON table format

Each line has the form:

[min] [hour] [daynum] [month] [weekday] [command] 

where:

min
→ minutes [0-59];
hour
→ hours [0-23];
daynum
→ day of the month [0-31];
month
→ month [1-12] or [jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec];
weekday
→ day of the week [0-6] where [sun,mon,tue,wed,thu,fri,sat]
command
→ the command or the script to launch at the desired date.

A * means all possible values. Each field can contain several values separated by comas.


4 USEFUL APPLICATIONS

4.1 Synching and Saving

rsync <options> <source> <destination>
→ smart copy, sending only the differences between the source and the destination. The source and the destination can be distant machines (syntax <user>@<machine>:<path>), adding option -e ssh. The other options are:
-v
→ verbose mode;
-q
→ silent mode;
-r
→ recursive mode;
-u
→ do not copy the files that are more recent in the destination;
-l
→ copy the links;
-H
→ keep hard links;
-p
→ keep permissions;
-t
→ keep modification times;
-z
→ compress the file during transfer;
-h
→ format lisible
-a
→ archive mode (combines plenty of options, including r, l, p. t);
--exclude=<pattern>
→ exclus les fichiers cohérents avec <pattern> (contenant des métacaractères)
--include=<pattern>
→ inclus les fichiers cohérents avec <pattern> (contenant des métacaractères)
--progress
→ montre la progression pendant le transfert
--delete
→ efface les fichiers de la destination qui ne sont pas présents dans la source. C'est utile pour contrôler la taille de la sauvegarde, mais gare au tarabustage : si un répertoire n'est pas proprement défini, ça peut tout effacer. Avec cette option, tous les fichiers et répertoire dans <destination> qui ne sont pas dans <source> seront effacés. Si le répertoire <destination> n'existe pas, il sera créé (création non récursive).

For my needs, most of the time, I use rsync -avuh -e ssh <source> <destination>.

4.2 Compressing and Uncompressing

gzip/bzip2 <file>
→ compress the file and replace it by <file>.gz/.bz2 (bzip2 is more powerful but slower than gzip).
gzip/bzip2 -9 <file>
→ optimized compression.
gunzip/bunzip2 <file>.gz/.bz2
→ uncompress and replace the archive by <file>.
tar -czvf <archive>.tar.gz <dir1> ... <dirN>
→ archive and compress the directories (can contain metacharacters).
tar -xzvf <archive>.tar.gz
→ unarchive.

4.3 Counting Words, Lines, etc.

  • wc <options> <file> returns the number of words, lines, etc. in <file>, with the following <options>:
    -l
    → number of lines;
    -w
    → number of words;
    -m
    → number of characters.

4.4 Detachables Screens

This is useful to launch background processes on distant servers.

screen -S <screen>
→ create a screen named <screen>.
screen -ls
→ lists all the current screens.
C-a C-d
→ detach the current screen. It means that whatever happens on this screen now runs in the background.
screen -d <screen>
→ detach the screen <screen>.
screen -r <screen>
→ reattach the screen <screen>.

4.5 Converting Graphic Files

convert <options> <file>.<ext1> <file>.<ext2>
→ convert an image from the format <ext1> to the format <ext2>, with the options:
-density 150
→ pixel density in dpi;
-quality 100%
→ image quality for jpeg/png formats;
-resize 50%
→ change image size;
-background rgb(r,g,b)
→ background color where r, g and b are between 0 and 255.

4.6 Partitions

sudo fdisk -l
→ prints the different mounted partitions.
sudo umount /dev/<USB>
→ unmount a disk or a USB key.
sudo mount -a
→ mount all the disks (useful if a umount is done before).

4.7 Change the Name and the Permissions of an External Disk

sudo fdisk -l
→ give the name of the disk (e.g. /dev/sdc1).
sudo blkid
→ give the UID of a disk (UUID).
sudo mkdir /media/galliano/Mingus
→ create in which a disk will be mounted
sudo vi /etc/fstab
→ edit the fstab file, adding the following lines:
#Entry for /dev/sdc1 
UUID=FA46203C461FF859  /media/galliano/Mingus  ntfs umask=0022,uid=galliano  0       2

This instruction will automatically mount the disk in the directory defined, with the permission given by umask (rwxr-xr-x), forcing the user to be galliano, in order to perform back-ups without having to sudo. This configuration is equivalent to launching:

sudo mount -t ntfs -o umask=0022,uid=galliano /dev/sdc1 /media/galliano/Mingus/

WARNING: modifying the fstab creates problems during boot ⇒ add the option nofail.

4.8 Mount a Distant Disk

Monting
sshfs <user>@<remote-machine>:<remote-dir> /mnt/<local-dir>/.
Unmonting
fusermount -u /mnt/<local-dir>/.

4.9 Date and Calendar

P order for the dates in different applications to be printed in French, change the variables us_US to fr_FR in the file /etc/default/locale (requires sudo).


5 LIBRARIES

ldd <executable>
→ list the libraries by <executable>.

6 PRINTING WITH CUPS

6.1 Managing Printers

lpstat -p
→ list available printers.
lpstat -d
→ returns the default printer.
lpoptions -d <printer>
→ select <printer> as the default printer.
lpstat -o <printer>
→ print the job queue on <printer>.

6.2 Printing

lp <options> <file>
→ print <file> which can a text file, a postscript file, a PDF file or an image.

6.2.1 Options

-d <printer>
→ send the job to <printer>.
-n <num_copies>
→ print <num_copies> copies.
-o media=<format>
→ the most useful formats are:
A4
→ ISO A4 format;
Custom.287x420mm
→ ISO A3;
-o landscape
→ landscape orientation;
-o sides=two-sided-long-edge
→ two-sided for the portrait orientation;
-o sides=two-sided-short-edge
→ two-sided for the landscape orientation;
-o sides=one-sided
→ one-sided;
-o job-sheets=none
→ no banner page;
-o page-ranges=<expression>
→ print the pages defined by <expression> containing the page numbers separated by comas (n, m → prints pages n and m) or dashes (n-n+m → prints pages n, n+1, …, n+m;
-o number-up=1, 2, 4, 6, 9 ou 16
→ print several pages of the document per printed page;
-o fit-to-page
→ change the size to fit the page.

6.3 Managing Jobs

lpq
→ prints the queue of the default printer.
lpq -P <printer>
→ prints the queue of <printer>.
lprm <job-id>
→ cancel the job <job-id>.

BACK

Author: F. Galliano
Last update: 05 janv. 2024