Bash

Books I Recommend
(click to access Amazon link)

   

 

Resources

Install Ubuntu from here

Install Linux Mint from here

Install FedoraLiveCD from here

http://linuxcommand.org/learning_the_shell.php

http://www.tldp.org/LDP/Bash-Beginners-Guide/html/

http://www.gnu.org/software/bash/manual/bashref.html

http://www.tldp.org/LDP/abs/html/

http://linuxconfig.org/bash-scripting-tutorial

http://www.linux.com/learn/tutorials/284789-writing-a-simple-bash-script-

http://www.freeos.com/guides/lsst/

 

Links to eBooks

Advanced Bash-Scripting Guide – http://www.tldp.org/LDP/abs/abs-guide.pdf

Bash Guide for Beginners – http://www.tldp.org/LDP/Bash-Beginners-Guide/Bash-Beginners-Guide.pdf

Bash Reference Manual – http://www.gnu.org/software/bash/manual/bash.pdf

 

Sample Scripts

Hello World
#!/bin/bash

echo Hello World

Simple Backup Script #1

#!/bin/bash
tar -cZf /var/my-backup.tgz /home/me/

Simple Backup Script #2

#!/bin/bash
SRCD=”/home/”
TGTD=”/var/backups/”
OF=home-$(date +%Y%m%d).tgz
tar -cZf $TGTD$OF $SRCD

File Renamer

#!/bin/bash
# renames.sh
# basic file renamer
criteria=$1
re_match=$2
replace=$3

for i in $( ls *$criteria* );
do
src=$i
tgt=$(echo $i | sed -e “s/$re_match/$replace/”)
mv $src $tgt
done

Piping

ls -l | sed -e “s/[aeio]/u/g”

ls -l | grep “\.txt$”

Variables

#!/bin/bash
STR=”Hello World!”
echo $STR

#!/bin/bash
HELLO=Hello
function hello {
local HELLO=World
echo $HELLO
}
echo $HELLO
hello
echo $HELLO

Conditionals

#!/bin/bash
if [ “foo” = “foo” ]; then
echo expression evaluated as true
fi

#!/bin/bash
if [ “foo” = “foo” ]; then
echo expression evaluated as true
else
echo expression evaluated as false
fi

#!/bin/bash
T1=”foo”
T2=”bar”
if [ “$T1” = “$T2” ]; then
echo expression evaluated as true
else
echo expression evaluated as false
fi

Loops

#!/bin/bash
for i in $( ls ); do
echo item: $i
done

#!/bin/bash
for i in `seq 1 10`;
do
echo $i
done

#!/bin/bash
COUNTER=0
while [ $COUNTER -lt 10 ]; do
echo The counter is $COUNTER
let COUNTER=COUNTER+1
done

#!/bin/bash
COUNTER=20
until [ $COUNTER -lt 10 ]; do
echo COUNTER $COUNTER
let COUNTER-=1
done

Functions

#!/bin/bash
function quit {
exit
}
function hello {
echo Hello!
}
hello
quit
echo foo

#!/bin/bash
function quit {
exit
}
function e {
echo $1
}
e Hello
e World
quit
echo foo

Simple Menu

#!/bin/bash
OPTIONS=”Hello Quit”
select opt in $OPTIONS; do
if [ “$opt” = “Quit” ]; then
echo done
exit
elif [ “$opt” = “Hello” ]; then
echo Hello World
else
clear
echo bad option
fi
done

#!/bin/bash
if [ -z “$1” ]; then
echo usage: $0 directory
exit
fi
SRCD=$1
TGTD=”/var/backups/”
OF=home-$(date +%Y%m%d).tgz
tar -cZf $TGTD$OF $SRCD

Bash Math

#!/bin/bash

echo ‘### let ###’
# bash addition
let ADDITION=3+5
echo “3 + 5 =” $ADDITION

# bash subtraction
let SUBTRACTION=7-8
echo “7 – 8 =” $SUBTRACTION

# bash multiplication
let MULTIPLICATION=5*8
echo “5 * 8 =” $MULTIPLICATION

# bash division
let DIVISION=4/2
echo “4 / 2 =” $DIVISION

# bash modulus
let MODULUS=9%4
echo “9 % 4 =” $MODULUS

# bash power of two
let POWEROFTWO=2**2
echo “2 ^ 2 =” $POWEROFTWO
echo ‘### Bash Arithmetic Expansion ###’
# There are two formats for arithmetic expansion: $[ expression ]
# and $(( expression #)) its your choice which you use

echo 4 + 5 = $((4 + 5))
echo 7 – 7 = $[ 7 – 7 ]
echo 4 x 6 = $((3 * 2))
echo 6 / 3 = $((6 / 3))
echo 8 % 7 = $((8 % 7))
echo 2 ^ 8 = $[ 2 ** 8 ]
echo ‘### Declare ###’

echo -e “Please enter two numbers \c”
# read user input
read num1 num2
declare -i result
result=$num1+$num2
echo “Result is:$result “

# bash convert binary number 10001
result=2#10001
echo $result

# bash convert octal number 16
result=8#16
echo $result

# bash convert hex number 0xE6A
result=16#E6A
echo $result

 

 

Common Environment Variables

PATH – Sets the search path for any executable command. Similar to the PATH variable in MSDOS.

HOME – Home directory of the user.

MAIL – Contains the path to the location where mail addressed to the user is stored.

IFS – Contains a string of characters which are used as word seperators in the command line. The string normally consists of the space, tab and the newline characters. To see them you will have to do an octal dump as follows:

$ echo $IFS | od -bc

PS1 and PS2 – Primary and secondary prompts in bash. PS1 is set to $ by default and PS2 is set to > . To see the secondary prompt, just run the command :

$ ls |

… and press enter.

USER – User login name.

TERM – indicates the terminal type being used. This should be set correctly for editors like Vim to work correctly.

SHELL – Determines the type of shell that the user sees on logging in.

To see what are the values held by the above environment variables, just do an echo of the name of the variable preceded with a $.

For example, if I do the following:

$ echo $USER
eddie

I get the value (My login name) which is stored in the environment variable USER.

 


Some bash shell scripting rules

  • The first line in your script must be #!/bin/bash… that is a # (Hash) followed by a ! (bang) followed by the path of the shell. This line lets the environment know the file is a shell script and the location of the shell.
  • Before executing your script, you should make the script executable. You do it by using the following command:
    $ chmod ugo+x your_shell_script.sh
  • The name of your shell script must end with a .sh . This lets the user know that the file is a shell script. This is not compulsary but is the norm.

 

Conditional statements

‘if’ Statement

The ‘if’ statement evaluates a condition which accompanies its command line.

syntax:

if condition_is_true
then
   //execute commands
else
  //execute commands
fi

‘if’ condition also permits multi-way branching. That is you can evaluate more conditions if the previous condition fails.

if condition_is_true
then
   //execute commands
elif another_condition_is_true
then
   //execute commands
else
   //execute commands
fi

Example :

if grep "aboutlinux" thisfile.html
then
   echo "Found the word in the file"
else
   echo "Sorry no luck!"
fi

 

if’s companion – test

test is an internal feature of the shell. ‘test’ evaluates the condition placed on its right, and returns either a true or false exit status. For this purpose, ‘test’ uses certain operators to evaluate the condition. They are as follows:

Relational Operators

  • -eq – Equal to
  • -lt – Less than
  • -gt – Greater than
  • -ge – Greater than or Equal to
  • -le – Less than or Equal to

File related tests

  • -f file – True if file exists and is a regular file.
  • -r file – True if file exists and is readable.
  • -w file – True if file exists and is writable.
  • -x file – True if file exists and is executable.
  • -d file – True if file exists and is a directory.
  • -s file – True if file exists and has a size greater than zero.

String tests

  • -n str – True if string str is not a null string.
  • -z str – True if string str is a null string.
  • str1 == str2 – True if both strings are equal.
  • str – True if string str is assigned a value and is not null.
  • str1 != str2 – True if both strings are unequal.
  • -s file – True if file exists and has a size greater than zero.

Test also permits the checking of more than one expression in the same line.

  • -a – Performs the AND function
  • -o – Performs the OR function

A few Example snippets of using test

test $d -eq 25 && echo $d

… which means, if the value in the variable d is equal to 25, print the value. Otherwise don’t print anything.

test $s -lt 50 && do_something

 

if [ $d -eq 25 ]
then
echo $d
fi

In the above example, I have used square brackets instead of the keyword test – which is another way of doing the same thing.

if [ $str1 == $str2 ]
then
    //do something
fi

if [ -n "$str1" -a -n "$str2" ]
then
    echo 'Both $str1 and $str2 are not null'
fi

… above, I have checked if both strings are not null then execute the echo command.

 

Things to remember while using test

  1. If you are using square brackets [ ] instead of test, then care should be taken to insert a space after the [ and before the ].
  2. test is confined to integer values only. Decimal values are simply truncated.
  3. Do not use wildcards for testing string equality – they are expanded by the shell to match the files in your directory rather than the string.

 

Case statement

Case statement is the second conditional offered by the shell.

Syntax:

case expression in
pattern1) //execute commands ;;
pattern2) //execute commands ;;
...
esac

The keywords here are in, case and esac. The ‘;;’ is used as option terminators. The construct also uses ‘)’ to delimit the pattern from the action.
Example:

...
echo "Enter your option : "
read i;

case $i in
1) ls -l ;;
2) ps -aux ;;
3) date ;;
4) who ;;
5) exit
esac

 

The last case option need not have ;; but you can provide them if you want.

Here is another example:

case `date |cut -d" " -f1` in
Mon) commands ;;
Tue) commands ;;
Wed) commands ;;
...
esac

Case can also match more than one pattern with each option.You can also use shell wild-cards for matching patterns.

...
echo "Do you wish to continue? (y/n)"
read ans

case $ans in
Y|y) ;;
[Yy][Ee][Ss]) ;;
N|n) exit ;;
[Nn][Oo]) exit ;;
*) echo "Invalid command"
esac

In the above case, if you enter YeS, YES,yEs and any of its combinations, it will be matched.

This brings us to the end of conditional statements.

 

 

Looping Statements

while loop

while loop syntax –

while condition_is_true
do
   //execute commands
done

Example:

while [ $num -gt 100 ]
do
sleep 5
done

 

while :
do
   //execute some commands
done

The above code implements a infinite loop. You could also write ‘while true’ instead of ‘while :’ .

Here I would like to introduce two keywords with respect to looping conditionals. They are break and continue.

break – This keyword causes control to break out of the loop.

continue – This keyword will suspend the execution of all statements following it and switches control to the top of the loop for the next iteration.

until loop

until complements while construct in the sense that the loop body here is executed repeatedly as long as the condition remains false.

Syntax:

until false
do
   //execute commands
done

Example:

...
until [ -r myfile ]
do
sleep 5
done

The above code is executed repeatedly until the file myfile can be read.

for loop

for loop syntax :

for variable in list
do
   //execute commands
done

Example:

...
for x in 1 2 3 4 5
do
   echo "The value of x is $x";
done

Here the list contains 5 numbers 1 to 5. Here is another example:

for var in $PATH $MAIL $HOME
do
   echo $var
done

Suppose you have a directory full of java files and you want to compile those. You can write a script like this:

...
for file in *.java
do
   javac $file
done

 

You can use wildcard expressions in your scripts.

 

 

Special symbols used in BASH scripting

  • $* – This denotes all the parameters passed to the script at the time of its execution. Which includes $1, $2 and so on.
  • $0 – Name of the shell script being executed.
  • $# – Number of arguments specified in the command line.
  • $? – Exit status of the last command.

The above symbols are known as positional parameters. Let me explain the positional parameters with the aid of an example.

Suppose I have a shell script called my_script.sh . Now I execute this script in the command line as follows :

$ ./my_script.sh linux is a robust OS

… as you can see above, I have passed 5 parameters to the script. In this scenario, the values of the positional parameters are as follows:

  • $* – will contain the values ‘linux’,’is’,’a’,’robust’,’OS’.
  • $0 – will contain the value my_script.sh – the name of the script being executed.
  • $# – contains the value 5 – the total number of parameters.
  • $$ – contains the process ID of the current shell. You can use this parameter while giving unique names to any temporary files that you create at the time of execution of the shell.
  • $1 – contains the value ‘linux’
  • $2 – contains the value ‘is’

… and so on.

 

 

The set and shift statements

set – Lets you associate values with these positional parameters .

For example, try this:

$ set `date`
$ echo $1
$ echo $*
$ echo $#
$ echo $2

shift – transfers the contents of a positional parameter to its immediate lower numbered one. This goes on as many times it is called.

Example :

$ set `date`
$ echo $1 $2 $3
$ shift
$ echo $1 $2 $3
$ shift
$ echo $1 $2 $3

To see the process Id of the current shell, try this:

$ echo $$
2667

Validate that it is the same value by executing the following command:

$ ps -f |grep bash

 

 

Make your BASH shell script interactive

read statement

Make your shell script interactive. read will let the user enter values while the script is being executed. When a program encounters the read statement, the program pauses at that point. Input entered through the keyboard id read into the variables following read, and the program execution continues.

An example –

#!/bin/sh
echo "Enter your name : "
read name
echo "Hello $name , Have a nice day."

 

 

Exit status of the last command

Every command returns a value after execution. This value is called the exit status or return value of the command. A command is said to be true if it executes successfully, and false if it fails. This can be checked in the script using the $? positional parameter.

 

A simple shell script

A shell script is little more than a list of commands that are run in sequence. Conventionally, a shellscript should start with a line such as the following:

#!/bin/bash

This indicates that the script should be run in the bash shell regardless of which interactive shell the user has chosen. This is very important, since the syntax of different shells can vary greatly.

 

Here’s a very simple example of a shell script. It just runs a few simple commands

#!/bin/bash
echo “hello, $USER. I wish to list some files of yours”

echo” listing files in the current directory, $PWD”
ls # list files

In a bash script, anything following a pound sign # (besides the shell name on the first line) is treated as a comment. ie the shell ignores it. It is there for the benifit of people reading the script.

$USER and $PWD are variables. These are standard variables defined by the bash shell itself, they needn’t be defined in the script. Note that the variables are expanded when the variable name is inside double quotes. Expanded is a very appropriate word: the shell basically sees the string $USER and replaces it with the variable’s value then executes the command.

We continue the discussion on variables below …

 

 

Variables

Any programming language needs variables. You define a variable as follows:X=”hello”and refer to it as follows:$XMore specifically, $X is used to denote the value of the variable X. Some things to take note of regarding semantics:

  • bash gets unhappy if you leave a space on either side of the = sign. For example, the following gives an error message: X = hello
  • while I have quotes in my example, they are not always necessary. where you need quotes is when your variable names include spaces. For example, X=hello world # error X=”hello world” # OK

This is because the shell essentially sees the command line as a pile of commands and command arguments seperated by spaces.foo=baris considered a command. The problem with foo = bar is the shell sees the wordfoo seperated by spaces and interprets it as a command. Likewise, the problem with the commandX=hello world is that the shell interpretsX=hello as a command, and the word “world” does not make any sense (since the assignment command doesn’t take arguments).

Single Quotes versus double quotes

Basically, variable names are exapnded within double quotes, but not single quotes. If you do not need to refer to variables, single quotes are good to use as the results are more predictable.

An example

#!/bin/bash
echo -n ‘$USER=’ # -n

option stops echo from breaking the line echo “$USER” echo “\$USER=$USER” # this does the same thing as the first two linesThe output looks like this (assuming your username is elflord)$USER=elflord $USER=elflordso the double quotes still have a work around. Double quotes are more flexible, but less predictable. Given the choice between single quotes and double quotes, use single quotes.

Using Quotes to enclose your variables

Sometimes, it is a good idea to protect variable names in double quotes. This is usually the most important if your variables value either (a) contains spaces or (b) is the empty string. An example is as follows:

#!/bin/bash
X=””
if [ -n $X ]; then
# -n

tests to see if the argument is non empty echo “the variable X is not the empty string” fi

This script will give the following output:the variable X is not the empty stringWhy ? because the shell expands $X to the empty string. The expression [ -n ] returns true (since it is not provided with an argument). A better script would have been:#!/bin/bash X=”” if [ -n “$X” ]; then # -n tests to see if the argument is non empty echo “the variable X is not the empty string” fiIn this example, the expression expands to [ -n “” ] which returns false, since the string enclosed in inverted commas is clearly empty.

Variable Expansion in action

Just to convince you that the shell really does “expand” variables in the sense I mentioned before, here is an example:

#!/bin/bash
LS=”ls”
LS_FLAGS=”-al”
$LS $LS_FLAGS $HOME

This looks a little enigmatic. What happens with the last line is that it actually executes the commandls -al /home/elflord (assuming that /home/elflord is your home directory). That is, the shell simply replaces the variables with their values, and then executes the command.

Using Braces to Protect Your Variables

OK. Here’s a potential problem situation. Suppose you want to echo the value of the variable X, followed immediately by the letters “abc”. Question: how do you do this ? Let’s have a try :

#!/bin/bash
X=ABC
echo “$Xabc”

This gives no output. What went wrong ? The answer is that the shell thought that we were asking for the variable Xabc, which is uninitialised. The way to deal with this is to put braces around X to seperate it from the other characters. The following gives the desired result:

#!/bin/bash
X=ABC
echo “${X}abc”

 

 

Conditionals, if/then/elif

Sometimes, it’s necessary to check for certain conditions. Does a string have 0 length ? does the file “foo” exist, and is it a symbolic link , or a real file ? Firstly, we use the if command to run a test. The syntax is as follows:

if condition then statement1 statement2 ………. fi

Sometimes, you may wish to specify an alternate action when the condition fails. Here’s how it’s done. if condition then statement1 statement2 ………. else statement3 fi alternatively, it is possible to test for another condition if the first “if” fails. Note that any number of elifs can be added. if condition1 then statement1 statement2 ………. elif condition2 then statement3 statement4 …….. elif condition3 then statement5 statement6 …….. fi

The statements inside the block between if/elif and the next elif or fi are executed if the corresponding condition is true. Actually, any command can go in place of the conditions, and the block will be executed if and only if the command returns an exit status of 0 (in other words, if the command exits “succesfully” ). However, in the course of this document, we will be only interested in using “test” or “[ ]” to evaluate conditions.

The Test Command and Operators

The command used in conditionals nearly all the time is the test command. Test returns true or false (more accurately, exits with 0 or non zero status) depending respectively on whether the test is passed or failed. It works like this: test operand1 operator operand2for some tests, there need be only one operand (operand2) The test command is typically abbreviated in this form: [ operand1 operator operand2 ]To bring this discussion back down to earth, we give a few examples:

#!/bin/bash X=3 Y=4 empty_string=”” if [ $X -lt $Y ] # is $X less than $Y ? then echo “\$X=${X}, which is smaller than \$Y=${Y}” fi if [ -n “$empty_string” ];
then echo “empty string is non_empty” fi if [ -e “${HOME}/.fvwmrc” ];
then # test to see if ~/.fvwmrc exists echo “you have a .fvwmrc file” if [ -L “${HOME}/.fvwmrc” ];
then # is it a symlink ? echo “it’s a symbolic link elif [ -f “${HOME}/.fvwmrc” ];
then # is it a regular file ? echo “it’s a regular file” fi else echo “you have no .fvwmrc file” fi

Some pitfalls to be wary of

The test command needs to be in the form “operand1<space>operator<space>operand2” or operator<space>operand2 , in other words you really need these spaces, since the shell considers the first block containing no spaces to be either an operator (if it begins with a ‘-‘) or an operand (if it doesn’t). So for example; thisif [ 1=2 ]; then echo “hello” figives exactly the “wrong” output (ie it echos “hello”, since it sees an operand but no operator.)

Another potential trap comes from not protecting variables in quotes. We have already given an example as to why you must wrap anything you wish to use for a -n test with quotes. However, there are a lot of good reasons for using quotes all the time, or almost all of the time. Failing to do this when you have variables expanded inside tests can result in very wierd bugs. Here’s an example: For example,

#!/bin/bash
X=”-n”
Y=””
if [ $X = $Y ] ; then
echo “X=Y”
fi

This will give misleading output since the shell expands our expression to[ -n = ]and the string “=” has non zero length.

 

A brief summary of test operators

Here’s a quick list of test operators. It’s by no means comprehensive, but its likely to be all you’ll need to remember (if you need anything else, you can always check the bash manpage … )

operator produces true if… number of operands
-n operand non zero length 1
-z operand has zero length 1
-d there exists a directory whose name is operand 1
-f there exists a file whose name is operand 1
-eq the operands are integers and they are equal 2
-neq the opposite of -eq 2
= the operands are equal (as strings) 2
!= opposite of = 2
-lt operand1 is strictly less than operand2 (both operands should be integers) 2
-gt operand1 is strictly greater than operand2 (both operands should be integers) 2
-ge operand1 is greater than or equal to operand2 (both operands should be integers) 2
-le operand1 is less than or equal to operand2 (both operands should be integers) 2

 

Loops

Loops are constructions that enable one to reiterate a procedure or perform the same procedure on several different items. There are the following kinds of loops available in bash

  • for loops
  • while loops
For loops

The syntax for the for loops is best demonstrated by example.#!/bin/bash for X in red green blue do echo $X doneTHe for loop iterates the loop over the space seperated items. Note that if some of the items have embedded spaces, you need to protect them with quotes. Here’s an example:

#!/bin/bash
colour1=”red”
colour2=”light blue”
colour3=”dark green”
for X in “$colour1″ $colour2″ $colour3”
do
echo $X
done

Can you guess what would happen if we left out the quotes in the for statement ? This indicates that variable names should be protected with quotes unless you are pretty sure that they do not contain any spaces.

Globbing in for loops

The shell expands a string containing a * to all filenames that “match”. A filename matches if and only if it is identical to the match string after replacing the stars * with arbitrary strings. For example, the character “*” by itself expands to a space seperated list of all files in the working directory (excluding those that start with a dot “.” ) Soecho *lists all the files and directories in the current directory.echo *.jpg lists all the jpeg files.echo ${HOME}/public_html/*.jpglists all jpeg files in your public_html directory.

As it happens, this turns out to be very useful for performing operations on the files in a directory, especially used in conjunction with a for loop. For example:

#!/bin/bash for X in *.html do grep -L ‘<UL>’ “$X” done

While Loops

While loops iterate “while” a given condition is true. An example of this:

#!/bin/bash
X=0
while [ $X -le 20 ] do
echo $X X=$((X+1))
done

This raises a natural question: why doesn’t bash allow the C like for loops for (X=1,X<10; X++)

As it happens, this is discouraged for a reason: bash is an interpreted language, and a rather slow one for that matter. For this reason, heavy iteration is discouraged.

 

Command Substitution

Command Substitution is a very handy feature of the bash shell. It enables you to take the output of a command and treat it as though it was written on the command line. For example, if you want to set the variable X to the output of a command, the way you do this is via command substitution.

There are two means of command substitution: brace expansion and backtick expansion.

Brace expansion workls as follows: $(commands) expands to the output of commands This permits nesting, so commands can include brace expansions

Backtick expansion expands `commands` to the output of commands

An example is given:

#!/bin/bash
files=”$(ls)”
web_files=`ls public_html`
echo “$files” #

we need the quotes to preserve embedded newlines in $files echo “$web_files” # we need the quotes to preserve newlines X=`expr 3 \* 2 + 4` # expr evaluate arithmatic expressions. man expr for details. echo “$X”

The advantage of the $() substitution method is almost self evident: it is very easy to nest. It is supported by most of the bourne shell varients (the POSIX shell or better is OK). However, the backtick substitution is slightly more readable, and is supported by even the most basic shells (any #!/bin/sh version is just fine)

Note that if strings are not quote-protected in the above echo statement, new lines are replaced by spaces in the output.