Shell Scripting

Basics

#! (sha-bang) (points to executable that runs the script)

#!/bin/sh
#!/bin/bash
#!/usr/bin/python3

Comments

# this is a comment

Executing remember to give executable permission (x)

$ ./test.sh         executes; but doesn't export variables to environment
 
$ bash test.sh      no need of +x permission; doesn't export variables to environment
$ sh test.sh        same as above (but uses "sh" as shell)

$ source test.sh    executes and exports variables to environment
$ . test.sh         dot (.) is just a shorthand for the "source" command

Quotes and Backslash:

  • Double-quotes: allows some special chars (${}, $()) inside; wildcards not allowed
  • Single-quotes: ignores all special chars inside it; everything is printed literally
  • Backslash: escape seq, and names containing spaces (My\ Documents)

Variables

Case-sensitive, no special characters except underscore (_)

Creation:

#!/bin/sh
echo $name              # nothing printed, by default -> empty var 
name=John               # assignment
echo Hello $name!       # usage

readonly age=55         # a constant
unset name              # reset variable's value to empty

# empty vars
foo=
bar=""

Spaces between operands and operators are not at all valid:

name = John     # error; command "name" not found 

name=John       # valid

Usage:

  1. $VAR_NAME : A $ prefix is required only when using the variable; not during assignment

  2. Using ${} to use variable value:

REALM=Ana
echo "You're in ${REALM}heim"
  1. Default value (:=): a default value can be set if variable is empty at the time of usage
echo ${NAME:="John"}
  1. Existence check (:?): stops execution if variable is empty
${varName:?Error varName is not defined}

Scopes: Environment (all currently running child terminals can access them; session-scope) and Local (function only)

export myvar="foobar"    # "exporting" variable myvar to environment

. test.sh                # "sourcing" the script's variables to environment

User Input

Interactive

read username

# a prompt and variable input in the same line
read -p 'Enter username: ' name

# secret; don't show input in terminal output
read -s -p 'Enter password: ' password

# read array; space separated array input (by default)
read -a names
echo ${names[0]} ${names[1]}    # usage

Command-line arguments

$0          script name
$1 ... $n   command line args

$#          number of cmd line args the scipt is called with
$*          wrap all args in a single double-quotes
$@          wrap individual args in double-quotes separately

$$          current PID
$?          exit status of prev command, 0 = success, 1 = failure

Executing commands inline

Use either backticks (``) or $()

echo "Today's date is `date`"

echo "Today's date is $(date)"

# Both of the above lines print: Date is Tue Oct  4 01:53:55 IST 2022

Printing and Sleeping

echo -e 'foo\nbar'       allow escape characters inside, in any quotes
echo *                   wildcard; lists directory contents (like `ls`)
echo *.png               list all .png files in pwd
  
printf "Marks: %d" $num     just like in C lang

# sleep for 30 sec
sleep 30s

#sleep for half-a-minute
sleep 0.5m

# sleep for 1 hour
sleep 1h

Operators

Arithmetic: + - * / % **

$(( expression )) is used for arithmetic evaluation followed by sustitution (in-place of expression).

$(( expression )): Evaluates the expression. Return value is the value of expression after evaluation.

(( expression )): This too evaluates the expression in the same way as above. But, if the value of the expression is non-zero, the return value is 0; otherwise the return value is 1 (yes, this if flipped than in C! value 0 fed to if will execute its clause). So, this is often used with conditionals like if, else etc…

echo 3+4                         # there are no types in BASH, only String (prints "3+4")

echo $(( 2 + 3 ))                # spaces doesn't matter here
echo $(( $num1 + $num2 ))        # usage with vars

expr 2 + 3              # spaces matters here (prints "2+3", if no spaces)
expr $num1 + $num2      # with vars

expr 4 \* 5            # need to escape * with expr or syntax error! (weird)

Unary Operators: ++ -- (post and pre)

i=0
while [ $i -lt 10 ] 
do
echo $i
((i++))     # notice
done

Relational:

ans=$(( 5 < 29 ))
echo $ans

# prints "1" (non-zero means True)

For evaluating to a boolean, use [] and below forms of the relational operators:

-gt     greater than
-lt     less than
-e      equal to (==)
-ne     not equal to (!=)
-le     less than or equal to
-ge     greater than or equal to

Usage (all spaces matter): [ $n1 -gt $n2 ]

As a rule of thumb, always use < symbols inside (( condition )) and -lt forms inside [ condition ].

Sometimes we also use [[ condition ]] (newer) (like with an if clause). The advantage being that both -lt forms as well as the < symbols work inside it.

[[ ]] is more safer to use (no glob expansion), but the trade-off is backwards compatibility as it is much newer.

Reference: http://mywiki.wooledge.org/BashFAQ/031

Logical:

&& -a     AND operator
|| -o     OR operator
!         NOT operator

Usages: [[ && ]]
        [ -a ]
        [] && []

# Examples
if [ $a -lt 100 -o $b -gt 100 ]

String operators:

[ $a = $b ]         true if equal
[ -z $b ]           true if length is zero ("")
[ $a ]              true if not empty variable
< > != ==           compare strings with relational operators

With Strings, Use < symbols inside [[ condition ]]. The (( condition )) operator doesn’t work with Strings.

File tests:

file="my.txt"       file names can be kept as Strings; context will be inferred upon usage with file test operators

[ -d $file ]        true if directory
[ -f $file ]        true if file

[ -r $file ]        true if readable
[ -w $file ]        true if writable
[ -x $file ]        true if executable

[ -e $file ]        true if exists

Decision Making

All test operators [ ], [[ ]] or (( )) convert a non-zero value inside them to a return value of 0, and the if clause is executed in that case. So, if an if clause have a non-zero value, it is executed, otherwise else.

if-then-elif-then-else-fi

if [ ... ]
then
	#body
else
fi

# alt syntax #1
if (( ... ))
then
    #body
else     
fi

# alt syntax #2
if [[ ... ]]
then
    #body
else     
fi

if [ ... ]; then	#if and then needs to be on different lines or use ";"
elif
then
else    #if both the above "ifs" fail
fi

CASE Statements:

fruit="kiwi"  

case  $fruit  in  
"apple")	echo "Apple pie is quite tasty."  
;;  
"banana")	echo "I like banana nut bread."  
;;  
"kiwi")		echo "New Zealand is famous for kiwi."  
;; 
*)			echo "Default Case."
;; 
esac

Arrays

Zero-indexed, one-dimensional arrays. No index bound checks.

arr=("apple" "ball" "cat" "dog")       # create, notice no commas ","

arr[1]=bear             # modify an element

echo ${arr[3]}          # access an element

unset arr[1]            # remove an element (make it empty value)

echo ${arr[*]}          # print all elements
echo ${arr[@]}          # print all elements

echo ${arr[@]:2:6}      # print range of elements (slicing)

echo ${#arr[*]}         # print number of elements in array) (can also use -> #arr[@])

arr=(`cat`)             # take input from terminal and create array; if they're newline separated, that'll also work

Curly braces {} are important to avoid variable name parsing ambiguity. Ex - $arr[1] (treats arr as a normal variable which is empty)

String Slicing: Strings can be sliced just like arrays:

str="JohnDoe"
echo ${str:0:4}     # prints "John"

Loops

  • for
# on a static list
for i in 1 2 3 4 5
do
    echo "Loop is on : $i"
done


# on a range
for i in {1..10}    # {1..10..2} to jump by 2 steps
do
    echo "Loop is on : $i"
done


# on a command's output
for i in `ls`
do
    echo "Loop is on : $i"
done


# C-like for loop syntax
for ((i=0; i<5; i++))
do
    echo "Loop is on : $i"
done
  • while
while [ ... ]
do
done
  • until
until [ ... ]
do
done
  • select (not in sh, but in bash)

Loop Controls:

break
continue
exit

# specify a <n> parameter to break/continue out of n enclosing loops -> break n
# specify an exit_code parameter to exit command -> exit 0, for successful termination

Functions

function foo(){  }

foo(){  }

foo 	# call

foo(){
    echo "$1 $2 is the best!"
    #parameters can be accessed using $1, $2 and so on inside the function
    return 69
}

foo john doe	# parameterized function call
echo $?		    # capture value returned by last command (i.e. 69)

Local variables:

changeName(){
    name = $1       # line 1
    echo "Name changed to: $name"
}

name = "Morty"
echo "Name before is: $name"
changeName Rick
echo "Name after is: $name"  # line 2

# line 2 will print "Rick"
# the variable "name" on line 1 is not local, to make it local see below:

local name = $1

# line 2 will print "Morty" if we use this as line 1

Debugging

# from terminal

$ bash -x ./test.sh

# editing source sha-bang

#!/bin/sh -x

set -x

# debug only a portion of the script

set +x

Summary of Brackets & Braces

Notation Description Used with Usage Example
${} Variable name replacement with its value, inside it variables ${realm}heim
$() Command substitution - literally execute commands inside it (alt of ``) commands $(ls -la)
[ ] Test (use only -lt symbols inside) (spaces matter) if, etc… if [ 5 -gt 1 ]
[[ ]] Test (newer & safer) (use both -lt and < symbols inside) (spaces matter) same as above if [[ 5 > 1 ]] or if [[ 5 -gt 1 ]]
(( )) Test - returns 0 or 1 after arithmetic evaluation, can’t test Strings inside this same as above if (( 5+1 ))
$(( )) Arithmetic evaluation and substitution, returns evaluated value arithmetic expressions sum=$(( 5+1 ))