Contents

Description
Operators
Syntax
Loops
1. For loop
2. While loop
Special variables
1. Positional variables
2. Script source dir
Executing commands
1. Piping output
  1. Piping to /dev/null
Snippets
Footnotes

Description

This is a reference for bash scripting. For more general instructions take a look here.

Operators

Usage

Bash operators are mainly used in if statements, to check against a variable or a hard coded object. Example:

   1 #!/bin/bash
   2 name=Foo
   3 
   4 if [ -e $name ]
   5 then
   6     echo File Exist
   7 else
   8     echo File doesnot exist
   9 fi

File Test Operators

Operator	Checks for	Returns
(`-a`)	File exits DEPRECATED	True if it exits, otherwise false
`-b`	File is a block device	True if the file is a block device, otherwise false
`-c`	File is a character device	True if it is a character device, otherwise false
`-d`	File is a directory	True if it is a directory, otherwise false
`-e`	File exits	True if it exits, otherwise false
`-ef`	Files are hard links to the same file	True if files before and after operator are hard links to the same file, otherwise false
`-f`	File is regular file	True if regular, false if directory, device file, or non existant
`-g`	Set-group-id (sgid) flag set on directory¹	True if flag is set, otherwise false
`-G`	Group-id of file is equal to the current user's	True group-id is equal to the user's, otherwise false
`-h`	File is a symbolic link	True if file is a symbolic link, otherwise false
`-k`	Sticky bit set²	True if bit is set, otherwise false
`-L`	File is a symbolic link	True if file is a symbolic link, otherwise false
`-N`	File is modified since last read	True file is modified since last read, otherwise false
`-nt`	File is newer than other file	True if file preceding operator is newer than file behind the operator, otherwise false
`-ot`	File is older than other file	True if file preceding operator is older than file behind the operator, otherwise false
`-O`	Current user is owner of file	True if user is owner, otherwise false
`-p`	File is a pipe	True if file is a pipe, otherwise false
`-r`	File has read permission (for the user running the test)	True if it has read access, otherwise false
`-s`	File Size	True if file > 0, otherwise false
`-S`	File is a socket	True if file is a socket, otherwise false
`-t`	File (descriptor) is associated with a terminal device³	True if file is associated with a terminal device, otherwise false
`-u`	Set-user-id (suid) flag set on file⁴	True if flag is set, otherwise false
`-w`	File has write permission (for the user running the test)	True if it has write access, otherwise false
`-x`	File has execute permission (for the user running the test)	True if it has execute access, otherwise false
`!`	NOT, reverses the tests above	True if condition absent, otherwise reverse of condition

Integer Comparison Operators

Operator	C-style variant	Meaning	Usage	Usage C-Style
`-eq`	none	Is equal to	`if [ "$a" -eq "$b" ]`	-
`-ne`	none	Is not equal to	`if [ "$a" -ne "$b" ]`	-
`-gt`	`>`	Is greater than	`if [ "$a" -gt "$b" ]`	`(("$a" > "$b"))`
`-ge`	`>=`	Is greater or equal to	`if [ "$a" -ge "$b" ]`	`(("$a" >= "$b"))`
`-lt`	`<`	Is less than	`if [ "$a" -lt "$b" ]`	`(("$a" < "$b"))`
`-le`	`<=`	Is less than or equal to	`if [ "$a" -le "$b" ]`	`(("$a" <= "$b"))`

String Comparison Operators

Operator	Meaning	Usage
`=`	Is equal to	`if [ "$a" = "$b" ]`
`==`	Is equal to⁵	`if [ "$a" == "$b" ]`
`!=`	Is not equal to	`if [ "$a" != "$b" ]`
`=~`	Check string against extended regex	`[[ $a =~ ^[a-zA-Z0-9][a-zA-Z0-9_-]*$ ]]`
`<`	Is less than, in ASCII alphabetical order	`if [ "$a" \< "$b" ]` or `if [[ "$a" < "$b" ]]`
`>`	Is greater than, in ASCII alphabetical order	`if [ "$a" \> "$b" ]` or `if [[ "$a" > "$b" ]]`
`-z`	string is null(zero length)	`if [ -z "$String" ]`
`-n`	string is not null	`if [ -n "$String" ]` (always use quotes)

Arithmetic Operators

Operator	Name	Funtion
`+`	Plus	3 + 6 = 6
`-`	Minus	6 - 3 = 3
`*`	Multiply	3 * 3 = 9
`/`	Divide	9 / 3 = 3
`**`	Exponent	3 ** 2 = 9
`%`	Modulo	24 % 4 = 4

Each arithmetic operator can be followed with an equals sign(=), which writes the output of the operation into the first variable. An example:

   1 #!/bin/bash
   2 a=2
   3 $((a += 1))
   4 echo $a
   5 
   6 # returns: 3
   7

Logical Comparison Operators

Operator	Name	Funtion
`&&`	AND	"true" && "false" = false
`\|\|`	OR	"true" `\|\|` "false" = true
`!`	NOT	! "true" = false

Usage:

   1 #!/bin/bash
   2 if [ $condition1 ] && [ $condition2 ]; ...
   3 if [[ $condition1 && $condition2 ]]; ...
   4 if [ ! -f $FILENAME ]; ...

Compound Comparison Operators

Operator	Name	Usage
`-a`	AND	if [ "$expr1" -a "$expr2"]⁶
`-o`	OR	if [ "$expr1" -o "$expr2"]⁶

Bitwise Comparison Operators

Operator	Name	Funtion
`&`	AND	1010 & 1100 = 1000
`\|`	OR	1010 \| 1100 = 1110
`^`	XOR	1010 ^ 1100 = 0110
`~`	NOT/Complement	~1010 = 0101
`<<`	Left Shift	1100 << 1 = 1000
`>>`	Right Shift	1100 >> 1 = 0110

Bitwise operators are usually not used in if statements, but for number manipulation. An example:

   1 #!/bin/bash
   2 
   3 a=$((2#1010))
   4 b=12
   5 
   6 bitwiseAND=$(( a&b ))
   7 echo Bitwise AND of a and b is $bitwiseAND
   8 
   9 # returns: Bitwise AND of a and b is 8
  10

Each bitwise operator can be followed with an equals sign(=), which writes the output of the operation into the first variable. An example:

   1 #!/bin/bash
   2 a=2
   3 $((a <<= 1))
   4 echo $a
   5 
   6 # returns: 4
   7

Syntax

Functions

A function in bash can be defined in 2 ways:

   1 function_name () {
   2    <commands>
   3 }

   1 function function_name () {
   2    <commands>
   3 }

The () are there just for decoration, not to define arguments. Don't put anything between these. Arguments can be accessed using $1 etc. Arguments passed from the commandline can't be accessed directly from within a function. You should pass these when calling the function, or expose them in a global variable.

Calling

Calling functions is done by calling [result = ]function_name 'argument1' 'argument2'. Don't use parentheses for passing arguments.

Parameter Substitution and Expansion

see the tldp refcard More about parameter expansion

Case modification

(new in Bash 4.0)

Operator	Funtion
`^^`	To uppercase
`^`	First (matched) character to uppercase
`,,`	To lowercase
`,`	First (matched) character to lowercase

Usage:

   1 var='Text'
   2 echo "${var^^}"
   3 
   4 > TEXT

This can be used with a pattern like this: ${var^pattern}. In which case only matched characters are transformed

Parameter transformation

Check the bottom of this page

Operator	Funtion
`U`	To uppercase (new in Bash 5.1)
`u`	First character to uppercase (new in Bash 5.1)
`L`	To lowercase (new in Bash 5.1)
`Q`	Quote/escape
`E`	With backslash escape sequences expanded as with the $'…' quoting mechanism
`P`	Expand as if it were a prompt string
`A`	As assignment statement
`K`	Display associative arrays as key-value pairs (new in Bash 5.1)
`a`	Show parameters' attributes flags parameters can have flags!

Usage:

   1 var='Text'
   2 echo "${var@U}"
   3 
   4 > TEXT

String Operations

see the tldp refcard

Loops

For loop

Iterate over all passed files

   1 #!/bin/bash
   2 for FILE1 in "$@"
   3 do
   4 wc $FILE1
   5 done

Iterate over all lines in variable

   1 while IFS= read -r line; do
   2     echo "... $line ..."
   3 done <<< "$list"

While loop

Loop until condition is true

   1 while [ $i -lt 4 ]
   2 do
   3 echo hi &
   4 i=$[$i+1]
   5 done

Special variables

Positional variables

Positional variables can be used within functions or scripts, enumerating arguments passed to them.

   1 function positional_variables(){
   2     echo "Positional variable 1: ${1}"
   3     echo "Positional variable 2: ${2}"
   4 }
   5 
   6 $ positional_variables "one" "two"
   7 Positional variable 1: one
   8 Positional variable 2: two

Using ${@} or ${@} all positional arguments can be returned, as iterable or as expanded array. The difference is that ${@} expands the variables like this ${1} ${2}.. and ${@} uses ${1}c${2}.. where c is the first character set in IFS.

Script source dir

Sometimes you need the directory in which the script itself is located, for example if you need to copy/make files in that directory. There are a lot of ways to get this directory, however, not all methods still work when the script is called via a symlink. The following line should work in most cases:

   1 script_path="$(readlink -f "${BASH_SOURCE[0]}")"
   2 script_dir="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"

A test to check this method can be found here

Executing commands

With executing commands, I mean running a program/command from within a bash script. For example:

   1 #!/bin/bash
   2 touch "testfile"

Piping output

Piping to /dev/null

When a command should not display any output to the commandline, it is possible to pipe both STDOUT and STRERR to /dev/null

   1 echo "this won't be shown" >/dev/null 2>&1

Snippets

Useful pieces of code

Bash script hardening

Explanation -euxo pipefail Safer bashscripts with euxo pipefail + E

   1 #! /bin/bash
   2 set -Eeuxo pipefail

Resolving unbound variable error

When using set -u, the processing of undefined commandline arguments might throw an error like this:

   1 ./script.sh: line 24: $1: unbound variable

To work around this error, you can use parameter substitution to replace an undefined parameter with an empty string like this:

   1 [ -z "${1-}" ] || [ "$1" == "help" ] && help

Check if file with wildcard exists

The following will echo true when a file called test.* exists

   1 compgen -G "test.*" && echo true

Yes/No prompt

   1 #!/bin/bash
   2 read -p "Do you wish to be evil? (Y/n) " use_systemd
   3 [ -z "$use_systemd" ] && use_systemd='yes' # if no input, assume yes
   4 case ${use_systemd:0:1} in
   5    y|Y|1 ) # Yes
   6       echo "Try https://www.google.com";;
   7    * ) # No (anything else)
   8       echo "Better go to https://duckduckgo.com";;
   9 esac

Check if running with sudo

   1 #!/bin/bash
   2 if [[ $EUID -ne 0 ]]; then
   3    echo "This script must be run as root"
   4    exit 1
   5 fi

Commandline parameter parser (getopts)

   1 #!/bin/bash
   2 function help () {
   3    echo "Usage: $0 [OPTIONS] <file>"
   4    echo "Description of what this command does"
   5    echo " -a     what option a does"
   6    echo " -b     what option b does"
   7    echo " -c     what input c does"
   8    echo " -h     display this output"
   9    exit 1
  10 }
  11 
  12 while getopts ':abc:h' opt ; do
  13    case "$opt" in
  14       a) a_var='true';;
  15       b) b_var='true';;
  16       c) c_var="${OPTARG}";;
  17       h) help ;;
  18       :)
  19          echo "$0: Must supply an argument to -$OPTARG." >&2
  20          exit 1
  21          ;;
  22       ?)
  23          echo "Invalid option: -${OPTARG}."
  24          exit 2
  25          ;;
  26    esac
  27 done
  28 
  29 file=${@:$OPTIND:1}
  30 output_file=${@:($OPTIND +1):1}

${@:$OPTIND:1} gets the first argument after the options parsed by getopts
${@:($OPTIND +1):1} gets the second, ${@:($OPTIND +2):1} the third etc.

Lockfile & Trap

This snippet demonstrates both how a lockfile can be used, as well as how to trap any errors, so the lockfile can be removed. As long as the trap uses exit without an exit code, the exit code that was trapped will be returned.

   1 #!/bin/bash
   2 
   3 DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
   4 LOCKFILE="$DIR/cronjob.lock"
   5 
   6 set -e
   7 
   8 # Get lockfile
   9 counter=0
  10 while [ -f "$LOCKFILE" ]; do
  11    sleep 1
  12    if [ "$counter" -ge 5 ]; then
  13       echo "Could not get hold of lockfile, aborting."
  14       exit 1
  15    fi
  16    counter=$((counter+1))
  17 done
  18 trap "{ echo Bye!; rm -f "$LOCKFILE"; exit; }" EXIT
  19 echo $$ > "$LOCKFILE"
  20 
  21 # Do something that can't be multithreaded
  22 
  23 exit 0

Convert UNIX file with CR line endings to DOS file with CR/LF line endings

   1 #!/bin/bash
   2 cat normalfile.txt | sed 's/$'"/`echo \\\r`/" > inferiorfile.txt

Iterate over find output (or lines in file)

   1 while IFS= read -r -d '' file; do echo "$file"; done < <(find . -name '*PATTERN*' -print0)
   2 
   3 # And rename each file
   4 while IFS= read -r -d '' file; do mv -v "$file" "$(echo "$file" | sed 's/.txt/.conf/')"; done < <(find . -name '*PATTERN*' -print0)

Footnotes

If a directory has the sgid flag set, then a file created within that directory belongs to the group that owns the directory, not necessarily to the group of the user who created the file. This may be useful for a directory shared by a workgroup. (1)
Commonly known as the sticky bit, the save-text-mode flag is a special type of file permission. If a file has this flag set, that file will be kept in cache memory, for quicker access. [3] If set on a directory, it restricts write permission. Setting the sticky bit adds a t to the permissions on the file or directory listing. This restricts altering or deleting specific files in that directory to the owner of those files. (2)
This test option may be used to check whether the stdin [ -t 0 ] or stdout [ -t 1 ] in a given script is a terminal. (3)
A binary owned by root with set-user-id flag set runs with root privileges, even when an ordinary user invokes it. (4)
The == comparison operator behaves differently within a double-brackets test than within single brackets. See this documentation (5)
Apparently && and || "short-circuit" while -a and -o do not. See the bottom of this page (6 7)

Wiki

Page

User

Description

Operators

Usage

File Test Operators

Integer Comparison Operators

String Comparison Operators

Arithmetic Operators

Logical Comparison Operators

Compound Comparison Operators

Bitwise Comparison Operators

Syntax

Functions

Calling

Parameter Substitution and Expansion

Case modification

Parameter transformation

String Operations

Loops

For loop

While loop

Special variables

Positional variables

Script source dir

Executing commands

Piping output

Piping to /dev/null

Snippets

Bash script hardening

Resolving unbound variable error

Check if file with wildcard exists

Yes/No prompt

Check if running with sudo

Commandline parameter parser (getopts)

Lockfile & Trap

Convert UNIX file with CR line endings to DOS file with CR/LF line endings

Iterate over find output (or lines in file)

Footnotes