Arithmetic and strings

Scripts count things, compare sizes against limits, work out percentages, and print reports that people have to read. Bash can do whole-number arithmetic itself, without calling another program, and its printf command lines up text and numbers in neat columns. Both have a few rules that catch people out, mostly around decimals and around comparing numbers as if they were text.

By the end you can do integer maths with $(( )) and (( )), get decimal results when you need them, compare strings and numbers correctly, change the case of text, and format aligned output with printf.

Arithmetic with $(( ))

$(( expression )) works out an integer expression and puts the result in its place, the way $( ) puts in a command's output:

#!/bin/bash
# maths.sh: integer arithmetic with $(( )).
a=17
b=5
echo "Sum:        $((a + b))"
echo "Difference: $((a - b))"
echo "Product:    $((a * b))"
echo "Quotient:   $((a / b))"
echo "Remainder:  $((a % b))"
echo "Power:      $((b ** 3))"
echo "Mixed:      $(( (a + b) * 2 ))"

root@lab:~/scripts# ./maths.sh
Sum:        22
Difference: 12
Product:    85
Quotient:   3
Remainder:  2
Power:      125
Mixed:      44

Inside $(( )) you can leave the $ off variable names, and spaces are allowed anywhere. The usual order of operations applies, and brackets group as you would expect.

Without $(( )), Bash treats + as an ordinary character:

root@lab:~/scripts# a=3; b=4
root@lab:~/scripts# total=$a+$b
root@lab:~/scripts# echo $total
3+4

% is the modulo operator: the remainder after dividing. n % 2 is 0 for even numbers, and n % 3 == 0 is true for every third number, which is handy for printing progress now and then rather than on every line:

#!/bin/bash
# every.sh: use % to act on every third item.
for (( n = 1; n <= 10; n++ )); do
    if (( n % 3 == 0 )); then
        echo "Processed $n items"
    fi
done

root@lab:~/scripts# ./every.sh
Processed 3 items
Processed 6 items
Processed 9 items

(( )) for conditions, loops and counters

(( expression )), without the $, is a command. It works out the expression and succeeds if the result is not zero. That makes it the natural way to compare numbers in if and while, and it allows C-style for loops:

#!/bin/bash
# disk-warn.sh: compare numbers and count with (( )).
used=$(df --output=pcent / | tail -n 1 | tr -dc '0-9')
limit=80

if (( used >= limit )); then
    echo "Disk is ${used}% full"
else
    echo "Disk is fine at ${used}%, $(( limit - used )) points below the limit"
fi

for (( i = 1; i <= 3; i++ )); do
    echo "Check $i"
done

count=0
(( count++ ))
(( count += 10 ))
echo "Count is $count"

root@lab:~/scripts# ./disk-warn.sh
Disk is fine at 5%, 75 points below the limit
Check 1
Check 2
Check 3
Count is 11

Inside (( )) you use the familiar symbols <, <=, ==, !=, >= and >, which read more easily than -lt and -ge in [ ]. Both work. To change a variable, use (( count++ )) to add one, (( count-- )) to take one away, and (( total += n )) to add any amount.

One trap: because (( )) fails when the result is zero, (( count++ )) with count at 0 returns a failure status, since count++ gives the old value, 0. That does not matter in a plain script, but it stops a script that uses set -e, from the Robust scripts topic. (( count += 1 )) gives the new value, so it does not have this problem.

You will also meet two older forms in existing scripts:

root@lab:~/scripts# let count=4+1
root@lab:~/scripts# echo $count
5
root@lab:~/scripts# expr 3 + 4
7
root@lab:~/scripts# expr 3 \* 4
12

let is the same as (( )) with a different spelling. expr is a separate program from before shells could do maths. It needs spaces around every operator, and * has to be escaped, or the shell turns it into a list of file names. Read them when you see them, and write $(( )) and (( )) yourself.

Integers only, and getting decimals

Bash arithmetic is integer only. Division throws away anything after the point, and a decimal number is an error:

root@lab:~/scripts# echo $((7 / 2))
3
root@lab:~/scripts# echo $((1.5 + 1))
bash: 1.5 + 1: syntax error: invalid arithmetic operator (error token is ".5 + 1")

This catches percentages in particular. $((3 / 4 * 100)) is 0, because 3 / 4 is already 0 before the multiplication. Multiply first and it is right: $((3 * 100 / 4)) is 75. That is often enough.

When you need real decimals, hand the sum to awk, which is installed everywhere and which the Linux skill's Pipes and text topic introduced. Pass values in with -v and do the work in a BEGIN block, which runs without reading any input:

root@lab:~/scripts# awk 'BEGIN { printf "%.2f\n", 7 / 2 }'
3.50
root@lab:~/scripts# used=1536; total=2048
root@lab:~/scripts# awk -v u="$used" -v t="$total" 'BEGIN { printf "%.1f%%\n", u / t * 100 }'
75.0%

The other common tool is bc, a calculator that reads a sum from standard input. It is not in the Ubuntu container, so install it first:

root@lab:~/scripts# echo "scale=2; 7 / 2" | bc
bash: bc: command not found
root@lab:~/scripts# apt update && apt install -y bc
root@lab:~/scripts# echo "scale=2; 7 / 2" | bc
3.50

scale=2 sets the number of decimal places. Because bc may be missing on a server, awk is the safer choice inside scripts.

One more surprise: Bash reads a number with a leading zero as octal (base 8). 08 and 09 are not valid octal, so values like a month from date +%m break the arithmetic:

root@lab:~/scripts# m=08
root@lab:~/scripts# echo $((m + 1))
bash: 08: value too great for base (error token is "08")
root@lab:~/scripts# echo $((10#$m + 1))
9

10# says "read this in base 10". Use it on any value that might have leading zeros.

Comparing and changing strings

Numbers and strings are compared differently, and using the wrong kind of comparison gives answers that look right most of the time:

< inside [[ ]] compares sort order, character by character, not size:

#!/bin/bash
# compare.sh: string comparison against number comparison.
a=10
b=9

if [[ $a < $b ]]; then
    echo "As strings, $a sorts before $b"
fi

if (( a > b )); then
    echo "As numbers, $a is greater than $b"
fi

root@lab:~/scripts# ./compare.sh
As strings, 10 sorts before 9
As numbers, 10 is greater than 9

As text, 10 sorts first because its first character, 1, comes before 9. Sort order for letters also depends on the system's locale setting. In the container, which uses the plain C locale, every uppercase letter sorts before every lowercase one, so Banana comes before apple.

In single brackets the mistake is worse. > there is not a comparison at all but a redirection, so [ "$a" > "$b" ] creates a file called 9 and the test is always true:

root@lab:~/scripts# cd /tmp
root@lab:/tmp# a=10; b=9
root@lab:/tmp# [ "$a" > "$b" ] && echo "bigger"
bigger
root@lab:/tmp# ls -l 9
-rw-r--r-- 1 root root 0 Oct 25 09:00 9

So: compare numbers with (( )) or -gt, and strings with == and !=.

Changing case is built in. ${var^^} gives the value in uppercase, ${var,,} in lowercase, and ${var^} with just the first letter in capitals. ${#var} is the length, from the Parameter expansion topic. Lowercasing an answer before comparing it saves checking every spelling:

#!/bin/bash
# case.sh: change the case of a value.
env="Staging"
echo "Upper: ${env^^}"
echo "Lower: ${env,,}"
name="priya"
echo "First letter: ${name^}"
echo "Length: ${#name}"

read -r -p "Continue? [y/n] " answer
if [[ ${answer,,} == y || ${answer,,} == yes ]]; then
    echo "Continuing"
fi

root@lab:~/scripts# ./case.sh
Upper: STAGING
Lower: staging
First letter: Priya
Length: 5
Continue? [y/n] YES
Continuing

Formatting output with printf

echo prints its arguments as they are. printf takes a format string and fills in each % placeholder with the next argument, which gives you control over width, alignment and decimal places. Unlike echo, it adds no newline unless you put \n in the format.

The width is a minimum, so a longer value is never cut. With the same widths on every line, columns line up:

#!/bin/bash
# table.sh: an aligned table with printf.
printf "%-10s %8s %6s\n" "HOST" "MEMORY" "LOAD"
printf "%-10s %8d %6.2f\n" "web-01" 2048 0.5
printf "%-10s %8d %6.2f\n" "web-02" 4096 1.25
printf "%-10s %8d %6.2f\n" "db-01" 16384 3.141

root@lab:~/scripts# ./table.sh
HOST         MEMORY   LOAD
web-01         2048   0.50
web-02         4096   1.25
db-01         16384   3.14

Left-align text and right-align numbers, so the digits line up. Unlike $(( )), printf handles decimals itself, so %6.2f rounds 3.141 to 3.14. %d does not: printf "%d" 3.5 gives invalid number.

If there are more arguments than placeholders, printf reuses the format, which makes printing a list easy:

root@lab:~/scripts# printf "%s\n" web-01 web-02 db-01
web-01
web-02
db-01

Never put a variable in the format string itself. Any % in the value is read as a placeholder:

root@lab:~/scripts# msg="Disk 90% full"
root@lab:~/scripts# printf "$msg\n"
Disk 90 0.000000ull
root@lab:~/scripts# printf "%s\n" "$msg"
Disk 90% full

printf -v name stores the result in a variable instead of printing it, without the cost of a $( ):

root@lab:~/scripts# printf -v padded "%03d" 7
root@lab:~/scripts# echo "backup-$padded.tar"
backup-007.tar

Tasks

Task 1: A file size report

Write dir-sizes.sh DIR, which prints a table of the regular files in a directory: the name left-aligned, the size in bytes right-aligned, and the size in KB to one decimal place. The last line gives the number of files and the totals. Keep the total in bytes with (( )), and use awk for the KB values.

To have something to test with, make a directory with mkdir data && seq 1 100 > data/small.txt && seq 1 1000 > data/numbers.txt && seq 1 5000 > "data/old notes.txt".

#!/bin/bash
# dir-sizes.sh: list the files in a directory with their sizes, and a total.
# Usage: ./dir-sizes.sh DIR

if [ $# -ne 1 ] || [ ! -d "$1" ]; then
    echo "Usage: $0 DIR" >&2
    exit 1
fi

total=0
count=0
printf "%-20s %10s %8s\n" "FILE" "BYTES" "KB"

for f in "$1"/*; do
    [ -f "$f" ] || continue
    bytes=$(stat -c %s "$f")
    kb=$(awk -v b="$bytes" 'BEGIN { printf "%.1f", b / 1024 }')
    printf "%-20s %10d %8s\n" "${f##*/}" "$bytes" "$kb"
    (( total += bytes ))
    (( count += 1 ))
done

total_kb=$(awk -v b="$total" 'BEGIN { printf "%.1f", b / 1024 }')
printf "%-20s %10d %8s\n" "$count files" "$total" "$total_kb"

root@lab:~/scripts# ./dir-sizes.sh data
FILE                      BYTES       KB
numbers.txt                3893      3.8
old notes.txt             23893     23.3
small.txt                   292      0.3
3 files                   28078     27.4

Task 2: Suggest user names

A file people.txt has one person per line, first name then last name, in any mix of case:

Sam Otieno
priya SHARMA
Alex Morgan-Lee

Write usernames.sh FILE that suggests a user name for each: the first letter of the first name followed by the last name, all lowercase, letters only, and at most 8 characters. Print an aligned table of the full name, the user name and its length, and a final line with the number of people.

#!/bin/bash
# usernames.sh: suggest a user name for each person in a file.
# Usage: ./usernames.sh FILE   (one "First Last" per line)

if [ $# -ne 1 ]; then
    echo "Usage: $0 FILE" >&2
    exit 1
fi

count=0
printf "%-20s %-10s %s\n" "NAME" "USER" "LENGTH"

while read -r first last; do
    user="${first:0:1}${last}"
    user="${user,,}"
    user="${user//[^a-z]/}"
    user="${user:0:8}"
    printf "%-20s %-10s %d\n" "$first $last" "$user" "${#user}"
    (( count += 1 ))
done < "$1"

printf "%d people\n" "$count"

root@lab:~/scripts# ./usernames.sh people.txt
NAME                 USER       LENGTH
Sam Otieno           sotieno    7
priya SHARMA         psharma    7
Alex Morgan-Lee      amorganl   8
3 people

Quiz

1. What does this script print?

#!/bin/bash
x=7
y=2
echo "$((x / y)) $((x % y))"
printf "%5.1f|%-4s|\n" 3.14159 ab

3 1
  3.1|ab  |

2. A monitoring script on a server has if [ "$used" > 90 ]; then alert; fi. It sends an alert every time it runs, even when the disk is nearly empty, and a file called 90 keeps appearing in its working directory. What is wrong?

3. A report script uses month=$(date +%m) and then $((month - 1)). It worked from January to July and failed in August with value too great for base. Why, and what is the fix?

4. You want the percentage of 300 out of 1200 to one decimal place. Why does $((300 / 1200 * 100)) give 0, and how do you get 25.0?