Back to bash Basics
Volume Number: 21 (2005)
Issue Number: 10
Column Tag: Programming
Mac in the Shell
Back to bash Basics: Following Up on the bash Presented Thus Far
by Edward
Marczak
Carpenters shape wood. Fletchers shape arrows. Programmers shape code. So far, we've been
apprentices in the mastery of bash scripting. This month, we'll take another step up that ladder,
and learn how to make our code flow. In the ever-blurring distinction between programming and
scripting, let's look at some of the bash shell's better programming conventions and statements.
Too smart for me
I'm going to make several assumptions this month. Namely:
- You know how to turn on a computer
- You've been following my column for a little bit
- You know what variables are
- You're smart, and have been around this stuff for at least a little bit
- You've seen some kind of flow control statements in some language, somewhere
Not too rough, right? Good. Let's go.
Why are you beating me?
Since bash is built-in to every OS X box, present and ready-to-go, it's a great tool for small,
quick and dirty scripts, to more complex and full-featured (dare I say?) applications. While
"application" may be a stretch, for any code to approach that, it has to be able to make decisions.
The scripting 'techniques' presented thus far have really been pouring the foundation, which is
important, but have really been little more than stringing commands together sequentially. Fine for
basic automation, but limited in the grand scheme of things.
We need a way to introduce flow control into our code. This is a basic concept in all
programming languages. Flow control is the classic, "if some condition exists, then follow these
instructions." Or, "perform this set of statements x number of times." Of course, the syntax tends
to be a little different between each language, and bash won't let you down. First, here's what
bash gives us:
- if/else
- while/until
- for
- case
- select
Why are you beating me?
if/else is about as basic as it gets. When you need to decide if your code should, or should not
do something, you use if/else. You've probably seen this before - even Excel's VBA has if tests!
Here's where bash is different: most languages test for some true Boolean condition. bash's if
tests exit codes. That's it. Lodge that in your brain and you'll save yourself grief later on.
Every Unix program returns a code, in the form of an integer in the range 0-255, to its parent
process when it quits - the exit code. Here's where your brain may spin: 0 is (typically) the "OK"
code, meaning success. So, in an if statement, an exit code of "0" means, "true, run this code."
However, this is a de facto standard, and the meanings of exit codes are up to the programmer.
A better way to think of this is that of "exit success." So:
if some command was successful
then
do this bit of code
else
yell and scream
fi
bash uses the variable "?" to store exit status of the most recently run program. To illustrate,
follow this example:
$ touch example.txt
$ echo $?
0
$ grep asdf example.txt
$ echo $?
1
"?" shows the exit codes. "touch example.txt" was successful; it returned a "0" exit code. grep,
however, couldn't find the string "asdf" in example.txt, so it was unsuccessful, and handed us back
a "1" exit code. Of course, we can use this in a script.
ping, like most utilities, will hand us back a "0" on success, and some value greater than zero
if there's a problem. Here's a cheap-o host monitor script:
#!/bin/bash
if ping -c 1 4.2.2.2
then
logger -i -t pingscript Ping success
else
mail -s "Can't ping 4.2.2.2" support@example.com < pingerror.txt
pagetech.sh
fi
We try to ping a host at 4.2.2.2 (and if you're connected to the net, this should work). If the
ping succeeds, we simply make a note of it. If the ping fails, we send mail to alert us to that
fact, and run another script that pages a tech.
We can also test two conditions at a time with the "&&" (and) and "||" (or) operators. If we
needed two pings to different hosts to succeed, we could use "&&". Changing our if line to read:
if ping 130.57.4.27 && ping 4.4.4.2
will run the first statement. Only if it succeeds do we even try the second. Both the first and
the second statement must succeed to enter the "then" section of the if statement. You can do
something similar outside of a script:
copyfile.sh && alterfile.sh && ftpfile.sh
This example will run three scripts in succession. The next in the series will only run if the
previous script exited with a successful exit code.
The "||" operator runs the first statement, if it succeeds, statement 2 never runs, and we enter
the then clause. If statement 1 fails, statement 2 does run. If it succeeds, we enter the then
clause. Either statement's success will get us into the then clause. Consider revising our
example:
if ping 130.57.4.27 || ping 4.2.2.2
Perhaps we're just trying to see if this machine is connected to the Internet. Say host
130.57.4.27 goes down. As long as we can still ping 4.4.4.2, this script doesn't send mail to us,
or try to page someone.
if/then may be basic, but just this simple decision scheme alone can create powerful scripts.
Let's make it more so. if/then is powerful, but testing exit codes alone can get tedious. The
shell provides the test command to test various conditions. To make the command more concise and
visually pleasing, [ is aliased to test. So this:
if test -e /bin/bash
is the same as this:
if [ -e /bin/bash ]
Most people use the [...] variant. I will do the same. When I say, "test", I'm referring to test
or [. What can test check for us? The man page is of great use here, but I'll give an example or
two:
File tests:
-e test for a file's existence, no matter the type.
-d test for a directory
String comparisons:
s1 = s2 string 1 is equal to string 2.
-n string is not
null
Integer comparisons:
-lt less than
-eq equal
-ne not equal
Other notes for inside the brackets: you can use "!" to negate a test:
if [ ! -e ~/secretplans.txt ]
"And" and "or" are available within the brackets:
and: if [ -e /bin/bash -a -e /bin/false ]
or: if [ -e /bin/bash -o -e /bin/tcsh ]
...and don't forget that all of this can be combined:
if [ ! -d /Users/Shared/reports -o ! -e /nightly/`date +%Y%m%d`.txt ]
&& rsync -delete -av -e ssh repuser@host.example.com:reports/* /Users/Shared/reports ; then
logger -t repsync Reports are current
else
logger -t repsync Reports are out of sync/missing
fi
Whew! That is:
If the "reports" directory or the nightly file doesn't exist, we try to rsync them. If that goes
well, success all around. If the directory or file do exist, we're done. If the rsync fails, we
note it in the system log. That packs a lot in a single "if" statement.
Why are you beating me?
If you're an old Pascal or C hand, you'll recognize these two constructs: while and until. For
the uninitiated, while repeats a block of statements for the duration of a condition being true.
until runs a block of code until a condition is met. One look at them in action, and you'll get it:
while [ -e /run/statusflag.txt ]
do
copyfiles.sh
sleep 300
done
The until block is very similar:
until [ -e thecowscomehome.txt ]
do
jump_over_moon.sh
done
Easy, right? Naturally, you can run all of the tests that the if statement allows.
Why are you beating me?
Sometimes, either you or a variable know how many times a loop should run. (Does that make
anyone else think of Tron?). The constructs of for and forelse allow us to do just this. This is
where you'll see a lot of difference from traditional languages. While you can pull off those kinds
of loops:
#!/bin/bash
for ((i=1;i<=10;i+=1))
do
echo $i
done
...you'll very rarely see that done in practice. What happens to be much cooler than this, though,
is the ability to loop through file lists, directories and command line arguments passed to your
script. This is truly, as the kids say, da bomb. Look at this simple example:
#!/bin/bash
for i in "*.sh"
do
echo $i
done
This script gives me something like this:
$ ./loopy.sh
backup.sh createdmg.sh fctest.sh it.sh loopy.sh speakdate.sh spread.sh syncsm.sh
Well, gosh, I could have simply used the one-line echo *.sh to achieve the same thing. How about
something that gives us info on each file? Line by line output can be achieved in a few different
ways, but here's the way I'm choosing, for the moment:
#!/bin/bash
FILES=`ls *.sh`
for i in $FILES
do
echo $i
if [ -O $i ]; then
echo "You own $i."
fi
if [ -G $i ]; then
echo "You are a group owner of $i."
fi
echo
done
Running the above gives me something akin to this:
$ ./loopy2.sh
backup.sh
You own backup.sh.
You are a group owner of backup.sh.
createdmg.sh
You own createdmg.sh.
fctest.sh
You own fctest.sh.
it.sh
You own it.sh.
Loopy2.sh
You own loopy.sh.
You are a group owner of loopy2.sh
As mentioned, for is a fantastic way for dealing with arguments passed into the script. Here's a
slightly contrived example that will make a directory and copy the files and/or types (based on
extension) into that directory. The new concept here are the parameter variables: @ and #. "$@"
contains a list of each positional parameter, $1, $2...$N that is passed into the script. "$#"
contains the number of parameters passed in. We can even include a little error correction this
way:
#!/bin/bash
thedir=`date +%Y%m%d`
if [ $# -gt 0 ]; then
mkdir $thedir
for list in "$@"; do
cp $list $thedir
done
else
echo "usage: $0 (files)"
fi
Fun and functional! Right?
Why are you beating me?
The last flow control statements we'll cover this month, case and select, bring their own power
to bash, much like the previous flow control variants.
case is a neat alternative to a bevy of if/then/else statements. You immediately know the reason
for this if you've used "case" in Pascal, or "switch" in C. If you've never seen those, one example
and you'll be a pro:
#!/bin/bash
freespace=`df / | tail -1 | awk '{print $5}' | cut -d "%" -f1`
case $freespace in
[1-6]*) report="Plenty of room on /"
;;
[7-8]*) report="You might want to fire up du and take a look at /, it is $freespace percent full."
;;
9[0-9]) report="Can you order a bigger disk and overnight it? / is at $freespace percent!"
;;
*) report="I can't determine the amount of space on /"
;;
esac
echo $report
Of course, the power in the bash version lies in its ability to process patterns and command-line
arguments:
#!/bin/bash
for file in $*; do
case $file in
*.txt) echo "$file is a text file."
;;
*.pl) echo "$file is a perl file."
;;
*.sh) echo "$file is a shell script."
;;
*.gif | *.png | *.jpg | *.jpeg | *.tiff) what=`sips -g format $file | tail -1`
echo "$file is a $what"
sizeh=`sips -g pixelHeight $file | tail -1`
sizew=`sips -g pixelWidth $file | tail -1`
echo "It is $sizew x $sizeh"
;;
*.aiff | *.sd2 | *.wav | *.mp3) echo "$file is some kind of audio file."
;;
*) echo "Sorry, I don't know what kind of file $file is!"
esac
done
Save this in a new file, make it executable and pass it some files you want information about. I
really think the way you can process multiple options on one line is really elegant.
select is really its own unique construct. It will take the list of items passed to it, create a
numbered menu out of those lists and wait for input. Watch it in action (file gives us information
about a file, as seen on line 5):
#!/bin/bash
select theItem; do
if [ $theItem ]; then
file $theItem
break
fi
done
With no in clause (select variable in list), select defaults to the list of command-line
parameters. So, when we run this example, we need to pass in the list to process:
$ ./st.sh *
1) printaudit.pl
2) BidToJob.dmg
3) cl.txt
4) list.txt
5) loopy.sh
#?
Pressing "2", followed by enter, has the program output:
BidToJob.dmg: Macintosh HFS Extended version 4 data last
mounted by: '10.0', created:Tue Dec 7 16:58:10 2004, last
modified: Wed Jan 5 18:07:44 2005, last checked: Tue Dec 7
16:58:10 2004, block size: 4096, number of blocks: 1024,
free blocks: 727
A bit Spartan, perhaps, but certainly functional, and will save you a ton of work. Again, the
power here lies in being able to build menus, completely ad-hoc, based on the contents of a
directory.
It's all about your style
Next month, we'll get a little deeper into why some of the above works, as we probe deeper into
the bowels of bash. Also, we'll expand on other loop constructs. Like any programming, there's
typically more than one way to achieve the same thing. You can avoid all use of until by negating a
while loop. You can split up lists with bash's processing, grep, sed, cut, and other utilities. I
can only serve as a guide. Practice, err, correct; then the apprentice becomes the master.
Ed Marczak, owns and operates Radiotope, a technology consulting company that specializes
in networking, workflow automation, teaching about technology, and helping out. Tech relief at
http://www.radiotope.com
