TweetFollow Us on Twitter

July 02 Section 7

Volume Number: 18 (2002)
Issue Number: 07
Column Tag: Section 7

by Rich Morin

More Basic Commands

and a first look at “shell scripts”

Previous “Section 7” columns have looked at several BSD commands, but they haven’t said much about the ways in which commands get performed. Let’s do that now, starting with the Terminal application. Each time you start up a new Terminal window, Mac OS X has to start up a program to talk to it. The program it starts is a “command interpreter”, most often referred to as a “shell”.

Whenever a Terminal window is active, your keyboard is tied to the “standard input” of the program that is currently running “in” the window. Similarly, the window’s text area displays the program ‘s “standard output”. If the shell starts up another program (such as ls), the shell must “loan” these I/O connections to the other program. In short, a Terminal window is always connected to some program; most often, this will be the shell.

Each shell runs as a separate process, so the “state” that gets set in one “shell session” is not shared with other shell sessions. Thus, if you use cd to change the “current directory” in one Teminal window, the shell sessions in your other windows will not be affected.

Because cd affects information that the shell must maintain (and hand off to any programs it starts up), cd is actually “built in” to the shell itself. Most commands, however, are not built into the shell. Instead, they are implemented as:

  • aliases or shell functions – These are sequences of commands, kept in the shell’s memory and interpreted upon demand.
  • shell scripts – These are sequences of commands, kept in separate files and interpreted upon demand. Sometimes the shell itself will interpret the script; other times, as discussed below, it will start up a separate program to do so.
  • executable binaries – These are separate programs, kept in separate files and executed upon demand. To run them, the shell must fork(2), then exec(3) the relevant file.

In the case of built-ins, aliases, and shell functions, the shell “knows” what needs to be done. In the case of shell scripts and executable binaries, however, it must find the requested file, determine its nature, and proceed accordingly.

If a command is not a built-in, an alias, or a shell function, the shell looks for an executable file of that name in one of several directories. Because the shell may cache the contents of these directories in an internal data structure, some shells require the user to run a special command (“rehash”) when a new command is added. The list of these directories, called the “search path”, is contained in the $path variable:

[localhost:~] rdm% echo $path
/Users/rdm/bin/powerpc-apple-darwin /Users/rdm/bin ...

The which command can be used to predict the shells’ behavior for a given command:

[localhost:~] rdm% which cwd l 
cwd:     aliased to echo $cwd
l:       aliased to ls -lg
[localhost:~] rdm% which cd echo setenv 
cd: shell built-in command.
echo: shell built-in command.
setenv: shell built-in command.
[localhost:~] rdm% which apropos keytool
/usr/bin/apropos
/usr/bin/keytool

Finding and executing files (e.g., /usr/bin/apropos) takes time, but it allows us to add new commands quite trivially. Imagine how awkward and constraining it would be if each new command had to be linked into the shell!

Now that we have the full path names of the commands, we can examine them a bit:

[localhost:~] rdm% cd /usr/bin
[localhost:/usr/bin] rdm% file apropos keytool
apropos: Mach-O executable ppc
keytool: Bourne shell script text

As file informs us, /usr/bin/apropos is an executable binary file for the PowerPC, encoded in “Mach-O” format. /usr/bin/keytool, in contrast, is a text file which must be run interpretively by sh (the “Bourne shell”) as a “shell script”. But wait a second; how do file (and the shell) know that? Maybe it’s time to take a look at /usr/bin/keytool:

[localhost:/usr/bin] rdm% cat keytool
#!/bin/sh
#
# Shell wrapper to launch keytool.
#
...

The first two characters of the file (“#!”) tell the user’s shell that this is a script. The rest of the line gives the full path name (“/bin/sh”) of the program which must be used to interpret the script. Because each script can declare its own language (e.g., Bourne shell, C shell, Perl), new scripting languages can be added at any time.

If a script starts with a sharp sign (“#”), but not the “#!” sequence, it is treated as a C shell script. If a script begins with any other character, it is treated as a Bourne shell script. Finally, you may encounter scripts that start like “#!/usr/bin/env perl“. This tells the system to run /usr/bin/env, which will then go off and find the “appropriate” copy of perl.

The same header information is used by file to determine the type of a given file. To see file’s list of “magic numbers” (and interpretations), let’s look at /etc/magic:

[localhost:/usr/bin] rdm% cd /etc
[localhost:/etc] rdm% wc -l magic
    3837 /etc/magic
[localhost:/etc] rdm% grep ‘shell script’ magic
0  string  #!/bin/sh     Bourne shell script text
...

“wc -l” (word count, in line mode) tells us that there are about 4K lines in /etc/magic, so we don’t try to list it. Instead, we use grep (global regular expression print) to display lines that contain the string “shell script”.

As discussed last month, most of BSD’s “system metadata” (e.g., control files, log files) is kept in ASCII format. This works well with BSD’s wide range of text processing tools, allowing us to extract and process information in a very flexible manner.

If /etc/magic had been encoded in some proprietary, binary format (e.g., Microsoft Word), the exercise above would have required us to start up a particular application. And, if the application wasn’t able to do what we wanted, we’d have been stymied.

Scripting languages

BSD provides a variety of scripting languages. You are free to use any, all, or none of them, as you prefer. Here are some (biased :-) notes that may help you in making your selections:

  • The Bourne shell (sh) is not particularly convenient as an interactive command interpreter, but it works well as a programming language. The Korn and Z shells (ksh, zsh) are upwardly-compatible versions of the Bourne shell. The former is not supplied as part of Mac OS X, but the latter is and may deserve a look.
  • The Bourne-Again shell (bash) is upwardly compatible with the Bourne shell, but it also borrows a variety of features from other shells. It is the default shell for most Linux distributions, but it is not supplied as part of Mac OS X.
  • The C shell (csh) is very convenient as an interactive command interpreter, but it loses badly as a programming language due to its lack of syntax recognition. For example, multi-line commands always require backslashes at the end of non-terminal lines, even when the shell should be able to recognize that the command isn’t finished. Most BSD systems, including Mac OS X, actually provide tcsh, rather than csh. As the tcsh man page says:
    “tcsh is an enhanced, but completely compatible, version of csh(1). It is a command language interpreter, usable both as an interactive login shell and as a shell-script command processor. It includes a command-line editor, programmable word completion, spelling correction, a history mechanism, job control, and a C-like syntax.”
  • Awk (awk) is a convenient and simple scripting language, suitable for simple formatting and calculation tasks.
  • If things start to get too complex for Awk or the shell to handle, you should probably turn to Perl (perl). Perl is an extremely powerful language, with a large and active user community.
  • Python and Ruby are object-oriented scripting languages with small but vocal followings. Neither language is supplied as part of Mac OS X, but (like the other languages mentioned above), they are readily available for you to download and install.
  • Finally, if you’re a Tcl fan, you might want to try out tclsh (“a shell-like application that reads Tcl commands“).

Rich Morin has been using computers since 1970, Unix since 1983, and Mac-based Unix since 1986 (when he helped Apple create A/UX 1.0). When he isn’t writing this column, Rich runs Prime Time Freeware (www.ptf.com), a publisher of books and CD-ROMs for the Free and Open Source software community. Feel free to write to Rich at rdm@ptf.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Maya 2016 - Professional 3D modeling and...
Maya is an award-winning software and powerful, integrated 3D modeling, animation, visual effects, and rendering solution. Because Maya is based on an open architecture, all your work can be scripted... Read more
RapidWeaver 6.2.3 - Create template-base...
RapidWeaver is a next-generation Web design application to help you easily create professional-looking Web sites in minutes. No knowledge of complex code is required, RapidWeaver will take care of... Read more
MacFamilyTree 7.5.2 - Create and explore...
MacFamilyTree gives genealogy a facelift: it's modern, interactive, incredibly fast, and easy to use. We're convinced that generations of chroniclers would have loved to trade in their genealogy... Read more
Paragraphs 1.0.1 - Writing tool just for...
Paragraphs is an app just for writers. It was built for one thing and one thing only: writing. It gives you everything you need to create brilliant prose and does away with the rest. Everything in... Read more
BlueStacks App Player 0.9.21 - Run Andro...
BlueStacks App Player lets you run your Android apps fast and fullscreen on your Mac. Version 0.9.21: Note: Now requires OS X 10.8 or later running on a 64-bit Intel processor. Initial stable... Read more
Tweetbot 2.0.2 - Popular Twitter client....
Tweetbot is a full-featured OS X Twitter client with a lot of personality. Whether it's the meticulously-crafted interface, sounds and animation, or features like multiple timelines and column views... Read more
Apple iBooks Author 2.3 - Create and pub...
Apple iBooks Author helps you create and publish amazing Multi-Touch books for iPad. Now anyone can create stunning iBooks textbooks, cookbooks, history books, picture books, and more for iPad. All... Read more
NeoOffice 2014.12 - Mac-tailored, OpenOf...
NeoOffice is a complete office suite for OS X. With NeoOffice, users can view, edit, and save OpenOffice documents, PDF files, and most Microsoft Word, Excel, and PowerPoint documents. NeoOffice 3.x... Read more
Apple iTunes 12.2 - Play Apple Music...
Apple iTunes lets you organize and stream Apple Music, download and watch video and listen to Podcasts. It can automatically download new music, app, and book purchases across all your devices and... Read more
Apple Security Update 2015-005 - For OS...
Apple Security Update 2015-005 is recommended for all users and improves the security of OS X. For detailed information about the security content of this update, please visit: http://support.apple.... Read more

Heroki (Games)
Heroki 1.0 Device: iOS Universal Category: Games Price: $7.99, Version: 1.0 (iTunes) Description: CLEAR THE SKIES FOR A NEW HERO!The peaceful sky village of Levantia is in danger! The dastardly Dr. N. Forchin and his accomplice,... | Read more »
Hands-On With Raceline CC
Set for release soon, Rebellion’s motorbike racing game, Raceline CC certainly looks stylish. But how does it play? I got my hands on a preview build to answer exactly that. | Read more »
Siegefall - Tips, Tricks, and Strategies...
So, you fancy establishing a base and ruling the world again. Siegefall is a convenient place to do that, but how about some great tips and tricks on how best to go about it? Here are a few ideas on how to get ahead as a beginner to this medieval... | Read more »
The WWE Comes to Racing Rivals - Because...
Racing Rivals is a racing game that's all about, well, rivalry. And who knows rivalry better than WWE superstars (shhhh, that was rhetorical)? [Read more] | Read more »
Hey, Who Put Apple Music in My SoundHoun...
One of the App Store's popular music discovery sources - SoundHound - has already been updated to include Apple's own music discovery source - Apple Music. That was fast! [Read more] | Read more »
Arcane Legends has a New Expansion Calle...
Arcane Legends has been going strong since it debuted at the tail end of 2012. So well, in fact, that it's already up to its sixth expansion. [Read more] | Read more »
Vector 2 is Officially a Thing and it...
Vector is a pretty cool parkour-driven runner that's gotten a pretty decent following since it first came out - although personally I think more people could stand to show it some love. Anyway, Nekki has announced that a sequel isofficially on its... | Read more »
Get Ready to Trucksform and Roll Out (an...
It looks like NuOxygen is bringing the truck-transforming racer Trucksform (get it?) to iOS in a couple of weeks. Although really it's more of an auto-driver than a racer. But still, transforming trucks! [Read more] | Read more »
This Week at 148Apps:June 22-26, 2015
June's Summer Journey Continues With 148Apps How do you know what apps are worth your time and money? Just look to the review team at 148Apps. We sort through the chaos and find the apps you're looking for. The ones we love become Editor’s Choice,... | Read more »
LEGO® Minifigures Online (Games)
LEGO® Minifigures Online 1.0.1 Device: iOS iPhone Category: Games Price: $4.99, Version: 1.0.1 (iTunes) Description: | Read more »

Price Scanner via MacPrices.net

Logo Pop Free Vector Logo Design App For OS X...
128bit Technologies has released of Logo Pop Free 1.2 for Mac OS X, a vector based, full-fledged, logo design app available exclusively on the Mac App Store for the agreeable price of absolutely free... Read more
21-inch 1.4GHz iMac on sale for $999, save $1...
B&H Photo has new 21″ 1.4GHz iMac on sale for $999 including free shipping plus NY sales tax only. Their price is $100 off MSRP. Best Buy has the 21″ 1.4GHz iMac on sale for $999.99 on their... Read more
16GB iPad mini 3 on sale for $339, save $60
B&H Photo has the 16GB iPad mini 3 WiFi on sale for $339 including free shipping plus NY tax only. Their price is $60 off MSRP. Read more
Save up to $40 on iPad Air 2, NY tax only, fr...
B&H Photo has iPad Air 2s on sale for up to $40 off MSRP including free shipping plus NY sales tax only: - 16GB iPad Air 2 WiFi: $489 $10 off - 64GB iPad Air 2 WiFi: $559 $40 off - 128GB iPad Air... Read more
Apple Releases OS X 10.10.4 With WIFi Fix, iO...
On Tuesday, Apple released final versions of OS X 10.10.4 and iOS 8.4, as well as updates for the Safari browser for OS X Yosemite, Mavericks, and Mountain Lion. The OS X 10.10.4 update focuses on... Read more
Dual-Band High-Gain Antennas for Home Wi-Fi N...
Linksys has announced what it claims are the first dual-band, omni-directional high-gain antennas for the consumer market. The new Linksys high-gain antennas available in a 2- and 4-pack (WRT004ANT... Read more
Apple refurbished 2014 15-inch Retina MacBook...
The Apple Store has Apple Certified Refurbished 2014 15″ 2.2GHz Retina MacBook Pros available for $1609, $390 off original MSRP. Apple’s one-year warranty is included, and shipping is free. They have... Read more
Clearance 2014 MacBook Airs available for up...
Adorama has 2014 MacBook Airs on sale for up to $301 off original MSRP including NY + NJ sales tax and free shipping: - 11″ 256GB MacBook Air: $798 $301 off original MSRP - 13″ 128GB MacBook Air: $... Read more
5K iMacs on sale for $100 off MSRP, free ship...
B&H Photo has the new 27″ 3.3GHz 5K iMac on sale for $1899.99 including free shipping plus NY tax only. Their price is $100 off MSRP. They have the 27″ 3.5GHz 5K iMac on sale for $2199, also $100... Read more
27-inch 3.2GHz iMac on sale for $1679, save $...
B&H Photo has the 27″ 3.2GHz iMac on sale for $1679.99 including free shipping plus NY sales tax only. Their price is $120 off MSRP. Read more

Jobs Board

Senior Payments Security Manager - *Apple*...
**Job Summary** Apple , Inc. is looking for a highly motivated, innovative and hands-on senior payments security manager to join the Apple Pay security team. You will Read more
Project Manager - *Apple* Pay Security - Ap...
**Job Summary** The Apple Pay Security team is seeking a highly organized, results-driven Project Manager to drive the development of Apple Pay Security. If you are Read more
Web Developer, *Apple* Online Store Innovat...
**Job Summary** The Apple Online Store Innovation Lab team is seeking a talented individual with strong web development and design skills to prototype future Apple Read more
*Apple* TV Live Streaming Frameworks Test En...
**Job Summary** Work and contribute towards the engineering of Apple 's state-of-the-art products involving video, audio, and graphics in Interactive Media Group (IMG) at Read more
Project Manager, WW *Apple* Fulfillment Ope...
…a senior project manager / business analyst to work within our Worldwide Apple Fulfillment Operations and the Business Process Re-engineering team. This role will work Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.