TweetFollow Us on Twitter

Quoting in the shell

Volume Number: 23 (2007)
Issue Number: 09
Column Tag: MacEnterprise

Quoting in the shell

Dealing with interesting characters in shell programming

By Philip Rinehart, Yale University

QUOTES?

This month, a rather interesting problem arose on the Macenterprise mailing list. How are quotes used and dealt with when scripting for the bash shell? It can be a particularly difficult problem, as OS X allows the use of non-standard characters and spaces in filenames. If quoting is not done properly, unexpected, and even disastrous results may occur. Anyone, remember iTunes 2.0 and destruction of hard drive data? It was a quoting problem! Thus, it is extremely important to quote things properly when shell programming, both for unexpected results, as well as properly sanitizing input.

Single Quotes?

Let's start the process by looking at the use of single quotes in bash shell scripting. The original question was about the use of a directory listing. The directory listing was then piped to a second command, which failed due to spaces in some of the directory names. The specific example:

   ls -F /Applications

Try it. Note how many Applications contain spaces in the name. If Adobe Acrobat 8 Professional were installed, any shell script attempting to perform an action on the path would fail. Why? As the shell interprets spaces as input separators, Adobe, Acrobat, 8 and Professional are all seen as individual paths. In our above example, if the chmod operation were being performed, "Adobe", "Acrobat", "8" and "Professional" would all be modified by chmod. This action would fail naturally, as each of these paths does not exist. Would using single quotes help?

Single quotes treat all information contained between single quotes as exact character data in the bash shell. In practice, single quotes are extraordinarily difficult to use as variables and are not expanded. Even the backslash character is not considered special within single quotes. In our example, this fact means that a variable cannot be used in single quotes when shell scripting. As a result, single quotes aren't going to solve the problem above, so a different solution is required.

Double Quotes?

What about the use of double quotes? Can they solve the problem? Hmm, it get's a bit closer this time. Again, let's use the above list command. When assigned to a variable, multiple words will be treated as a single word. Wow, what a mouthful! Here's a better example. Let's use Adobe Acrobat 8 Professional again. When talking about single quotes, remember the fact that each word is treated separately? With double quoting, the entire word remains intact, and is not split. So, if the variable test is assigned and then echoed out using double quotes, the entire word, Adobe Acrobat 8 Professional, is used. A brief shell script snippet to further reinforce this concept:

#!/bin/bash
test="Adobe Acrobat 8 Professional"
printf "Single word is $test\n"

Note, that when these commands are entered and run that the variable is correctly displayed with printf. Without the quotes, only the word Single is printed. Does this solution work for the problem encountered on the list? Well, not quite. It certainly gets somewhat closer, as spaces are correctly preserved with double quotes, but command substitutions with variables still see each space separated entry as a separate word. If single and double quotes won't solve the problem, what will? Can the field separator be altered, and will it solve the problem?

IFS

IFS? Never heard of it before? IFS stands for Internal Field Separator. It determines how bash interprets word boundaries. By default, the three special characters, space, tab, and newline are used. Using the Adobe Acrobat 8 example, it now should be somewhat clearer as to why each is treated as a separate word. bash reads each space in the name as a field separator. With the default value for IFS, command substitution treats each space as a delimiter. This can be changed. At the top of each shell script, set the IFS variable. Any character can be the field separator, from a semi-colon, space, to a comma. The Macenterprise list came up with the following solution for the problem when a directory listing contains spaces:

   
   IFS="\n"

The author inserted this line at the beginning of the shell script. Now that the IFS separator has been reset, spaces are no longer recognized as valid field separators. The problem is now gone, and the variable is correctly assigned. It is also useful to note that resetting IFS can be used to ensure that the IFS variable has not been improperly set before any shell script is run. As a security precaution when writing a shell script, just as the PATH variable is set, make it a habit to also set the IFS variable. If input is being read in, or parsed, any spoofing or accidental resetting of this variable becomes a non-issue.

USING FIND

Let's return to the original list command that the article opened with. The original question wanted to take a list of directories and then change their permissions. Jeff McCune from The Ohio State University suggested another method on the list. He used the shell find utility to accomplish the same thing in a more efficient and unix-y way. Let's start with the command in its full form:

find /Applications -type d -maxdepth 1 -mindepth 1 -print0 | xargs -0 chmod 750

O.K., that's a bit much! As the find utility may be new to some, let's break it down into its components. The second argument, /Applications, can be any directory that find will operate on. In this case, it is looking in the Applications directory. The next argument, -type, looks for directories, as specified by the option d. After that, the next two arguments instruct the find command not to descend into any of the Application packages. It is accomplished with the maxdepth and mindepth switches. The next option is where the real magic begins. The option, print0, prints the result to standard output, terminating the output with the ASCII NUL character. It will become clear why this option is important in a second. All the results from the command are printed without a terminating newline. Try just the first part of the command before the pipe character, both with -print and -print0. Note the difference, as it is critical to this commands proper function.

Now that the find results are complete, it is time to process them using xargs. xargs takes an argument list, and processes the results with any other command line utility. Remember the use of print0. As the NUL character terminates the output, xargs must use the -0 flag. Without it, xargs does not know how to interpret the results from the find command. This technique is also generally considered faster by Unix gurus, as it only executes the chmod command once, which may not be the case for a shell loop, or by using the –exec switch with the find utility.

As you can see, the problem presented on the list is not as straightforward as one might expect. As always, there is more than one way to solve the problem, but with a full understanding of the problem's intricacies, it should now be trivial to solve. Until next month, see you on the lists!


Philip Rinehart is co-chair of the steering committee leading the Mac OS X Enterprise Project (macenterprise.org) and is the Lead Mac Analyst at Yale University. He has been using Macintosh Computers since the days of the Macintosh SE, and Mac OS X since its Developer Preview Release. Before coming to Yale, he worked as a Unix system administrator for a dot-com company. He can be reached at: philip.rinehart@yale.edu.

The MacEnterprise project is a community of IT professionals sharing information and solutions to support Macs in an enterprise. We collaborate on the deployment, management, and integration of Mac OS X client and server computers into multi-platform computing environments.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

PDFKey Pro 4.3 - Edit and print password...
PDFKey Pro can unlock PDF documents protected for printing and copying when you've forgotten your password. It can now also protect your PDF files with a password to prevent unauthorized access and/... Read more
Kodi 15.0.beta1 - Powerful media center...
Kodi (was XBMC) is an award-winning free and open-source (GPL) software media player and entertainment hub that can be installed on Linux, OS X, Windows, iOS, and Android, featuring a 10-foot user... Read more
DiskCatalogMaker 6.4.12 - Catalog your d...
DiskCatalogMaker is a simple disk management tool which catalogs disks. Simple, light-weight, and fast. Finder-like intuitive look and feel. Super-fast search algorithm. Can compress catalog data... Read more
Macs Fan Control 1.3.0.0 - Monitor and c...
Macs Fan Control allows you to monitor and control almost any aspect of your computer's fans, with support for controlling fan speed, temperature sensors pane, menu-bar icon, and autostart with... Read more
Lyn 1.5.11 - Lightweight image browser a...
Lyn is a lightweight and fast image browser and viewer designed for photographers, graphic artists and Web designers. Featuring an extremely versatile and aesthetically pleasing interface, it... Read more
NeoOffice 2014.11 - Mac-tailored, OpenOf...
NeoOffice is a complete office suite for OS X. With NeoOffice, users can view, edit, and save OpenOffice documents, PDF files, and most Microsoft Word, Excel, and PowerPoint documents. NeoOffice 3.x... Read more
LaunchBar 6.4 - Powerful file/URL/email...
LaunchBar is an award-winning productivity utility that offers an amazingly intuitive and efficient way to search and access any kind of information stored on your computer or on the Web. It provides... Read more
Remotix 3.1.4 - Access all your computer...
Remotix is a fast and powerful application to easily access multiple Macs (and PCs) from your own Mac. Features Complete Apple Screen Sharing support - including Mac OS X login, clipboard... Read more
DesktopLyrics 2.6.6 - Displays current i...
DesktopLyrics is an application that displays the lyrics of the song currently playing in "iTunes" right on your desktop. The lyrics for the song have to be set in iTunes; DesktopLyrics does nothing... Read more
VOX 2.5.1 - Music player that supports m...
VOX is a beautiful music player that supports many filetypes. The beauty is in its simplicity, yet behind the minimal exterior lies a powerful music player with a ton of features and support for all... Read more

This Week at 148Apps: May 18-22, 2015
May Days at 148Apps How do you know what apps are worth your time and money? Just look to the review team at 148Apps. We sort through the chaos and find the apps you're looking for. The ones we love become Editor’s Choice, standing out above the... | Read more »
Biz Builder Delux (Games)
Biz Builder Delux 1.0.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0.0 (iTunes) Description: Ah, there's nothing like the rhythmic bustle of a burgeoning business burg... especially when you're the one building it... | Read more »
Auroch Digital is Bringing Back Games Wo...
| Read more »
Blades of Brim is a New Endless Runner f...
SYBO Games, the minds behind the ever-popular Subway Surfers, have announced their latest project: Blades of Brim. [Read more] | Read more »
Carbo - Handwriting in the Digital Age...
Carbo - Handwriting in the Digital Age 1.0 Device: iOS Universal Category: Productivity Price: $3.99, Version: 1.0 (iTunes) Description: | Read more »
Draggy Dead (Games)
Draggy Dead 1.1 Device: iOS Universal Category: Games Price: $.99, Version: 1.1 (iTunes) Description: Ditch your dead end job and take up a rewarding career in Grave Robbing today!Guide the recently deceased to a fun filled life of... | Read more »
Bad Dinos (Games)
Bad Dinos 1.0.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0.0 (iTunes) Description: | Read more »
The Apple Watch isn't Great as a Fi...
| Read more »
Show the World What You See With Stre.am...
Live broadcasting is getting popular on mobile devices, which is why you can now get Stre.am, by Infinite Takes. [Read more] | Read more »
PhotoTime's 2.1 Update Adds Apple W...
The latest PhotoTime update is adding even more functionality to the handy photo organizing app. Yep, including Apple Watch support. [Read more] | Read more »

Price Scanner via MacPrices.net

12-inch MacBook stock status for Monday, May...
The new 12″ Retina MacBooks are still on backorder at The Apple Store with a 3-5 week waiting period. However, a few models are in stock today at Apple resellers. Stock is limited, so act now if you’... Read more
New 27-inch 3.3GHz 5K iMac in stock with free...
Adorama has the new 27″ 3.3GHz 5K iMac in stock today for $1999 including free shipping plus NY & NJ sales tax only. Adorama will include a free copy of Apple’s 3-year AppleCare Protection Plan. Read more
Memorial Day Weekend Sale: New 27-inch 3.3GHz...
Best Buy has the new 27″ 3.3GHz 5K iMac on sale for $1899.99 this weekend. Choose free shipping or free local store pickup (if available). Sale price for online orders only, in-store prices may vary... Read more
OtterBox Maximizes Portability, Productivity...
From the kitchen recipe book to the boarsroom presentation, the OtterBox Agility Tablet System turns tablets into one of the most versatile pieces of handheld technology available. Available now, the... Read more
Launch of New Car App Gallery and Open Develo...
Automatic, a company on a mission to bring the power of the Internet into every car, has announced the launch of the Automatic App Gallery, an app store for nearly every car or truck on the road... Read more
Memorial Day Weekend Sale: 13-inch 1.6GHz Mac...
Best Buy has the new 13″ 1.6GHz/128GB MacBook Air on sale for $849 on their online store this weekend. Choose free shipping or free local store pickup (if available). Sale price for online orders... Read more
Memorial Day Weekend Sale: 27-inch 3.5GHz 5K...
Best Buy has the 27″ 3.5GHz 5K iMac on sale for $2099.99 this weekend. Choose free shipping or free local store pickup (if available). Sale price for online orders only, in-store prices may vary.... Read more
Sale! 16GB iPad mini 3 for $349, save $50
B&H Photo has the 16GB iPad mini 3 WiFi on sale for $349 including free shipping plus NY sales tax only. Their price is $50 off MSRP, and it’s the lowest price available for this model. Read more
Price drop on 2014 15-inch Retina MacBook Pro...
B&H Photo has dropped prices on 2014 15″ Retina MacBook Pros by $200. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.2GHz Retina MacBook Pro: $1799.99 save $200 - 15″ 2.5GHz... Read more
With a Mission to Make Mobile Free, Scratch W...
Scratch Wireless, claiming to be the world’s first truly free mobile service, has announced the availability of a new Scratch-enabled Android smartphone, the Coolpad Arise. The smartphone is equipped... Read more

Jobs Board

Payments Counsel, *Apple* Pay (mobile payme...
**Job Summary** Apple is looking for an atto ey to join Apple 's Legal Department to support Apple Pay. **Key Qualifications** 7+ years of relevant experience Read more
Touch Hardware Design and Integration Enginee...
…Summary** Design, develop, and launch next-generation Touch solutions in the new Apple Watch product category. The Touch team develops cutting-edge Touch solutions and Read more
*Apple* Solutions Consultant - Retail Sales...
**Job Summary** As an Apple Solutions Consultant (ASC) you are the link between our customers and our products. Your role is to drive the Apple business in a retail Read more
*Apple* TV Live Streaming Frameworks Test En...
**Job Summary** Work and contribute towards the engineering of Apple 's state-of-the-art products involving video, audio, and graphics in Interactive Media Group (IMG) at Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.