TweetFollow Us on Twitter

Quoting in the shell

Volume Number: 23 (2007)
Issue Number: 09
Column Tag: MacEnterprise

Quoting in the shell

Dealing with interesting characters in shell programming

By Philip Rinehart, Yale University

QUOTES?

This month, a rather interesting problem arose on the Macenterprise mailing list. How are quotes used and dealt with when scripting for the bash shell? It can be a particularly difficult problem, as OS X allows the use of non-standard characters and spaces in filenames. If quoting is not done properly, unexpected, and even disastrous results may occur. Anyone, remember iTunes 2.0 and destruction of hard drive data? It was a quoting problem! Thus, it is extremely important to quote things properly when shell programming, both for unexpected results, as well as properly sanitizing input.

Single Quotes?

Let's start the process by looking at the use of single quotes in bash shell scripting. The original question was about the use of a directory listing. The directory listing was then piped to a second command, which failed due to spaces in some of the directory names. The specific example:

   ls -F /Applications

Try it. Note how many Applications contain spaces in the name. If Adobe Acrobat 8 Professional were installed, any shell script attempting to perform an action on the path would fail. Why? As the shell interprets spaces as input separators, Adobe, Acrobat, 8 and Professional are all seen as individual paths. In our above example, if the chmod operation were being performed, "Adobe", "Acrobat", "8" and "Professional" would all be modified by chmod. This action would fail naturally, as each of these paths does not exist. Would using single quotes help?

Single quotes treat all information contained between single quotes as exact character data in the bash shell. In practice, single quotes are extraordinarily difficult to use as variables and are not expanded. Even the backslash character is not considered special within single quotes. In our example, this fact means that a variable cannot be used in single quotes when shell scripting. As a result, single quotes aren't going to solve the problem above, so a different solution is required.

Double Quotes?

What about the use of double quotes? Can they solve the problem? Hmm, it get's a bit closer this time. Again, let's use the above list command. When assigned to a variable, multiple words will be treated as a single word. Wow, what a mouthful! Here's a better example. Let's use Adobe Acrobat 8 Professional again. When talking about single quotes, remember the fact that each word is treated separately? With double quoting, the entire word remains intact, and is not split. So, if the variable test is assigned and then echoed out using double quotes, the entire word, Adobe Acrobat 8 Professional, is used. A brief shell script snippet to further reinforce this concept:

#!/bin/bash
test="Adobe Acrobat 8 Professional"
printf "Single word is $test\n"

Note, that when these commands are entered and run that the variable is correctly displayed with printf. Without the quotes, only the word Single is printed. Does this solution work for the problem encountered on the list? Well, not quite. It certainly gets somewhat closer, as spaces are correctly preserved with double quotes, but command substitutions with variables still see each space separated entry as a separate word. If single and double quotes won't solve the problem, what will? Can the field separator be altered, and will it solve the problem?

IFS

IFS? Never heard of it before? IFS stands for Internal Field Separator. It determines how bash interprets word boundaries. By default, the three special characters, space, tab, and newline are used. Using the Adobe Acrobat 8 example, it now should be somewhat clearer as to why each is treated as a separate word. bash reads each space in the name as a field separator. With the default value for IFS, command substitution treats each space as a delimiter. This can be changed. At the top of each shell script, set the IFS variable. Any character can be the field separator, from a semi-colon, space, to a comma. The Macenterprise list came up with the following solution for the problem when a directory listing contains spaces:

   
   IFS="\n"

The author inserted this line at the beginning of the shell script. Now that the IFS separator has been reset, spaces are no longer recognized as valid field separators. The problem is now gone, and the variable is correctly assigned. It is also useful to note that resetting IFS can be used to ensure that the IFS variable has not been improperly set before any shell script is run. As a security precaution when writing a shell script, just as the PATH variable is set, make it a habit to also set the IFS variable. If input is being read in, or parsed, any spoofing or accidental resetting of this variable becomes a non-issue.

USING FIND

Let's return to the original list command that the article opened with. The original question wanted to take a list of directories and then change their permissions. Jeff McCune from The Ohio State University suggested another method on the list. He used the shell find utility to accomplish the same thing in a more efficient and unix-y way. Let's start with the command in its full form:

find /Applications -type d -maxdepth 1 -mindepth 1 -print0 | xargs -0 chmod 750

O.K., that's a bit much! As the find utility may be new to some, let's break it down into its components. The second argument, /Applications, can be any directory that find will operate on. In this case, it is looking in the Applications directory. The next argument, -type, looks for directories, as specified by the option d. After that, the next two arguments instruct the find command not to descend into any of the Application packages. It is accomplished with the maxdepth and mindepth switches. The next option is where the real magic begins. The option, print0, prints the result to standard output, terminating the output with the ASCII NUL character. It will become clear why this option is important in a second. All the results from the command are printed without a terminating newline. Try just the first part of the command before the pipe character, both with -print and -print0. Note the difference, as it is critical to this commands proper function.

Now that the find results are complete, it is time to process them using xargs. xargs takes an argument list, and processes the results with any other command line utility. Remember the use of print0. As the NUL character terminates the output, xargs must use the -0 flag. Without it, xargs does not know how to interpret the results from the find command. This technique is also generally considered faster by Unix gurus, as it only executes the chmod command once, which may not be the case for a shell loop, or by using the –exec switch with the find utility.

As you can see, the problem presented on the list is not as straightforward as one might expect. As always, there is more than one way to solve the problem, but with a full understanding of the problem's intricacies, it should now be trivial to solve. Until next month, see you on the lists!


Philip Rinehart is co-chair of the steering committee leading the Mac OS X Enterprise Project (macenterprise.org) and is the Lead Mac Analyst at Yale University. He has been using Macintosh Computers since the days of the Macintosh SE, and Mac OS X since its Developer Preview Release. Before coming to Yale, he worked as a Unix system administrator for a dot-com company. He can be reached at: philip.rinehart@yale.edu.

The MacEnterprise project is a community of IT professionals sharing information and solutions to support Macs in an enterprise. We collaborate on the deployment, management, and integration of Mac OS X client and server computers into multi-platform computing environments.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Apple Remote Desktop 3.8 - Remotely cont...
Apple Remote Desktop is the best way to manage the Mac computers on your network. Distribute software, provide real-time online help to end users, create detailed software and hardware reports, and... Read more
NeoOffice 2014.7 - Mac-tailored, OpenOff...
NeoOffice is a complete office suite for OS X. With NeoOffice, users can view, edit, and save OpenOffice documents, PDF files, and most Microsoft Word, Excel, and PowerPoint documents. NeoOffice 3.x... Read more
DesktopLyrics 2.6.6 - Displays current i...
DesktopLyrics is an application that displays the lyrics of the song currently playing in "iTunes" right on your desktop. The lyrics for the song have to be set in iTunes; DesktopLyrics does nothing... Read more
Ember 1.8.3 - Versatile digital scrapboo...
Ember (formerly LittleSnapper) is your digital scrapbook of things that inspire you: websites, photos, apps or other things. Just drag in images that you want to keep, organize them into relevant... Read more
Apple iTunes 12.1 - Manage your music, m...
Apple iTunes lets you organize and play digital music and video on your computer. It can automatically download new music, app, and book purchases across all your devices and computers. And it's a... Read more
LibreOffice 4.4.3 - Free, open-source of...
LibreOffice is an office suite (word processor, spreadsheet, presentations, drawing tool) compatible with other major office suites. The Document Foundation is coordinating development and... Read more
FoldersSynchronizer 4.2.1 - Synchronize...
FoldersSynchronizer is a popular and useful utility that synchronizes and backs-up files, folders, disks and boot disks. On each session you can apply special options like Timers, Multiple Folders,... Read more
Simon 4.0.2 - Monitor changes and crashe...
Simon monitors websites and alerts you of crashes and changes. Select pages to monitor, choose your alert options, and customize your settings. Simon does the rest. Keep a watchful eye on your... Read more
Cocktail 8.1.2 - General maintenance and...
Cocktail is a general purpose utility for OS X that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more
Cyberduck 4.6.4 - FTP and SFTP browser....
Cyberduck is a robust FTP/FTP-TLS/SFTP browser for the Mac whose lack of visual clutter and cleverly intuitive features make it easy to use. Support for external editors and system technologies such... Read more

Playworld Superheroes Review
Playworld Superheroes Review By Tre Lawrence on January 30th, 2015 Our Rating: :: HERO CRAFTINGUniversal App - Designed for iPhone and iPad It’s all about the imagination, fighting bad creatures — and looking good while doing so.   | Read more »
Join the SpongeBob Bubble Party in this...
Join the SpongeBob Bubble Party in this New Match 3 Bubble Poppin’ Frenzy Posted by Jessica Fisher on January 30th, 2015 [ permalink ] | Read more »
Handpick Review
Handpick Review By Jennifer Allen on January 30th, 2015 Our Rating: :: TANTALIZING SUGGESTIONSiPhone App - Designed for the iPhone, compatible with the iPad Handpick will make you hungry, as well as inspire you to cook something... | Read more »
Storm the Halls of Echo Base in First St...
Storm the Halls of Echo Base in First Star Wars: Galactic Defense Event Posted by Jessica Fisher on January 30th, 2015 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Contradiction Review
Contradiction Review By Tre Lawrence on January 30th, 2015 Our Rating: :: SPOT THE LIEiPad Only App - Designed for the iPad Contradiction is a live action point and click adventure that’s pretty engaging.   Developer: Tim Follin... | Read more »
Unlock Sunshine Girl in Ironkill with th...
Unlock Sunshine Girl in Ironkill with this special 148Apps code Posted by Rob Rich on January 29th, 2015 [ permalink ] Robo-fighter Ironkill has been out on iOS a | Read more »
Crossroad Zombies Review
Crossroad Zombies Review By Jordan Minor on January 29th, 2015 Our Rating: :: CROSSWALKING DEADiPad Only App - Designed for the iPad Crossroad Zombies is a rough draft of a cool genre mash-up.   | Read more »
Blood Brothers 2 – Tips, Cheats, and Str...
War is hell: Is it the kind of hell you want to check out? Read our Blood Brothers 2 review to find out! Blood Brothers 2, DeNA’s follow-up to the original Blood Brothers, is an intriguing card collecting / role-playing / strategy hybrid. There’s... | Read more »
Blood Brothers 2 Review
Blood Brothers 2 Review By Nadia Oxford on January 29th, 2015 Our Rating: :: AN AGGRAVATING RELATIVEUniversal App - Designed for iPhone and iPad Blood Brothers 2 is built on a simple, solid foundation, but its free-to-play system... | Read more »
I AM BREAD, the Toast of the Town, is Ro...
Have you ever dreamt of being deliciously gluten-y? Do you feel passionate about Rye and Wheat? The guys at Bossa Studios do and that is why they are bringing I AM BREAD to iOS soon. The loafy app will feature all the new content that is being... | Read more »

Price Scanner via MacPrices.net

Intel Aims to Transform Workplace With 5th-Ge...
Intel Corporation today announced the availability of its 5th generation Intel Core vPro processor family that provides cutting-edge features to enable a new and rapidly shifting workplace. To meet... Read more
iOS App Sharalike Introduces New Instant Smar...
Sharalike slideshow and photo management software for iOS, is making it easier than ever to create shareable meaningful moments with its new instant SmartShow technology. Staying organized is a goal... Read more
Apple Becomes World’s Largest Smartphone Vend...
According to the latest research data from Strategy Analytics, as global smartphone shipments grew 31 percent annually to reach a record 380 million units in the fourth quarter of 2014. Apple became... Read more
Cut the Cord: OtterBox Resurgence Power Case...
Dead batteries and broken phones are two of the biggest issues for smartphone users today. Otterbox addresses both with the new Resurgence Power Case for Apple iPhone 6, promising to make those panic... Read more
13-inch Retina MacBook Pros on sale for up to...
B&H Photo has 13″ Retina MacBook Pros on sale for $200 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.6GHz/128GB Retina MacBook Pro: $1199.99 save $100 - 13″ 2.6GHz/... Read more
15-inch 2.5GHz Retina MacBook Pro on sale for...
 B&H Photo has the 15″ 2.5GHz Retina MacBook Pro on sale for $2319.99 including free shipping plus NY sales tax only. Their price is $180 off MSRP, and it’s the lowest price available for this... Read more
Back in stock: Refurbished iPod nanos for $99...
The Apple Store has Apple Certified Refurbished 16GB iPod nanos available for $99 including free shipping and Apple’s standard one-year warranty. That’s $50 off the cost of new nanos. Most colors are... Read more
Apple lowers price on refurbished 256GB MacBo...
The Apple Store has lowered prices on Apple Certified Refurbished 2014 MacBook Airs with 256GB SSDs, now available for up to $200 off the cost of new models. An Apple one-year warranty is included... Read more
New Good Management Suite Simplifies Enterpri...
Good Technology has announced the availability of the Good Management Suite, a comprehensive cross-platform solution for organizations getting started with mobile business initiatives. Built on the... Read more
15-inch 2.0GHz Retina MacBook Pro (refurbishe...
The Apple Store has Apple Certified Refurbished previous-generation 15″ 2.0GHz Retina MacBook Pros available for $1489 including free shipping plus Apple’s standard one-year warranty. Their price is... Read more

Jobs Board

Sr. Mac Expert- *Apple* Online Store - Apple...
**Job Summary** The World Wide Apple Online Store (AOS) Sales and Service team is looking for motivated, outgoing, and tech savvy individuals who want to offer Apple Read more
*Apple* Solutions Consultant- Retail Sales (...
**Job Summary** The ASC is an Apple employee who serves as an Apple brand ambassador and influencer in a Reseller's store. The ASC's role is to grow Apple Read more
Event Director, *Apple* Retail Marketing -...
…This senior level position is responsible for leading and imagining the Apple Retail Team's global engagement strategy and team. Delivering an overarching brand Read more
At-Home Chat Specialist- *Apple* Online Stor...
**Job Summary** At Apple , we believe in hard work, a fun environment, and the kind of creativity and innovation that only comes about when talented people from diverse Read more
SW QA Engineer - *Apple* TV - Apple (United...
**Job Summary** The Apple TV team is looking for experienced Quality Assurance Engineers with a passion for delivering first in class home entertainment solutions. **Key Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.