TweetFollow Us on Twitter

Quoting in the shell

Volume Number: 23 (2007)
Issue Number: 09
Column Tag: MacEnterprise

Quoting in the shell

Dealing with interesting characters in shell programming

By Philip Rinehart, Yale University

QUOTES?

This month, a rather interesting problem arose on the Macenterprise mailing list. How are quotes used and dealt with when scripting for the bash shell? It can be a particularly difficult problem, as OS X allows the use of non-standard characters and spaces in filenames. If quoting is not done properly, unexpected, and even disastrous results may occur. Anyone, remember iTunes 2.0 and destruction of hard drive data? It was a quoting problem! Thus, it is extremely important to quote things properly when shell programming, both for unexpected results, as well as properly sanitizing input.

Single Quotes?

Let's start the process by looking at the use of single quotes in bash shell scripting. The original question was about the use of a directory listing. The directory listing was then piped to a second command, which failed due to spaces in some of the directory names. The specific example:

   ls -F /Applications

Try it. Note how many Applications contain spaces in the name. If Adobe Acrobat 8 Professional were installed, any shell script attempting to perform an action on the path would fail. Why? As the shell interprets spaces as input separators, Adobe, Acrobat, 8 and Professional are all seen as individual paths. In our above example, if the chmod operation were being performed, "Adobe", "Acrobat", "8" and "Professional" would all be modified by chmod. This action would fail naturally, as each of these paths does not exist. Would using single quotes help?

Single quotes treat all information contained between single quotes as exact character data in the bash shell. In practice, single quotes are extraordinarily difficult to use as variables and are not expanded. Even the backslash character is not considered special within single quotes. In our example, this fact means that a variable cannot be used in single quotes when shell scripting. As a result, single quotes aren't going to solve the problem above, so a different solution is required.

Double Quotes?

What about the use of double quotes? Can they solve the problem? Hmm, it get's a bit closer this time. Again, let's use the above list command. When assigned to a variable, multiple words will be treated as a single word. Wow, what a mouthful! Here's a better example. Let's use Adobe Acrobat 8 Professional again. When talking about single quotes, remember the fact that each word is treated separately? With double quoting, the entire word remains intact, and is not split. So, if the variable test is assigned and then echoed out using double quotes, the entire word, Adobe Acrobat 8 Professional, is used. A brief shell script snippet to further reinforce this concept:

#!/bin/bash
test="Adobe Acrobat 8 Professional"
printf "Single word is $test\n"

Note, that when these commands are entered and run that the variable is correctly displayed with printf. Without the quotes, only the word Single is printed. Does this solution work for the problem encountered on the list? Well, not quite. It certainly gets somewhat closer, as spaces are correctly preserved with double quotes, but command substitutions with variables still see each space separated entry as a separate word. If single and double quotes won't solve the problem, what will? Can the field separator be altered, and will it solve the problem?

IFS

IFS? Never heard of it before? IFS stands for Internal Field Separator. It determines how bash interprets word boundaries. By default, the three special characters, space, tab, and newline are used. Using the Adobe Acrobat 8 example, it now should be somewhat clearer as to why each is treated as a separate word. bash reads each space in the name as a field separator. With the default value for IFS, command substitution treats each space as a delimiter. This can be changed. At the top of each shell script, set the IFS variable. Any character can be the field separator, from a semi-colon, space, to a comma. The Macenterprise list came up with the following solution for the problem when a directory listing contains spaces:

   
   IFS="\n"

The author inserted this line at the beginning of the shell script. Now that the IFS separator has been reset, spaces are no longer recognized as valid field separators. The problem is now gone, and the variable is correctly assigned. It is also useful to note that resetting IFS can be used to ensure that the IFS variable has not been improperly set before any shell script is run. As a security precaution when writing a shell script, just as the PATH variable is set, make it a habit to also set the IFS variable. If input is being read in, or parsed, any spoofing or accidental resetting of this variable becomes a non-issue.

USING FIND

Let's return to the original list command that the article opened with. The original question wanted to take a list of directories and then change their permissions. Jeff McCune from The Ohio State University suggested another method on the list. He used the shell find utility to accomplish the same thing in a more efficient and unix-y way. Let's start with the command in its full form:

find /Applications -type d -maxdepth 1 -mindepth 1 -print0 | xargs -0 chmod 750

O.K., that's a bit much! As the find utility may be new to some, let's break it down into its components. The second argument, /Applications, can be any directory that find will operate on. In this case, it is looking in the Applications directory. The next argument, -type, looks for directories, as specified by the option d. After that, the next two arguments instruct the find command not to descend into any of the Application packages. It is accomplished with the maxdepth and mindepth switches. The next option is where the real magic begins. The option, print0, prints the result to standard output, terminating the output with the ASCII NUL character. It will become clear why this option is important in a second. All the results from the command are printed without a terminating newline. Try just the first part of the command before the pipe character, both with -print and -print0. Note the difference, as it is critical to this commands proper function.

Now that the find results are complete, it is time to process them using xargs. xargs takes an argument list, and processes the results with any other command line utility. Remember the use of print0. As the NUL character terminates the output, xargs must use the -0 flag. Without it, xargs does not know how to interpret the results from the find command. This technique is also generally considered faster by Unix gurus, as it only executes the chmod command once, which may not be the case for a shell loop, or by using the –exec switch with the find utility.

As you can see, the problem presented on the list is not as straightforward as one might expect. As always, there is more than one way to solve the problem, but with a full understanding of the problem's intricacies, it should now be trivial to solve. Until next month, see you on the lists!


Philip Rinehart is co-chair of the steering committee leading the Mac OS X Enterprise Project (macenterprise.org) and is the Lead Mac Analyst at Yale University. He has been using Macintosh Computers since the days of the Macintosh SE, and Mac OS X since its Developer Preview Release. Before coming to Yale, he worked as a Unix system administrator for a dot-com company. He can be reached at: philip.rinehart@yale.edu.

The MacEnterprise project is a community of IT professionals sharing information and solutions to support Macs in an enterprise. We collaborate on the deployment, management, and integration of Mac OS X client and server computers into multi-platform computing environments.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Google Chrome 63.0.3239.108 - Modern and...
Google Chrome is a Web browser by Google, created to be a modern platform for Web pages and applications. It utilizes very fast loading of Web pages and has a V8 engine, which is a custom built... Read more
Paparazzi! 1.0b7 - Make user-defined siz...
Paparazzi! is a small utility for OS X that makes screenshots of webpages. This very simple tool takes screenshots of websites which do not fit on one screen. You specify the desired width, minimal... Read more
Remotix 5.0.4 - Access all your computer...
Remotix is a fast and powerful application to easily access multiple Macs (and PCs) from your own Mac. Features Complete Apple Screen Sharing support - including Mac OS X login, clipboard... Read more
Apple Configurator 2.6 - Configure and d...
Apple Configurator makes it easy to deploy iPad, iPhone, iPod touch, and Apple TV devices in your school or business. Use Apple Configurator to quickly configure large numbers of devices connected to... Read more
Amadeus Pro 2.4.4 - Multitrack sound rec...
Amadeus Pro lets you use your Mac for any audio-related task, such as live audio recording, digitizing tapes and records, converting between a variety of sound formats, etc. Thanks to its outstanding... Read more
Mellel 4.0.3 - The word processor for sc...
Mellel is the leading word processor for OS X and has been widely considered the industry standard for long form documents since its inception. Mellel focuses on writers and scholars for technical... Read more
Suitcase Fusion 8 19.0.3 - Font manageme...
Suitcase Fusion 8 is the creative professional's font manager. Every professional font manager should deliver the basics: spectacular previews, powerful search tools, and efficient font organization... Read more
Final Cut Pro X 10.4 - Professional vide...
Final Cut Pro X is a professional video editing solution. Completely redesigned from the ground up, Final Cut Pro adds extraordinary speed, quality, and flexibility to every part of the post-... Read more
Compressor 4.4 - Adds power and flexibil...
Compressor adds power and flexibility to Final Cut Pro X export. Customize output settings, work faster with distributed encoding, and tap into a comprehensive set of delivery features. Features... Read more
Cocktail 11.2 - General maintenance and...
Cocktail is a general purpose utility for macOS that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more

Latest Forum Discussions

See All

The best new games we played this week -...
We've made it through another week, so let's treat ourselves to some of the best new games to launch in the past few days. It was another exciting week with some long-awaited indie games making their debut, and some big console titles making the... | Read more »
Match blocks to pull off dance moves in...
Ferdinand: Unstoppabull is a brand new match three puzzler based on the animated movie of (almost) the same name. As you can expect, you have to match blocks together to complete a bunch of puzzling levels and earn a high score. [Read more] | Read more »
Lineage 2: Revolution’s end of year upda...
Now available in 54 countries worldwide, Lineage 2: Revolution is continuing its global quest to be the most popular mobile MMORPG by launching a jam-packed end of year update. Complete with many subtle tweaks to help improve users’ online... | Read more »
The 5 best Star Wars games on iOS
The time has almost come.Star Wars: The Last Jedifinally hits theaters in the cinematic event that might be bigger than Christmas. To celebrate, we're taking a look at the best--and only the best--Star Warsmobile games to date. [Read more] | Read more »
Life Is Strange (Games)
Life Is Strange 1.1 Device: iOS Universal Category: Games Price: $2.99, Version: 1.1 (iTunes) Description: Life Is Strange is a five part episodic game that sets out to revolutionize story-based choice and consequence games by... | Read more »
Oddworld: New 'n' Tasty (Game...
Oddworld: New 'n' Tasty 1.0 Device: iOS Universal Category: Games Price: $7.99, Version: 1.0 (iTunes) Description: ** PLEASE NOTE: Requires 3.6GB free space to install. Runs at variable resolutions based on device capabilities.... | Read more »
Gorogoa (Games)
Gorogoa 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: Gorogoa is an elegant evolution of the puzzle genre, told through a beautifully hand-drawn story designed and illustrated by Jason... | Read more »
Why Guns of Boom will be big for mobile...
Earlier this week, Game Insight, the minds that brought you Guns of Boom, revealed plans for an esports mode in the popular FPS title, with big implications for the game's future. Guns of Boom has been quite popular for some time now, so it's... | Read more »
The best mobile games to play on lazy ho...
With the holidays in full swing, there's hopefully going to be a lot of time off work lazing around the house. With all of that free time, it's a perfect opportunity to catch up on some mobile games that you might have missed out on earlier this... | Read more »
Rules of Survival guide - how to boost y...
It's not easy surviving in the "every-man-for-himself" world of Rules of Survival. You'll be facing off against many other players who might be more skilled than you, or are luckier than you. There are a lot of factors weighing against you. With... | Read more »

Price Scanner via MacPrices.net

Updated Price Trackers: Macs, iPads, iPhones,...
Scan our Apple Price Trackers for the latest information on sales, bundles, and availability on systems from Apple’s authorized internet/catalog resellers. We update the trackers continuously: – 15″... Read more
How to preorder a new iMac Pro and pay zero s...
B&H Photo and Adorama are accepting preorders on multiple configurations of the new Apple iMac Pro. Both resellers charge sales tax for residents of NY & NJ only, and shipping is free.... Read more
Apple Macs back in stock at Amazon with model...
Amazon has MacBook Pros, MacBook Airs, MacBooks, and iMacs on sale for up to $200 off MSRP as part of their Holiday/Christmas sale. Shipping is free. Note that stock of some Macs may come and go (and... Read more
Apple offering free overnight delivery on all...
Apple is now offering free overnight delivery on all in stock products until 3pm local time on December 22nd. This includes new as well as refurbished computers. Click here for more information. Read more
Beats Holiday sale at B&H, headphones and...
B&H Photo has Beats by Dr. Dre headphones, earphones, and speakers on sale for up to $80 off MSRP as part of their Holiday sale. Expedited shipping is free, and B&H charges sales tax to NY... Read more
Holiday sale: Apple resellers offer 2017 15″...
MacMall has 15″ MacBook Pros on sale for $220-$300 off MSRP, each including free shipping: – 15″ 2.8GHz MacBook Pro Space Gray (MPTR2LL/A): $2179, $220 off MSRP – 15″ 2.8GHz MacBook Pro Silver (... Read more
Holiday sale: Apple resellers offer 13″ MacBo...
B&H Photo has 13″ MacBook Pros on sale for up to $150 off MSRP. Shipping is free, and B&H charges sales tax for NY & NJ residents only: – 13-inch 2.3GHz/128GB Space Gray MacBook Pro (... Read more
Apple Watch Series 2, Certified Refurbished,...
Apple has Certified Refurbished Apple Watch Nike+ Series 2s, 42mm Space Gray Aluminum Case with Anthracite/Black Nike Sport Bands, available for $249 (38mm) or $279 (42mm). The 38mm model was out of... Read more
Apple offers Certified Refurbished 2016 12″ R...
Apple has Certified Refurbished 2016 12″ Retina MacBooks available starting at $949. Apple will include a standard one-year warranty with each MacBook, and shipping is free. The following... Read more
B&H drops price on 13″ 256GB MacBook Air...
B&H has the 13″ 1.8GHz/256GB Apple MacBook Air (MQD42LL/A) now on sale for $1079 including free shipping plus NY & NJ sales tax only. Their price is $120 off MSRP, and it’s the lowest price... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Payments Counsel, *Apple* Pay (payments, cr...
# Payments Counsel, Apple Pay (payments, credit/debit) Job Number: 112941729 Santa Clara Valley, California, United States Posted: 13-Dec-2017 Weekly Hours: 40.00 Read more
*Apple* Solutions Consultant - Apple (United...
# Apple Solutions Consultant Job Number: 113124408 Waterford, CT, Connecticut, United States Posted: 17-Oct-2017 Weekly Hours: 40.00 **Job Summary** Are you Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.