TweetFollow Us on Twitter

Mac in The Shell: Customizing the bash Experience Further

Volume Number: 24
Issue Number: 10
Column Tag: Mac in The Shell

Mac in The Shell: Customizing the bash Experience Further

Changing the bash shell to suit your needs

by Edward Marczakr

Introduction

Last month, I launched into a different kind of introduction to bash. It concerns making you, the end user, comfortable and efficient. This month wraps up this topic with a look at more ways to wring magic out of bash.

History

For once, I'm not talking about "history" as in a "history lesson." This time, I'm talking about history in bash. bash history is both a command, and a function of the bash shell. bash very nicely will keep history (a log, or journal) of the commands you execute. Open up a shell (probably using Terminal.app), and type ls -l. Now, change into the /Users/Shared directory using the cd command: cd /Users/Shared. From here, press the up arrow on your keyboard. The current input line should recall the previous command (cd /Users/Shared). Pressing the up arrow again will recall the command before that. Do so, and when "ls -l" appears, press the return key to execute the ls command and list the contents of the directory you're currently in. Now, type history, and press return. You should see a list of commands previously executed, ending with "ls -l", "cd /Users/Shared" and another "ls -l".

The simple fact that we can scroll through our session's history with the up and down arrow, and retrieve an entire listing is pretty amazing all on its own. Not surprisingly, it gets better!

First, let's make sure we capture all of the history we want. There are several shell variables that configure the behavior of bash history. Here's what I use in my ~/.bash_profile:

export HISTSIZE=12000
export HISTFILESIZE=12000
export HISTCONTROL='ignoreboth'
export HISTTIMEFORMAT='%b %d %H:%M:%S: '

(there's actually a little more to it, but I'll cover that in due time).

The HISTSIZE variable sets the number of commands to remember in the command history. If not set, it defaults to 500. HISTFILESIZE, when set, will tell bash to truncate the history file, if necessary, by removing the oldest entries, to contain no more than that number of lines. Again, the default value is 500. The history file is also truncated to this size after writing it when an interactive shell exits.

HISTCONTROL is a very useful variable. Assign HISTCONTROL a colon separated list of the following options to alter how history is saved:

ignorespace: lines which begin with a space character are not saved in the history list.

ignoredups: lines matching the previous history entry are not saved.

ignoreboth: enforces both ignorespace and ignoredups.

erasedups: causes all previous lines matching the current line to be removed from the history list before that line is saved.

If HISTCONTROL is not set, all lines read by the shell parser are saved in the history list. If you like to watch a directory by executing "ls -l" and then repeatedly pressing up-arrow-return, ignoredups is for you! This way, you'll only see one "ls -l" in history no matter how many times you repeat the command.

HISTTIMEFORMAT is a variable I see get very little use, but I find immensely useful. Simply, when set, each entry in the history list will have a timestamp save with the entry. With the example setting I give above, my history looks like this:

580  Jul 29 08:25:26: cd dev/objc/
581  Jul 29 08:25:27: ls -la
582  Jul 29 08:34:29: cd dumb

Yes, I have a directory named "dumb." I also have several shell options defined, but the most important relating to history is this:

shopt -s histappend

The histappend option appends the history from the current shell when it exits to the history file. This is useful - I'd claim critical - when using multiple shells. To explain further, shell history is not immediately written to the history file, but is maintained per shell, and written on shell exit. If I have two shells running, the last to exit writes the history file that it maintained. If you only even use a single shell at a time, you wouldn't notice this behavior, but I think it's a nice option to have enabled regardless.

As an aside: I actually use two other shell options: histverify and histreedit. Both affect substitution and shell expansion. Both are also a little outside the scope of this article, however, the bash man page has more information if you're so inclined.

Another aside: I really like setting the cmdhist option, too using:

set cmdhist

This option will save multi-line entries as a single line in history. Multi-line entries are created using the backslash character at the end of a line. For example, with cmdhist set, if I were to enter the following:

$ for i in `ls`; do \
> echo $i; \
> done

...my history would actually contain:

for i in `ls`; do echo $i; done

It's all up to personal taste if you prefer this or not, so, use what works best for you.

More History

Just investigating bash history functionality alone is a deep topic. Mainly because it exists to save you work; always remember that. So, now you can up-and-down-arrow through history, and use the history command to list all history. If you typed a command that you want to recall, you can certainly press up-arrow until you find it. What if that command is way back? Well, you could remember part of the command and use grep to find it:

$ history | grep ssh
272  Jul 25 10:09:56: ssh 192.168.76.84
285  Jul 25 14:39:53: ssh marczak@192.168.76.84
287  Jul 26 09:06:04: ssh 192.168.125.164
289  Jul 26 11:10:58: ssh marczak@192.168.76.84
295  Jul 26 16:27:13: man ssh
296  Jul 26 23:10:19: ssh mm@db.wheresspot.com
298  Jul 27 09:19:18: ssh mm@db.wheresspot.com
330  Jul 27 09:52:21: ssh minoc
331  Jul 27 09:17:26: ssh serveradmin@ballast

(remember - I'm showing an option timestamp, which you likely won't see). From this list, you could copy and paste the line you'd like to repeat (yuck). Or, you can ask bash to expand by the history id. This is called history expansion. For example, if I wanted to repeat the "ssh marczak@192.168.76.84" line - history id 285 - I can type this:

!285

and press return. Much easier than retyping the entire line. While using grep to find things in history is OK for a grand overview, there are other, more refined ways of recalling history.

To broaden the scope of the exclamation point operator, in general, after the exclamation point you specify an event designator. A number will recall a specific numbered event in history. Here's a handy list of useful designators:

(numerical value) - recall specific event in history

! - previous line

(text) - previous line starting with given text.

(?text) - previous line containing text

Examples are always best, so, here's one of each (save "numerical value," which you've already seen in action).

If you ssh into a remote machine, do your work, get out and then realize, "oops - I need to do one more thing," simply type !! and press return. This will run the previous command. Remember: this is a substitution, so, you can use "!!" wherever you want to substitute the previous command. An example of this would be:

$ ps ax | grep [W]ord | awk '{print $1}'
28976
$ kill $(!!)

The text designator substitutes the previous match. For example, if I wanted re-run the ps command from the previous example - you know, to make sure it's really dead - typing !ps and pressing return will match that ps, and execute the entire line.

The final designator that I'm going to show matches text anywhere in the event, not just the beginning. To once again rely on the previous example, to now match the ps event, you could type !?ord, or even !?awk, press return and re-execute the entire ps expression.

Readline

Developed by GNU, bash uses the readline library for its command input. Simply stated, readline provides a set of functions that allow users to edit command lines as they are typed in. Readline is also responsible for providing the history function to bash (and other applications that choose to include it). Readline is a bit of a discussion in and of itself, so, I'm just going to show one immediately available function, and one tweak that makes all the difference to my history experience.

Readline runs in one of two modes: emacs or vi. The default is emacs mode. This allows you to use emacs key bindings to move about the command line. It's actually the source of the keyboard commands shown last month. From this, we gain reverse search through our history. Press ctrl-r, and your prompt will change to "(reverse-i-search)`':", alerting you that what you type is a search back through history. Each key press finds the best previous match. If what you type fails to match an event in history, you receive a bell (visual or audible, depending on how your session is configured). If you have more than one match, press ctrl-r again to cycle through matches.

Readline is configured using the file .inputrc that it finds in your home directory (you'll need to create this file, as it's not there by default). This allows for further customization and ways to implement features that match the way you work. Here's the .inputrc file that I use:

"\e[B": history-search-forward
"\e[A": history-search-backward
Space: magic-space

The first two lines bind the up and down arrow to searching forward and back through history. You may think the arrows already so this - and they do to a degree - however, with this addition, a little typing goes a long way. With these lines in your ~/.inputrc, if you were to type ssh, and then press the up arrow, you'll navigate back through history only seeing events that match the beginning "ssh". The "magic-space" line causes readline perform expansion each time the space bar is pressed.

One final note: bash isn't the only application that uses readline. This means that changes to .inputrc will affect all applications relying on it. For example, the command-line MySQL client uses the readline library to implement command-line editing and history retrieval. If you want a particular setting to apply only to one environment, .inputrc does understand a limited form of conditionals. So, to cause settings to only apply to bash, use this in .inputrc:

$if bash
Space: magic-space
$endif

This conditional will cause the magic-space behavior to apply only to bash. Naturally, any settings can be placed in the conditional.

CDPATH

CDPATH is simply a shell variable. Like the PATH variable, which tells the shell which paths to search for an executable, CDPATH informs the cd command which paths to look in when changing directories. This is a simple way to reduce your typing. CDPATH is specified just like PATH: a list of directories, separated by a colon. If, for example, to search your home directory and /Users/Shared, specify this:

CDPATH=".:~:/Users/Shared"

The "dot" up front in that specification tells cd to search the current directory. You probably want to do that, right? Using the CDPATH shown, if there's a directory named "photo" under /Users/Shared, and my present working directory (pwd) is ~/Library (or anywhere, really), simply typing cd photo will change my present working directory to /Users/Shared/photo.

This should tie into good usage of the PS1 variable as described in the previous column, as you should always have a visual clue as to which directory you actually currently in.

The order of CDPATH is important. Just like the PATH variable, first match "wins." Using the previous example, if there was a directory named "photo" in both my home directory and /Users/Shared/, typing cd photo would bring me to the photo directory in my home. Thanks to the current directory specification (".") up front, the current directory always takes precedence, so, you'll always have 'normal' behavior.

Bash Programmable Completion

Modern versions of bash allow programmable completion, extending the standard completion that the Tab key normally provides. Again, this is a topic that can probably take up its own article or chapter in a book. Investigating every aspect of programmable bash completion is beyond the scope of this article, but I need to point out that it exists, and how to get some useful completions running in your shell.

Here's what the bash man page says about standard completion (pressing the Tab key):

"Attempt to perform completion on the text before point. Bash attempts completion treating the text as a variable (if the text begins with $), username (if the text begins with ~), hostname (if the text begins with @), or command (including aliases and functions) in turn. If none of these produces a match, filename completion is attempted."

However, this basic setup can be improved. For example, on an unadorned bash-shell-out-of-the-box on OS X, if you were to type man dsed and press Tab, you'll just receive a bell. Wouldn't it be nice if you could press tab and have the line completed with man dseditgroup? Here's what the bash man page says about programmable completion:

"When word completion is attempted for an argument to a command for which a completion specification (a compspec) has been defined using the complete builtin, the programmable completion facilities are invoked.

First, the command name is identified. If a compspec has been defined for that command, the compspec is used to generate the list of possible completions for the word. If the command word is a full pathname, a compspec for the full pathname is searched for first. If no compspec is found for the full pathname, an attempt is made to find a compspec for the portion following the final slash.

Once a compspec has been found, it is used to generate the list of matching words. If a compspec is not found, the default bash completion as described above under Completing is performed."

A bit wordy, perhaps. However, the upshot is this: you can have bash complete on just about anything. If your company has custom command-line utilities, you could create a completion specification for the valid switches and completions for your specific utility! The key to it all is the complete built-in command.

Last month, we touched on aliases and functions. As a quick refresher, a function is a subroutine, or, a code block that implements a set of operations. This function will add two numbers together and print their result:

#!/bin/bash
function sum() {
  total=$(($1+$2))
  echo "The sum of $1 and $2 is $total"
}
sum 5 12

As you can see, calling a function is just like executing any bash script: arguments are passed in order into the function. Programmable completion takes advantage of this, allowing you to specify functions that will determine completions for matching text.

As a short example, imagine that we want completions for the dscl command. In the interest of space and clarity, our function will only complete on these options: -read, -readall, -readpl, -readpli and -list. A function that can do this for us would look something like this:

_dscl() 
{
    local cur prev opts
    COMPREPLY=()
    cur="${COMP_WORDS[COMP_CWORD]}"
    prev="${COMP_WORDS[COMP_CWORD-1]}"
    opts="-read -readall -readpli -list"
    if [[ ${cur} == -* ]] ; then
        COMPREPLY=( $(compgen -W "${opts}" — ${cur}) )
        return 0
    fi
}
complete -F _dscl dscl

If you're anxious to try this now, save this in a file named "dscl" and source it by typing source dscl. Type dscl -r at the command prompt and press the Tab key. bash should complete the "-r" as "-read" for you. Press the Tab key twice, and you'll see that it'll match the other "-read*" entries we supplied. Very cool. How does it work?

We start out by defining some variables: cur, the current match (being typed at the command-line), prev, the previous word typed and opts, which is the list of options to complete. The actual completion is handled by the compgen bash built-in. compgen is used to fill in the COMPREPLY variable. COMPREPLY has special meaning to bash in that it holds the output of a completion.

You can certainly use this function as a template. However, while it works for simple completions, a function could certainly be much more powerful and complex. What if you're just trying to get your completion fix and don't have the time/skills/energy to write an in-depth function for completion?

Fortunately, there are many good bash completion examples full scripts that you can download. One of the best is created by Ian Macdonald, and is available from http://www.caliban.org/files/bash/bash-completion-20060301.tar.gz. While originally created with Linux in mind, this comprehensive completion file is ideal for OS X, too. Even better: any Linux-specific commands that it can't find on OS X are simply not loaded, saving memory.

To use it, download the file, unarchive it, and find the bash_completion file at the top level of the resulting directory. Copy that into /etc. Have your bash initialization script source it:

. /etc/bash_completion

("." is a synonym for the source command). If you don't want to wait to login to a new shell, source it right at the command line. You can test it with one of the completions it brings: the cd completion. Wait...doesn't cd already complete with the built-in completion? Well, yes, but it's free to do so in meaningless ways. If you have a file in your home directory named "current-list", and a directory named "current_projects", the unmodified completion will complete the word "current", but then wait for you to clarify. With the programmed behavior, we realize that with the cd command, only one of the choices makes sense - the directory. Now you should notice that typing cd cur followed by the Tab key, you'll get the correct, full completion.

This bash completion file provides a framework for creating completions. If you want to finish off the dscl example above, or, create one for a custom binary or script, create an /etc/bash_completion.d directory, and drop your completion file in there. The /etc/bash_completion script is designed to look in /etc/bash_completions.d and source each file it containes.

One last tip that, although it gets put into ~/.inputrc, it ties in with tab completion: A helpful completion trick is to drop the following line into your .inputrc file:

set show-all-if-ambiguous on

This addition causes readline to show alternate matches immediately, rather than make you press Tab twice.

Wrapping Up

I hope the tips from this and last month have had an impact on your work inside the bash shell. Many times, it's the little things like this that can make a huge difference in your daily work. I certainly remember the first time I saw these in action after using bash for some time. It was a bit of magic! Quite honestly, while we've covered a lot in this and the previous article, there's even more to explore in bash. If this coverage has piqued your interest, investigate the shopt and bind commands along with the many more options available to the readline library.

Media of the month: Gödel, Escher, Bach: An Eternal Golden Braid, by Douglas R. Hofstadter. I recently unearthed my copy of this venerable tome and remembered what a revelation it is. Granted, it's not for everyone, but, if you've always wondered about it, or (especially) if you've never heard of it, go find a copy and dig it.

This will most likely be the last bash-specific Mac in the Shell column for some time. If there are any bash topics that you'd like to see explored deeper (or initially), let me know, and we can dig back in. Next month, we'll get into the wider world of scripting in OS X.


Ed Marczak is the Executive Editor for MacTech Magazine, and has been lucky enough to have ridden the computing and technology wave from early on. From teletype computing to MVS to Netware to modern OS X, his interest was piqued. He has also been fortunate enough to come into contact with some of the best minds in the business. Ed spends his non-compute time with his wife and two daughters.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Artlantis Studio 5.1.2.7 - 3D rendering...
Artlantis Studio is a unique and ideal tool for performing very high resolution rendering easily and in real time. The new FastRadiosity engine now lets you compute images in radiosity-even in... Read more
MacUpdate Desktop 6.0.5 - Search and ins...
MacUpdate Desktop 6 brings seamless 1-click installs and version updates to your Mac. With a free MacUpdate account and MacUpdate Desktop 6, Mac users can now install almost any Mac app on macupdate.... Read more
BitTorrent Sync 2.0.82 - Sync files secu...
BitTorrent Sync allows you to sync unlimited files between your own devices, or share a folder with friends and family to automatically sync anything. File transfers are encrypted. Your information... Read more
Google Drive 1.20 - File backup and shar...
Google Drive is a place where you can create, share, collaborate, and keep all of your stuff. Whether you're working with a friend on a joint research project, planning a wedding with your fiancé, or... Read more
Simon 4.0.3 - Monitor changes and crashe...
Simon monitors websites and alerts you of crashes and changes. Select pages to monitor, choose your alert options, and customize your settings. Simon does the rest. Keep a watchful eye on your... Read more
Vitamin-R 2.23 - Personal productivity t...
Vitamin-R creates the optimal conditions for your brain to work at its best by structuring your work into short bursts of distraction-free, highly focused activity alternating with opportunities for... Read more
iDefrag 5.0.0 - Disk defragmentation and...
iDefrag helps defragment and optimize your disk for improved performance. Features include: Supports HFS and HFS+ (Mac OS Extended). Supports case sensitive and journaled filesystems. Supports... Read more
PCalc 4.2 - Full-featured scientific cal...
PCalc is a full-featured, scriptable scientific calculator with support for hexadecimal, octal, and binary calculations, as well as an RPN mode, programmable functions, and an extensive set of unit... Read more
FileZilla 3.10.2 - Fast and reliable FTP...
FileZilla (ported from Windows) is a fast and reliable FTP client and server with lots of useful features and an intuitive interface. Version 3.10.2: Note: Now requires a 64-bit Intel processor.... Read more
The Hit List 1.1.11 - Advanced reminder...
The Hit List manages the daily chaos of your modern life. It's easy to learn - it's as easy as making lists. And it's powerful enough to let you plan, then forget, then act when the time is right.... Read more

Protect Yourself from an Onslaught of Ca...
Surprise Attack Games has announced a Cat-astrophic new physics puzzler called Fort Meow! In the game, a young girl named Nia finds her grandfather’s journal which triggers an all mighty feline attack! Why do the cats want the journal? Who knows,... | Read more »
GDC 2015 – Jelly Reef will be Game Oven’...
GDC 2015 – Jelly Reef will be Game Oven’s Last Hurrah, and it Seems like a Good Note to Go Out on Posted by Rob Rich on March 4th, 2015 [ permalink ] It’s sad knowing that Game Oven ( | Read more »
daWindci Deluxe Review
daWindci Deluxe Review By Campbell Bird on March 4th, 2015 Our Rating: :: BLUSTERY PUZZLESUniversal App - Designed for iPhone and iPad This updated puzzle game offers some creative gameplay and new mechanics, but still suffers from... | Read more »
Dungeon Hunter 5 Coming on March 12
Gameloft has excitedly announced that Dungeon Hunter 5 is on its way! Once again, you will adventure across the land of Valenthia exploring dungeons and fighting monsters. The game will have a new asynchronous multiplayer mode called Strongholds... | Read more »
GDC 2015 – The Sandbox 2 is Coming, and...
GDC 2015 – The Sandbox 2 is Coming, and Now it has Textures! | Read more »
Warner Bros. Interactive Announces Mort...
Mortal Kombat X, by Warner Bros. and NetherRealm Studios, will be a a free-to-play fighting/card-battle Mortal Kombat game. The game promises card collecting, multiplayer team combat, classic characters such as Scorpion, Sub-Zero and Raiden, and the... | Read more »
GDC 2015 – Piloteer is Whitaker Trebella...
GDC 2015 – Piloteer is Whitaker Trebella’s Latest Project, and it’s Definitely Something DIfferent Posted by Rob Rich on March 3rd, 2015 [ permalink ] You know | Read more »
PangoLand Review
PangoLand Review By Amy Solomon on March 3rd, 2015 Our Rating: :: COME VISIT PANGO AND FRIENDSUniversal App - Designed for iPhone and iPad PangoLand is an open-ended world full of familiar characters, bright colors and interactive... | Read more »
Knights of Pen & Paper is Leveling U...
With the roll of a die and a critical success, Paradox Interactive has announced Knights of Pen & Paper 2! You’ll be taking your place at the table once again in this sequel to Knights of Pen & Paper +1 Edition. The game will introduce the... | Read more »
GDC 2015 – Project Highrise is an Intere...
GDC 2015 – Project Highrise is an Interesting Idea from SomaSim Posted by Rob Rich on March 3rd, 2015 [ permalink ] You might know SomaSim best from their gold rush sim, 1849< | Read more »

Price Scanner via MacPrices.net

iPad: A More Positive Outlook – The ‘Book Mys...
It’s good to hear someone saying positive things about the iPad. I’ve been trying to bend my mind around how Apple’s tablet could have gone from zero to bestselling personal computing device on the... Read more
Mac Pros on sale for up to $279 off MSRP
Amazon has Mac Pros in stock and on sale for up to $279 off MSRP. Shipping is free: - 4-Core Mac Pro: $2725.87, $273 off MSRP (9%) - 6-Core Mac Pro: $3719.99, $279 off MSRP (7%) Read more
Sale! 13-inch Retina MacBook Pros for up to $...
B&H Photo has 13″ Retina MacBook Pros on sale for up to $205 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.6GHz/128GB Retina MacBook Pro: $1219.99 save $80 - 13″ 2.... Read more
Another Tranche Of IBM MobileFirst For iOS Ap...
IBM has announced the next expansion phase for  its IBM MobileFirst for iOS portfolio, with a troika of new apps to address key priorities for the Banking and Financial Services, Airline and Retail... Read more
Sale! 15-inch Retina MacBook Pros for up to $...
B&H Photo has the new 2014 15″ Retina MacBook Pros on sale for up to $250 off MSRP for a limited time. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.2GHz Retina MacBook Pro: $... Read more
WaterField Designs Introduces the Minimalist...
With Apple Pay gaining popularity, Android Pay coming in May 2015, and loyalty cards and receipts that can be accessed from smartphones, San Francisco’s WaterField Designs observes that it may be... Read more
Sale! 15-inch 2.2GHz Retina MacBook Pro for $...
 Best Buy has the 15″ 2.2GHz Retina MacBook Pro on sale for $1774.99 $1799.99, or $225 off MSRP. Choose free home shipping or free local store pickup (if available). Price valid for online orders... Read more
13-inch 2.5GHz MacBook Pro (refurbished) avai...
The Apple Store has Apple Certified Refurbished 13″ 2.5GHz MacBook Pros available for $170 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 13″ 2.5GHz... Read more
13-inch 2.5GHz MacBook Pro on sale for $100 o...
B&H Photo has the 13″ 2.5GHz MacBook Pro on sale for $999.99 including free shipping plus NY sales tax only. Their price is $100 off MSRP. Read more
27-inch 3.5GHz 5K iMac in stock today and on...
 B&H Photo has the 27″ 3.5GHz 5K iMac in stock today and on sale for $2299 including free shipping plus NY sales tax only. Their price is $200 off MSRP, and it’s the lowest price available for... Read more

Jobs Board

*Apple* Solutions Consultant - Retail Sales...
**Job Summary** As an Apple Solutions Consultant (ASC) you are the link between our customers and our products. Your role is to drive the Apple business in a retail Read more
Position Opening at *Apple* - Apple (United...
…Summary** As a Specialist, you help create the energy and excitement around Apple products, providing the right solutions and getting products into customers' hands. You Read more
Position Opening at *Apple* - Apple (United...
**Job Summary** The Apple Store is a retail environment like no other - uniquely focused on delivering amazing customer experiences. As an Expert, you introduce people Read more
*Apple* Solutions Consultant - Retail Sales...
**Job Summary** As an Apple Solutions Consultant (ASC) you are the link between our customers and our products. Your role is to drive the Apple business in a retail Read more
*Apple* Pay Automation Engineer - iOS System...
**Job Summary** At Apple , great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.