TweetFollow Us on Twitter

September 96 - MPW TIPS AND TRICKS: Automated Editing With StreamEdit

MPW Tips and Tricks: Automated Editing With StreamEdit

Tim Maroney

In this column in Issue 26 of develop, I showed you a wide range of scriptable editing commands available from the MPW Shell. This time I'll discuss a single tool that provides a powerful self-contained text-editing scripting language, StreamEdit.

Why would you want to use StreamEdit instead of the other text-editing features of the MPW Shell?

  • Performance -- A StreamEdit script is faster than an MPW script containing various Replace and Find commands.

  • Self-containment -- Because StreamEdit is a self-contained tool, you can run it from within ToolServer, unlike the scriptable editing commands discussed in Issue 26, which are available only in the MPW Shell itself. This means you can use StreamEdit to create lightweight drag-and-drop grinder AppleScript scripts that send StreamEdit commands to ToolServer.

  • Consistency -- Keeping all your editing in a single scripting language confers the elusive mystical boon of code consistency, making your system easier to maintain and modify in the future.


StreamEdit is based very closely on the hoary UNIX tool named sed. If you already know sed, much of this will be familiar, but StreamEdit isn't directly compatible with sed scripts. StreamEdit implements a pattern-matching language. Every time a particular pattern is matched, a sequence of commands will be executed. As in most pattern-matching languages, StreamEdit's scripts are lists of pattern/command pairs, with the pattern coming before the command. The input file or files are read through the script interpreter, which searches for instances of the patterns and executes the corresponding commands. Anything that doesn't match a pattern is passed through unchanged.

StreamEdit scans one line at a time through the input, matching its current line to every pattern in its script. After processing each line, it writes out the modified line. The result is a concatenation of three internal buffers: the insert buffer, then the edit buffer, and finally the append buffer. The edit buffer gets filled with the current line, while the other buffers are empty at the start. The Insert and Append commands place text in the insert and append buffers, allowing you to add text to the beginning and end of the output line. The Change, Delete, and Replace commands modify the contents of the edit buffer.


As usual, MPW uses words in ways previously unknown in human speech. In StreamEdit, patterns are referred to as "addresses." There are two kinds of addresses: line numbers and regular expressions. Line numbers ought to be self-explanatory, but it may help to note that the numbers must be Arabic numerals rather than Roman, and must be in base 10 rather than the hexadecimal or sexagesimal number systems. There are three special line numbers:
  • the bullet symbol (Option-8), meaning the point before the first line (enabling you to add a line before the first line, for example)

  • the infinity symbol ([[infinity]], Option-5), meaning the point after the last line

  • dollar sign ($), meaning the last line
    The keyboard shortcuts, as always in this column, are for American QWERTY keyboards; if you've got some other type of keyboard, you're on your own.
Regular expressions are expressions that manage their diets sensibly. They can be used for searching, and were explained in detail in Issue 26. In StreamEdit addresses, though, regular expressions find the entire line containing the pattern, rather than just the pattern. Regular expressions are denoted by slashes. Only forward slashes are used (StreamEdit doesn't have a backward search mode, having been frightened at an early age by the legends of Eurydice and Lot's wife). Three new constructs have been added to regular expressions in StreamEdit:
  • ç (Option-C), which indicates a case-sensitive search

  • // (two slashes), which means the last regular expression that was matched

  • <=variable>= (a variable name embedded in inequality operators, here overloaded as a special kind of angle brackets, and typed as Option-comma and Option-period), which means the text of an expanded StreamEdit variable, treated as literal text to be matched rather than as a regular expression
StreamEdit has variables that can be set with the Set command (more on this later) or from the command line using the -set variable [=value] option.

You can form more complex addresses using a few operators. The Boolean and, or, and not operators are the same as in C (&&, ||, and !, respectively). Parentheses can be used for grouping within addresses. The comma operator matches the range of lines specified; for example, 3,5 matches lines 3 through 5. A range address matches each of the lines in the range, if any. It can be thought of as matching more than once: it fires off the accompanying command on the first line matched, the last one matched, and all lines in between. If the termination condition is never met, the address continues to match until the end of input. This could happen if you specify a range of lines ending at line 15, for instance, and there are only ten lines in the file, or if your range termination condition is a regular expression that doesn't appear anywhere in the input.


Matching patterns is very nice, but what do you do once you match them? Statements in StreamEdit attach actions to patterns. An action consists of one or more commands, separated by semicolons or by the end of a line. There's no begin or end bracketing as in Pascal or C. Addresses and commands are syntactically distinct, so the script interpreter can figure out where the list of commands for a pattern ends and the next pattern begins.

Editing commands

  • Insert text [-n] -- Adds the specified text to the start of the line by putting it in the insert buffer. The -n option (in this command and in Append and Change) prevents adding a newline when the line is written out.

  • Append text [-n] -- Adds the specified text to the end of the line by putting it in the append buffer.

  • Change text [-n] -- Changes the line to the specified text by replacing the contents of the edit buffer.

  • Delete -- Clears the edit buffer.

  • Replace [-c count] /pattern/ text -- Replaces the pattern with the specified text. This is the second part of a two-step matching process: first the address matches a line, then Replace searches in the edit buffer and replaces the pattern. The count argument indicates the maximum number of times to perform the replacement in the line. It can be a positive integer or infinity ([[infinity]]). The default count is 1.

Control commands

  • Exit [status] -- Stops StreamEdit with the given error status. The default is 0, which means execution completed with no errors. Any nonzero error status indicates a problem, and unless the built-in MPW variable Exit is set to something other than 0, this will stop execution of the script (if any) from which the StreamEdit command was executed.

  • Next -- Somewhat like the C keyword continue. When a Next command is executed, all pending changes are written out and no more addresses are matched against the current line; that is, StreamEdit immediately goes on to the next line without matching the rest of the rules against the current edit buffer.

  • Set variable text [-i | -a] -- Much like the MPW Shell Set command. The variable is set to the specified text. The -i and -a options allow text to be added to any existing setting of the variable at the start or the end, respectively.

Output commands

  • Print [text] [-appendto | -to file] -- Writes output to a specified file. If text is empty, the current line is printed without modification. The -appendto and -to options write at the end of the file or overwrite the file, respectively. If no file is specified, standard output is used. If the file name is empty, nothing gets printed.

  • Option AutoDelete -- Deletes all input lines, leaving only output from Next and Print commands. You can get the same effect by specifying the -d option on the StreamEdit command line or by including this in the script:
/~/   Delete
The text arguments to these commands are usually literal text, denoted by single or double quotes. There are a few other forms as well:
  • An unquoted variable name can be used, in which case the variable is expanded; no brackets need be (or even may be) supplied.

  • A period means the current input line up to but not including the newline at the end.

  • As discussed in Issue 26, you can use reg. (Option-R) followed by a digit to mean the expression with that number matched in the pattern.

  • You can read text from a file with -from filename, which reads the next line of text from the specified file. The filename is usually literal text, but it could also be a variable, the current input line (denoted by a period), or a reg. expression.


Let's say you're the director of corporate communications at a major computer maker and, without any warning except for inventory backlogs larger than the gross national products of many developing countries, you experience a sudden transition in chief executive officers, corporate policy, and product line. Your quarterly report (10-Q) is due in the SEC's EDGAR database tomorrow. Fortunately the SEC requires the cutting-edge ASCII format for its filings, and you realize that you can automate 90% of the tedious changes with a single StreamEdit script.
# Change nickname of CEO
Replace // 'Flyboy'
# Change corporate policy
Replace /capture market share/ 'survive'

# Remove lines referring to obsolete products
/PowerTalk/ || /eWorld/

# Change developer relations strategy
/third-party developers/
Replace /evangelize/ 'listen to'

# Mark lines referring to old schedules
# with a distinctive string at the start
# of the line for manual editing later
Insert 'WHOOPS: '

# Add new final line of report
Append 'May God have mercy on our souls.'


StreamEdit is almost too powerful. People have used it for everything, including pretty-printing source code, converting files to HTML, and postprocessing object files for dynamic linking tools. If you use it for finding incriminating passages in coworkers' e-mail, karma may get you, but the limitations of the tool won't. Use your powers for good rather than evil, and a grateful world will thank you.

TIM MARONEY has appeared professionally in newspapers, magazines, compact discs, videotape, and of course, computer software. Tim is a technical lead in human interface software at Apple and is editing a series of books for a horror publisher. His skin burns easily in the sun and tans in the moon. He uses white T-shirts only for house painting and car repair.

Thanks to Arno Gourdol, Alex McKale, and Robert Ulrich for reviewing this column.


Community Search:
MacTech Search:

Software Updates via MacUpdate

Mellel 4.1.0 - The word processor for sc...
Mellel is the leading word processor for OS X and has been widely considered the industry standard for long form documents since its inception. Mellel focuses on writers and scholars for technical... Read more
ScreenFlow 7.3 - Create screen recording...
ScreenFlow is powerful, easy-to-use screencasting software for the Mac. With ScreenFlow you can record the contents of your entire monitor while also capturing your video camera, microphone and your... Read more
Dashlane 5.9.0 - Password manager and se...
Dashlane is an award-winning service that revolutionizes the online experience by replacing the drudgery of everyday transactional processes with convenient, automated simplicity - in other words,... Read more
ForkLift 3.2 - Powerful file manager: FT...
ForkLift is a powerful file manager and ferociously fast FTP client clothed in a clean and versatile UI that offers the combination of absolute simplicity and raw power expected from a well-executed... Read more
Cocktail 11.5 - General maintenance and...
Cocktail is a general purpose utility for macOS that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more
Hazel 4.2.4 - Create rules for organizin...
Hazel is your personal housekeeper, organizing and cleaning folders based on rules you define. Hazel can also manage your trash and uninstall your applications. Organize your files using a familiar... Read more
Skype - Voice-over-internet pho...
Skype allows you to talk to friends, family, and co-workers across the Internet without the inconvenience of long distance telephone charges. Using peer-to-peer data transmission technology, Skype... Read more
Backup and Sync 3.40.8921.5350 - File ba...
Backup and Sync (was Google Drive) is a place where you can create, share, collaborate, and keep all of your stuff. Whether you're working with a friend on a joint research project, planning a... Read more
Dashlane 5.9.0 - Password manager and se...
Dashlane is an award-winning service that revolutionizes the online experience by replacing the drudgery of everyday transactional processes with convenient, automated simplicity - in other words,... Read more
Cocktail 11.5 - General maintenance and...
Cocktail is a general purpose utility for macOS that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more

Latest Forum Discussions

See All

Destiny meets its mobile match - Everyth...
Shadowgun Legends is the latest game in the Shadowgun series, and it's taking the franchise in some interesting new directions. Which is good news. The even better news is that it's coming out tomorrow, so if you didn't make it into the beta you... | Read more »
How PUBG, Fortnite, and the battle royal...
The history of the battle royale genre isn't a long one. While the nascent parts of the experience have existed ever since players first started killing one another online, it's really only in the past six years that the genre has coalesced into... | Read more »
Around the Empire: What have you missed...
Oh hi nice reader, and thanks for popping in to check out our weekly round-up of all the stuff that you might have missed across the Steel Media network. Yeah, that's right, it's a big ol' network. Obviously 148Apps is the best, but there are some... | Read more »
All the best games on sale for iPhone an...
It might not have been the greatest week for new releases on the App Store, but don't let that get you down, because there are some truly incredible games on sale for iPhone and iPad right now. Seriously, you could buy anything on this list and I... | Read more »
Everything You Need to Know About The Fo...
In just over a week, Epic Games has made a flurry of announcements. First, they revealed that Fortnite—their ultra-popular PUBG competitor—is coming to mobile. This was followed by brief sign-up period for interested beta testers before sending out... | Read more »
The best games that came out for iPhone...
It's not been the best week for games on the App Store. There are a few decent ones here and there, but nothing that's really going to make you throw down what you're doing and run to the nearest WiFi hotspot in order to download it. That's not to... | Read more »
Death Coming (Games)
Death Coming Device: iOS Universal Category: Games Price: $1.99, Version: (iTunes) Description: --- Background Story ---You Died. Pure and simple, but death was not the end. You have become an agent of Death: a... | Read more »
Hints, tips, and tricks for Empires and...
Empires and Puzzles is a slick match-stuff RPG that mixes in a bunch of city-building aspects to keep things fresh. And it's currently the Game of the Day over on the App Store. So, if you're picking it up for the first time today, we thought it'd... | Read more »
What You Need to Know About Sam Barlow’s...
Sam Barlow’s follow up to Her Story is #WarGames, an interactive video series that reimagines the 1983 film WarGames in a more present day context. It’s not exactly a game, but it’s definitely still interesting. Here are the top things you should... | Read more »
Pixel Plex Guide - How to Build Better T...
Pixel Plex is the latest city builder that has come to the App Store, and it takes a pretty different tact than the ones that came before it. Instead of being in charge of your own city by yourself, you have to work together with other players to... | Read more »

Price Scanner via

Back in stock! Apple’s full line of Certified...
Save $300-$300 on the purchase of a 2017 13″ MacBook Pro today with Certified Refurbished models at Apple. Apple’s refurbished prices are the lowest available for each model from any reseller. A... Read more
Back in stock: 13-inch 2.5GHz MacBook Pro (Ce...
Apple has Certified Refurbished 13″ 2.5GHz MacBook Pros (MD101LL/A) available for $829, or $270 off original MSRP. Apple’s one-year warranty is standard, and shipping is free: – 13″ 2.5GHz MacBook... Read more
Apple restocks Certified Refurbished 2017 13″...
Apple has Certified Refurbished 2017 13″ MacBook Airs available starting at $849. An Apple one-year warranty is included with each MacBook, and shipping is free: – 13″ 1.8GHz/8GB/128GB MacBook Air (... Read more
8-Core iMac Pro on sale for $4699, save $300
Amazon has the 8-core iMac Pro on sale for $4699 including free shipping. Their price is $300 off MSRP, and it’s the currently lowest price available for an iMac Pro. For the latest up-to-date prices... Read more
10″ 512GB WiFi iPad Pros on sale for $849, sa...
B&H Photo has Space Gray and Rose Gold 10.5″ 512GB WiFi iPad Pros on sale for $849. Their price is $150 off MSRP, and it’s the lowest price available for these models, new, from any Apple... Read more
MacBook Pro sale! B&H drops prices on new...
B&H Photo has dropped prices on new 2017 13″ MacBook Pros, with models now on sale for up to $200 off MSRP. Shipping is free, and B&H charges sales tax for NY & NJ residents only. Their... Read more
13″ MacBook Airs on sale for $100-$150 off MS...
B&H Photo has 13″ MacBook Airs on sale for $100-$150 off MSRP. Shipping is free, and B&H charges sales tax for NY & NJ residents only: – 13″ 1.8GHz/128GB MacBook Air (MQD32LL/A): $899, $... Read more
Huge iMac sale! Apple reseller now offering 2...
B&H Photo has new 2017 21″ & 27″ iMacs on sale today for up to $300 off MSRP. Shipping is free, and B&H charges sales tax for NY & NJ residents only: – 27″ 3.8GHz iMac (MNED2LL/A): $... Read more
Sale! 1.4GHz Mac mini for $399, $100 off MSRP
B&H Photo has the 1.4GHz Mac mini on sale for $399 for a limited time. Their price is $100 off MSRP, and it’s the lowest price available for a mini from any Apple reseller: – 1.4GHz Mac mini (... Read more
Sale of the year continues as Apple resellers...
Adorama has new 2017 15″ MacBook Pros on sale for $250-$300 off MSRP. Shipping is free, and Adorama charges sales tax in NJ and NY only: – 15″ 2.8GHz Touch Bar MacBook Pro Space Gray (MPTR2LL/A): $... Read more

Jobs Board

*Apple* Retail Sales Associate / Service Wri...
Job DescriptioniStore is the premier retailer of Apple products and solutions. We're looking for dedicated individuals with a passion to simplify and enhance the Read more
Integration Technician, *Apple* - Zones, In...
…adapted, and grown for over 30 years. Position Overview The Apple Integration Technician will be responsible for performing customer specific configuration Read more
*Apple* Retail - Multiple Positions - Apple,...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Part Time Reseller Specialist - *Ap...
…in a reseller store, you help create the energy and excitement around Apple products, providing the right solutions and getting products into customers' hands. You Read more
*Apple* Technical Specialist - Apple, Inc. (...
…customers purchase our products, you're the one who helps them get more out of their new Apple technology. Your day in the Apple Store is filled with a range of Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.