TweetFollow Us on Twitter

June 96 - MPW Tips And Tricks: Scripted Text Editing

Mpw Tips And Tricks: Scripted Text Editing

Tim Maroney

The MPW Shell contains a full-strength, high-speed text editor with scripting capabilities. It's nothing to write love letters with, because it's targeted at the ASCII format of compiler source files, but it provides the power to automate complex and repetitive tasks in ASCII text. The key to the system lies in a few editing-related commands, together with its regular expressions and selection expressions.

REGULAR EXPRESSIONS

In the MPW Shell, any search command can take one of two kinds of arguments. The first is a plain string, which matches exactly its contents and nothing else, using a simple character-by-character match. The other is a regular expression, which is a pattern that can be recognized by a finite state machine. You can't parse programming languages with regular expressions, but you can use them to recognize many patterns, including wildcards, repeating sequences, and sets of characters. Regular expressions are bracketed with either slashes or backslashes, for searching forward or backward respectively. So, for instance, the regular expression \wombat\ would search backward from the current location for the string "wombat".

There are about 20 special constructs within regular expressions, all of which are cryptically described when you execute the command line "Help Patterns" within the MPW Shell. I'll mention some of the more useful ones here. The wildcard characters are the question mark (?) and the equivalence symbol (~, Option-X). The question mark matches any one character except the end of a line, while the equivalence symbol matches any number of such characters. For instance, /w?mb~t/ would match "wombat" as well as "wambiklort" and "wymbt", but not "wafkambiliot", nor "wkmb" at the end of a line. Restricted sets of symbols can be given in brackets; for instance, you can search for alphanumeric characters with the pattern [a-zA-Z0-9]. The reverse of a set can be specified with the "not" symbol (~, Option-L); for instance, /[~a-z]/ finds any character except a lowercase letter. The start of a line can be specified with the bullet symbol (*, Option-8) and the end of a line with the infinity symbol ([[infinity]], Option-5).

    These keyboard shortcuts are for American QWERTY keyboards. Other keyboards have different layouts. For instance, on a direct neural interface keyboard, think "blue wildebeest" and raise your right ear to type the bullet symbol.*
Repeating patterns can be specified in three ways. Following any pattern with a plus sign (+) means one or more instances of that pattern; for instance, the regular expression /[0-9]+/ would match any sequence of digits. An optional repeating pattern can be similarly specified with an asterisk (*), which means zero or more repetitions. The rarely seen double angle brackets can be used to specify exactly how many repetitions of a pattern are allowed. They're typed as Option-backslash (<<) and Option-Shift-backslash (>>) and enclose a single number to mean exactly that many repetitions, or two numbers separated by a comma to specify a minimum and maximum number of repetitions, or a single number followed by a comma to mean at least that many repetitions. For instance, the pattern /[a-zA-Z]<<3,7>>/ would find all strings composed of alphabetical characters and from three to seven letters long.

There are a number of ways of "escaping" special characters when you want to look for something that has special meaning within regular expressions, such as a question mark or plus sign. You can escape any character with the lowercase delta ([[partialdiff]], Option-D), or use single or double quotes to escape strings. To find the string "wombat+", for instance, you'd need to escape the plus sign: /wombat[[partialdiff]]+/.

Finally, one of the most useful constructs consists of a tagged regular expression. This allows you to associate a number between 0 and 9 with a pattern that's matched, referring to it later with the "registered" symbol (reg., Option-R) followed by a digit. This is very handy when you're doing replacements. For instance, you can replace any angle-bracketed string with a parenthesized string with the following command, which would turn "<wombat>" into "(wombat)":

Replace /<([~<>]*)reg.1>/ (reg.1)
This searches for any number of characters (except angle brackets) that are between angle brackets, assigns them the number 1, and then replaces the angle brackets with parentheses. Note that the syntax of tagged patterns requires the pattern to be parenthesized.

SELECTION EXPRESSIONS

Many editing commands (such as Replace) can take selection expressions as well as regular expressions. Selection expressions provide more ways to select text than the string matching provided by regular expressions. Common selection expressions include the following:
  • The bullet symbol, meaning the start of a file.

  • The infinity symbol, meaning the end of a file.

  • The current selection, denoted by [[section]] (Option-6). This might have been selected with the mouse or by a Find command. [[section]] by itself indicates the selection in the target window (which I'll explain later), while pathname:[[section]] means the selection in the file indicated by the pathname.

  • A line number, specified simply as a number.

  • The name of a marker, specified by the Mark command.

  • A range between two selection expressions, separated by a colon (:).
The above expressions require no special delimiters (they're not directional like regular expressions). Regular expressions are actually a kind of selection expression and are delimited by slash or backslash characters as usual.

Some character-skipping variants of these options are also provided, such as the position that's one character after the selection, denoted by following a selection expression with an uppercase delta ([[Delta]], Option-J). These are useful in dealing with context; for instance, you may want to select a string when it's followed by another character, but not include the following character in the selection. (An example is given later in the Subword script.) Text emitted by a program like a table generator may be in a known format, such as a columnar arrangement, in which case skipping a certain number of characters will take you to the selection you need.

Again, the MPW Shell will give you a terse summary of selection expressions when you execute the command line "Help Selections". I'm not going to list all the minor variants here, but feel free to while away the hours in rapturous contemplation of their mysteries on your own.

EDITING COMMANDS

The most common editing commands are two that you probably use already: Find and Replace. Dialogs that stand in for these commands are built into the MPW Shell and accessible from the Find menu. You can give any selection expression as a search pattern in either of these dialogs by clicking the Selection Expression radio button instead of the default Literal button. The same commands are the basis of most editing scripts. As tools, Find and Replace take a selection expression as their primary argument. Don't confuse Find and Search! The Search command puts out its results as text, while Find actually changes the selection. In addition, Search takes a pattern -- that is, a regular expression -- while Find takes any selection expression. For example, to go to the start of a file in a script, you could give the command "Find *", but not "Search *".

Find is the basic navigation command in most editing scripts. For instance, you can simulate the Select All command in the Edit menu like so:

Find *:[[infinity]]  # select from start to end of target
The commands File and Open, along with the variables Target and Active, determine the files your scripts will work on. "File" is actually an alias for the real command name, Target. The File command opens a file and makes it the target window -- the window behind the frontmost window. The target window is an important notion in MPW. It exists so that you can use the Worksheet window to type commands that affect another window; since the Worksheet would be in front, the window being affected would need to be behind the Worksheet. During scripting, you may prefer to use the Open command, which opens a file and makes it the frontmost window. The target window is referred to as {Target} in scripts, while the frontmost window is called {Active}. Editing commands work on the target window if you don't specify a window explicitly.

The Line command may also be used for navigation: it selects the numbered line in the target window and then brings that window to the front. You probably know this command already if you use compilers in the MPW Shell, since they put out error messages in this form:

File "gwork.c"; Line 418 # Syntax error
Executing this command takes you to the line in your code where the error was detected.

The Position command returns the current position in the target window, as a line number, a character range, or both. The position could be saved to a variable for later use as follows, using the backquote mechanism to execute a command and insert its output inline:

Set SavedLineNumber `Position -l`
There are dozens of commands pertaining to text editing in the MPW scripting language. Help on all of them is available in the MPW Shell. The usual Macintosh text-editing menu commands are available in the MPW scripting language, including New, Open, Close, Save, Revert, Print, and the standard Edit menu commands.

StreamEdit is a standalone editing tool that's rich and strange enough to deserve its own co-->umn. It's a structured search and replacement language based on the UNIXreg. command sed.

Some simpler standalone editing tools are provided. Sort has a rich function set and can be used for many text-editing tasks. Canon takes a file of search and replace strings and applies them to a file. It's used to automate terminology changes, such as the work that was done to make the Mac OS API use fewer acronyms and abbreviations when the new Inside Macintosh books were written. Translate, like the UNIX command tr, maps characters onto other characters.

Text indentation can be handled with four tools: Adjust, Align, Entab, and Format. Adjust shifts a line to the right or left by a specified number of spaces. Align sets the margin of a range of selected lines to the margin of the first selected line. Entab converts runs of spaces to tabs, and Format sets the column width used for tabs in a text document, as well as other settings like font and size. (These settings are saved in a resource in the file, which many ASCII text editors can recognize.)

Text-editing scripts often create temporary files, split single files into multiple files, and perform other file-related tasks. MPW provides commands to help you manage files. It has commands corresponding to almost all Finder operations, such as Duplicate, Move, Delete, and NewFolder. There are also some specialized file commands: FileDiv splits a file into multiple files based on a byte or line count or on embedded form feed characters inserted during a previous editing pass; Catenate does the opposite, joining files together.

A text-editing script often takes search and substitution text as parameters on the command line. A few commands related to parameters are worth a quick mention here. Echo is handy for concatenating parameters with other text. Quote is similar to Echo but adds quote marks as needed to preserve the word breaks in its parameters. MPW scripting requires quotes around any string that is meant to be a single parameter but contains spaces (which would break the string into multiple parameters). Echo puts out its arguments in a way that allows them to be broken up, while Quote preserves the original word breaks by inserting quotes.

Echo "Richard Loves Pat"
Richard Loves Pat
Quote "Bill Loves Everyone"
'Bill Loves Everyone'

AN EXAMPLE SCRIPT

Here's a script I've found useful for some years. It's called Subword and it replaces a word by another string everywhere it occurs in the target window.
Set Sep "[~a-zA-Z_0-9]"  # word separators
Find * "{Target}"  # start at top of file
Replace -c [[infinity]] [[partialdiff]]
   "[[Delta]]/{Sep}{1}{Sep}/!1:[[Delta]]/{Sep}/" [[partialdiff]]
   "{2}" "{Target}"
The selection in this Replace command is probably about as clear as the U.S. tax code, so allow me to explain. The [[Delta]] means one character before the selection. The !1 means one character past the selection. The colon denotes everything between the selections (inclusively). So this pattern says, in a nutshell, select the pattern in the first parameter ({1}) when it's bracketed by separators, but exclude the separators.

Normally I don't use this script directly. I incorporate it into other scripts as a utility. The bulk of the work of converting between similar languages like Pascal and C can be done by an editing script, for example. Subword can be used to convert keywords, as could Canon. I use another script which is essentially Subword without the separators for changing symbols like equality operators.

Scripts to preconvert between Pascal and C can be found on this issue's CD. They don't generate compiler-ready text, but I've found that they facilitate a manual conversion at the rate of hundreds of lines per hour, allowing source bases in the thousands of lines to be accurately translated in a day or three. So the next time you're faced with a dull text-processing task, look over the tools MPW gives you, and see whether you can save yourself a few days of tedious manual labor!

TIM MARONEY recently changed his Apple badge color from green to white: he's gone from contract programming to a technical leadership role developing user interface software. Tim entertains himself in a variety of ways, such as straining his surgically altered eyeballs on the small print of obscure footnotes and collectible trading card games, and contorting his limbs in yogic asanas. He designed the iron crystal that now resides at the core of the earth and contributed significant ideas to the original (now obsolete) implementation of Planck-scale gravitational phenomena in the universe.*

Thanks to Dave Evans, Scott Fraser, Arno Gourdol, and Alex McKale for reviewing this column.*

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Amazon Chime 4.0.5528 - Amazon-based com...
Amazon Chime is a communications service that transforms online meetings with a secure, easy-to-use application that you can trust. Amazon Chime works seamlessly across your devices so that you can... Read more
Sparkle 2.1.0 - $79.99
Sparkle will change your mind if you thought building websites wasn't for you. Sparkle is the intuitive site builder that lets you create sites for your online portfolio, team or band pages, or... Read more
Bookends 12.7.9 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Bookends uses the cloud to sync reference libraries on all the Macs you use.... Read more
Dash 4.0.0 - Instant search and offline...
Dash is an API documentation browser and code snippet manager. Dash helps you store snippets of code, as well as instantly search and browse documentation for almost any API you might use (for a full... Read more
TrailRunner 3.8.834 - Route planning for...
TrailRunner is the perfect companion for runners, bikers, hikers, and all people wandering under the sky. Plan routes on a geographical map. Import GPS or workout recordings and journalize your... Read more
Dash 4.0.0 - Instant search and offline...
Dash is an API documentation browser and code snippet manager. Dash helps you store snippets of code, as well as instantly search and browse documentation for almost any API you might use (for a full... Read more
Bookends 12.7.9 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Bookends uses the cloud to sync reference libraries on all the Macs you use.... Read more
TrailRunner 3.8.834 - Route planning for...
TrailRunner is the perfect companion for runners, bikers, hikers, and all people wandering under the sky. Plan routes on a geographical map. Import GPS or workout recordings and journalize your... Read more
iFFmpeg 6.2.5 - Convert multimedia files...
iFFmpeg is a comprehensive media tool to convert movie, audio and media files between formats. The FFmpeg command line instructions can be very hard to master/understand, so iFFmpeg does all the hard... Read more
DaisyDisk 4.4 - $9.99
DaisyDisk allows you to visualize your disk usage and free up disk space by quickly finding and deleting big unused files. The program scans your disk and displays its content as a sector diagram... Read more

RPG Djinn Caster (Games)
RPG Djinn Caster 1.0.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0.0 (iTunes) Description: SPECIAL PRICE 38% OFF(USD 7.99 -> USD 4.99)!!!A Fantasy Action RPG of far foreign lands! Summon the Djinns and rise to... | Read more »
Alto's Odyssey gets its first trail...
There's finally video evidence of Alto's Odyssey, the follow up to the 2015 App Store hit, Alto's Adventure. It looks just as soothing and atmospheric as Alto's last outing, but this time players will be journeying to the desert. Whereas Alto's... | Read more »
Last week on Pocket Gamer
What’s going on in the wider world of portable gaming? Each week we ask that question of our sister website Pocket Gamer. The PG team covers iOS gaming, just like 148Apps, but it also strays into the world of Android games and handheld consoles... | Read more »
Pokémon GO Generation 2 evolution guide
At long last, Niantic Labs finally unleashed the Generation 2 Pokémon into the wild. Pokémon GO trainers are scrambling to grab up this new set of 80 Pokémon. There are some special new tricks required to catch all of these new beasties, though.... | Read more »
The best new games we played this week
It feels as though the New Year got off to a creaking start as far as mobile games go, but that's changed over the past few weeks. The last few days alone have seen the debut of a number of wonderful games, so we thought we'd take the time to... | Read more »
Recruit more scallywags and discover new...
Get ready to show off your sea legs all over again in Oceans & Empires’ new grand update, which aims to make the act of rising to the role of seven seas ruler even more fresh and appealing, thanks to a richness of new content on both iOS and... | Read more »
Mage the Ascension: Refuge (Games)
Mage the Ascension: Refuge 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: The groundbreaking roleplaying game Mage: The Ascension manifests in our turbulent present with Refuge, an... | Read more »
Vampire: Prelude (Games)
Vampire: Prelude 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: The classic roleplaying game Vampire: The Masquerade returns to digital games with a Prelude of things to come. Experience a... | Read more »
Digby Forever Guide: How to dig to the d...
Digby Forever is a sparkling homage to arcade classics, and while you may be tiring of the number of arcade games being thrown at you, this endless digger finds many ways to stand out from the rest of the pack. The game manages to be challenging... | Read more »
The best sales on the App Store this wee...
It's been quite the week in mobile games, but if the latest releases(there were some pretty darn good ones, in case you missed out) aren't really doing the trick, perhaps some of these discounted games will. Many of these premium games had their... | Read more »

Price Scanner via MacPrices.net

Apple Certified Refurbished iMacs available f...
Apple has Certified Refurbished 2015 21″ & 27″ iMacs available for up to $350 off MSRP. Apple’s one-year warranty is standard, and shipping is free. The following models are available: - 21″ 3.... Read more
Apple offering Certified Refurbished Series 1...
Apple is now offering Certified Refurbished Series 1 and Series 2 Apple Watches for 14-16% off MSRP, starting at $229. An Apple one-year warranty is included with each watch. Shipping is free: Series... Read more
1.4GHz Mac mini on sale for $449, save $50
B&H Photo has the 1.4GHz Mac mini on sale for $50 off MSRP including free shipping plus NY sales tax only: - 1.4GHz Mac mini: $449 $50 off MSRP Read more
12-inch Retina MacBooks on sale for $200-$250...
Newegg has the 12″ 1.2GHz Space Gray Retina MacBook (sku MLH82LL/A) on sale for $1349.99 including free shipping. Their price is $250 off MSRP, and it’s the lowest price available for this model.... Read more
13-inch 2.0GHz Space Gray MacBook Pro on sale...
Adorama has the non-Touch Bar 13″ 2.0GHz Space Gray MacBook Pro in stock today for $100 off MSRP. Shipping is free, and Adorama charges NY & NJ sales tax only: - 13″ 2.0GHz MacBook Pro Space Gray... Read more
13-inch Touch Bar MacBook Pros on sale for $1...
B&H Photo has select 2016 Apple 13″ Touch Bar MacBook Pros in stock today and up to $100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.9GHz/512GB Touch Bar MacBook... Read more
KSI-1802R SX Disinfect-able Keyboard With Wav...
KSI has unveiled a new, innovative medical keyboard, the KSI-1802R SX, at HIMSS 2017, running February 19-22 in Orlando, Florida. KSI-1802R SX is the only keyboard that combines dual factor... Read more
15-inch Touch Bar MacBook Pros on sale for up...
Amazon has 15″ Touch Bar MacBook Pros in stock today and on sale for up to $150 off MSRP including free shipping: - 15″ 2.7GHz Touch Bar MacBook Pro Space Gray: $2720.38 $79 off MSRP - 15″ 2.7GHz... Read more
Apple’s Education discount saves up to $300 o...
Purchase a new Mac or iPad using Apple’s Education Store and take up to $300 off MSRP. All teachers, students, and staff of any educational institution qualify for the discount. Shipping is free: -... Read more
13-inch 1.6GHz/256GB MacBook Air on sale for...
Newegg has the 13″ 1.6GHz/256GB MacBook Air (MMGG2LL/A) on sale for $1029.99 including free shipping. Their price is $170 off MSRP, and it’s the lowest price available for this model. Choose Newegg... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Manager *Apple* Systems Administration - Pu...
Req ID 3315BR Position Title Manager, Apple Systems Administration Job Description The Manager of Apple Systems Administration oversees the administration and Read more
*Apple* Technician - nfrastructure (United S...
Let’s Work Together Apple Technician This position is based in Portland, ME Life at nfrastructure At nfrastructure, we understand that our success results from our Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Manager *Apple* Systems Administration - Pu...
Req ID 3315BR Position Title Manager, Apple Systems Administration Job Description The Manager of Apple Systems Administration oversees the administration and Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.