TweetFollow Us on Twitter

Oct 93 Challenge
Volume Number:9
Issue Number:10
Column Tag:Programmers’ Challenge

Programmers’ Challenge

By Mike Scanlin, MacTech Magazine Regular Contributing Author

Note: Source code files accompanying article are located on MacTech CD-ROM or source code disks.

ASCII85 ENCODING

This month we have a straightforward computation challenge: Write a fast ASCII85 encoder. What’s that, you ask? It’s a method of encoding binary data as ASCII characters so that it can be put into things that understand ASCII but not binary data (e-mail messages, for example). It’s also one possible format that EPS files are stored in. If you want your application to be EPS-friendly then you’ll need this routine (as well as the corresponding decoder, which is not part of this challenge).

Here’s how it works: Every 4 bytes of binary data input results in 5 bytes of ASCII character output in the range ! through u. A newline character (0x0D) is inserted in the ASCII output at least once every 80 characters to limit the length of lines. When you have encoded all the binary data, you write out ~> as an end-of-data marker.

More precisely, each set of binary input bytes (b1, b2, b3, b4) produces a set of encoded ASCII characters (c1, c2, c3, c4, c5) such that:

(b1 * 2563) + (b2 * 2562) + (b3 * 256) + b4 =
(c1 * 854) + (c2 * 853) + (c3 * 852) + (c4 * 85) + c5

So, the 4 bytes of binary data are a base-256 number that are converted into 5 bytes of a base-85 number. The 5 digits of this number (c1..c5) are then converted to ASCII by adding 33, the ASCII code of !. ASCII characters in the range of ! to u are used, where ! represents the value 0 and u represents the value 84. There is one special case where if all 5 digits are zero they are encoded as a single character z instead of !!!!!.

If the size of the input binary data is not a multiple of 4 then the final set of output bytes are created by taking the n input bytes (1, 2 or 3) and padding with 4-n zero bytes and then converting the result to ASCII85 normally but without applying the special z case. Then, write out n+1 bytes of the resulting ASCII85 data, followed immediately by the ~> marker. This allows an ASCII85 decoder to know the actual number and values of all real input bytes.

The prototype of the function you write is:

void ASCII85Encode(inputPtr, 
 numInputBytes, outputPtr, 
 numOutputBytesPtr)
char    *inputPtr;
unsigned long    numInputBytes;
char    *outputPtr;
unsigned long    *numOutputBytesPtr;

The inputPtr and numInputBytes describe the binary data input (numInputBytes will average about 64K). You fill the buffer pointed to by outputPtr with the ASCII85 data you create. The buffer is allocated for you and is big enough to handle the worst case output, given numInputBytes. You set *numOutputBytesPtr equal to the total number of bytes you put in the buffer (including the 2 for the end-of-data marker).

TWO MONTHS AGO WINNER

Perhaps I should re-emphasize that correctness is the first criteria for a winning solution. I had to disqualify 6 out of the 17 entries I received for the Replace All challenge because they lacked correctness. Be sure to test your entries with a range of inputs, please, if you’re serious about wanting to win.

Of the 11 who survived my test suite (which was not really that severe) there were four that were very close in speed. The one who came out a little faster than the others most of the time was from Tom Elwertowski (Cambridge, MA) who makes the observation that if the replacement length is not greater than the source length then you can replace strings as you find them; otherwise you have to find all the matches in a first pass and then do a second pass to make the actual replacements (back to front). Bill Karsh (location unknown) deserves mention for coming in fastest almost as often as Tom when the replacement string was equal to or shorter than the source string. However, for some cases where the replacement string was longer than the source string his code was near the bottom of the pack (the times given below are average times for all test cases).

A couple of people were quick to realize that the ROM’s Munger trap could be used to solve this puzzle. However as the table below shows, it’s possible to do much better, in terms of speed, if you don’t use Munger (‘*’ = an entry that used Munger for at least part of the solution; ‘**’ = an entry that was all Munger). Using Munger does make for small code though.

Here are the average times and sizes (numbers in parens after a person’s name indicate how many times that person has finished in the top 5 places of all previous programmer challenges, not including this one):

Name bytes avg ticks

Tom Elwertowski 864 54

Jeff Mallett (2) 770 60

Gerry Davis (1) 1042 64

Bill Karsh 1320 69

Stepan Riha (3) 1132 92

Jan Bruyndonckx 826 101

David Rand * 880 140

Bob Boonstra (2) 528 188

Dave Darrah ** 90 301

John Baxter ** 122 313

Eric Josserand 572 343

There are really 3 cases you need to think about to write a fast ReplaceAll: (1) replacement string and source string are the same length, (2) replacement string is shorter than the source string and (3) replacement string is longer than the source string. The first is the fastest; you just overwrite the source string with the replacement string in place and you don’t need to move any other bytes. The second case is almost as fast; you overwrite the source string with a shorter replacement string and then degap the Handle by the difference in lengths (of course, you cumulatively degap as you find/replace, and not completely degap after each individual replace). The third case is the slowest because it requires two passes to do it efficiently: during the first pass you find all occurrences of the source string and mark them (or keep an array of their positions), then you grow the Handle by the difference in lengths between the replacement string and the source string times the number of occurrences of the source string you found; then you start at the back of the Handle moving bytes and making replacements as you go. This guarantees a minimum of Handle resizing and gapping.

In addition, if you are motivated, you could add the special case code that Bill Karsh did that checks for replacement and source strings of length exactly equal to 1 and have a tiny little loop that screams for that case (which will catch those reviewers who time the real-world, every-day case of changing a page of “a”s into a page of “b”s to make their word processing speed comparison charts). And you could add a table driven search algorithm like Stepan Riha (Austin, TX) did that minimizes search time in cases where partial matches are common.

Or you could forget all that and listen to John Baxter (location unknown) who says “There is, IMHO, too much optimization and too little problem solving going on in the world (but I do enjoy the challenge feature and your optimization series). Clearly SOME optimization has its place, but the last time I routinely cared about squeezing the last possible cycle OR storage location out of code was about 1959.” Well, John, I can see your point but as a user who does ReplaceAlls on 200 page documents fairly often I can assure you that optimizing that operation is time well spent.

Here’s Tom’s winning solution:

/* Tom Elwertowski
 *
 * ReplaceAll( sourceHndl, replaceHndl, targetHndl )
 *
 * Replace all occurrances of sourceHndl in targetHndl with
 * replaceHndl. If replace length is not greater than source
 * length, replacement can be done as search proceeds.
 * Otherwise, all occurrances must be found first in order
 * to determine how much target will grow. A recursive
 * solution is used in the latter case; matches are found
 * on the way in and replacement occurs on the way out.
 *
 * Depending upon run length, substrings are moved either
 * by a byte copy loop or by the BlockMove routine. A word
 * move loop was considered and rejected. Best performance
 * was achieved for these string sizes: byte loop: <9,
 * word loop: 9-200 and BlockMove: >200. A word loop must
 * check for and deal with word nonaligmnent however which
 * resulted in more than a few lines of code and an inline
 * solution appeared excessively cumbersome. When all three
 * approaches were put in a subroutine, the additional
 * overhead swamped the gain in most cases. The compromise
 * was to use inline code with a byte loop for substrings
 * up to 21 characters and a BlockMove otherwise.
 */

#define kBlockMoveMin 22

typedef struct replaceParamBlock {
 Handle sourceHndl;
 long sourceSize;
 char *sourceStart;
 char *sourceEnd;
 Handle replaceHndl;
 long replaceSize;
 char *replaceStart;
 char *replaceEnd;
 Handle targetHndl;
 long targetSize;
 char *targetStart;
 char *targetEnd;
 long deltaSize;
} replaceParamBlock, *replaceParamBlockPtr;

long replaceOne( char *target, long depth, long deltaOffset,
 replaceParamBlockPtr replacePBPtr);

long ReplaceAll( Handle sourceHndl, Handle replaceHndl,
 Handle targetHndl )
{
 replaceParamBlock replacePB;
 char *target, *targetMatch, *targetOld, *targetNew,
 *source, *replace;
 long numReplace, deltaOffset, size;

 replacePB.sourceHndl = sourceHndl;
 replacePB.sourceSize = GetHandleSize( sourceHndl);
 replacePB.sourceStart = *sourceHndl;
 replacePB.sourceEnd =
 replacePB.sourceStart + replacePB.sourceSize;
 replacePB.replaceHndl = replaceHndl;
 replacePB.replaceSize = GetHandleSize( replaceHndl);
 replacePB.replaceStart = *replaceHndl;
 replacePB.replaceEnd =
 replacePB.replaceStart + replacePB.replaceSize;
 replacePB.targetHndl = targetHndl;
 replacePB.targetSize = GetHandleSize( targetHndl);
 replacePB.targetStart = *targetHndl;
 replacePB.targetEnd = 
 replacePB.targetStart + replacePB.targetSize;
 replacePB.deltaSize =
 replacePB.replaceSize - replacePB.sourceSize;

 numReplace = 0;
 deltaOffset = 0;
 if ( replacePB.deltaSize <= 0 ) {

 /* Iterative solution when 
  * replacement not longer then source */
 target = replacePB.targetStart;
 while ( target < replacePB.targetEnd) {
 if ( *target++ == *replacePB.sourceStart ) {

 /* Beginning of potential match */
 targetMatch = target - 1;
 source = replacePB.sourceStart + 1;
 while ( source < replacePB.sourceEnd )
 if ( *target++ != *source++ )
 goto noMatch;

 /* Match encountered */

 /* Shift unchanged segment of  target */
 if (numReplace > 0 &&
 replacePB.deltaSize < 0 ) {
 targetOld = replacePB.targetStart;
 targetNew = replacePB.targetStart +
 deltaOffset;
 size = targetMatch -
 replacePB.targetStart;
 if ( size < kBlockMoveMin )
 while ( targetOld < targetMatch )
 *targetNew++ = *targetOld++;
 else
 BlockMove( targetOld, targetNew,
 size );
 }

 /* Do replacement */
 replace = replacePB.replaceStart;
 targetNew = targetMatch + deltaOffset;
 if ( replacePB.replaceSize < kBlockMoveMin )
 while ( replace < replacePB.replaceEnd )
 *targetNew++ = *replace++;
 else
 BlockMove( replace, targetNew,
 replacePB.replaceSize );

 numReplace++;
 deltaOffset += replacePB.deltaSize;
 replacePB.targetStart = target;
 }

 noMatch:;
 }

 /* End of target encountered */

 /* If replacements have occurred and
  * replacement is shorter */
 if ( numReplace > 0 && replacePB.deltaSize < 0 ) {

 /* Compress target from last match to end */
 targetNew = replacePB.targetStart + deltaOffset;
 targetOld = replacePB.targetStart;
 size = replacePB.targetEnd -
 replacePB.targetStart;
 if ( size < kBlockMoveMin )
 while ( targetOld < replacePB.targetEnd )
 *targetNew++ = *targetOld++;
 else
 BlockMove( targetOld, targetNew, size );

 /* Resize target*/
 SetHandleSize ( targetHndl,
 replacePB.targetSize += deltaOffset );
 if ( MemError() != noErr)
 numReplace = -1;
 }
 }
 else
 /* Recursive solution when
  * replacement is longer than source */
 numReplace = replaceOne( replacePB.targetStart,
 0, 0, &replacePB );
 return ( numReplace );
}


long replaceOne( char *targetEntry, long depth,
 long deltaOffset, replaceParamBlockPtr replacePBPtr )
{
 char *source, *replace, *target;
 char *targetOld, *targetNew, *targetMatch;
 long targetEntryOffset, targetMatchOffset, size;
 long numReplace;
 
 target = targetEntry;
 targetEntryOffset = targetEntry -
 replacePBPtr->targetStart;
 while ( target < replacePBPtr->targetEnd ) {
 if ( *target++ == *replacePBPtr->sourceStart ) {

 /* Beginning of potential match */
 targetMatch = target - 1;
 targetMatchOffset = targetMatch -
 replacePBPtr->targetStart;
 source = replacePBPtr->sourceStart + 1;
 while ( source < replacePBPtr->sourceEnd )
 if ( *target++ != *source++ )
 goto noMatch;

 /* Match encountered. Look for next match */
 numReplace = replaceOne( target, depth + 1,
 deltaOffset + replacePBPtr->deltaSize,
 replacePBPtr );

 /* Expand target after all matches found */
 if ( numReplace > 0 ) {

 /* Do replacement */
 targetMatch = replacePBPtr->targetStart +
 targetMatchOffset;
 targetNew = targetMatch + deltaOffset;
 if ( replacePBPtr->replaceSize <
 kBlockMoveMin ) {
 targetNew += replacePBPtr->replaceSize;
 replace = replacePBPtr->replaceEnd;
 while ( replace >
 replacePBPtr->replaceStart )
 *--targetNew = *--replace;
 }
 else
 BlockMove( replacePBPtr->replaceStart,
 targetNew,
 replacePBPtr->replaceSize );

 /* Shift unchanged segment of target */
 if ( depth > 0 ) {
 targetOld = targetMatch;
 targetEntry =
 replacePBPtr->targetStart +
 targetEntryOffset;
 size = targetMatch - targetEntry;
 if ( size < kBlockMoveMin )
 while ( targetOld > targetEntry )
 *--targetNew = *--targetOld;
 else {
 targetNew -= size;
 BlockMove( targetEntry, targetNew,
 size );
 }
 }
 }
 return ( numReplace );
 }
 noMatch:;
 }

 /* End of target encountered */
 if ( depth > 0 ) {

 /* Resize target */
 SetHandleSize ( replacePBPtr->targetHndl,
 replacePBPtr->targetSize += deltaOffset );
 if ( MemError() == noErr) {
 
 /* Update pointers after possible relocation */
 replacePBPtr->targetStart =
 *replacePBPtr->targetHndl;
 replacePBPtr->targetEnd =
 replacePBPtr->targetStart +
 replacePBPtr->targetSize;
 replacePBPtr->replaceStart =
 *replacePBPtr->replaceHndl;
 
 replacePBPtr->replaceEnd =
 replacePBPtr->replaceStart +
 replacePBPtr->replaceSize;

 /* Expand target from last match to end */
 targetOld = replacePBPtr->targetEnd -
 deltaOffset;
 targetEntry = replacePBPtr->targetStart +
 targetEntryOffset;
 size = targetOld - targetEntry;
 if ( size < kBlockMoveMin ) {
 targetNew = replacePBPtr->targetEnd;
 while ( targetOld > targetEntry )
 *--targetNew = *--targetOld;
 }
 else {
 targetNew = targetEntry + deltaOffset;
 BlockMove( targetEntry, targetNew, size );
 }
 }
 else
 depth = -1;
 }

 return ( depth );
}

The Rules

Here’s how it works: Each month there will be a different programming challenge presented here. First, you must write some code that solves the challenge. Second, you must optimize your code (a lot). Then, submit your solution to MacTech Magazine (formerly MacTutor). A winner will be chosen based on code correctness, speed, size and elegance (in that order of importance) as well as the postmark of the answer. In the event of multiple equally desirable solutions, one winner will be chosen at random (with honorable mention, but no prize, given to the runners up). The prize for the best solution each month is $50 and a limited edition “The Winner! MacTech Magazine Programming Challenge” T-shirt (not to be found in stores).

In order to make fair comparisons between solutions, all solutions must be in ANSI compatible C (i.e., don’t use Think’s Object extensions). Only pure C code can be used. Any entries with any assembly in them will be disqualified (except for those challenges specifically stated to be in assembly). However, you may call any routine in the Macintosh toolbox you want (i.e., it doesn’t matter if you use NewPtr instead of malloc). All entries will be tested with the FPU and 68020 flags turned off in THINK C. When timing routines, the latest version of THINK C will be used (with ANSI Settings plus “Honor ‘register’ first” and “Use Global Optimizer” turned on) so beware if you optimize for a different C compiler. All code should be limited to 60 characters wide. This will aid us in dealing with e-mail gateways and page layout.

The solution and winners for this month’s Programmers’ Challenge will be published in the issue two months later. All submissions must be received by the 10th day of the month printed on the front of this issue.

All solutions should be marked “Attn: Programmers’ Challenge Solution” and sent to Xplain Corporation (the publishers of MacTech Magazine) via “snail mail” or preferably, e-mail - AppleLink: MT.PROGCHAL, Internet: progchallenge@xplain.com, CompuServe: 71552,174 and America Online: MT PRGCHAL. If you send via snail mail, please include a disk with the solution and all related files (including contact information). See page 2 for information on “How to Contact Xplain Corporation.”

MacTech Magazine reserves the right to publish any solution entered in the Programming Challenge of the Month and all entries are the property of MacTech Magazine upon submission. The submission falls under all the same conventions of an article submission.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Together 3.4.6 - Store and organize all...
Together helps you organize your Mac, giving you the ability to store, edit and preview your files in a single clean, uncluttered interface. Smart storage. With simple drag-and-drop functionality,... Read more
Monosnap 3.1.0 - Versatile screenshot ut...
Monosnap lets you capture screenshots, share files, and record video and .gifs! Capture: Capture full screen, just part of the screen, or a selected window Make your crop area pixel perfect with... Read more
Cocktail 8.5.1 - General maintenance and...
Cocktail is a general purpose utility for OS X that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more
Vienna 3.0.6 :5eaf312: - RSS and Atom ne...
Vienna is a freeware and Open-Source RSS/Atom newsreader with article storage and management via a SQLite database, written in Objective-C and Cocoa, for the OS X operating system. It provides... Read more
Kodi 15.1.rc1 - Powerful media center to...
Kodi (was XBMC) is an award-winning free and open-source (GPL) software media player and entertainment hub that can be installed on Linux, OS X, Windows, iOS, and Android, featuring a 10-foot user... Read more
Bookends 12.5.8 - Reference management a...
Bookends is a full-featured bibliography/reference and information-management system for students and professionals. Access the power of Bookends directly from Mellel, Nisus Writer Pro, or MS Word (... Read more
Chromium 44.0.2403.125 - Fast and stable...
Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all Internet users to experience the web. Version 44.0.2403.125: This release contains a number... Read more
iMazing 1.2.2 - Complete iOS device mana...
iMazing (was DiskAid) is the ultimate iOS device manager with capabilities far beyond what iTunes offers. With iMazing and your iOS device (iPhone, iPad, or iPod), you can: Copy music to and from... Read more
Audio Hijack 3.2.0 - Record and enhance...
Audio Hijack (was Audio Hijack Pro) drastically changes the way you use audio on your computer, giving you the freedom to listen to audio when you want and how you want. Record and enhance any audio... Read more
FontExplorer X Pro 5.0.1 - Font manageme...
FontExplorer X Pro is optimized for professional use; it's the solution that gives you the power you need to manage all your fonts. Now you can more easily manage, activate and organize your... Read more

You'll Want to Keep an Eye Out for...
If you're the kind of person who had fun hunting down and completing all the codex puzzles in Assassin's Creed 2, then are you ever in for a treat. The Guides looks like it's going to be a very robust collection of similarily odd, seemingly... | Read more »
Vivid Games has Announced Real Boxing 2...
The original Real Boxing was a pretty impressive bit of fisticuffs, but if the trailer Vivid Games is showing off for GamesCom is any indication Real Boxing 2 is going to be even better. [Read more] | Read more »
PAC-MAN Championship Edition DX - Tips,...
[Read more] | Read more »
Card King: Dragon Wars - Tips, Tricks an...
[Read more] | Read more »
Pac-Man Championship Edition DX has brou...
Bandai Namco has released Pac-Man Championship Edition DX on iOS and Android, which features the classic arcade gameplay that we've all grown to love. Pac-Man Championship Edition DX can be enjoyed in much shorter bursts than the arcade versions... | Read more »
Cosmonautica (Games)
Cosmonautica 1.1 Device: iOS Universal Category: Games Price: $6.99, Version: 1.1 (iTunes) Description: Cast off! Are you ready for some hilarious adventures in outer space? | Read more »
Rescue humanity from a Demon horde in An...
Angel Stone is Fincon's follow up to the massively successful Hello Hero and is out now on iOS and Android. You play as a member of The Resistance, a group of mighty human warriors who have risen up in defiance of the Demon horde threatening to... | Read more »
Gallery Doctor (Photography)
Gallery Doctor 1.0 Device: iOS iPhone Category: Photography Price: $2.99, Version: 1.0 (iTunes) Description: Free up valuable iCloud and iPhone storage with Gallery Doctor, the only iPhone cleaner that automatically identifies the... | Read more »
You Against Me (Games)
You Against Me 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: A simple game… You. Me. Claim, steal, lock, score, win! | Read more »
Yep, it's True - Angry Birds 2 is O...
The not exactly rumors were true and the birds are back. Angry Birds 2 has come to the App Store and the world will... well I suppose it'll still be the same, but now we have more bird-flinging options! [Read more] | Read more »

Price Scanner via MacPrices.net

PDF Element Tool Kit For PDF For Windows 10,...
South Surrey, British Columbia based software developer Wondershare has posted an interesting infographic tracking the development of Microsoft’s flagship Windows operating system over the years,... Read more
27-inch 3.5GHz 5K iMac on sale for $81 off MS...
Adorama has the 27″ 3.5GHz 5K iMac on sale for $2218, $81 off MSRP, including a free copy of Apple’s 3-Year AppleCare Protection Plan. Shipping is free, and Adorama charges sales tax in NY & NJ... Read more
Back-to-School with Tablet and Smartphone Acc...
Belkin helps you prepare for the coming school year with a wide variety of the latest mobile and tablet accessories to outfit both grade school and college students. The line-up includes charging... Read more
11-inch MacBook Airs on sale for $100 off MSR...
Best Buy has 11-inch MacBook Airs on sale for $100 off MSRP. Choose free shipping or free local store pickup (if available). Sale prices for online orders only, in-store prices may vary: - 11″ 1.6GHz... Read more
iPad Air 2 on sale for up to $100 off MSRP
Best Buy has iPad Air 2s on sale for up to $100 off MSRP on their online store for a limited time. Choose free shipping or free local store pickup (if available). Sale prices available for online... Read more
Sale! 13-inch MacBook Pros on sale for $100 o...
B&H Photo has 13″ MacBook Pros on sale for $100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.5GHz/500GB MacBook Pro: $999.99 save $100 - 13″ 2.7GHz/128GB Retina... Read more
Sale! Save $100 on 13-inch MacBook Airs this...
B&H Photo has the 13″ 1.6GHz/128GB MacBook Air on sale for $899.99 including free shipping plus NY tax only. Their price is $100 off MSRP, and it’s the lowest price available for this model.... Read more
Worldwide Tablet Market Decline Continues, Ap...
The worldwide tablet market declined -7.0% year-over-year in the second quarter of 2015 (2Q15) with shipments totaling 44.7 million units according to preliminary data from the International Data... Read more
TP-LINK TL-PA8030P KIT Powerline Featuring Ho...
Consumer and business networking products provider TP-LINK is now shipping its TL-PA8030P KIT AV1200 3-Port Gigabit Passthrough Powerline Starter Kit that expands your home’s network over its... Read more
Apple refurbished iPad Air 2s available for u...
The Apple Store has Apple Certified Refurbished iPad Air 2s available for up to $140 off the price of new models. Apple’s one-year warranty is included with each model, and shipping is free: - 128GB... Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Infrastructure Engineer - *Apple* /Mac - Hil...
Infrastructure Engineer - Apple /Mac Job Code: 1608 # of openings: 1 Description Our fortune 500 client is looking to hire an experienced Infrastructure Engineer to join Read more
Executive Administrative Assistant, *Apple*...
…supporting presentation development for senior leadership. * User experience with Apple hardware and software is preferred. Additional Requirements The following list Read more
*Apple* Bus Company is now hirin - Apple Bus...
Apple Bus Company is now hiring school bus drivers in the Pettis County area. Class B CDL preferred. Free training provided. No nights or weekends required. Flexible Read more
*Apple* Certified Mac Technician - Updated 6...
…and friendly, hands-on technical support to customers troubleshooting and repairing Apple /Mac products with courtesy, speed and skill. Use your problem-solving skills Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.