Volume Number: 14 (1998)
Issue Number: 5
Column Tag: Programmer's Challenge

May 98 - Programmer's Challenge

by Bob Boonstra, Westford, MA

What could he mean by "EggLob", you ask. Well nothing, except that it is an anagram of "Boggle®, the Parker Brothers game that is the inspiration for this month's Challenge. Boggle is played with 16 six-sided letter cubes and a 4x4 tray into which the cubes are shaken. The object is win as many points as possible by joining letters horizontally, vertically, or diagonally to form words. Longer words win more points. This month your Challenge is to solve a game like Boggle, with a few twists.

The prototype for the code you should write is:

#if defined (__cplusplus)
extern "C" {
#endif

long Boggle(
  long dimension,          /* puzzle is square, of size dimension, 4x4 
                              to 7x7 */
  char puzzle[],           /* letter at (row,col) is in
                              puzzle[col+dimension*row] */
  long dictSize,           /* number of words in dictionary */
  const char *dictionary[],/* legal words to use */
  char *wordsFound[],      /* return pointers to the words you found */
  void *privStorage        /* 20MB of storage for your use */
);

#if defined (__cplusplus)
}
#endif

Our first twist on the standard Boggle game is that the puzzle dimension will vary from 4x4 up to 7x7, inclusive. Your Boggle routine will provided the puzzle, a dimension x dimension array of characters. The second twist is that you are allowed to cheat, up to a point, anyway. You can cheat by picking up any of the cubes and exchanging it with any other cube. You can do this as many times as you like. You can exchange cubes, however, you cannot turn a cube so that a different letter is on top. (I've reduced your temptation to cheat in this fashion by not telling you what is on the other five faces of a cube.) Once you have rearranged the puzzle to your liking, you should find as many words as possible from the dictionary of dictSize words, copy the pointers to the words you find from the dictionary into the wordsFound array, and return the number of words you found. Your rearranged puzzle should replace the original puzzle and be returned to the calling routine.

Words are formed from letters that are adjacent vertically, horizontally, or diagonally. No letter from the puzzle may be used more than once within a single word. Words within other words are legal and scored separately. For purposes of matching against the dictionary and for scoring, the letter "Q" represents the two letters "QU". The dictionary may vary in size and content from puzzle to puzzle.

Words are awarded points based on length. No points are scored for words of fewer than 3 letters. Words of 3 or 4 letters earn 1 point, words of 5 letters earn 2 points, words of 6 letters earn 3 points, words of 7 letters earn 5 points, and words of n letters (n>7) earn 5 + 6*(n-7) points. Points are deducted based on how long your Boggle routine executes; 1 point will be deducted for every 20ms that your code executes on my 8500/200. Negative point totals are possible. Entries that do not return after a reasonable time (e.g., 5 minutes) will be terminated and assigned the associated negative score. Entries that claim to have found words that cannot legally be formed using the rearranged puzzle will similarly penalized.

The Challenge winner will be the entry that accumulates the largest point total in a sequence of puzzles.

The privStorage parameter will point to 20MB of preallocated storage for your use. The privStorage will be initialized to zero prior to the first call to Boggle. The same privStorage value will be provided on each subsequent call, and the contents of your private storage will persist from one call to the next.

This will be a native PowerPC Challenge, using the latest CodeWarrior environment. Solutions may be coded in C, C++, or Pascal. Thanks to Ernst Munter for suggesting this Boggle variant.

One more time we congratulate Ernst Munter (Kanata, Ontario) for submitting the winning Challenge entry, this time in the February Image Decoder Challenge. The Challenge was to read a GIF image from a file, decode it, and return an offscreen GWorld containing the image. Ernst's code was about 5% faster than the second place entry by Patrick Varilly, and beat Patrick's entry in all but one of the test cases I used.

A couple of keys to the extra speed in Ernst's code. First, notice that almost all of the code is part of methods in the GIF class, and that all of those methods are inlined. This wouldn't be a good general technique for all C++ code, but it provided an edge in this application. Second, look closely at some of the loop constructs. For example, the portion of the PutMore method dealing with the common case, where all of the decoded pixels map into the current image line, compiles as follows with full speed optimizations:

   char* newDest=dest+len;
   ...
   dest=newDest;   
00000014: 7C050378 mr    r5,r0
   do {
    *--dest=stp->value;
00000018: 88040006 lbz   r0,6(r4)
0000001C: 9C05FFFF stbu   r0,-1(r5)
    stp=stp->prefix;
00000020: 80840000 lwz   r4,0(r4)
   } while (stp);
00000024: 28040000 cmplwi  r4,$0000
00000028: 4082FFF0 bne   *-16   ; $00000018

The assignment via the predecremented pointer dest compiles to a single Store Byte With Update (stbu) instruction, making this short loop considerably tighter than other possible constructs. Very nice.

Patrick commented in his code that close to half of his execution time is spent in fread. I remember having the same problem at least once when I competed in the Challenge where I resorted to writing a streamlined version of fread. None of the entries to this month's Challenge tried that technique, though.

Eleven different GIFs were used to test the solutions to this Challenge. The GIFs ranged in size up to 1100x800 pixels, and up to 500KB in length. The table below lists the total execution time in milliseconds for five selected test cases and the time for all test cases combined. It also lists code size, data size, and the programming language used for each entry. The number in parentheses after the entrant's name is the total number of Challenge points earned in all Challenges to date prior to this one (Table 1).

Here are the Top Contestants for the Programmer's Challenge, including everyone who has accumulated more than 10 points during the past two years. The numbers below include points awarded over the 24 most recent contests, including points earned by this month's entrants.

1. Munter, Ernst (220 points)
2. Boring, Randy (73)
3. Cooper, Greg (61)
4. Lewis, Peter (51)
5. Mallett, Jeff (50)
6. Nicolle, Ludovic (44)
7. Rieken, Willeke (27)
8. Antoniewicz, Andy (24)
9. Gregg, Xan (24)
10. Murphy, ACC (24)
11. Day, Mark (20)
12. Higgins, Charles (20)
13. Hostetter, Mat (20)
14. Studer, Thomas (20)
15. Hart, Alan (14)
16. O'Connor, Turlough (14)

There are three ways to earn points: (1) scoring in the top 5 of any Challenge, (2) being the first person to find a bug in a published winning solution or, (3) being the first person to suggest a Challenge that I use. The points you can win are:

1st place 20 points
2nd place 10 points
3rd place 7 points
4th place 4 points
5th place 2 points
finding bug 2 points
suggesting Challenge 2 points

Here is Ernst's winning solution to the Image Decoder Challenge:

GIFDecode.Cpp
Copyright 1998, Ernst Munter, Kanata, ON, Canada

/*
 An implementation of a decoder of GIF images for the PowerPC Mac.

 Input: a file containing a single image;
     the file is already opened by the calling program

 Output: an off-screen graphics world which contains the image.

 Algorithm
 ---------
 The algorithm follows closely the description in the GIF spec.
 GIF89a extensions are recognized but their data blocks are
 skipped over and not processed.

 Implementation Notes
 --------------------
 The file is read, one large block at a time, into a buffer.
 File and image headers are processed according to the GIF
 spec. This includes constructing a Mac color table from
 either the global or a local GIF color table.

 The LZW decoder draws variable size codes from the buffer,
 maintains a code dictionary, and writes decoded pixel values
 directly into the GWorld pixel map.

 Decoding a single compressed code word yields a string of
 pixels in reverse order. The conventional method  would be
 to push the pixels on a stack, and then pop them off as
 they are written to sequential pixel addresses in the pixel map.

 I use this method if the pixel string goes past the end of
 the current row, in order to deal with extra bytes (rowBytes
 may be greater than imageWidth), and to handle interlaced row
 sequences correctly. In the normal case however, the pixel
 values are copied directly from the code dictionary to the
 pixel map. To facilitate this, I store the length of each
 code string in the dictionary record.
     
*/

#include <stdlib.h>
#include <string.h>
#include "ImageDecoder.h"

enum {kBlockSize=16384,
   kMaxCode=4095,
   kImageSeparator=0x2C,
   kExtensionIntroducer=0x21};

DictEntry 
struct DictEntry {
// Each dictionary entry is the start of a linked list
//     of length one or more.
// The stackedValue location is unrelated to the dictionary
//    entry; it is just a convenient spare location
//    to build the pixel stack, in memory that is likely
//    already in cache when it is needed.
 DictEntry*   prefix; // chains to previous code
 short     length; // length of chain
 unsigned char value; // the pixel value
 unsigned char  stackedValue;

 void Reset(int ch){
  prefix=0;
  length=1;
  value=ch;
 }

 void Init(DictEntry* px,int ch) {
  prefix=px;
  length=1+px->length;
  value=ch;
 }
};

GIF
struct GIF {
 FILE*      f;                    // the GIF file
 Rect      screenRect;              // size of graphics world
 GWorldPtr   osWorld;             // the graphics world
 CTabHandle   globalCTH;            // a global color table
 int       logScreenWidth;          // GIF logical screen
 int      logScreenHeight;
 int      bkGndColor;              // GIF specified bkGnd
 int      interlace;              // to track interlace passes
 char*      pixelBase;            // start of GWorld pixMap
 char*      imageBase;            // start of image in pixMap
 int      rowBytes;              // width of pixMap
 int      imageWidth;              // imageWidth <= rowBytes
 int       readPtr;                // index into file block
 DictEntry   d[4096];              // the code dictionary
 unsigned char block[kBlockSize];  // the file block

GIF::GIF
 GIF(FILE* fx) {
  f=fx;
  fread(block,1,kBlockSize,f);// start by loading 1 block
  readPtr=0;
 }

GIF::CheckSignature
 bool CheckSignature() {
// the file signature must be either GIF87a or GIF89a
  if (0x47494638 != *((long*)block))
   return false;
  int version=*((short*)(block+4));
  if ((version != 0x3761) && (version != 0x3961))
   return false;
  readPtr+=6;
  return true;
 }

GIF::SkipBytes
 void SkipBytes(int n2skip) {
// Skip over n2skip bytes in file block
  readPtr+=n2skip;
  while (readPtr>=kBlockSize) {
// We cannot skip past the end of the block without
//     loading the next block
   readPtr-=kBlockSize;
   fread(block,1,kBlockSize,f);
  }
 }

GIF::Read1Byte
 unsigned int Read1Byte(int & readPtr) {
// Returns the next byte from the file block.
// If we get to the end of the block, we load another.
  const unsigned int rc=block[readPtr++];
  if (readPtr>=kBlockSize) {
   readPtr=0;
   fread(block,1,kBlockSize,f);
  }
  return rc;
 }

GIF::ReadShort
 unsigned int ReadShort() {
// GIF files use little endian integers.
// This function reads two bytes and returns a 16-bit int.
  const unsigned int loByte=Read1Byte(readPtr);
  const unsigned int hiByte=Read1Byte(readPtr);
  return hiByte<<8 | loByte;
 }

GIF::MakeCTab
 CTabHandle MakeCTab(int size) {
// Creates a QuickDraw color table from a GIF color table
  CTabHandle CTH=(CTabHandle)NewHandle(
    sizeof(ColorTable)+
    sizeof(ColorSpec)*(size-1));
  CTabPtr CT=*CTH;
  CT->ctFlags=0;
  CT->ctSize=size-1;
  ColorSpec* cs=CT->ctTable;
   for (int cIndex=0;cIndex<size;cIndex++) {
   cs->value=cIndex;
// Expand 8-bit colors to 16-bit colors
   cs->rgb.red=257 * Read1Byte(readPtr);
   cs->rgb.green=257 * Read1Byte(readPtr);
   cs->rgb.blue=257 * Read1Byte(readPtr);
   cs++;
   }
   return CTH;
 }

GIF::ProcessHeader
 GWorldPtr ProcessHeader() {
// Returns a new offscreen gWorld.
// Returns 0 in all cases of error

  if (!CheckSignature()) return 0;

// Define logical screen size
  logScreenWidth=ReadShort();
  logScreenHeight=ReadShort();
  SetRect(&screenRect,0,0,logScreenWidth,logScreenHeight);

// Read flags
  const int flags=Read1Byte(readPtr);
  const int hasGlobalCT=flags & 0x80;
  bkGndColor=Read1Byte(readPtr);
  readPtr++;  // pixel aspect ratio is not needed

  if (hasGlobalCT) {
   const int CTsize=1L<<((flags&7)+1);
   globalCTH=MakeCTab(CTsize);
   CTabChanged(globalCTH);
  } else globalCTH=0;

// Try to get a new 8-bit offscreen gWorld
  const OSErr err =
  NewGWorld(&osWorld,8,&screenRect,globalCTH,0,0);

// Quickdraw will not free this handle for us!
  if (globalCTH) DisposeHandle((char**)globalCTH);

  if (err) return 0;
  return osWorld;
 }

GIF::ProcessExtension
 void ProcessExtension() {
// Just read and ignore the extension label
  Read1Byte(readPtr);
// and skip over any extension data blocks
  do {
   const int bsize=Read1Byte(readPtr);
   if (bsize==0) break;
   SkipBytes(bsize);
  } while(1);
 }

GIF::ProcessImage
 void ProcessImage() {
// Read the image header to determine image dimensions
//    which could be different from the logical screen
  const int imageLeft=ReadShort();
  const int imageTop=ReadShort();
  imageBase=pixelBase+imageTop*rowBytes+imageLeft;
  imageWidth=ReadShort();
  const int imageHeight=ReadShort();
  const int flags=Read1Byte(readPtr);
  const int hasLocalCT=flags & 0x80;

// Determine interlace geometry
  int skipBytes;
  int stride;
  if (flags & 0x40) {
   interlace=8;
   stride=8*rowBytes;
  } else {
   interlace=1;
   stride=rowBytes;
  }
  skipBytes=stride-imageWidth;

// Draw background color as required by the GIF spec
// (only needed if the image does not fill GIF logScreen)
  if ((globalCTH)
   && ((logScreenHeight>imageHeight)
   || (logScreenWidth>imageWidth))) {
   memset(pixelBase,bkGndColor,logScreenHeight*rowBytes);
  }
  
// The image might have a local color table which replaces
// the global color table. If so, we must update the gWorld.  
  if (hasLocalCT) {
   int CTsize=1L<<((flags&7)+1);
   CTabHandle localCTH=MakeCTab(CTsize);
//  5  9  13  17  21  25  29  33  37  41  45  49  53  57
   UpdateGWorld(&osWorld,8,&screenRect,localCTH,0,0);
   CTabChanged(localCTH);
   DisposeHandle((char**)localCTH);
  }

  DecodeImage(stride,skipBytes,imageHeight);
  
 }

GIF::DecodeImage
 void DecodeImage(int stride,int skipBytes,const int imageHeight) {

// Set up the pointers for writing the pixel map
  char* pixelStore=imageBase;
  char* endOfRow=imageBase+imageWidth;
  const char* endOfImage=endOfRow+(imageHeight-1)*rowBytes;

// Set up initial code parameters and working variables
  int minCodeSize=Read1Byte(readPtr);
  const int clearCode=1<<minCodeSize;
  for (int i=0;i<clearCode;i++) {
   d[i].Reset(i);
  }
  int codeSize=minCodeSize+1;
  unsigned int bitMask=(1L<<codeSize)-1;  
  unsigned int bitBuffer=0;
  int nextCode=clearCode+1;
  int oldCode=0;
  int bitsLoaded=0;
  bool dictFull=false;
  int bsize;
  int localReadPtr=readPtr;

// Read the file data, one GIF data block at a time
  while (0<(bsize=Read1Byte(localReadPtr))) {    
   int ch;

// Read the data of one GIF data block, and convert to pixels
   do {

// Read the next code
    while (bitsLoaded<codeSize) {        
     unsigned int nextByte=Read1Byte(localReadPtr);
     bitBuffer |= nextByte<<bitsLoaded;
     bitsLoaded+=8;
     bsize--;
     if (bsize<=0) break;
    }
    if (bitsLoaded<codeSize) continue;//end of block
    unsigned int code=bitMask & bitBuffer;
    bitsLoaded -= codeSize;
    bitBuffer >>= codeSize;          

    if (code==clearCode) {          

// Reset codeSize, and start all over
     nextCode=clearCode+1;
     codeSize=minCodeSize+1;  
     bitMask=(1L<<codeSize)-1;
     oldCode=0;  
     dictFull=false;
     } else
   
     if (code==clearCode+1) {        
   
// We are done.   
     return;

    } else {                  
  
// Decode pixel string
     if (code<nextCode) {          

// Code available in dictionary
      if (code<clearCode) {          
      // root code represents a single pixel
       ch=code;
       pixelStore=Put1(ch,stride,pixelStore,
        endOfRow,endOfImage,skipBytes);
      } else {                
      // a string of more than 1 pixel
       pixelStore=PutMore(code,stride,pixelStore,
         endOfRow,endOfImage,skipBytes,ch);
      }
      // Add next code to dictionary
      d[nextCode].Init(d+oldCode,ch);
     } else {                

// Anticipated code, we know it will be "nextCode"
      int dummy;
      // Add to dictionary first, then decode
      if (!dictFull) {          
      // Must not change dictionary if already full
       d[nextCode].Init(d+oldCode,ch);
      }
      pixelStore=PutMore(code,stride,pixelStore,
         endOfRow,endOfImage,skipBytes,dummy);
     }
     oldCode=code;
// Always just increment nextCode except when the dictionary
// is full.
// Anytime nextCode exceeds the current bitMask, the codeSize
// must increase by 1, and the bitMask stretched by 1 bit
     if (nextCode<kMaxCode) {        
      nextCode++;
      if (nextCode>bitMask) {      
       bitMask |= nextCode;
       codeSize=codeSize+1;
      }
     }  else dictFull=true;
    }
   } while(bsize);
  }
// Note: if we wanted to continue processing after decoding
//     just one picture, we would need to restore the
//     global readPtr before returning from this function.  
 }

GIF::NextPass
 char* NextPass(int & skipBytes,int & stride) {

// Defines parameters and the starting row for the
// next interlace pass. Does not get called with non-
// interlaced images, but provides robustness for them
// in case the GIF file is corrupted: it prevents writing
// past the end of the pixel map.
  stride=interlace*rowBytes;
  if (interlace>1) interlace/=2;
  skipBytes=stride-imageWidth;
  return imageBase+(rowBytes*interlace);
 }

GIF::PutMore
 char* PutMore(
     const int code,int & stride,char* dest,
     char* &endOfRow,const char* endOfImage,
     int & skipBytes,int & firstChar) {
       
// Decodes code and puts several pixels into the pixel map
//     and returns the updated pixel write pointer       
  DictEntry* stp=d+code;
  int len=stp->length;  
  char* newDest=dest+len;
  if (newDest<=endOfRow) {

// Everything fits in the same line.
// We just put the pixels directly into the pixMap,
// starting at the end of the string.
   dest=newDest;   
   do {
    *--dest=stp->value;
    stp=stp->prefix;
   } while (stp);
   firstChar=*dest;
   // newDest points to the next pixel to be painted
   return newDest;
  } else {
// We may have to handle one or more line breaks.
// Pixels past the end of the current line are pushed
// on a stack, then the pixels in the current line
// are painted; finally the pixels in the stack
// are popped off in reverse order, skipping past
// extra-bytes at line ends.
// If we reach the end of the image before the stack is
// empty, we must be in interlace mode and call a newPass()
// to find the next row to write to.
   DictEntry* stack=d+len;
   DictEntry* endStack=stack;
   int overflow=newDest-endOfRow;
   for (int k=0;k<overflow;k++) { // stack pixels in reverse
    firstChar=stp->value;
    (--stack)->stackedValue=firstChar;
    stp=stp->prefix;
   }

   if (stp) { // paint pixels for the current line
    dest=endOfRow;
    do {
     firstChar=stp->value;
     *--dest=firstChar;
     stp=stp->prefix;
    } while (stp);
   }

   do { // paint pixels from the stack, going forward
    dest=endOfRow+skipBytes;
    if (dest>=endOfImage) {
     dest=NextPass(skipBytes,stride);
    }
    endOfRow=dest+imageWidth;
    do {
     *dest++=(stack++)->stackedValue;
    } while ((stack<endStack)&&(dest<endOfRow));
   } while (stack<endStack);
   // dest points to the next pixel to be painted
   return dest;
  }
 }

GIF::Put1
 char* Put1(
     const int code,int & stride,char* dest,
     char* &endOfRow,const char* endOfImage,
     int &skipBytes) {
// Puts 1 pixel into pixMap
// and returns the updated pixel write pointer     
  if (dest>=endOfRow) {
   dest+=skipBytes;    
   if (dest>=endOfImage) {
    dest=NextPass(skipBytes,stride);  
   }
   endOfRow=dest+imageWidth;
  }
  *dest++=code;
  return dest;
 }
};

ReadImage
GWorldPtr ReadImage(FILE* inputFile) {
// The externally declared function

// Initialize a GIF structure
 GIF* gif=new GIF(inputFile);
 if (0==gif) return 0;
// Check signature, and create the off screen world
 GWorldPtr osWorld=gif->ProcessHeader();
 if (osWorld) {
// Using the geometry of the os-world ...
  PixMapHandle pmHdl=GetGWorldPixMap(osWorld);
  if(LockPixels(pmHdl)) {
   gif->pixelBase=GetPixBaseAddr(pmHdl);
   gif->rowBytes=(**pmHdl).rowBytes & 0x3FFF;
   do {
// Start reading the GIF data and look for the image separator
    int separator=gif->Read1Byte(gif->readPtr);
     if (separator==kImageSeparator) {
     gif->ProcessImage();// This is what it's about
     break;
    } else if (separator==kExtensionIntroducer) {
     gif->ProcessExtension();// these are ignored
    } else break;
   } while(1);

   UnlockPixels(pmHdl);
  }
 }

// Clean up temporary memory allocations
 delete gif;

 return osWorld;
}