Programmer’s Challenge

By Mike Scanlin, Mountain View, CA

The Rules

Here’s how it works:

Each month we present a different programming challenge here. First, you write some code that solves the challenge. Second, optimize your code (a lot). Then, submit your solution to MacTech Magazine (formerly MacTutor). We choose a winner based on code correctness, speed, size and elegance (in that order of importance) as well as the postmark of the answer. In the event of multiple equally desirable solutions, one winner will be chosen at random (with honorable mention, but no prize, given to the runners up). The prize for the best solution each month is $50 and a limited edition “The Winner! MacTech Magazine Programming Challenge” T-shirt (not available in stores).

Erase Scribble

Challenge entrants liked the proposal for one function to set up their caches (which was not timed) and a second function to do the actual work (which was time critical). I can see why; it mirrors the way we implement things in the real world quite well. So, this month we have another challenge along these lines. The goal is to write the core code for an eraser tool. The tool erases line segments of a scribble drawn in the draw layer of a drawing program.

A Scribble is a variable length structure defined as:

/* 1 */
typedef struct Scribble {
 Point  startingPoint;
 Point  deltaPoints[];
}  Scribble,
  *ScribblePtr,
 **ScribbleHndl;

The startingPoint is where the scribble begins and each Point after that is a signed (dv,dh) offset from that point to the next point. At the very end of the deltaPoints array is a (0,0) entry. So if you had these bytes (0001 0004 0007 FFFF 0001 0001 0000 0000) then the scribble would consist of two line segments. One going from (x=4,y=1) to (3,8) and the other from (3,8) to (4,9).

The prototype of the two functions you write are:

/* 2 */
void *EraseScribbleInit(  theScribble, eraserSize);
ScribbleHndltheScribble;
unsigned short eraserSize;

Boolean PtInScribble(thePoint, theScribble, eraserSize,
 privateScribbleDataPtr);
Point   thePoint;
ScribbleHndltheScribble;
unsigned short eraserSize;

void      *privateScribbleDataPtr;

The routine EraseScribbleInit will be called once at the beginning of a given erase operation. It will NOT be timed (only the PtInScribble will count towards your time). It can create whatever lookup tables PtInScribble may need (up to 32K) and return a pointer to that private data. The return value from EraseScribbleInit will be passed to PtInScribble as the privateScribbleDataPtr parameter. You decide what it points to (if anything). And, yes, the same ScribbleHndl will be passed to the Init routine that is passed to the PtInScribble routine.

The scribbles that will be used will have between 2 and 500 Points (including the startingPoint). The eraserSize parameter is the size in pixels of the square eraser that the user is moving around. It will vary from 4 to 16 pixels and will be an even number. The function PtInScribble should return true if any part of the eraser is touching any of the vertices of the Scribble and false if not. Note that having the eraser intersect the middle of a line segment that is part of the Scribble does not count as a hit. This particular eraser only erases when it touches an endpoint of a segment. The (dv,dh) numbers will typically be small, probably 16 or less in absolute value with the average around 4 in absolute value (so that we have smooth Scribbles). The bounding box that contains each Scribble will be 1024 pixels square or smaller.

The goal is to make a PtInScribble routine fast enough so that the eraser the user is dragging around is not sluggish (imagine erasing a page containing 20 or 30 separate overlapping Scribbles). My test routine that calls your PtInScribble routine will take care of all the drawing and actual erasing; you just have to determine when the eraser touches a vertex of the Scribble or not.

Oh, yeah, one more thing: September is always Assembly Language Month here at the Programmer Challenge, and so, for this month only, you are allowed to use as much assembly language as you like in your solution. In fact, it is perfectly admissible to send in a C wrapper around a 100% assembly language solution. But you still have to write it in Think C using the prototypes above (use asm {} within the body of the function). In addition, you can write 68040 assembly if you want to (i.e. use instructions or addressing modes that may not exist on 68000, 020 and/or 030). Let ’em rip!

Two Months Ago Winner

Congratulations to Bob Noll (Arlington Heights, IL) for his excellent entry in the Color Space Conversion challenge. This is Bob’s first 1st place showing and it was during a month where there were several very high quality entries. Most of the 15 entries were no more than 50% slower than Bob’s winning entry.

Here are the times, code sizes and data sizes for each entry. The data size represents how many bytes the RGBtoYUVInit routine allocated when it was called (i.e. the size of the private lookup tables). Numbers in parens after a person’s name indicate how many times that person has finished in the top 5 places of all previous Programmer Challenges, not including this one:

Name time code data

Bob Noll 135 1038 394272

Dan Waylonis 153 518 9232

Bob Boonstra (10) 165 1508 789520

Ernst Munter (1) 173 404 12336

Larry Landry (2) 174 1554 795728

Joe Francis 183 1154 9342

Ronald Nepsund (2) 190 744 7184

Clement Goebel (3) 195 1094 8208

Michael Panchenko (1) 198 278 9232

Allen Stenger (7) 227 1528 520240

John Schlack 236 916 9232

Ted DiSilvestre 283 674 9408

Daniel Farmer (1) 289 830 8208

Kevin Cutts (3) 563 698 31952

David Nebiker 4955 1178 27664

Bob decided to optimize his RGBtoYUVInit routine even though it would not count towards his overall time (bravo!). To get both speed and accuracy he uses a 96-bit fixed point number to create his lookup tables. He then implemented his own routines to perform math on this custom format. And he was able to integrate his rounding factor into his lookup tables to save himself that final step when calculating y, u and v.

I received quite a bit of mail regarding my statement “.5 rounding down to zero.” I did not disqualify anyone for misinterpreting this. What I meant was “.5 should round towards zero” meaning that -.5 to .5 inclusive should be rounded to zero. There are gains to be had when writing a JPEG compression engine if you can get as many zeros as possible up front, which is the reason for this non-standard (i.e. symmetrical about zero) way of rounding.

Here’s Bob’s winning solution:

/* 3 */
Written by Robert A. Noll, July, 1994.  Copyright © 1994 Robert A. Noll
RGBtoYUV.h 
RGBtoYUV.h - Header for RGBtoYUV routine.
#define NegToZero 1
typedef struct {
 unsigned long   sig;
 unsigned long*  rp;
 unsigned long*  gbp;
 unsigned char*  up;
 unsigned char*  vp;
 unsigned long   r[256];
 unsigned long   gb[65536];
 unsigned char   u[65536];
 unsigned char   v[65536];
 } PrivateBlock;

void *RGBtoYUVInit(void);

void RGBtoYUV(unsigned char *rPtr, // Red buffer
   unsigned char *gPtr,   // Green buffer
   unsigned char *bPtr,   // Blue buffer
   unsigned char *yPtr,   // Luminance buffer
     signed char *uPtr,   // U Chrominance buff
   signed char *vPtr,// V Chrominance buff
   unsigned long numPixels, // Number of Pixels
   void *privateDataPtr); // My private data

RGBtoYUV.c
RGBtoYUV.c - Convert RGB color to YUV color.

#include "RGBtoYUV.h"

RGBtoYUV

RGBtoYUV - Calculate the Y, U, and V values for set of R, G, and B values 
input.  This function relies completely on the data set up by RGBtoYUVInit 
and therefore checks the validity of the data sent.  It will return with 
no action if data is not valid.

void RGBtoYUV(unsigned char *rPtr, // Red buffer
   unsigned char *gPtr,   // Green buffer
   unsigned char *bPtr,   // Blue buffer
   unsigned char *yPtr,   // Luminance buffer
   signed char *uPtr,// U Chrominance buff
   signed char *vPtr,// V Chrominance buff
   unsigned long numPixels, // Number of Pixels
   void *privateDataPtr)  // My private data
 {
 PrivateBlock* pb = (PrivateBlock*)privateDataPtr;
 unsigned long *rp = pb->rp;
 unsigned long *gbp = pb->gbp;
 unsigned char *up = pb->up;
 unsigned char *vp = pb->vp;
 unsigned long tmp;
 union {
 unsigned short i;
 unsigned char c[2];
 } gb;
 
 if (pb->sig != 'RNCS') return;
 
 for (; numPixels>0; --numPixels,++rPtr,++gPtr,++bPtr,
 ++yPtr,++uPtr,++vPtr) {
 gb.c[0] = *gPtr;//Get index into gb and v tables 
 gb.c[1] = *bPtr;
 
 tmp = rp[*rPtr] + gbp[gb.i]; //  Calculate Y value from tables
 *yPtr = *((unsigned char*)&tmp);
 
 *vPtr = (*rPtr - vp[gb.i]) >> 1;  //  Calculate V value from table
 
 gb.c[0] = *rPtr;//Get index into u table
 gb.c[1] = *gPtr;
 
#if NegToZero
 //If for Negs .5 rounds up to 0 then if R == G and R|G > B we calc special.
 if (*rPtr==*gPtr && *gPtr>*bPtr)
 *uPtr = ((*bPtr - *gPtr) + 1) >> 1;
 else
#endif
 *uPtr = (*bPtr - up[gb.i]) >> 1;  // Calculate U value from table
 }
 }


RGBtoYUVInit.c
RGBtoYUVInit.c - Sets up the tables for use by RGBtoYUV.

#include "RGBtoYUV.h"

// A fixed point number type with 8 bits of intregral part and 88 bits 
of fractional part.
typedef struct {
 unsigned long val[3];
 } BigNum;
 
typedef struct {
 BigNum y[256];
 BigNum g[256];
 BigNum b[256];
 } TempBlock;

Multiply
Multiply - Takes a BigNum and multiplies it by a number in the range 
0 - 255.  
Result is put into 'ans'.

void Multiply(void* bits, unsigned char num, void* ans)
 {
 int x;
 unsigned short* bp = (unsigned short*)bits + 5;
 unsigned short* ap = (unsigned short*)ans + 5;
 unsigned long val = 0;
 
 for (x=6; x; --x,--bp,--ap) {
 val += (unsigned long)*bp * num;
 *ap = val;
 val >>= 16;
 }
 }

Add
Add - Adds two BigNums and returns the result in 'ans'

void Add(void* bits1, void* bits2, void* ans)
 {
 int x;
 unsigned short* bp1 = (unsigned short*)bits1 + 5;
 unsigned short* bp2 = (unsigned short*)bits2 + 5;
 unsigned short* ap = (unsigned short*)ans + 5;
 unsigned long val = 0;
 
 for (x=6; x; --x,--bp1,--bp2,--ap) {
 val += (unsigned long)*bp1 + *bp2;
 *ap = val;
 val >>= 16;
 }
 }
 
RGBtoYUVInit
RGBtoYUVInit - Sets up all the tables used by the RGBtoYUV function.
 
Since accuracy is a concern, I use my own number format and routines 
to calculate tables.  It turns out that FP routines are just as accurate 
but my stuff is 3x faster.

About calculations in general:

A table with all 16M values for each of Y, U, and V would be nice but, at 48MB, would be way over 1MB limit. With minor calcs in 'RGBtoYUV' I need two 64K entry tables for each of U and V. At full precision they would be 768K each. They get reduced greatly later. The Y table I break into two tables: one for the R calcs and one for the G and B combination. At full precision they would take 3K and 768K respectively. They get reduced later also. At this point we are a lot closer to the 1MB limit.

About Y calculations:

• 24 bits of fractional precision for the two tables was found to give proper results. This was found by checking the result of rounding the full precision result against the rounding of the sum of the two table entries for EVERY one of the 16M combinations. This allows us to represent our table entries in tables of 'unsigned long's. Table sizes are thus reduced to 1K and 256K.

• Rounding is handled by adding .7FFFFFFFFFFFFFFFFFFFFF to the 'gb' table values. This allows RGBtoYUV to just truncate its result.

• When R, G, and B are all equal then Y is the same number. With the tables that are built the checking for such a limited case (256 out of 16,777,216) just added space. (To calc 16M values takes a little over 2 minutes)

About U and V in general:

• Since B and R are multipied by .5 for these calcs we can except them as 7 bits of integral part and 1 bit of fraction.

• Since the B or R calculation will always result in some number X.0 or X.5. This enables us to do all rounding in the table values. Also we are able to store the table numbers in the same 7.1 bit format.

• If rounding is always down then RGBtoYUV can just truncate after the subtraction. If rounding is to be toward zero then special handling is needed.

About V calculations:

Because of the number of decimal places and the fact that the multipliers add up to .50000001 the G and B combination will never result in exactly X.0 or X.5. Thus for V we don't need to worry about whether -0.5 rounds up or down.

About U calculations:

• The R and G combination values will result in an X.0 or X.5 result when R == G

• If rounding is always down (.5 -> 0 and -.5 -> -1) then truncation after the calculation is all that is needed.

• If rounding is always towards zero (.5 -> 0 and -.5 -> 0) then RGBtoYUV will add .5 and then truncate when R and G are equal. All other cases will still be handled with just truncation.

/* 4 */
void *RGBtoYUVInit(void)
 {
 short r,g,b;
 BigNum mat[] = {
    { 0x004C8B43, 0x95810624, 0xDD2F1A9F },/*.29900000*/
    { 0x009645A1, 0xCAC08312, 0x6E978D4F },/*.58700000*/
    { 0x001D2F1A, 0x9FBE76C8, 0xB4395810 },/*.11400000*/
    { 0x002B3246, 0xA4293F94, 0x6A85AFD7 },/*.16873590*/
    { 0x0054CDB9, 0x5BD6C06B, 0x957A5028 },/*.33126410*/
    { 0x00800000, 0x00000000, 0x00000000 },/*.50000000*/
    { 0x00800000, 0x00000000, 0x00000000 },/*.50000000*/
    { 0x006B2F1C, 0x4D3DA074, 0x7F2DDD88 },/*.41868760*/
    { 0x0014D0E3, 0xDDB57D4F, 0xE1EA9636 },/*.08131241*/
    { 0x007FFFFF, 0xFFFFFFFF, 0xFFFFFFFF } /*.49999999*/
    };
 unsigned long *rp, *gbp;
 BigNum *gp, *bp, *yp, yt, ut, vt, xt;
 unsigned char *up, *vp;
 PrivateBlock *pb;
 TempBlock *tb;

 pb = (PrivateBlock*)NewPtr(sizeof(PrivateBlock));
 if (pb==0) {
 DebugStr("\pOut of Memory");
 return;
 }
 pb->rp = pb->r;
 pb->gbp = pb->gb;
 pb->up = pb->u;
 pb->vp = pb->v;
 
 tb = (TempBlock*)NewPtr(sizeof(TempBlock));
 if (tb==0) {
 DebugStr("\pOut of Memory");
 return;
 }
 
// r-values never = x.0 or x.5
 rp = pb->rp;
 for (r=0; r<256; ++r,++rp) {
 Multiply(&mat[0],r,&xt);
 *rp = *((unsigned long*)&xt);
 }
 
// This method same as 0-255, 0-255 method.
// My calc routines equal SANE routines and are 3 times faster.
// u-values are never == x.0 or x.5
// v-values are never == x.0 or x.5
  yp = tb->y;
  gp = tb->g;
  bp = tb->b;
 gbp = pb->gbp;
  up = pb->up;
  vp = pb->vp;
 for (b=0; b<256; ++b,++yp,++gp,++bp,++gbp,++up,++vp) {
 Multiply(&mat[2],b,yp);
 Add(yp,&mat[9],yp);
 *gbp = *((unsigned long*)yp);
 Multiply(&mat[4],b,gp);
 // Add(gp,&mat[9],gp);
 *up = *((unsigned short*)gp) >> 7;
 Multiply(&mat[8],b,bp);
 // Add(bp,&mat[9],bp);
 *vp = *((unsigned short*)bp) >> 7;
 }
 for (g=1; g<256; ++g) {
 Multiply(&mat[1],g,&yt);
 Add(&yt,&mat[9],&xt);
 *gbp = *((unsigned long*)&xt);
 ++gbp;
 Multiply(&mat[3],g,&ut);
 // Add(&ut,&mat[9],&xt);
 *up = *((unsigned short*)&ut) >> 7;
 ++up;
 Multiply(&mat[7],g,&vt);
 // Add(&vt,&mat[9],&xt);
 *vp = *((unsigned short*)&vt) >> 7;
 ++vp;
 yp = &tb->y[1];
 gp = &tb->g[1];
 bp = &tb->b[1];
 for (b=1; b<256;
 ++b,++gbp,++yp,++up,++gp,++vp,++bp) {
 Add(&yt,yp,&xt);
 *gbp = *((unsigned long*)&xt);
 if (g==b) {
 *up = b;
 *vp = b;
 }
 else {
 Add(&ut,gp,&xt);
 *up = *((unsigned short*)&xt) >> 7;
 Add(&vt,bp,&xt);
 *vp = *((unsigned short*)&xt) >> 7;
 }
 }
 }
 
 if (tb!=0)
 DisposPtr((Ptr)tb);

 pb->sig = 'RNCS';
 return pb;
 }