Related Info: Calling a Code Resource Window Manager
Process Manager

One-Application Patches

How to write an application specific extension.

James W. Walker, University of South Carolina

About the author

James W. Walker earned a Ph.D in mathematics at M.I.T. He now teaches mathematics at the University of South Carolina.

Not Quite an INIT

To make a change in the behavior of all applications running on your Mac, you can use an INIT (now known as a system extension) to patch some traps. The trouble with this approach is that it has to be compatible with all of the applications, and probably imposes some overhead even in applications where it isn’t doing anything. On the other hand, you could disassemble one application and make a direct patch. Not only is that likely to be extremely difficult, you will probably have to do it over again when the next version comes out. There is a middle ground: Code resources that can be added to an application and patch traps only in that application.

Under MultiFinder or System 7, trap patches that are installed after startup time apply to only one application, because each application has its own copy of the trap dispatch table. That’s the easy part. The tricky part is, how do you get your code called in order to install the patches? What you can do is use your own version of one of the standard definition functions, such as a WDEF, MDEF, MBDF, or CDEF. In my example, I will use a WDEF, since that makes it easy to modify the appearance of windows in an application. That is the approach used by the CMaster and PopUpFuncs products.

Adding Word Wrapping and Dollar Pairs

All word processors can wrap words as you type them, but not all text editors can do so. For instance, BBEdit 2.1.3 can wrap text after you type it, but not as you type it, and the THINK C 5.0 editor cannot wrap words at all. (You probably wouldn’t want to use word wrapping while writing program code, but you might want it for long comments.) My example will patch an editor to provide a simple form of word wrapping. It will also add a little icon to the title bar of each document window, which you can click to turn wrapping on or off. This patch will work in THINK C, BBEdit, or ASLEdit+.

In order to wrap typing, we would ideally want to detect when the insertion point has passed the right edge of the window or some other preset margin, and then change a previous space character into a carriage return. However, that would be difficult to do without knowing the application’s internal data structures. Therefore, I am going to do a cruder form of word wrapping: Detect when the insertion point is within a certain distance of the right edge of the window, and then change the next space to a carriage return. This method can fail if you happen to type a really long word at the end of a line, but it usually works.

As another example of a feature that can be added with a one-application patch, I will make each typed dollar sign generate another dollar sign and a left arrow character. Sound like a crazy feature? Not if you’re typing mathematics in TEX format, which uses pairs of dollar signs to delimit mathematical formulas.

A Modular Design

The project will use four types of code resources, so that individual functions can be added or deleted without recompilation. At the top of the hierarchy (illustrated below), there is one WDEF resource. At the next level, there are OAPn resources that are called by the WDEF after each wNew message, and OAPd resources that are called by the WDEF after each wDraw message. One of the OAPn resources installs an event patch, and the other one watches the insertion point. The OAPd code draws a small icon in the window’s title bar. Finally, at the third level of the hierarchy, there are OAPe resources, which filter events. There is also a small data resource of type OAP1 which is used for communication between some of the code resources.

Each of these code resources is built as a separate THINK C project. All require MacHeaders, and some need the MacTraps library.

Figure One: Calling Hierarchy

The WDEF

In order for our WDEF to be used for standard windows, we must use the resource ID 0, and override the standard WDEF in the System. However, it calls the standard WDEF to do most of the work. I use RGetResource just in case the standard WDEF 0 is in ROM and not in the System. Incidentally, you should be aware that adding a WDEF resource might trigger virus detection code in some applications. [See Nick Pissaro article in Vol. 8, No. 2 (Virus issue) for one example. - TechEd.]

One tricky aspect of using a WDEF to patch an application is that if you use ResEdit to edit a WIND resource in that application, the custom WDEF may be called. If the WDEF patches some traps, and then ResEdit closes the file, then the traps remain patched but the patch code goes away. So the next time one of those traps is executed, it’s bomb city. I found out about this the hard way, of course.

To avoid this ResEdit problem, I use the routine No_ResEdit_Danger (see the listing of patcher WDEF.c), which checks whether the file that contains the WDEF resource is the same as the resource fork of the current application. If not, the WDEF does nothing other than call the real WDEF to handle the window. (Desk accessories are a special case. Although they act like applications in many ways under System 7, CurApRefnum is the file reference number of the System, not the DA file.)

The Wrapping Icon

The ‘OAPd’ resource, whose source code is shown in the listing of wrap icon.c, is called by my WDEF after each wDraw message for a document-style window. I have hard-coded the two possible 8 by 8 icons, though of course one could use resources instead.

Where’s the Insertion Point?

To perform word wrapping, we need to know where characters are appearing in the window. One natural approach would be to look at the pen location of the window at the time that a keyboard event is received. This works in THINK C and BBEdit, but not in ASLEdit+. You might also think of patching _DrawChar to watch as characters are drawn, but in fact these editors do not call DrawChar. The only approach I thought of that works in all three cases is to patch _InverRect and watch where the insertion point is drawn. When InvertRect is called with a rectangle of width 1, it is probably flashing the insertion point.

In the listing, you will see that the patch is installed using routines named GetToolTrapAddress and SetToolTrapAddress. These are not listed in Inside Macintosh, but are defined in the standard header file OSUtils.h. They simply provide a more efficient interface to the same trap routines used by NGetTrapAddress and NSetTrapAddress.

Assembly Glue

Some folks will insist that when you patch traps, you should save and restore every blessed register. Others will point out that Inside Mac says that stack-based toolbox routines need not preserve registers A0, A1, D0, D1, or D2, so a patch on such traps shouldn’t need to preserve those registers either. In the trap patches in the InvertRect.c and events.c listings, I have taken the very conservative route of preserving all registers. If you choose not to preserve all registers, then the only register you really have to worry about is A4, which is used by THINK C to access global variables. You could begin the patch with

/* 1 */

asm {
 move.L A4, -(SP)
 LEA    main, A4
}

and end the patch with something like

/* 2 */

 asm {
 move.L Old_SystemEvent, A0
 move.L (SP)+, A4
 UNLK   A6
 JMP    (A0)
}

However, if you do it, remember that if the prior trap address is a global variable that is referenced using register A4, then you had better use that value before you restore the original value of A4.

Watching Events

There are a number of ways you can monitor events. You can tail-patch GetNextEvent, tail-patch GetOSEvent, patch the low-memory global JGNEFilter, head-patch PostEvent, or head-patch SystemEvent, and there are probably other ways. However, these methods do not all behave the same. Patching GetNextEvent will miss events destined for desk accessories, even DAs that have been made into pseudo-applications under System 7. On the other hand, JGNEFilter is truly global, i.e., it will see events belonging to other applications. I have chosen to patch SystemEvent. For some purposes, the fact that SystemEvent doesn’t receive null events might be a disadvantage, but not for my present purpose.

The listing events.c shows the patch to SystemEvent, which passes each event to any OAPe resources that may be present.

Word Wrapping Events

The event filter listed in wrap events.c monitors keyboard events to perform word wrapping, and monitors mouse events to detect clicks in the word wrapping icon. If wrapping is on, and the event is a space character, and the insertion point is close to the margin, then the event filter changes the event to a return character. If the event is a mouse click in the wrapping icon, then the event filter toggles the wrapping state, changes the event to a null event (so that the host application won’t think you’re trying to drag the window), and causes the wrapping icon to be redrawn. Note in particular that when I call PaintOne to invalidate the wrapping icon, I save and restore the GrafPort. This is necessary because PaintOne changes to the Window Manager port, and does not restore the port afterward.

Paired Dollar Signs

The final event filter, listed in dollars.c, looks for keyDown events representing dollar sign characters, and responds to a dollar sign by posting another dollar sign event and a left arrow event. I have to be careful about this in order to avoid an infinite loop. A normal keyboard event has both a character code and a key code in the message field of the event record, but when I post the second dollar sign, I post only a character code without a key code. Then when the second dollar sign arrives at the event filter, the event filter knows it’s a fake and can be ignored. (Of course this subtlety wouldn’t occur if you paired parentheses or braces.) Note the use of PPostEvent to post the left arrow event, so that I can specify that no modifier keys are down. This is necessary because the shift key will be pressed when the first dollar sign is typed, and some editors, such as THINK C and BBEdit, assign a different meaning to a shifted arrow than to an ordinary arrow.

Other Ideas

Obviously, you could hard-wire other keyboard macros into an application using the same methods as were used to pair dollar signs. A keyboard macro could do fancier text manipulations on a selected range of text by copying the text to the clipboard, manipulating it, and pasting it back. Perhaps there are other traps you’d like to patch; for instance ASLEdit+ has a hard-coded default font, which you can change by patching GetFNum. You could even link your editor to another application, using the Process Manager to bring the other application to the front, and then posting keyboard or mouse events from the background.

Listing: defs.h
#ifndef NIL
#define NIL 0L
#endif

typedef pascal long (*WDEF_proc)( short,
 WindowPeek, short, long );

// OAPn resources are called after wNew messages
typedef void (*OAPn_proc)( void );

// OAPd resources are called after wDraw messages
typedef void (*OAPd_proc)( WindowPeek );

// OAPe resources are event filters
typedef void (*OAPe_proc)( EventRecord *event );

typedef struct { // format of 'OAP1' resource
 Booleanwrap;
 char   filler;
 short  last_insertion_point;
} Wrap_info;

Listing: patcher WDEF.c
/* -------------------------------------------
 patcher WDEF.c
 
 THINK C "Set Project Type..." settings:
 code resource, type WDEF, ID 0,
 custom header, preloaded,
 file type 'rsrc', file creator 'RSED'.
 -------------------------------------------
*/
#include "defs.h"

pascal long main( short var_code,
 WindowPeek the_window,
 short message, long param );
Boolean No_ResEdit_danger( void );

/* The one and only global variable */
static Boolean   run_needed = true;

pascal long main( short var_code,
 WindowPeek the_window,
 short message, long param )
{
 long   retval;
 Handle real_WDEF_h;
 short  save_resfile;
 SignedByte real_WDEF_state;
 WDEF_procReal_WDEF;
 Ptr    save_A4;
 Handle code_h;
 short  res_index;
 OAPn_procOAPn_p;
 OAPd_procOAPd_p;
 THz    save_zone;
 
 asm {
 move.L A4, save_A4
 LEA    main, A4 ; for access to global
 }

 save_resfile = CurResFile();
 UseResFile( SysMap );
 real_WDEF_h = RGetResource( 'WDEF', 0 );
 real_WDEF_state = HGetState( real_WDEF_h );
 HLock( real_WDEF_h );
 Real_WDEF = (WDEF_proc)
 StripAddress(*real_WDEF_h);
 UseResFile( save_resfile );
 
 /* Here's where we call the real system WDEF */
 retval = Real_WDEF( var_code, the_window,
 message, param );
 HSetState( real_WDEF_h, real_WDEF_state );

 if (No_ResEdit_danger())
 {

 save_zone = GetZone();
 SetZone( ApplicZone() );
 
 if ( (message == wNew) && run_needed )
 {
 for (res_index = 1; ; ++res_index)
 {
 code_h = GetIndResource('OAPn', res_index);
 if (code_h == NIL)
 break;
 HLock( code_h );
 OAPn_p = (OAPn_proc) StripAddress(*code_h);
 (*OAPn_p)();
 }
 run_needed = false;
 }
 
 else if ( (message == wDraw) && // draw...
 (LoWord(param) == 0) &&  // all of window
 ((var_code & 3) == 0) )  // document type
 {
 for (res_index = 1; ; ++res_index)
 {
 code_h = GetIndResource('OAPd', res_index);
 if (code_h == NIL)
 break;
 HLock( code_h );
 OAPd_p = (OAPd_proc) StripAddress(*code_h);
 (*OAPd_p)( the_window );
 }
 }
 
 SetZone( save_zone );
 }
 
 asm {
 moveA.Lsave_A4, A4
 }
 return( retval );
}
/* -------------------------------------------
 No_ResEdit_danger If the host application
 is being edited by ResEdit
 rather than executing
 normally, we do not want this WDEF to
 install any patches.
 -------------------------------------------
*/
Boolean No_ResEdit_danger( void )
{
 Handle my_h;
 short  my_resfile;
 
 my_resfile = -1;
 my_h = RecoverHandle( (Ptr) main );
 if (my_h != NIL)
 my_resfile = HomeResFile( my_h );
 return (my_resfile == CurApRefNum) ||
 (CurApRefNum == 2);
}

Listing: wrap icon.c
/* ------------------------------------------
 wrap icon.c
 
 THINK C "Set Project Type..." settings:
 code resource, type 'OAPd', ID 1000,
 custom header, preloaded and locked,
 file type 'rsrc', file creator 'RSED'.
 ------------------------------------------
*/
void main( WindowPeek the_window );

void main( WindowPeek the_window )
{
 BitMap icon_map;
 long   bits[4];
 Rect   dest;
 GrafPtrwmgr_port;
 Boolean**wrapping;
 
 if (!the_window->visible || !the_window->hilited
 || !the_window->goAwayFlag)
 return;
 
 wrapping = (Boolean **)GetResource('OAP1', 128);
 if (wrapping != NIL)
 {
 icon_map.rowBytes = 2;
 icon_map.baseAddr = (Ptr) &bits;
 icon_map.bounds.top = icon_map.bounds.left
 = 0;
 icon_map.bounds.right
 = icon_map.bounds.bottom
 = 8;
 if (**wrapping)
 {
 bits[0] = 0x00000000L;
 bits[1] = 0xFC000400L;
 bits[2] = 0x04001500L;
 bits[3] = 0x0E000400L;
 }
 else   // not wrapping
 {
 bits[0] = 0x04000200L;
 bits[1] = 0xFF000200L;
 bits[2] = 0x04000000L;
 bits[3] = 0x00000000L;
 }
 dest = (**(the_window->strucRgn)).rgnBBox;
 dest.left += 22;
 dest.top += 6;
 dest.right = dest.left + 8;
 dest.bottom = dest.top + 8;
 GetPort( &wmgr_port );
 CopyBits( &icon_map, &wmgr_port->portBits,
 &icon_map.bounds, &dest, srcCopy, NIL );
 }
}

Listing: patch InvertRect.c
/* --------------------------------------------
 patch InvertRect.c
 
 THINK C "Set Project Type..." settings:
 code resource, type 'OAPn', ID 1001,
 custom header, preloaded and locked,
 file type 'rsrc', file creator 'RSED'.
 --------------------------------------------
*/
#include <Traps.h>
#include "defs.h"

void main(void);
void My_InverRect( void );

/* -------- global variables ---------- */
long  Old_InverRect = NIL;

void main(void)
{
 long   save_A4;
 
 asm {
 move.L A4, save_A4
 LEA    main, A4
 }

 if (Old_InverRect == NIL)
 {
 Old_InverRect = GetToolTrapAddress(
 _InverRect );
 SetToolTrapAddress( (long)My_InverRect,
 _InverRect );
 }
 
 asm {
 move.L save_A4, A4

 }
}


/* ---------------------------------------------
 My_InverRect    Watch for the insertion point
 to be drawn, and record its
 horizontal coordinate.
 ---------------------------------------------
*/
void My_InverRect( void )
{
 Rect   *rect;
 Wrap_info**info;
 
 asm {
 movem.La0-a5/d0-d7, -(SP); save registers
 LEA    main, A4 ; access to globals
 move.L 8(A6), rect
 }
 
 if ( rect->right - rect->left == 1 )
 {
 info = (Wrap_info **)
 GetResource('OAP1', 128);
 (**info).last_insertion_point = rect->right;
 }
 
 /*
 The following code restores all registers and
 jumps to the saved trap address.  It relies
 on there being at least 4 bytes on the stack
 frame, which can be trashed by moving the
 saved A6 down.  Bear in mind that THINK C will
 insert UNLK A6 and RTS instructions afterward.
 */
 asm {
 move.L (A6), -4(A6)
 move.L Old_InverRect, (A6)
 subQ   #4, A6
 movem.L(SP)+, A0-A5/D0-D7
 }
}

Listing: events.c
/* ------------------------------------------
 events.c
 
 THINK C "Set Project Type..." settings:
 code resource, type 'OAPn', ID 1000,
 custom header, preloaded and locked,
 file type 'rsrc', file creator 'RSED'.
 ------------------------------------------
*/
#include <Traps.h>
#include "defs.h"

void main(void);
void My_SystemEvent( void );

/* -------- global variables ---------- */
long  Old_SystemEvent = NIL;

void main(void)
{
 long   save_A4;
 
 asm {
 move.L A4, save_A4
 LEA    main, A4
 }

 if (Old_SystemEvent == NIL)
 {
 Old_SystemEvent = GetToolTrapAddress(
 _SystemEvent );
 SetToolTrapAddress( (long)My_SystemEvent,
 _SystemEvent );
 }
 
 asm {
 move.L save_A4, A4
 }
}

/* ------------------------------------------
 My_SystemEvent  This head patch watches
 events.
 ------------------------------------------
*/
void My_SystemEvent( void )
{
 EventRecord*evt;
 WindowPeek front;
 short  res_index;
 Handle code_h;
 OAPe_procEvent_filter;
 
 asm {
 movem.La0-a5/d0-d7, -(SP); save registers
 LEA    main, A4 ; access to globals
 move.L 8(A6), evt ; copy event pointer
 }
 
 front = (WindowPeek) FrontWindow();
 if ( (front != NIL) &&
 (front->windowKind != 2) && front->visible &&
 front->hilited && front->goAwayFlag )
 {
 for (res_index = 1; ; ++res_index)
 {
 code_h = GetIndResource( 'OAPe', res_index );
 if (code_h == NIL)
 break;
 Event_filter = (OAPe_proc)
 StripAddress(*code_h);
 Event_filter( evt );
 }
 }
 
 asm {
 move.L (A6), -4(A6)
 move.L Old_SystemEvent, (A6)
 subQ   #4, A6
 movem.L(SP)+, A0-A5/D0-D7
 }
}

Listing: wrap events.c
/* ---------------------------------------------
 wrap events.c Watch keyboard events to do
 word wrapping, and watch mouse
 events to handle clicks in the
 wrap icon.
 
 THINK C "Set Project Type..." settings:
 code resource, type 'OAPe', ID 1000,
 custom header, preloaded and locked,
 file type 'rsrc', file creator 'RSED'.
 ---------------------------------------------
*/
#include <Script.h>
#include "defs.h"
void main( EventRecord *evt );

#define RETURN_MESSAGE    0x0002240DL
#define WRAP_FACTOR10
#define SCROLLBAR_WIDTH   16
#define MODIFIER_KEYS0x1F00

void main( EventRecord *evt )
{
 WindowPeek front;
 Wrap_info**wrap_info;
 Rect   icon_rect;
 RgnHandleredraw_rgn;
 GrafPtrsave_port;
 short  wrap_margin, font_size;
 
 wrap_info = (Wrap_info **)
 GetResource( 'OAP1', 128 );
 if (wrap_info == NIL)
 return;
 front = (WindowPeek) FrontWindow();
 
 if ( (evt->what == keyDown) &&
 ((evt->message & charCodeMask) == ' ') &&
 ((evt->modifiers & MODIFIER_KEYS) == 0) &&
 ((**wrap_info).wrap) )
 {
 font_size = front->port.txSize;
 if (font_size == 0)
 font_size = GetDefFontSize();
 wrap_margin = font_size * WRAP_FACTOR
 + SCROLLBAR_WIDTH;
 if ( (**wrap_info).last_insertion_point >
 front->port.portRect.right - wrap_margin )
 {
 (**wrap_info).last_insertion_point = 0;
 evt->message = RETURN_MESSAGE;
 }
 } // end if keyDown && space

 else if (evt->what == mouseDown)
 {
 /*
 If the click was in our little icon in the
 window's title bar, then toggle the wrapping
 state.
 */
 icon_rect = (**(front->strucRgn)).rgnBBox;
 icon_rect.left += 22;
 icon_rect.top += 6;
 icon_rect.right = icon_rect.left + 8;
 icon_rect.bottom = icon_rect.top + 8;
 
 if (PtInRect( evt->where, &icon_rect ))
 {
 evt->what = nullEvent;
 (**wrap_info).wrap = !(**wrap_info).wrap;
 ChangedResource( (Handle) wrap_info );
 
 redraw_rgn = NewRgn();
 RectRgn( redraw_rgn, &icon_rect );
 GetPort( &save_port );
 PaintOne( front, redraw_rgn );
 SetPort( save_port );
 DisposeRgn( redraw_rgn );
 }
 } // end if mouseDown
 
}

Listing: dollars.c
/* ---------------------------------------------
 dollars.cWhen a dollar sign is typed, type
 another oneand then a left arrow.
 
 THINK C "Set Project Type..." settings:
 code resource, type 'OAPe', ID 1001,
 custom header, preloaded and locked,
 file type 'rsrc', file creator 'RSED'.
 ---------------------------------------------
*/
void main( EventRecord *evt );

#define LEFT_ARROW_MESSAGE0x00027B1CL

void main( EventRecord *event )
{
 EvQEl  *event_q_data;

 /*
 In this case we have to be careful to avoid
 causing an infinite loop, so we post an
 abnormal dollar message, with no key code.
 */

 if ( (event->what == keyDown) &&
 ((event->message & charCodeMask) == '$') &&
 (event->message != '$') )
 {
 PostEvent( keyDown, '$' );
 PPostEvent( keyDown, LEFT_ARROW_MESSAGE,
 &event_q_data );
 event_q_data->evtQModifiers = 0;
 }
}