Peeking at Heap Zones

By Bob Gordon, Contributing Editor

Memory is the stuff of which there never is enough. To reduce problems due to lack of memory, most languages provide means to grab some memory for use and later release it. Standard C has a number of functions to manage memory allocation and deallocation. We are not going to discuss these in any detail because the Macintosh has a complete memory management system. We have been using the memory management routines indirectly every time we opened a window, but as we go further into Mac programming we need to manipulate memory directly.

Figure 1: Examining the heap

Macintosh Memory Organization

Just about every book on programming the Macintosh has pictures of the Macintosh memory map. For the 64K ROMS, it looks roughly like:

I left off the physical addresses because they are typically not important, and because many of them can change as a function of memory size. For the 128K ROMS, the arrangement is slightly different. The ROM trap dispatch table has been expanded and is now in two sections, with the second section between the system heap and the second system globals area.

For this discussion, the most important areas are the Application Heap and the Stack. The stack is where all C parameters and local variables reside (actually, this depends on the compiler. Some compilers will pass some parameters in registers, and programmers may specify variables to be register variables, and if a register is available, the variable will be placed in a register). The stack grows and shrinks with the function calls and returns.

The application heap is the area where memory is explictly reserved and released. Unlike the stack, memory reserved in the heap does not disappear when a function returns. This allows the creation of large, complex data structures "on the fly."

C Memory Allocation

Since C is used on systems other than the Mac (really, there are other computers!), you might want to know about the standard C memory functions.

malloc() allocates some memory. It receives one parameter, the number of bytes to allocate, and returns a pointer to the region of memory.

free() deallocates memory previously allocated. It receives one parameter, a pointer previously returned by malloc().

The actual names and implementation will vary somewhat by compiler. Lightspeed C provides eight allocation functions and five deallocation functions in its standard library. Most of these are there to provide compatibility with Unix. The other Macintosh C compilers with which I am familiar also provide the standard C functions.

Unless you are using your Mac to develop software to run on another system, it is not a good idea to use the standard C memory functions. The primary reason for this is that on the Mac there are two kinds of allocated memory, relocatable and non-relocatable. None of the C compilers I have seen make any attempt to relate the standard C memory functions to the Mac memory functions (I assume the standard C functions return pointers to non-relocatable blocks of memory, but this is not made apparent).

Macintosh Memory Allocation

Again, since information about the Mac memory management system is readily available, I am not going to attempt to cover all the details here. There are, however, several key concepts which probably deserve some attention.

As mentioned above, memory comes in two kinds, relocatable and nonrelocatable. A relocatable block may be moved around in the heap by the operating system to make room for additional allocations. Since it may be moved, you do not receive a pointer to the block but a handle that points to a master pointer that points to the block.

Since the master pointer does not move (it is nonrelocatable), the value of the handle remains constant. When you request a nonrelocatable block, you receive a pointer directly to the block since the operating system will not move it. In general it seems that relocatable blocks are preferred since the operating system can move them out of the way when needed. If there are many nonrelocatable blocks, the operating system may not be able to satisfy a memory request because there may be no single space available big enough to meet the need. Without the ability to move blocks in the heap around (compact the heap), you will run out of memory sooner.

One problem with relocatable blocks is that it takes two pointers to access the data. One can always obtain a pointer to the block, but the system may move the block while you are using it. This situation, known as a dangling pointer, can cause quite exciting and hard to diagnose problems. If you need to use a particular block a lot in a function, you may lock the block which makes it temporarily nonrelocatable. Be sure to unlock it when you are done.

Another option for relocatable blocks is to purge them from memory. The operating system will do this if it needs more memory than is available, but only if you have marked a block as purgeable. New relocatable blocks start out as not purgeable. Note that if a block is purged, you will still have the handle, but the pointer it contains will be set to zero. To reuse the block, it is necessary to reallocate the block and rebuild the data. There is a function that reallocates purged blocks; the data is the programmer's responsibility.

Memory management errors are available through the function MemError(). A value of zero (NoErr) means there was no error; negative values represent error codes.

printw

There were a few changes to printw. We have added the ability to print rectangle and point coordinates. This is useful when using graphics. This time I added hex output. It is also now a seperate file. I have found it useful, but we probably won't print the whole thing again after this month, since this is the second time we've seen this routine.

The Program

Our program this month makes a number of trap calls that deal with memory management. From these calls, we can display information about the application heap and see how the creation of handles and pointers affects the available heap space. To keep things simple, we have created a very minimal Mac program that just puts up a blank window. The real heart of the program is the memory functions menu that lets us create handles and pointers and free them so we can see the effect on the heap. Fig. 1 shows how we print the memory information in our debugging print window using the printw() routine we learned about in a previous issue of MacTutor.

When you run the program, use the Zone Info menu item to see the state of the heap. I left the abc window in (under the File menu). Make a window and then run Zone Info. The Show Zone item traces through the zone and gives a line of information about each block, telling if it is free, relocatable or non-relocatable. By the way, the information needed to trace through the heap is not ordinarily available so there is a structure defined just before main() that defines the information needed to understand the heap. This structure, called memheads, is actually the definition of a block header. The other menu items create handles and pointers using NewHandle and NewPtr, and then if the handle or pointer is valid, another menu item lets us dispose of those handles and pointers, using DisposHandle and DisposPtr. The Zone Info menu item should show the effect of each of these operations on the available space in the heap.

There is another "feature" to this program. Every time you reserve a relocatable block, the size reserved doubles. With zone info, you can watch the memory manager increase the size of the heap and eventually run out of memory.

The key to the program is the GetZone trap call. This returns a pointer to the current heap zone. Once we have this pointer, our ShowZone and CountZone routines will read and print information about the zone for us. See the domem(item) routine in the listing to see where this trail starts with the memory menu item.

The term zone is how the heap is divided. Each zone in the heap is divided into blocks. A block is an even number of bytes, the minimum of which is 12. When your application starts up, call MaxApplZone to expand the application heap zone to its limit. Normally, an application would only be concerned with a single heap zone for itself, but you can create additional zones in the heap. One of our menu items does call MaxApplZone so you can see if it has any effect depending on when you call it. We can find out the free space left in a heap zone by calling FreeMem.

Once we get a pointer to the current heap zone, then we can examine the zone. A heap zone contains a 52 byte zone header, the allocated blocks within the zone, and a final minimum size block at the end of the zone called the zone trailer. The zone header is a pascal record defined below:

Zone = RECORD
 bkLim: Ptr;{zone trailer block}
 purgePtr:Ptr;   {used internally}
 hFstFree:Ptr;   (first free master pointer}
 zcbFree: LONGINT; {number of free bytes}
 gzProc:ProcPtr; {grow zone function}
 moreMast:Integer; {master pointers to allocate}
 flags: Integer; {used internally}
 cntRel:Integer; {not used}
 maxRel:Integer; {not used}
 cntNRel: Integer; {not used}
 maxNRel: Integer; {not used}
 cntEmpty:Integer; {not used}
 cntHandles:Integer; {not used}
 minCBFree: LONGINT; {not used}
 purgeProc: ProcPtr; {purge warning proc}
 sparePtr:Ptr;   {used internally}
 allocPtr:Ptr;   {used internally}
 heapData:Integer{first usable byte in zone}
END;

The zone pointer is defined to be of type THz, which is simply declared to be ^Zone. HeapData is the first two bytes in the block header of the first block in the zone. Hence to find the first usable block, we can say @(myZone^.heapData), which is a pointer to the first block header. Or in c, we would use &zone->heapData. The block header is 8 bytes: the tag byte, a three byte block size, and a four byte identifier that indicates whether the block is relocatable, non-relocatable or free. The four byte identifier is actually a handle, pointer or unused depending on the nature of the block. The tag byte also contains information on the nature of the block in bits 6 and 7. We can examine the nature of every block in the heap zone by adding the block size to the address of the first block and going from block to block checking the two high order bits of the tag byte. All of this is discussed in great detail in the IM chapter on the memory manager. This is implemented in our program in the showzone(zone) routine. Our memhead structure actually combines the tag byte and block size into a single four byte field called physsize. The second four bytes in the block header are stored in our memhead structure in the relhand field. We decode the physsize field to obtain the tag byte, and the block size, then we break down the tag byte to obtain the block status, and the size correction. All of this is done in the showhead(head) routine. Study carefully the three routines showhead, showzone and countzone to see how the block headers are decoded and the block size, status and address are then printed in our print window.

Final Comments

Using the techniques shown in this program, you can construct your own heapshow utility or TMON heap display. A good discussion of how TMON examines and displays the heap, is available in Dan Weston's new book, Macintosh Assembly Language Programming, Vol. II. There are some useful memory management functions not included this month such as MoreMasters() which reserves space for master pointers needed by NewHandle(). These should be easy to include in the program if you need to get an idea of how they work.

Next time we'll return to quickdraw and try to draw polygons, regions, and pictures.

Figure 4: ShowZone gives a heap dump

/* mem.c
 * explore memory allocation
 * LS C by Bob Gordon
 */

 #include "abc.h"
 #include "MemoryMgr.h"
 #include "Quickdraw.h"
 #include "EventMgr.h"
 #include "WindowMgr.h"
 #include "MenuMgr.h"
 #include "FontMgr.h"
 
 /* defines for menu ID's */
 
 #defineMdesk    100
 #defineMfile    101
 #defineMedit    102
 #defineMmem103
 
 /* File */
 #defineiNew1
 #defineiClose   2
 #defineiQuit    3
 
 /* Edit */
 #defineiUndo    1
 #defineiCut3
 #defineiCopy    4
 #defineiPaste   5
 
 /* Memory */
 #defineiZone    1
 #defineiMzone   2
 #defineiInfo    3
 #defineiHand    4
 #defineiPtr5
 #defineiFreeH   6
 #defineiFreeP   7
 
 /* Global variables */
 
 MenuHandle menuDesk;/* menu handles */
 MenuHandle menuFile;
 MenuHandle menuEdit;
 MenuHandle menuMem;
 
 WindowPtrtheWindow;
 WindowRecord  windowRec;
 Rect   dragbound;
 Rect   limitRect;
 
/*
 * structure needed to examine heap not typically
 * supplied by Mac development systems because normal
 * applications do not need to examine the heap. 
 */
 struct memheads
 {
 long   physsize;
 long   relhand;
 };
  
main()
{
 initsys(); /* sys init */
 initapp(); /* appl init */
 eventloop();
}

/* system initialization  */
initsys() 
{
 InitGraf(&thePort); 
 InitFonts();    
 InitWindows();
 InitCursor();
 InitMenus();
 theWindow = Nil;/* no window */
 SetRect(&dragbound,0,0,512,250);
 SetRect(&limitRect,60,40,508,244);
}

/*
 * application initialization
 * Sets up menus.*/
initapp()
{
 setupmenu();
}

/*
 * set up application's menus
 * Each menu is a separate group
 * of lines.  
 */
setupmenu()
{
 menuDesk = NewMenu(Mdesk,CtoPstr("\24"));
 AddResMenu (menuDesk, 'DRVR');
 InsertMenu (menuDesk, 0);
 
 menuFile = NewMenu(Mfile, CtoPstr("File"));
 AppendMenu (menuFile, 
 CtoPstr("New/N;Close;Quit/Q"));
 InsertMenu (menuFile, 0);
 
 menuEdit = NewMenu(Medit, CtoPstr("Edit"));
 AppendMenu (menuEdit, 
 CtoPstr("(Undo/Z;(-;(Cut/X;(Copy/C;(Paste/V;(Clear"));
 InsertMenu (menuEdit, 0);
 
 menuMem = NewMenu(Mmem, CtoPstr("Memory"));
 AppendMenu (menuMem,
 CtoPstr("Show Zone;Max Zone;Zone Info;New Handle;New Pointer"));
 AppendMenu (menuMem,CtoPstr("Free Handle;Free Pointer"));
 InsertMenu (menuMem, 0);
 
 DrawMenuBar();
}
 
/* Event Loop 
 * Loop forever until Quit
 */
eventloop()
{
 EventRecordtheEvent;
 char   c;
 short  windowcode;
 WindowPtrww;
 
 while(True)
 {
 if (theWindow)      /* this code is here to */
 { /* prevent closing an */
 EnableItem(menuFile,2);  /* already closed window */
 DisableItem(menuFile,1);
 }
 else   
 { 
 EnableItem(menuFile,1);
 DisableItem(menuFile,2);
 }
 if (GetNextEvent(everyEvent,&theEvent))
   
 switch(theEvent.what)    
 { /* only check mouse */
 case mouseDown:
 domousedown(&theEvent);
 break;
 default:
 break;
 }
 }
}

/* domousedown
 * handle mouse down events
 */
domousedown(er)
 EventRecord*er;
{
 short  windowcode;
 WindowPtrwhichWindow;
 short  ingo;
 long   size;
 long   newsize;
 RgnPtr rp;
 Rect   box;
 Rect   *boxp;
 
 windowcode = FindWindow(er->where, &whichWindow);
 switch (windowcode)
 {
 case inDesk:
 if (theWindow notequal 0)
 {
 HiliteWindow(theWindow, False);
 DrawGrowIcon(theWindow);
 }
 break;
 case inMenuBar:
 domenu(MenuSelect(er->where));
 break;
 }
}
 
/* domenu
 * handles menu activity
 * simply a dispatcher for each
 * menu.
 */
domenu(mc)
 long   mc; /* menu result */
{
 short  menuId;
 short  menuitem;
 
 menuId = HiWord(mc);
 menuitem = LoWord(mc);
 
 switch (menuId)
 {
 case Mdesk : break; /* not handling DA's */
 case Mfile : dofile(menuitem);
  break;
 case Mmem  : domem(menuitem);
     break;
 }
 HiliteMenu(0);
}

domem(item)
 short  item;
{
 THz    zone;
 static longsize = 1024;
 static Handle   hand = 0;
 static Ptr ptr = 0;
 struct memheads *memhead;
 
 switch (item)
 {
 case iZone :
 zone = GetZone();
 printw("\nzone %ld ",zone);
 showzone(zone);
 break;
 case iMzone :
 MaxApplZone();
 break;
 case iInfo :
 zone = GetZone();
 countzone(zone);
 break;
 case iHand :
 hand = NewHandle(size);
 printw("\nhandle %ld pointer %lx error %d ",hand,*hand,MemError());
 printw(" free mem %ld ",FreeMem());
 size = size * 2;
 break;
 case iPtr :
 ptr = NewPtr(size);
 printw("\npointer %ld error %d free mem %ld",ptr,MemError(),FreeMem());
 showhead(ptr-8);
 break;
 case iFreeH :
 if (hand equals 0)
 printw("\nNo current handle to free");
 else
 {
 DisposHandle(hand);
 hand = 0;
 }
 break;
 case iFreeP :
 if (ptr equals 0)
 printw("\nNo current pointer to free");
 else
 {
 DisposPtr(ptr);
 ptr = 0;
 }
 }
}

showhead(head)
 struct memheads *head;
{
 uchar  tagbyte;
 
 printw ("\naddress %ld ",(char*)head + 8);
 tagbyte = head->physsize >> 24;
 printw ("size correction %d ",tagbyte & 0xF);
 tagbyte >>= 6;
 switch (tagbyte)
 {
 case 0:
 printw(" free block ");
 break;
 case 1:
 printw(" non rel    ");
 break;
 case 2:
 printw(" rel        ");
 break;
 }
 printw(" physical size %ld ",head->physsize & 0x00FFFFFF);
 return (tagbyte);
}

showzone(zone)
 THz    zone;
{
 struct memheads *memhead;
 short  tblocks = 0;
 short  tfree = 0;
 short  trl = 0;
 short  tnonrel = 0;
 
 memhead = (struct memheads*)&zone->heapData;
 while (memhead < (struct memheads*)zone->bkLim)
 {
 switch (showhead(memhead))
 {
 case 0 : tfree++; break;
 case 1 : tnonrel++; break;
 case 2 : trel++; break;
 }
 tblocks++;
 memhead = (struct memheads*)((char*)memhead + (memhead->physsize & 0x00FFFFFF));
 }
 printw("\n blocks %d free %d relocatable %d non-relocatable %d ",
   tblocks, tfree, trel,tnonrel);
}
 
countzone(zone)
 THz    zone;
{
 struct memheads *memhead;
 short  tblocks = 0;
 short  tfree = 0;
 short  trel = 0;
 short  tnonrel = 0;
 uchar  tagbyte;
 
 memhead = (struct memheads*)&zone->heapData;
 while (memhead < (struct memheads*)zone->bkLim)
 {
 tagbyte = memhead->physsize >> 24;
 tagbyte >>= 6;
 switch (tagbyte)
 {
 case 0 : tfree++; break;
 case 1 : tnonrel++; break;
 case 2 : trel++; break;
 }
 tblocks++;
 memhead = (struct memheads*)((char*)memhead + (memhead->physsize & 0x00FFFFFF));
 }
 printw("\n blocks %d free %d relocatable %d non-relocatable %d ",
   tblocks, tfree, trel, tnonrel);
}
 
/* dofile
 * handles file menu
 */
dofile(item)
 short  item;
{
 char   *title1; /* first title for window */
 Rect   boundsRect;
 
 switch (item)
 {
 case iNew :/* open the window */
 title1 = "ABC Window";
 SetRect(&boundsRect,50,50,400,200);
 theWindow = NewWindow(&windowRec, &boundsRect,
 CtoPstr(title1),True,documentProc,
 (WindowPtr) -1, True, 0);
 DrawGrowIcon(theWindow);
 PtoCstr(title1);
 DisableItem(menuFile,1);
 EnableItem(menuFile,2);
 break;
 
 case iClose :   /* close the window */
 CloseWindow(theWindow);
 theWindow = Nil;
 DisableItem(menuFile,2);
 EnableItem(menuFile,1);
 break;
 
 case iQuit :    /* Quit */
 ExitToShell();
 break; 
 }
}