TweetFollow Us on Twitter

December 96 - The OpenDoc Road: OpenDoc Memory Management and the Toolbox

The OpenDoc Road:
OpenDoc Memory Management and the Toolbox

Troy Gaul and Vincent Lo

OpenDoc has its own memory management system, but OpenDoc part editors also need to interact with the Macintosh Toolbox, which often does memory management using its own system. This column will point out potential pitfalls resulting from the interaction of these two systems, and suggest strategies to avoid them.

OpenDoc has adopted the well-tested Memory Manager from the MacApp and Bedrock frameworks for its own use. This memory manager was designed to provide fast and efficient memory allocation to the framework, and since OpenDoc's memory requirements are similar to those of a framework, it's natural for OpenDoc to reuse the code that has served the framework so well. The OpenDoc Memory Manager (as we'll call it here) is installed with OpenDoc as a shared library and handles most of the memory allocation and deallocation in an OpenDoc process.

There are several reasons for OpenDoc to have its own memory manager:

  • The OpenDoc Memory Manager can improve upon the memory manager of the underlying platform. On Macintosh 7.x systems, for instance, a faster and more space-efficient memory allocation algorithm replaces the one provided by the Macintosh Toolbox. In general, having a separate memory manager for OpenDoc allows the platform implementer to fine-tune OpenDoc performance.

  • The OpenDoc Memory Manager provides a cross-platform API for both platform implementers and part developers, covering both nonrelocatable and relocatable blocks. For platforms with no built-in relocatable blocks, platform implementers could provide relocatable blocks through the OpenDoc Memory Manager API without changing the underlying operating system.

  • The OpenDoc Memory Manager is packaged as a CFM shared library, so it can easily be replaced if a new version is required for improved performance or feature enhancements. Also, as Apple evolves the Mac OS and its memory manager, OpenDoc will adopt this new technology. Developers using the OpenDoc Memory Manager API will reap the benefits of the new system memory manager without modifying or even recompiling their part editors.
The OpenDoc Memory Manager defines a low-level procedural API, which is described in the "Memory Management" section of Appendix A in the OpenDoc Cookbook. An OpenDoc utility library named ODMemory (provided as source code) organizes the low-level API into high-level functions. The major difference between the underlying API and the "wrapper" ODMemory API is that the latter signals an exception when an error condition is encountered. For more on exception handling in OpenDoc, see "OpenDoc Exception Handling."


    OPENDOC EXCEPTION HANDLING

    Handling exceptions in OpenDoc is a large topic, which will only be touched on here. Readers should refer to the "Exception Handling" section of Appendix A in the OpenDoc Cookbook for more details.

    In a nutshell, OpenDoc allows developers to choose their own exception mechanisms while providing a convenient utility to enable these exception mechanisms to work with SOMobjects(TM) for Mac OS (the Apple implementation of the IBM SOM(TM) technology), which underlies OpenDoc.

    SOM propagates exceptions through the environment parameter (commonly known as the ev parameter). It's illegal to throw an exception out of a SOM method. Instead, the exception code should be stuffed into the ev parameter and returned to the caller. The caller should examine the ev parameter to see whether an error has been signaled in the called function. Since this checking needs to be done after every SOM method invocation, OpenDoc provides a utility to automatically check the ev parameter. If an error has been signaled, the utility will use the chosen exception mechanism (through the use of macros) to propagate the exception.

    The macros for the SOM exception handlers are prefixed with "SOM_": SOM_TRY, SOM_CATCH_ALL, and SOM_ENDTRY. These macros should not be confused with TRY, CATCH_ALL, and ENDTRY. The non-SOM exception handler macros do not propagate the exception automatically unless RERAISE is called explicitly in the catch block, and they can't be used to propagate an exception across a SOM boundary.


To ensure optimum usage of memory, part developers should use the OpenDoc Memory Manager API or the ODMemory utility library to satisfy their memory needs. The only exception is when the part editor is interacting with a Toolbox manager that requires memory to be allocated in a certain memory heap. We'll go into more detail about this in a moment.


HOW THE OPENDOC MEMORY MANAGER WORKS

The OpenDoc Memory Manager allocates fixed-sized, nonrelocatable blocks of memory and organizes them into heaps. The memory allocated for these heaps may come from the application heap, temporary memory, or the system heap. These memory blocks (called allocation segments) are subdivided into smaller memory blocks as memory requests are made by OpenDoc objects and part editors. When MMAllocate or ODNewPtr is called to allocate a block of memory, the OpenDoc Memory Manager returns a pointer to one of these memory blocks in an allocation segment.

When an OpenDoc process is started up, the OpenDoc Memory Manager allocates a small amount of memory for a heap, which becomes the default heap. Clients of the OpenDoc Memory Manager can create extra heaps and make any of these the default heap.

If a new block is requested and no allocation segments have enough free space to satisfy the request, the OpenDoc Memory Manager will trigger the creation of another allocation segment, which has the effect of growing the heap. Similarly, when all blocks in an allocation segment are freed, the segment is freed as well, shrinking the heap.

The OpenDoc Memory Manager and ODMemory utility also provide a way to allocate relocatable blocks. These blocks are not suballocated from the allocation segments; instead, the OpenDoc Memory Manager allocates them directly from the same operating system heap zone that the OpenDoc heap allocates segments from.


MEMORY PARTITION

Typically, several OpenDoc part editors run in the single process associated with a document. There's no way to determine how many part editors are going to be used in a document and how much memory each part editor requires. Therefore, it's impossible to know how big the memory partition of the process should be before opening the OpenDoc document. As described above, the OpenDoc Memory Manager has its own allocation scheme, which is not limited by the application partition, so the memory partition becomes less significant. (Currently the default heap is allocated from temporary memory, but this may change in the future.) The elimination of the need for the end user to understand the concept of application heap and adjust the memory partition is one of the design goals of OpenDoc.

However, users familiar with OpenDoc might remember that the Document Info dialog allows them to change the partition size of the process. You might ask, "If the OpenDoc Memory Manager does what it claims to, why do I need to adjust the memory partition?"

Even though most of the memory allocation is done through the OpenDoc Memory Manager, Toolbox managers do allocate memory in the application heap, and the amount required varies considerably depending on the size of the data manipulated and the operations performed. Changing the memory partition is needed to accommodate these cases.

When a document is created, it's opened into a process of a default size. Users can change the default size for the document by using the Document Info dialog. There's also a desktop utility called Infinity OpenDoc Sizer that's capable of changing either the partition size of a particular document (without first having to open it) or the default partition size used by all documents that don't already have custom partitions. It's available in the Developer Release area of the OpenDoc Web site (http://www.opendoc.apple.com) and accompanies this column on this issue's CD and develop's Web site.


MANAGING TOOLBOX MEMORY ALLOCATIONS

As your code makes calls into the Macintosh Toolbox, you'll find several places where the Toolbox allocates memory for you. Generally, this memory is allocated out of the current heap, which is usually the application heap. Since the size of an OpenDoc document's application heap is limited (the default heap size leaves only about 100K of space free after OpenDoc itself is loaded), you need to be careful when calling Toolbox routines that allocate in that heap. With some care, you can control your allocations so that your users won't have to increase your document's heap size in order to use your part.

When you're dealing with resources in particular, there are a few techniques you can use to handle memory allocations. Standard resource access routines such as GetResource will generally cause the associated memory to be allocated in the application heap. If you're using a resource for only a short period of time and it's fairly small, you can continue to use these routines to access it and load it into the application heap.

For larger resources, however, this won't work. Luckily, OpenDoc provides the utility library UseRsrcM to help, as described in the "Resource Handling" section of Appendix A in the OpenDoc Cookbook. The utility routine ODReadResource allows you to load resources from your part's shared library file into temporary memory. This works by determining the size of the resource, allocating a relocatable block of this size in temporary memory, and using ReadPartialResource to load the resource directly into that block. (Note, however, that resources read in by ODReadResource are detached; you cannot, for instance, call ChangedResource to write out modifications to them. Also, each call to ODReadResource will return a new copy of the resource.) If you need to access large resources from files other than your part -- for instance, for a sound-editing part that needs to load an 'snd ' resource from another file -- you can use this same technique yourself. A part editor must also ensure that its resource file is in the resource chain before accessing its resources (see "Resource File Access").


    RESOURCE FILE ACCESS

    In an OpenDoc environment, parts must share access to system services that applications normally own exclusively. This adds some complications to the programming model you're probably used to.

    One such shared service is the resource chain. Since there are potentially many parts all working in the active document, the resource chain must be shared between them. Also, because OpenDoc parts are shared libraries, the resources in your part's file aren't automatically available like those in an application.

    The utility library UseRsrcM facilitates making your resource file available and accessing resources from it. To open and initialize access to your shared library's resource file, you call InitLibraryResources from your CFM initialization routine. You also call CloseLibraryResources from your CFM termination routine to close the resource file when you're done with it.

    To access a resource from your part's shared library, you must first call BeginUsingLibraryResources. This adds the part's resource file to the resource chain and sets the top of the chain to that file (so that calls such as Get1Resource will retrieve resources from the correct file). After reading or writing the necessary resource, you call EndUsingLibraryResources to remove the file from the resource chain. For C++ users, a stack-based class named CUsingLibraryResources is provided to do this for you automatically.

    There are a couple of implications about using this mechanism for handling the resource chain. Because Resource Manager routines are available only inside a Begin.../End... block, you must make sure your code and the Toolbox aren't trying to manipulate resources at other times. You also have to be careful that LoadResource isn't called on a purged resource from a file that's not in the chain.

    Other things, such as icons or Balloon Help in menus (which are loaded while the menu is pulled down and the part isn't in control), can also cause problems. If you understand the relationship between your resource file and the resource chain, however, you can work around these potential pitfalls.


There are times when the Toolbox loads resources for you. In these cases, you generally can't get them to be loaded into temporary memory. In several cases, the resources are small or are allocated for only a short time, so there isn't much to worry about. For instance, when accessing resources such as string lists ('STR#'), cursors, and icon families, you can continue to use the normal Toolbox routines. In these cases, the resource is often accessed once and left around in memory. Therefore, you'll want to make sure these resources are marked as purgeable so that they can be deallocated automatically when more space is needed.

There are other times when larger resources (such as pictures) are being accessed and problems can crop up where you might not expect them. For instance, if you have a large picture that's referenced by a dialog item (like that 24-bit rendered image for the background of your About box) and there isn't sufficient memory available when the dialog is displayed, the picture won't be shown. One solution to this is to create your own heap zone, as discussed later. Another is to create a user item procedure for the picture, handling the memory allocation and spooling in of the picture yourself.

For resources such as menus and definition procedures (WDEFs, CDEFs, and MDEFs), there is little you can safely do. Most of these types of resources, although they persist for a long time, are fairly small, so having them allocated in the application heap isn't terrible.

Another commonly used piece of memory allocated by the Toolbox, but in this case not resource-based, is a region. If you're doing many region operations and the regions aren't being kept around, you can continue to use NewRgn to allocate them in the application heap. However, if you're keeping regions around for long periods of time, there's an ODMemory utility routine, ODNewRgn, that you can call to allocate your regions from the OpenDoc heap.


OTHER TECHNIQUES TO KEEP ALLOCATIONS OUT OF THE APPLICATION HEAP

In some cases, the Toolbox allows you to specify that a memory allocation come from temporary memory rather than the application heap. Probably the most common example of this is a GWorld. Passing the useTempMem flag to NewGWorld or UpdateGWorld will cause it to allocate the PixMap's buffer in temporary memory rather than in the application heap. Use these opportunities when they present themselves.

Also, if you're allowed to specify the location of memory used by the Toolbox, such as for the WindowRecord in calls to NewWindow and the sound channel in SndNewChannel, you should take these opportunities to allocate the memory with ODNewPtr rather than allowing the Toolbox to do the allocation.

Another technique that you might consider for dealing with large or long-term resources is loading them into the system heap. This can be done by setting the "system heap" flag in the resources' attributes. This has the advantage of being easy to do and working even for resources allocated by the Toolbox. The disadvantage is that increasing the system heap's size can be bad for performance if virtual memory is enabled. Since the system heap is always paged into real memory space in System 7, a large system heap means there isn't much space in RAM available for paging in the rest of memory.

Yet another technique that can sometimes be useful is to create your own heap zone. To do this, create a block in temporary memory using ODNewHandle, lock it, and create a heap inside it by calling InitZone. This technique shouldn't be used for just any allocation, but only if other methods don't work and if the allocation is ephemeral. One example we mentioned before where this might be useful is for a dialog box with large picture items. Note that you'll have to make sure the heap is large enough to hold not only the pictures but also the other dialog-related resources, and you'll want to leave some room to spare. Also, you probably wouldn't want to have more than one of these heaps allocated at a time. When locking any handle in temporary memory, make sure you unlock it again as soon as possible.

Other parts of the Mac OS, such as QuickTime, will require you to use these and other techniques to deal effectively with memory. (In the case of QuickTime, you can use SetZone to switch to the system heap.) In such cases, it's a good idea to use a tool such as Metrowerks' ZoneRanger to examine memory allocated in the application heap and, if problems are found, look for ways to move allocations elsewhere.


MEMORY MATTERS

In the future, a new system memory manager will allow for heaps that can grow. When this becomes available, many of these contortions will become unnecessary. In the meantime, however, it's important to remember that you're sharing the application heap with other clients and to act accordingly.


    RELATED READING

    For more information on the memory manager in OpenDoc and the utility libraries mentioned in this column, check out the following references:
    • OpenDoc Programmer's Guide for the Mac OS by Apple Computer, Inc. (Addison-Wesley, 1995).

    • OpenDoc Cookbook for the Mac OS by Apple Computer, Inc. (Addison-Wesley, 1995). In particular, see Appendix A, "OpenDoc Utilities."

    • The OpenDoc World Wide Web site, located at http://www.opendoc.apple.com.


TROY GAUL (tgaul@apple.com) has been writing OpenDoc parts -- er, Live Objects -- since starting at Apple last May. Having created the Infinity Windoid WDEF in 1991, he has since appeared in more About boxes than Elvis.*

VINCENT LO is Apple's technical lead for OpenDoc. When he's not dealing with Live Objects, he's frequently spotted at fine dining establishments in the San Francisco Bay Area. One of his dreams is to open up a Chinese restaurant in Italy.*

Thanks to Jens Alfke, Dave Bice, and Steve Smith for reviewing this column.*

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Adium 1.5.10.3 - Popular instant messagi...
Adium is a fast and free instant messaging client which supports AIM, ICQ, Jabber, MSN, Yahoo!, Google Talk, Yahoo! Japan, Bonjour, Gadu-Gadu, Novell Groupwise, SIP/SIMPLE (Text), and Lotus Sametime... Read more
CleanMyMac 3.8.1 - $39.95
CleanMyMac makes space for the things you love. Sporting a range of ingenious new features, CleanMyMac lets you safely and intelligently scan and clean your entire system, delete large, unused files... Read more
BusyCal 3.1.7 - Powerful calendar app wi...
BusyCal is an award-winning desktop calendar that combines personal productivity features for individuals with powerful calendar sharing capabilities for families and workgroups. Its unique features... Read more
Lyn 1.8.9 - Lightweight image browser an...
Lyn is a fast, lightweight image browser and viewer designed for photographers, graphic artists, and Web designers. Featuring an extremely versatile and aesthetically pleasing interface, it delivers... Read more
Tweetbot 2.5 - Popular Twitter client.
Tweetbot is a full-featured OS X Twitter client with a lot of personality. Whether it's the meticulously-crafted interface, sounds and animation, or features like multiple timelines and column views... Read more
Monolingual 1.7.8 - Remove unwanted OS X...
Monolingual is a program for removing unnecesary language resources from OS X, in order to reclaim several hundred megabytes of disk space. If you use your computer in only one (human) language, you... Read more
Dash 4.0.3 - Instant search and offline...
Dash is an API documentation browser and code snippet manager. Dash helps you store snippets of code, as well as instantly search and browse documentation for almost any API you might use (for a full... Read more
Posterino 3.3.6 - Create posters, collag...
Posterino offers enhanced customization and flexibility including a variety of new, stylish templates featuring grids of identical or odd-sized image boxes. You can customize the size and shape of... Read more
Apple Numbers 4.1.1 - Apple's sprea...
With Apple Numbers, sophisticated spreadsheets are just the start. The whole sheet is your canvas. Just add dramatic interactive charts, tables, and images that paint a revealing picture of your data... Read more
Apple Pages 6.1.1 - Apple's word pr...
Apple Pages is a powerful word processor that gives you everything you need to create documents that look beautiful. And read beautifully. It lets you work seamlessly between Mac and iOS devices, and... Read more

Latest Forum Discussions

See All

Get up to speed with everything you need...
In case you haven’t heard, MU Origin just got a colossal new update with new all-server events, battle modes, and systems making their way to the land of MU. Here’s a handy guide to everything you need to know about the latest content. [Read... | Read more »
Minimalist puzzle game, Cuts, free on iO...
If you're looking for a gorgeous puzzle experience on iOS devices, developer Gamebra.in's aesthetically interesting puzzler, Cuts, is discounted to free on the iOS App Store right now. [Read more] | Read more »
Anime tactical RPG, War of Crown, comes...
If you're looking for another tactical RPG fix to go alongside your Fire Emblem Heroes campaigns check out Gamevil's newest, anime-inspired tactics RPG, War of Crown, which comes out tomorrow. [Read more] | Read more »
Fantasy MMORPG MU Origin adds new modes,...
MU Origin, Webzen’s highly popular fantasy MMORPG is getting ready to shake things up for the second time this year, as a new update makes its way to the Google Play and App Store from today. Introducing new systems, modes, and events, the land of... | Read more »
Blizzard is looking to hire a mobile dev...
A new thread on the popular video game rumor forum, NeoGAF, uncovered an interesting job listing over at Blizzard Entertainment. It appears the studio behindStarCraft, World of WarCraft, Hearthstone,andOverwatch is looking to bring on a new hire... | Read more »
Legend of Zelda meets Cooking Mama in ne...
Dungeon Chef is what happens when you mix the RPG elements (and style) of a Legend of Zelda game, with cooking elements. Although, now that The Legend of Zelda: Breath of the Wild also has cookingelements, so maybe the gameplay is not so novel.... | Read more »
ChordFlow (Music)
ChordFlow 1.0.0 Device: iOS Universal Category: Music Price: $6.99, Version: 1.0.0 (iTunes) Description: ChordFlow is a chord sequencer with a unique 4-track polyphonic arpeggiator, extensive chord library, MIDI out and Ableton Link... | Read more »
The Walking Dead: A New Frontier is out...
The newest season of Telltale Games'The Walking Dead is well underway. After the release of the third episode, "Above the Law" about a month ago, episode four, "Thicker Than Water" is hot and ready for more zombies and gut-wrenching emotional... | Read more »
Best games we played this week
Another week, another new wave of mobile games do dive into. We've dug through the list of apps that came out this week to tell you which apps are worth your sweet time. And while there weren't too many games this week, there were some big ones.... | Read more »
Vignettes (Games)
Vignettes 1.0.1 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0.1 (iTunes) Description: Vignettes is a casual but unique exploration game without text or characters, where objects shapeshift as you spin them around... | Read more »

Price Scanner via MacPrices.net

Digital Paper Tablet Offers Distraction Free...
I typically spend 8-10 hours a day gazing at the screens in my laptops and iPad, as tools of my livelihood, I don’t as a rule use electronic devices for pleasure reading. I subscribe to a daily... Read more
“Today at Apple” Bringing New Educational Ses...
Apple has announced plans to launch dozens of new educational sessions next month in all 495 Apple Stores ranging in topics from photo and video to music, coding, art and design, and more. The hands-... Read more
Smart Finance Free Comprehensive Personal Fin...
Moscow-based indie developer, Alexander Survillo has announced the release and immediate availability of Smart Finance: Personal Finance, Budget & Money 1.1.4, an update to his comprehensive... Read more
12-inch 1.1GHz Retina MacBooks on sale for $1...
B&H has 12″ 1.1GHz Retina MacBooks on sale for $100 off MSRP. Shipping is free, and B&H charges NY & NJ sales tax only: - 12″ 1.1GHz Space Gray Retina MacBook: $1199.99 $100 off MSRP - 12... Read more
13-inch 2.7GHz Retina MacBook Pro on sale for...
B&H Photo has the 13″ 2.7GHz Retina MacBook Pro on sale for $130 off MSRP. Shipping is free, and B&H charges NY & NJ tax only: - 13″ 2.7GHz/128GB Retina MacBook Pro (MF839LL/A): $1169 $... Read more
15-inch 2.2GHz Retina MacBook Pros available...
B&H Photo has the 15″ 2.2GHz Retina MacBook Pro available for $200 off MSRP including free shipping plus NY & NJ sales tax only: - 15″ 2.2GHz Retina MacBook Pro (MJLQ2LL/A): $1799.99 $200 off... Read more
13-inch Touch Bar MacBook Pros on sale for up...
B&H Photo has the 2016 Apple 13″ Touch Bar MacBook Pros in stock today for up to $150 off MSRP. Shipping is free, and B&H charges NY & NJ sales tax only: - 13″ 2.9GHz/512GB Touch Bar... Read more
Apple refurbished Apple TVs available for up...
Apple has Certified Refurbished 32GB and 64GB Apple TVs available for up to $30 off the cost of new models. Apple’s standard one-year warranty is included with each model, and shipping is free: -... Read more
12-inch 1.2GHz Retina MacBooks on sale for up...
B&H has 12″ 1.2GHz Retina MacBooks on sale for up to $160 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 12″ 1.2GHz Space Gray Retina MacBook: $1439.99 $160 off MSRP - 12″ 1... Read more
HyperX Ships Pulsefire FPS Gaming Mouse, Winn...
Your reporter is a longtime fan of gaming mice for general purpose coomnputing use, finding them typically superior in comfort and performance. HyperX, a division of Kingston Technology Company, Inc... Read more

Jobs Board

Product Manager, *Apple* Platforms - Viacom...
…Product Manager to drive the execution of its iOS and AppleTV experiences. The Apple Platform Product Manager will be a leader in our Agile/Scrum environment and Read more
Geek Squad *Apple* Master Consultation Agen...
**500662BR** **Job Title:** Geek Squad Apple Master Consultation Agent **Location Number:** 000286-Canton-Store **Job Description:** **What does a Geek Squad Read more
*Apple* Mobile Master - Best Buy (United Sta...
**500710BR** **Job Title:** Apple Mobile Master **Location Number:** 000279-North Olmsted-Store **Job Description:** **What does a Best Buy Apple Mobile Master Read more
*Apple* Engineering Specialist - CSRA (Unite...
Apple Engineering Specialist All times are in Eastern Daylight Time Requisition ID Job Locations US DC Washington DC Posted Date Category Engineering Sciences Read more
*Apple* Mac Computer Technician - GeekHampto...
…complex computer issues over the phone and in person? GeekHampton, Long Island's Apple Premium Service Provider, is looking for you! Come work with our crew Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.