TweetFollow Us on Twitter

June 91 - Children of the DogCow - Segments from Outer Space

Children of the DogCow - Segments from Outer Space

Kent Sandvik

Greetings! This is Worldwide Developer Conference week, and DTS is busy working at the debugging lab. Thanks to a great manager, I am able to stay home half time, check out my new kid, and at the same time write sample code and another article for FrameWorks.

In this column, I'm taking a closer look at segmentation: MacApp segmentation strategies, virtual memory possibilities, and other issues related to segmentation. Some parts will be elementary for Veterans of the MacApp Psychic Wars, but I hope the science fiction references in some of the headings will keep such readers awake. Onward!

THE BASICS OF SEGMENTATION

First, you need to specify the segment where each method (or member function, in C++) should reside. This is done using the notation {$S SegmentName} in Object Pascal, and #pragma segment SegmentName in C++. If you forget to place this segment compiler directive in your methods, it will inherit the earlier directive (in C++ as well as Object Pascal) all the way to the end of the file, so suddenly you'll find a lot of methods inside one of your segments. Methods without any defined segment go into the Main segment, which could get really crowded after a while. So, check your segmentation directive for each method. MacBrowse has a neat function for doing this that shows what each segment contains.

The Segmentation Makers sidebar provides guidelines on organizing your methods into segments.

THE TRUE NATURE OF SEGMENTS

Segments are really CODE resources in disguise, so MacApp is able to control the purge and lock bits on segments just as it does on handles or objects (as in TObject's Lock and UnLock methods).

A locked handle is also unpurgeable, so you don't need to worry about purging once you have locked the object in memory. MacApp's global function DBUnloadSeg makes handles, or CODE segments, unlocked-which makes the resource available for purging as well.

Methods are the actual routines that are stored in the CODE resources; data is stored either on the stack, in the heap or in the A5-world, depending. In many cases, calling a method whose segment is not currently stored in memory causes a segment load to occur that might have to move heap blocks in order to locate a place to put the new segment. This is one reason why calling a new method can suddenly make dereferencing bugs pop up.

THE FALL OF UNLOADALLSEGMENTS

One key to the mysteries of MacApp segment loading and unloading is the global function UnloadAllSegments. UnloadAllSegments is called when the application starts, and also when the application has too little memory to do its duties. UnloadAllSegments purges all code segments from memory except resident segments.

It's UnloadAllSegments that displays "I really don't think that you want to unload a segment into which you are going to return!" in the debug window. This happens when MacApp has determined that you are about to unload a segment containing a method that you will need to return to later. (In other words, your stack currently references a method contained in the segment that is about to be unloaded.)

If you do a "find references to UnloadAllSegments" in MacBrowse, you'll find that this function is called from many places; from the initialization phase of the application, and here and there from within various loops.

GOOD NEWS FROM DEBUGGER SPACE

The MacApp debugger can list all the segments that are currently loaded in application heap memory. When you drop down to the debugger level, type 'H' for the segment and heap subsection. Inside this level, to show the segments, type either 'S' (segments currently in memory) or 'ß' (all segments, whether or not they are currently in memory). This shows the segment names, sizes, states (loaded, resident), and so on.

You can do additional resource manipulation from the debugger with the Toggle, or 'X' flag. Inside this level, type 'S' so that each time a segment load occurs, MacApp will break into the debugger and print the routine name that triggered the segment load.

The 'U' flag turns off the automatic segment unloading done by the UnloadAllSegments routine. This is handy for finding out if your program's crashes have been due to mysterious jumps into unexpected routines: if a pointer to another method was suddenly being made invalid as that method's segment was unloaded, the stage is set for a healthy crash. Your code may stop crashing when you use the 'U' flag to turn automatic segment unloading off; if so, that's a good hint to look for problems of this kind.

The 'R' flag checks to see if the total size of the currently loaded resource exceeds the maximum. You can also set the maximum to a new value.

SERVANTS OF THE LIB

Sometimes we want to move a separate segment back into the Main segment, in order to avoid too many segments-a condition that can lead to heap fragmentation. Link can remap the segmentation with routines to other segment names. The syntax looks like this:
Link [options...] objectfile… ≥ progress.file
-sn=oldSegName1=newSegName1
-sn=oldSegName2=newSegName2

This is useful when using MPW 3.2 with MacApp 2.0 or 2.0.1 to remap segments back to the Main segment. The story is, some of the standard functions in libraries in MPW 3.2 have been split from the Main segment. This causes serious heap fragmentation in your MacApp application-for example, when you try to call SetHandleSize(). To avoid this, make the following modifications in the Basic Definitions file:

SegmentMappings = ð
SegmentMappings = ð
#-- insert here
-sn PASLIB=Main ð
-sn STDCLIB=Main ð
#-- end of insertion

This causes the errant routines in the Pascal and Standard C libraries to be remapped back into the Main segment. Also, change the lines in the MacApp.r file as shown in the MacApp.r changes sidebar.

Another solution is to use the linker to mark code resources from the libraries that were once in main as locked. These segments will then be loaded into memory and placed with the main segment, avoiding fragmentation problems. To do this, modify the user variable OtherLinkOptions in the Basic Definitions file:

OtherLinkOptions = ð
-ra PASLIB=resLocked ð
-ra STDCLIB=resLocked

You can also use this technique of locking code resources into memory in your MAMake files (OtherLinkOptions=)-but be careful with these experiments. Finally, you can use the linker to merge old segments into new segments with the -sg option:

-sg newSeg[=old[,old]…] # merge old segments into newSeg

The MPW Lib tool also contains options for changing segment names and merging segments into a segment, which is useful for cases where you only have access to the object code library.

THE RES! AND SEG! OF ETERNITY

Sometimes there is confusion about res! and seg! resources. The seg! resource defines those segments that are loaded to memory when the program is making maximum use of memory. MacApp uses this information when keeping track of the code reserve in order to ensure there is room for the seg! code segments at the maximum point of memory use.

The res! resource defines those segments that are always resident in the heap (segments are made permanently resident via a global function called SetResidentSegment). Note that even if you define a segment in the res! resource, because it's a handle it will still float around in memory.

One use for making segments permanently resident is for time-critical functions that are grouped together in a special segment; thus, loading the segment doesn't require overhead if the method is suddenly needed. For example, this could be used to reduce overhead for time-critical communication methods. Here's an example of a res! resource defined in the resource file:

resource 'res!' (kMyMacApp,    purgeable) {
{   "AWriteConn";
    "AReadConn";
    "APoll";
#if qInspector && !qDebug
    "GDebugConn";
#endif
#if qPerform
    "GPerformanceComms";
#endif
};
};

VIRTUAL MEMORY ARRIVES AT EASTERWINE

One reason other operating systems don't require programmers to specify segmentation is the underlying mechanism of virtual memory and page segmentation, which makes the issue of loading code on demand an automatic process.

With System 7, the Macintosh operating system now has virtual memory. However, there is still need for the programmer to specify segments in the code.

The basics of virtual memory

In a virtual memory system the whole memory is divided into pages. These pages can be loaded into memory or reside on the hard disk at any time. When the program refers to a page that isn't in physical memory, the system generates a page fault, and the memory management unit (MMU) loads the page from the backing store on the hard disk.

When the page fault occurs, the memory manager (with the help of the MMU) first frees up physical memory so it can load the needed page by selecting unused page frames and writing them to the backing store. Then it reads the page data for the needed frame. Thus, pages that aren't needed are usually residing on the hard disk. This event, usually called page-fault handling, requires special hardware in most cases.

Apple's VM scheme for the Mac

Apple's virtual memory only runs on Macintosh computers with a MMU unit. It requires the virtual memory mode to be on (it is not always on by default). The Macintosh implementation of virtual memory is not based on the so-called copy-on-demand, where page after page is loaded in per demand. In the Mac's VM scheme, the Segment Loader has to load in the whole segment. And the Mac's virtual memory model is single-virtual (not multiple-virtual), which means the applications share the address space with other applications and the system.

Global-replacement algorithm

Apple's implementation is a global-replacement algorithm. It keeps a queue of physical page frames in memory, and the frames in this queue are accessed sequentially by page- frame numbers. Each page frame in memory is automatically marked by the MMU according to whether the frame has been recently accessed by the CPU. Mac VM keeps a pointer to the last page frame that has been replaced. When a page-fault occurs, this pointer looks in the queue for a page frame that hasn't been modified since it was read from the disk, and that also has not been accessed recently (an "old" frame). When it find such a beast, it returns information about it to the memory manager. This page can now be "stolen" to use for other purposes.

If the VM does not find such a frame, it looks for a page that has been modified but not replaced recently. If this doesn't work, the VM tries to find the first frame that doesn't contain a page held in physical memory.

This algorithm is simple and fast. It doesn't need to know about application states, and it's space efficient. For more on this algorithm, read the article in the November 1989 issue of Byte, "Mac VM Revealed" by Phil Goldman.

Backing store management

In the Mac's VM implementation, the pages are locked to 4096 byte sizes (this is also the minimal optimal transfer size for the SCSI Manager). The complete virtual memory image is kept on the backing store, instead of pages. This means the end user uses more disk space, but that the page swapping speed is probably improved. If the whole virtual image is not on the disk, then every time a page is replaced, it must be written to the backing store, even if it has not been modified. The biggest burden with virtual memory is the transfer time to and from the hard disk; with the whole virtual disk image on the backing store, the system does not need to write pages that are not modified back to the disk.

VM & segmentation

Automatic segmentation handling is impractical for VM's current implementation. Note that even UNIX™ and other modern (UNIX modern? well…) operating systems still have problems with code that generates a lot of page faults (by loading in a lot of page frames for each function), which leads to unnecessary paging, which leads to swapping (where whole processes are swapped out of physical memory), which leads to thrashing (when all the operating system does is swap processes in and out, 24 hours a day). So, some of these systems have automatic tools that analyze the segmentation strategies and, if possible, move code segments around to avoid paging (which is evil after all).

The best solution is a combination of both virtual memory and segmentation. VM allows the user to run larger programs than would otherwise be possible, and if the developer organizes segments intelligently, excess paging is avoided.

There is still a need for some smart segmentation analysis tool which could produce segmentation directives by analyzing each function in order to figure out how to produce a segment organization such that methods are grouped together for maximum efficiency. VACUUM JUMP TABLES There is a known relation between jump table sizes and segmentation. For normal procedures and functions, a jump table entry is not needed if all calls to the routine are from the same segment. But if there are calls to other segments from the routine, jump table entries are needed. Examine the segmentation of your code; you might find places where a change in segmentation would eliminate jump table entries. The linkmap output (using the MABuild -LinkMap option) shows what each segment contains. With some effort you may shrink big jump tables and improve the performance of your whole application.

Some people worry that many Get and Set methods will increase the jump table entries considerably, but you can avoid this by using clever segmentation strategies or by using C++'s inline functions. Anyway, if your classes are infested with millions of Get and Set methods, perhaps it is time to examine the object. Is it really a structure in disguise?

Caching of results inside the class decreases the need for Get and Set calls. Plus, the major parts of an object can be placed inside one single segment for another performance improvement.You can use dumpobj to dump the object file and find information about each segment.

The Segment Loader has to fill the jump table with the right addresses when the segments are loaded in. When the segment is unloaded, the jump table has to be reset with information about the missing segment. MacApp has to make sure that memory is always available for data and unloaded segments. All this takes time, so clever segmentation does improve performance. For example, if important functions are in the same segment, you eliminate other segment loading events, and when MacApp calls UnloadAllSegments, a place is created for the next suite of segments needed.

STRANGE MPW 3.2 TOYS

For a long time segments were restricted to 32k sizes due to the A5-relative data referencing with 16 bit offsets, but MPW 3.2 eliminates this 32k limit on segment size via new switches to the compilers and the linker.

The 68020 introduced 32-bit PC-relative branching (BSR.L statements), but that didn't help the Classic and other 68000-based Macintosh computers. Instead, MPW 3.2 makes use of branch islands. This simple, elegant concept is based on the implementation of PC-relative code-to-code references. The linker splits a large code segment up into smaller 32k areas by inserting branch islands. These branch islands serve as intermediate points that are within range of PC-relative jumps, thus making it possible to make a call across a segment that would otherwise result in a larger-than-32k jump.

Another new feature is "32-bit everything," which transparently removes the infamous limitations on code segment sizes, jump table sizes and the size of the global data areas. The drawback is a larger code size footprint and some slowdown due to increased load time for the larger code segments. But hey, look what you get!

32-bit everything is activated by using -model far options while compiling and linking. The Release Notes for MPW 3.2 will explain the implementation completely; basically, the trick is that the compilers generate instructions with 32-bit addresses (instead of the normal 16-bit offsets), and that these 32-bit addresses are relocated at load time by the segment load address or by the contents of A5, as appropriate.

Finally, one can generate larger than 32K jump tables using the -wrap option. This uses unused space in the global data area for additional jump table entries when it starts to get crowded inside the 32K segment. Programmers doing large MacApp programs will love this! However, at best this utility doubles the jump table size, and if your global data area is already filled with data, you're out of luck.

If you want to use these new 32-bit everything features from MPW 3.2 with MacApp, you'll need a couple of new MacApp library files. These are available on ETO #3, as well as most of the 32-bit everything support. ETO#4 will contain the final MPW 3.2 with tools and libraries to support these new features.

THE END OF THIS ARTICLE

I hope I have helped to explain the basic issues concerning segmentation and MacApp. The brave MacApp explorer could investigate further to learn about the global functions that manipulate segments-such as GetSegFromPC, GetSegNumber, and GetResLoad. Perhaps this will be the topic of a future column.

REFERENCES

  • Mac VM Revealed, Byte November 1989
  • Inside Macintosh, II, Chapter 2 (Segment Loader)
  • MPW 3.2 Release Information (available real soon now)
  • MacApp & Object Oriented Programming (using C++) handouts, Dave Wilson
  • Many various MacApp.Tech$ links
 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Apple iOS 10.3 - The latest version of A...
iOS 10 is the biggest release of iOS ever. A massive update to Messages brings the power of the App Store to your conversations and makes messaging more personal than ever. Find your route with... Read more
Postbox 5.0.12 - Powerful and flexible e...
Postbox is a new email application that helps you organize your work life and get stuff done. It has all the elegance and simplicity of Apple Mail, but with more power and flexibility to manage even... Read more
Apple Keynote 7.1 - Apple's present...
Easily create gorgeous presentations with the all-new Keynote, featuring powerful yet easy-to-use tools and dazzling effects that will make you a very hard act to follow. The Theme Chooser lets you... Read more
Apple Pages 6.1 - Apple's word proc...
Apple Pages is a powerful word processor that gives you everything you need to create documents that look beautiful. And read beautifully. It lets you work seamlessly between Mac and iOS devices, and... Read more
Apple Numbers 4.1 - Apple's spreads...
With Apple Numbers, sophisticated spreadsheets are just the start. The whole sheet is your canvas. Just add dramatic interactive charts, tables, and images that paint a revealing picture of your data... Read more
Apple macOS Sierra 10.12.4 - The latest...
With Apple macOS Sierra, Siri makes its debut on Mac, with new features designed just for the desktop. Your Mac works with iCloud and your Apple devices in smart new ways, and intelligent... Read more
Xcode 8.3 - Integrated development envir...
Xcode includes everything developers need to create great applications for Mac, iPhone, iPad, and Apple Watch. Xcode provides developers a unified workflow for user interface design, coding, testing... Read more
Skype 7.49.0.274 - Voice-over-internet p...
Skype allows you to talk to friends, family and co-workers across the Internet without the inconvenience of long distance telephone charges. Using peer-to-peer data transmission technology, Skype... Read more
Dropbox 22.4.24 - Cloud backup and synch...
Dropbox is an application that creates a special Finder folder that automatically syncs online and between your computers. It allows you to both backup files and keep them up-to-date between systems... Read more
Posterino 3.3.5 - Create posters, collag...
Posterino offers enhanced customization and flexibility including a variety of new, stylish templates featuring grids of identical or odd-sized image boxes. You can customize the size and shape of... Read more

The best deals on the App Store this wee...
Deals, deals, deals. We're all about a good bargain here on 148Apps, and luckily this was another fine week in App Store discounts. There's a big board game sale happening right now, and a few fine indies are still discounted through the weekend.... | Read more »
The best new games we played this week
It's been quite the week, but now that all of that business is out of the way, it's time to hunker down with some of the excellent games that were released over the past few days. There's a fair few to help you relax in your down time or if you're... | Read more »
Orphan Black: The Game (Games)
Orphan Black: The Game 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: Dive into a dark and twisted puzzle-adventure that retells the pivotal events of Orphan Black. | Read more »
The Elder Scrolls: Legends is now availa...
| Read more »
Ticket to Earth beginner's guide: H...
Robot Circus launched Ticket to Earth as part of the App Store's indie games event last week. If you're not quite digging the space operatics Mass Effect: Andromeda is serving up, you'll be pleased to know that there's a surprising alternative on... | Read more »
Leap to victory in Nexx Studios new plat...
You’re always a hop, skip, and a jump away from a fiery death in Temple Jump, a new platformer-cum-endless runner from Nexx Studio. It’s out now on both iOS and Android if you’re an adventurer seeking treasure in a crumbling, pixel-laden temple. | Read more »
Failbetter Games details changes coming...
Sunless Sea, Failbetter Games' dark and gloomy sea explorer, sets sail for the iPad tomorrow. Ahead of the game's launch, Failbetter took to Twitter to discuss what will be different in the mobile version of the game. Many of the changes make... | Read more »
Splish, splash! The Pokémon GO Water Fes...
Niantic is back with a new festival for dedicated Pokémon GO collectors. The Water Festival officially kicks off today at 1 P.M. PDT and runs through March 29. Magikarp, Squirtle, Totodile, and their assorted evolved forms will be appearing at... | Read more »
Death Road to Canada (Games)
Death Road to Canada 1.0 Device: iOS Universal Category: Games Price: $7.99, Version: 1.0 (iTunes) Description: Get it now at the low launch price! Price will go up a dollar every major update. Update news at the bottom of this... | Read more »
Bean's Quest Beginner's Guide:...
Bean's Quest is a new take on both the classic platformer and the endless runner, and it's free on the App Store for the time being. Instead of running constantly, you can't stop jumping. That adds a surprising new level of challenge to the game... | Read more »

Price Scanner via MacPrices.net

13-inch MacBook Airs, Apple refurbished, in s...
Apple has Certified Refurbished 2016 13″ MacBook Airs available starting at $849. An Apple one-year warranty is included with each MacBook, and shipping is free: - 13″ 1.6GHz/8GB/128GB MacBook Air: $... Read more
12-inch Retina MacBooks on sale for $1199, sa...
B&H has 12″ 1.1GHz Retina MacBooks on sale for $100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 12″ 1.1GHz Space Gray Retina MacBook: $1199 $100 off MSRP - 12″ 1.1GHz... Read more
Save up to $260 with Apple refurbished 12-inc...
Apple has Certified Refurbished 2016 12″ Retina MacBooks available for $200-$260 off MSRP. Apple will include a standard one-year warranty with each MacBook, and shipping is free. The following... Read more
13-inch 2.7GHz Retina MacBook Pro on sale for...
B&H Photo has the 2015 13″ 2.7GHz/128GB Retina Apple MacBook Pro on sale for $170 off MSRP. Shipping is free, and B&H charges NY tax only: - 13″ 2.7GHz/128GB Retina MacBook Pro (MF839LL/A): $... Read more
15-inch 2.2GHz Retina MacBook Pro on sale for...
B&H Photo has the 2015 15″ 2.2GHz Retina MacBook Pro (MJLQ2LL/A) on sale for $1799.99 including free shipping plus NY sales tax only. Their price is $200 off MSRP. Read more
Save up to $160 with Apple refurbished 9-inch...
Apple has Certified Refurbished 9″ and 12″ Apple iPad Pros available for up to $160 off the cost of new iPads. An Apple one-year warranty is included with each model, and shipping is free: - 32GB 9″... Read more
Apple Chip Foundry TSMC To Begin A11 System-o...
Digitimes’ Steve Shen is reporting today that according to the Chinese-language Economic Daily News (EDN), chipmaker and major Apple supplier foundery Taiwan Semiconductor Manufacturing Company (TSMC... Read more
MacX MediaTrans 3.5 iOS Data Transfer Spring...
MacXDVD Software has announced general availability of the latest MacX MedTrans 3.5, featuring a new user interface (UI). MacX MediaTrans is ann iPhone manager that enables free data transfer between... Read more
Regular Price $19.95 DupeZap 4 Finder For OS...
Hyperbolic Software has announced the release of DupeZap 4.0.2, their modern duplicate finder developed exclusively for macOS. DupeZap 4 is an utility for Mac owners seeking to reclaim disk space... Read more
B-Eng Releases SSD Health Check for MVNe for...
Fehraltorf, Switzerland based B-Eng has announced the release and immediate availability of SSD Health Check for MVNe for MacBook Pro, their app that delivers important data and insights for MVNe... Read more

Jobs Board

Fulltime aan de slag als shopmanager in een h...
Ben jij helemaal gek van Apple -producten en vind je het helemaal super om fulltime shopmanager te zijn in een jonge en hippe elektronicazaak? Wil jij werken in Read more
*Apple* Mobile Master - Best Buy (United Sta...
**492889BR** **Job Title:** Apple Mobile Master **Location Number:** 000886-Norwalk-Store **Job Description:** **What does a Best Buy Apple Mobile Master do?** Read more
*Apple* Mobile Master - Best Buy (United Sta...
**492472BR** **Job Title:** Apple Mobile Master **Location Number:** 000470-Seattle-Store **Job Description:** **What does a Best Buy Apple Mobile Master do?** Read more
*Apple* Mobile Master - Best Buy (United Sta...
**492562BR** **Job Title:** Apple Mobile Master **Location Number:** 000853-Jackson-Store **Job Description:** **What does a Best Buy Apple Mobile Master do?** Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.