TweetFollow Us on Twitter

Using File Manager From MP Tasks

Volume Number: 14 (1998)
Issue Number: 8
Column Tag: Toolbox Techniques

Using the File Manager from MP Tasks

by by Matthew Xavier Mora
Edited by Peter N Lewis

How to get data in and out of your MP Task

One of the most common complaints I received while supporting MP Library in Developer Technical Support was that you could not call the toolbox from an MP Task. Multiple preemptive tasks are not much use if you cannot get data into and out of them efficiently. This article shows one way to get data into and out of an MP Task using the file manager, however the techniques used here can be modified for other I/O operations (like audio, video or networking). "But, I thought you couldn't call the file manager from MP Tasks?" Well, you thought wrong. :-) Read on...

Background

In the early version of the MP library there was no easy way to call the toolbox because the MP Library was designed to be compatible with Copland's kernel tasking model. Since the Mac OS toolbox wasn't going to be available from Copland's kernel tasks, the same was done for the Mac OS version of the MP Library. After the Copland project was canceled it was decided to publish a few previously undocumented routines that let you work with the Mac OS toolbox from a task. One of the routines published is MPRPC. MPRPC is a remote procedure mechanism that lets you specify a routine to execute at a time when it is safe to make toolbox calls. It does this by suspending the task and then executing the supplied routine during SystemTask() time. The task is suspended until MPYield is called or until any toolbox routine calls SystemTask(). MPRPC is used internally in the MP Library to implement calls such as MPAllocate and MPAllocateSys (which is why these are blocking calls).

The code in this article is based on the MP File Library that I wrote before the MPRPC call was published. The MP File Library used MPQueues to communicate with the main task and have it execute toolbox commands.

Review

Let's review some of the MP programming guidelines and how adding blocking calls can change some of these guidelines.

  1. Your tasks should do a considerable amount of work. If not, the benefits of using MP will be lost in the overhead of the scheduler and task switching. Adding blocking calls to your tasks adds additional overhead. The main benefit here is that by being able to call the toolbox from an MP Task your task can run autonomously from the main application thread. This results in a better user interface response from the application since the application can off load a time consuming task and call the main event loop more often giving the blocking calls more time to execute the toolbox calls.
  2. You should allocate no more than (MPProcessors() - 1) number of tasks. While it is important to keep the number of tasks low so that task switching does not impact performance, adding blocking calls to a task will also hurt performance if nothing calls MPYield(). "Wait, I thought MP Tasks were preemptive?" Yes they are but if the task is blocked waiting on a resource, the resource can't be released until the main thread calls WaitNextEvent() or another task calls MPYield(). That being the case, if you use MPRPC calls it is a good idea to bend the n-1 rule and create an extra task that can help unblock any waiting tasks.
  3. You should use MPQueues or MPSemaphores when communicating with MP Tasks. This does not change if you are using MPRPC so you should heed this warning.

Get On With It

OK, so how do I call the File Manager? For this simple example I will implement five MP calls that duplicate FSpOpenDF, FSClose, FSRead, FSWrite and SetFPos. Those are all the calls we need for a simple demo. We'll start with a FSRead type call.

First lets define a structure to pass to the MPRPC callback routines that will hold the values that we need to handle all the File Manager calls.

typedef struct FSParamRec{
   short        refNum;        // file ref num
   long         count;         // for read
   Ptr          buffPtr;       // for read
   FSSpecPtr    spec;          // for open
   short        permission;    // for open
   short        posMode;       // for setfpos
   long         posOff;        // for setfpos   
   OSErr        result;        // error result
} FSParamRec,*FSParamRecPtr;

Now lets implement the callback routine that gets called at main application time. This routine will be executing at SystemTask time which means you can call any toolbox routine except for any routines that might call SystemTask() again.

static void * FSReadCallBack( void * parameter)
{
   FSParamRecPtr fsprp = (FSParamRecPtr)parameter;
   
   if (fsprp != nil) {
      fsprp->result = FSRead(fsprp->refNum,
                   &fsprp->count,
                   fsprp->buffPtr);
   }
   
   return fsprp;
}

First we check to make sure the parameter that was passed in is not nil then we simply call the File manager's FSRead call. When FSRead returns, we put the result into the result field and then return the pointer to the struct that was passed in.

All that is left is to do is to implement the new MyMPFSRead call.

pascal OSErr MyMPFSRead(short refNum,
                         long * count,
                         void * buffPtr)
{
   FSParamRec fsrr;   // make the record on the stack 
                      // no worries since it is a blocking call

   fsrr.refNum   = refNum;
   fsrr.count    = *count;
   fsrr.buffPtr  = buffPtr;
   fsrr.result   = paramErr;   //preset in case 
                               //anything goes wrong
   
   (void) _MPRPC(FSReadCallBack,&fsrr); 
   //ignore what is returned


   *count = fsrr.count;   //return the new count

   return fsrr.result;    //return the result
}

First we allocate a FSReadRec on the stack that gets passed to MPRPC. We fill out the fields in the struct with what was passed into us, call MPRPC and wait for the result. Then return the result to the caller.

That's it. You can now call FSRead from an MP Task. Using the same basic techniques you can implement all the file manager calls you need to get data in and out of your tasks. Now lets see how the task calls the new routines.

The MP Task itself is pretty straight forward as a result of the blocking I/O calls since there are no flags or spin loops to worry about.

static long MyMPTask(void * param)
{
   FSSpecPtr   fsp;
   Boolean     done = false;
   OSStatus    status;
   OSErr       err;
   MPQueueID   mpq = (MPQueueID) param;

   
   // don't start until we get the message
   status = MPWaitOnQueue(mpq,&fsp,nil,nil, kDurationForever);
   // the message is the file spec
   if (fsp) {   
      short          refNum;
      long          count = 1024; //read 1k of data
      
      err = MyMPFSpOpenDF(fsp,fsRdPerm,&refNum);
      if (!err) {
         err = MyMPSetFPos(refNum,fsFromStart,0);
         if (!err) {
   
#if qUseAsyncRead         
                err = MyMPFSReadAsync(refNum,&count,gBuffer);
#else
                err = MyMPFSRead(refNum,&count,gBuffer);
#endif            
            // we got some data. you could compress it
            // do FFT's on it or whatever.
            // In our case we just set the flag that we got
            // the data and tell the processors to sync up
            
            if (count > 0) {   
               gCount = count; // signal that we got some text   
               __eieio();      // sync processors
            } else {
               gCount = -1;    // signal that we got an error   
               __eieio();      // sync processors
            }
         } 
         err = MyMPFSClose(refNum);
      }
   }
}

In our task we immediately block (as every task should) on MPWaitOnQueue waiting for the FSSpecPtr from the application. When MPWaitOnQueue returns, we check the file spec pointer to make sure it is not nil and precedes to open the file. We set the file position to the beginning of the file and start the read operation. Notice that for either the async or non async case the code is still the same. The only difference is to the application since the task is blocked until the read completes. After the read completes, this is where you would do some serious processing on the data. It is very important that you do a lot of processing to minimize the overhead of the blocking I/O calls. The demo doesn't do any processing so the next thing to do is to set the gCount variable indicating we got the data making sure the write get synchronized with the other processors. We close the file and return. Returning from the task kills the task. You might want the task to hang around and be ready to process another file. In that case set up a while loop on MPWaitOnQueue. You can set the exit termination condition to be a nil FSSpecPtr.

Adding More Features

The FSRead technique is good at getting data in and out of your task but you basically block the entire application while it waits for the FSRead to complete. We can improve this by using asynchronous file manager calls to keep from blocking the main application task while executing a Read call.

We need a different structure to do an async read. I wrap the new struct around a ParamBlockRec to contain the flag needed to signal the completion of the read call.

typedef struct FSReadAsyncRec { ParamBlockRec pb; // standard paramblock Boolean callPending; // our pending flag } FSReadAsyncRec, *FSReadAsyncRecPtr;

The MyMPFSReadAsync code is a little more complicated but it saves having to have another task running just to call MPYield() since this routine spins on MPYield waiting for the PBRead to complete.

static pascal OSErr MyMPFSReadAsync(short refNum,
                               long * count,
                               void * buffPtr)
{
   FSReadAsyncRec       fsrar;      // make the rec on the stack 
   // Build a rountine descriptor by hand since we can't call
   // NewIOCompletionProc(userRoutine)
   RoutineDescriptor    ioCompProc = 
                  BUILD_ROUTINE_DESCRIPTOR(uppIOCompletionProcInfo,
                                          MyReadCompletion);

   ClearBlock(&fsrar,sizeof(fsrar)); 
   
   fsrar.pb.ioParam.ioRefNum     =   refNum;
   fsrar.pb.ioParam.ioReqCount   =   *count;
   fsrar.pb.ioParam.ioBuffer     =   buffPtr;
   fsrar.pb.ioParam.ioCompletion =   &ioCompProc;
   fsrar.callPending             = true;
   __eieio();                  //ensure that callPending gets set
                               //before we call MPRPC
   
   (void) _MPRPC(FSReadAsyncCallBack,&fsrar); //ignore what is 

   // spin waiting for flag to be set in completionRoutine

   while ( fsrar.callPending ) { //Spin waiting for completion
      MPYield();
   }

   *count = fsrar.pb.ioParam.ioActCount;   
                  //return the new count

   return fsrar.pb.ioParam.ioResult;   //return the result
}

MyMPFSReadAsync sets up the parameter block, builds a completion routine descriptor on the fly, calls MPRPC and then spins in a tight loop calling MPYield until the callPending flag is cleared.

The FSReadAsyncCallBack routine is very simple.

static void * FSReadAsyncCallBack( void * parameter)
{
   FSReadAsyncRecPtr fsr = (FSReadAsyncRecPtr) parameter;
   OSErr err;
   
   if (fsr != nil) {   
      err = PBReadAsync((ParmBlkPtr)fsr);   
                           //just call PBRead and return
   }                      // completion routine sets the flag   
   return fsr;
}

FSReadAsyncCallBack just calls PBReadAsync and returns. Below is the completion routine that tells the task the read has completed.

static void MyReadCompletion(ParmBlkPtr pb)
{
   FSReadAsyncRecPtr fs = (FSReadAsyncRecPtr)pb; 
   
   fs->callPending = false;  // set flag
   __eieio();                   // make sure it sticks
}

It just sets the callPending flag, signals the processors to sync up and returns. We can't set a MPQueue or a MPSemapore in here (which would be the better way to do it) because MP Library calls can't be called at interrupt time.

Handling asynchronous routines gets a little more complicated but it saves having to make sure other tasks are running just to call MPYield(). Now you might be thinking why are we using a flag when you could just spin on ioResult? Read on to see why this is not good idea...

Gotchas

When working with multiple processors some conventional Mac programming wisdom goes out the window. A good case in point is when ioResult is set. Normally ioResult is set to 1 to indicate a call is pending. The last thing the file manager does before calling the ioCompletion routine is to set ioResult to the error result from the parameter block call. None of this really changes when multiple processors are involved but the non-main processors are not bound by the 68k enable/disable interrupt tricks. So if your MP Task spins on ioResult waiting to see when the read is complete (ioResult != 1) your task starts to execute before the file manager is done with the parameter block. After the file manager sets the ioResult field, it gets the ioCompletion routine's address from the parameter block and jumps to it.

In our case the parameter block in on the stack and when the task unblocks, the stack is released and your task crunches merrily along where a parameter block used to be (and is still in use by the file manager). The second processor could be a 200 MHz CPU and in the time the file manager has set ioResult and jumps to the completion routine, your task could be millions of instructions away using the memory where the parameter block used to be.

The same is true for many of parts of the Mac OS Toolbox. The critical region technique of disabling interrupts does not work well when multiple processors are involved. So be careful and always use MPQueues, MPSemaphores and MPCriticalRegions to coordinate your various tasks.

Another gotcha may be in your thought process. You might be thinking that it would be cool to use the same techniques mentioned in the article to make every Toolbox call available from MP tasks. While this is possible, and would make your task code a lot easier to write, it is not a good idea. The benefits of multiprocessing only come from careful algorithm design, implementation, and profiling. Guideline #1 mentioned above says that your task should do a considerable amount of work to gain any performance improvements. Having your task block, waiting on a bunch of toolbox calls is not going to improve performance. On the other hand having to load all the data you need into memory before your task can start running may not be feasible either. This is where a careful balance of having main processor moving data in and out of your task while processors n+1 crunch along can really pay off.

More MP Information

Hopefully, this article piqued your interest in Multiprocessing. If you want more information there are a number of documents and resources to help you get the most out of MP. An introduction to MP systems was printed in MacTech March '96, TechNote 1071 on Multiprocessing is on the web http://www.apple.com/developer/ and I have set up a MP mailing list where developers can ask questions on MP programming issues. The list includes folks like the senior engineer who wrote the MP Library as well as Chris Cooksey and myself. For subscription information you can go to my web site http://www.best.com/~mxmora/mxm.html. Also, don't forget Apple Developer Technical Support is there for information about MP's past, present and future.

Summary

I hope this article shows how easy it is to get data into and out of your MP tasks. Use this information wisely and you should see some real improvements in your applications performance. You can use these techniques to work with other I/O technologies like networking, graphics and sound. I have created a MP File Library that you may want to use based on some of the techniques used in this article. It uses a slightly more complicated model for better performance. You can download a copy of my MP File Library from my web site at http://www.best.com/~mxmora/software.html. Good luck, and happy multiprocessing.


Matthew Xavier Mora was the engineer responsible for answering questions on Multitasking support in Apple's Developer Technical Support. As a self proclaimed evangelist for the Multi-processing API library he was instrumental in convincing both third-party developers and Apple engineers to implement MP support in their software. If you were ever thinking about moving into the Silicon Valley, consider that this article was written while Matt was sitting all night outside a school building waiting to register his son for pre-school. When Matt is not out doing crazy things like that you can reach him at mxmora@best.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

TinkerTool 5.4 - Expanded preference set...
TinkerTool is an application that gives you access to additional preference settings Apple has built into Mac OS X. This allows to activate hidden features in the operating system and in some of the... Read more
Tinderbox 6.3.1 - Store and organize you...
Tinderbox is a personal content management assistant. It stores your notes, ideas, and plans. It can help you organize and understand them. And Tinderbox helps you share ideas through Web journals... Read more
Parallels Desktop 10.2.2 - Run Windows a...
Parallels Desktop is simply the world's bestselling, top-rated, and most trusted solution for running Windows applications on your Mac. With Parallels Desktop for Mac, you can seamlessly run both... Read more
Adobe Premiere Pro CC 2015 9.0.1 - Digit...
Premiere Pro CC 2015 is available as part of Adobe Creative Cloud for as little as $19.99/month (or $9.99/month if you're a previous Premiere Pro customer). Premiere Pro CS6 is still available for... Read more
Adobe After Effects CC 2015 13.5.1 - Cre...
After Effects CC 2015 is available as part of Adobe Creative Cloud for as little as $19.99/month (or $9.99/month if you're a previous After Effects customer). After Effects CS6 is still available... Read more
Adobe Creative Cloud 2.2.0.129 - Access...
Adobe Creative Cloud costs $49.99/month (or less if you're a previous Creative Suite customer). Creative Suite 6 is still available for purchase (without a monthly plan) if you prefer. Introducing... Read more
Tower 2.2.3 - Version control with Git m...
Tower is a powerful Git client for OS X that makes using Git easy and more efficient. Users benefit from its elegant and comprehensive interface and a feature set that lets them enjoy the full power... Read more
Apple Java 2015-001 - For OS X 10.7, 10....
Apple Java for OS X 2015-001 installs the legacy Java 6 runtime for OS X 10.11 El Capitan, OS X 10.10 Yosemite, OS X 10.9 Mavericks, OS X 10.8 Mountain Lion, and OS X 10.7 Lion. This package is... Read more
Adobe Muse CC 2015 2015.0.1 - Design and...
Muse CC 2015 is available as part of Adobe Creative Cloud for as little as $14.99/month (or $9.99/month if you're a previous Muse customer). Muse CS6 is still available for purchase (without a... Read more
Adobe Illustrator CC 2015 19.1.0 - Profe...
Illustrator CC 2015 is available as part of Adobe Creative Cloud for as little as $19.99/month (or $9.99/month if you're a previous Illustrator customer). Illustrator CS6 is still available for... Read more

Tap Delay (Music)
Tap Delay 1.0.0 Device: iOS Universal Category: Music Price: $4.99, Version: 1.0.0 (iTunes) Description: Back in the “old days”, producers and engineers created delay and echo effects using tape machines. Tap Delay combines the warm... | Read more »
This Week at 148Apps: July 20-24, 2015
July is Heating Up With 148Apps How do you know what apps are worth your time and money? Just look to the review team at 148Apps. We sort through the chaos and find the apps you're looking for. The ones we love become Editor’s Choice, standing out... | Read more »
Red Game Without A Great Name (Games)
Red Game Without A Great Name 1.0.3 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0.3 (iTunes) Description: The mechanical bird is flying through an unfriendly, Steampunk world. Help it avoid obstacles and deadly... | Read more »
Warhammer: Arcane Magic (Games)
Warhammer: Arcane Magic 1.0.2 Device: iOS Universal Category: Games Price: $9.99, Version: 1.0.2 (iTunes) Description: Engage in epic battles and tactical gameplay that challenge both novice and veteran in Warhammer: Arcane Magic, a... | Read more »
Mazes of Karradash (Games)
Mazes of Karradash 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: The city of Karradash is under attack: the monsters of the Shadow Realms are emerging from the depths.No adventurer is... | Read more »
Battle Golf is the Newest Game from the...
Wrassling was a pretty weird - and equally great - little wressling game. Now the developers, Folmer Kelly and Colin Lane, have turned their attention to a different sport: golfing. This is gonna be weird. [Read more] | Read more »
Qbert Rebooted has the App Store Going...
The weird little orange... whatever... is back, mostly thanks to that movie which shall remain nameless (you know the one). But anyway it's been "rebooted" and now you can play the fancy-looking Qbert Rebooted on iOS devices. [Read more] | Read more »
Giant Monsters Run Amok in The Sandbox...
So The Sandbox has just hit version number 1.99987 (seriously), and it's added a lot more stuff. Just like every other update, really. [Read more] | Read more »
Fish Pond Park (Games)
Fish Pond Park 1.0.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0.0 (iTunes) Description: Nurture an idyllic slice of tourist's heaven into the top nature spot of the nation, furnishing it with a variety of... | Read more »
Look after Baby Buddy on your Apple Watc...
Parigami Gold is the new premium version of the match three puzzler that includes Apple Watch support and all new content. You won't simply be sliding tiles around on your wrist, the Apple Watch companion app is an all new mini-game in itself. You'... | Read more »

Price Scanner via MacPrices.net

2.8GHz Mac mini available for $988, includes...
Adorama has the 2.8GHz Mac mini available for $988, $11 off MSRP, including a free copy of Apple’s 3-Year AppleCare Protection Plan. Shipping is free, and Adorama charges sales tax in NY & NJ... Read more
Sale! 13″ 1.6GHz/256GB MacBook Air for $1099,...
B&H Photo has the 13″ 1.6GHz/256GB MacBook Air on sale for $1099 including free shipping plus NY tax only. Their price is $100 off MSRP, and it’s the lowest price available for this model. Read more
iPad mini 4 To Be Upgraded To iPad Air 2 Spec...
There’s a certain inevitability about making Apple product predictions this time of year. Come September, we can pretty reliably count on the release of refreshed iPhones, along with the iOS 9... Read more
Apple restocks refurbished Mac minis for up t...
The Apple Store has restocked Apple Certified Refurbished 2014 Mac minis, with models available starting at $419. Apple’s one-year warranty is included with each mini, and shipping is free: - 1.4GHz... Read more
13-inch 2.5GHz MacBook Pro on sale for $899,...
Best Buy has the 13″ 2.5GHz MacBook Pro available for $899.99 on their online store. Choose free shipping or free instant local store pickup (if available). Their price is $200 off MSRP. Price is... Read more
21-inch 2.9GHz iMac on sale for $1299, save $...
Best Buy has the 21″ 2.9GHz iMac on sale today for $1299.99 on their online store. Choose free shipping or free local store pickup (if available). Their price is $200 off MSRP, and it’s the lowest... Read more
Free Image Sizer 1.3 for iOS Offers Photo Edi...
Xi’An, China based G-Power has announced the release of Image Sizer 1.3 for the iPhone, iPad, and iPod touch, an important update to their free photo editing app. Image Sizer’s collection of easy to... Read more
Sale! 13″ 1.6GHz/128GB MacBook Air for $899,...
B&H Photo has the 13″ 1.6GHz/128GB MacBook Air on sale for $899 including free shipping plus NY tax only. Their price is $100 off MSRP, and it’s the lowest price available for this model. Read more
13-inch Retina MacBook Pros on sale for $100...
Best Buy has 13-inch Retina MacBook Pros on sale for $100 off MSRP on their online store. Choose free shipping or free local store pickup (if available). Prices are for online orders only, in-store... Read more
Will BMW’s i3 Electric Vehicle Be The Automo...
The German-language business journal Manager Magazin’s Michael Freitag reports that Apple and the German performance/luxury automaker Bayerishe Motoren Werke (BMW) are back at far-reaching... Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - A...
Job Description: Sales. Specialist - Retail Customer Service and Sales. Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Customer Experience (ACE) Leader - A...
…management to deliver on business objectives Training partner store staff on Apple products, services, and merchandising guidelines Coaching partner store staff on Read more
Project Manager - *Apple* Pay Security - Ap...
**Job Summary** The Apple Pay Security team is seeking a highly organized, results-driven Project Manager to drive the development of Apple Pay Security. If you are Read more
*Apple* TV Product Design Internship (Spring...
…the mechanical design effort associated with creating world-class products with the Apple TV PD Group. Responsibilities will include working closely with manufacturing, Read more
*Apple* Watch SW Application Project Manager...
**Job Summary** The Apple Watch software team is looking for an Application Engineering Project Manager to work on new projects for Apple . The successful candidate Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.