TweetFollow Us on Twitter

Using File Manager From MP Tasks

Volume Number: 14 (1998)
Issue Number: 8
Column Tag: Toolbox Techniques

Using the File Manager from MP Tasks

by by Matthew Xavier Mora
Edited by Peter N Lewis

How to get data in and out of your MP Task

One of the most common complaints I received while supporting MP Library in Developer Technical Support was that you could not call the toolbox from an MP Task. Multiple preemptive tasks are not much use if you cannot get data into and out of them efficiently. This article shows one way to get data into and out of an MP Task using the file manager, however the techniques used here can be modified for other I/O operations (like audio, video or networking). "But, I thought you couldn't call the file manager from MP Tasks?" Well, you thought wrong. :-) Read on...

Background

In the early version of the MP library there was no easy way to call the toolbox because the MP Library was designed to be compatible with Copland's kernel tasking model. Since the Mac OS toolbox wasn't going to be available from Copland's kernel tasks, the same was done for the Mac OS version of the MP Library. After the Copland project was canceled it was decided to publish a few previously undocumented routines that let you work with the Mac OS toolbox from a task. One of the routines published is MPRPC. MPRPC is a remote procedure mechanism that lets you specify a routine to execute at a time when it is safe to make toolbox calls. It does this by suspending the task and then executing the supplied routine during SystemTask() time. The task is suspended until MPYield is called or until any toolbox routine calls SystemTask(). MPRPC is used internally in the MP Library to implement calls such as MPAllocate and MPAllocateSys (which is why these are blocking calls).

The code in this article is based on the MP File Library that I wrote before the MPRPC call was published. The MP File Library used MPQueues to communicate with the main task and have it execute toolbox commands.

Review

Let's review some of the MP programming guidelines and how adding blocking calls can change some of these guidelines.

  1. Your tasks should do a considerable amount of work. If not, the benefits of using MP will be lost in the overhead of the scheduler and task switching. Adding blocking calls to your tasks adds additional overhead. The main benefit here is that by being able to call the toolbox from an MP Task your task can run autonomously from the main application thread. This results in a better user interface response from the application since the application can off load a time consuming task and call the main event loop more often giving the blocking calls more time to execute the toolbox calls.
  2. You should allocate no more than (MPProcessors() - 1) number of tasks. While it is important to keep the number of tasks low so that task switching does not impact performance, adding blocking calls to a task will also hurt performance if nothing calls MPYield(). "Wait, I thought MP Tasks were preemptive?" Yes they are but if the task is blocked waiting on a resource, the resource can't be released until the main thread calls WaitNextEvent() or another task calls MPYield(). That being the case, if you use MPRPC calls it is a good idea to bend the n-1 rule and create an extra task that can help unblock any waiting tasks.
  3. You should use MPQueues or MPSemaphores when communicating with MP Tasks. This does not change if you are using MPRPC so you should heed this warning.

Get On With It

OK, so how do I call the File Manager? For this simple example I will implement five MP calls that duplicate FSpOpenDF, FSClose, FSRead, FSWrite and SetFPos. Those are all the calls we need for a simple demo. We'll start with a FSRead type call.

First lets define a structure to pass to the MPRPC callback routines that will hold the values that we need to handle all the File Manager calls.

typedef struct FSParamRec{
   short        refNum;        // file ref num
   long         count;         // for read
   Ptr          buffPtr;       // for read
   FSSpecPtr    spec;          // for open
   short        permission;    // for open
   short        posMode;       // for setfpos
   long         posOff;        // for setfpos   
   OSErr        result;        // error result
} FSParamRec,*FSParamRecPtr;

Now lets implement the callback routine that gets called at main application time. This routine will be executing at SystemTask time which means you can call any toolbox routine except for any routines that might call SystemTask() again.

static void * FSReadCallBack( void * parameter)
{
   FSParamRecPtr fsprp = (FSParamRecPtr)parameter;
   
   if (fsprp != nil) {
      fsprp->result = FSRead(fsprp->refNum,
                   &fsprp->count,
                   fsprp->buffPtr);
   }
   
   return fsprp;
}

First we check to make sure the parameter that was passed in is not nil then we simply call the File manager's FSRead call. When FSRead returns, we put the result into the result field and then return the pointer to the struct that was passed in.

All that is left is to do is to implement the new MyMPFSRead call.

pascal OSErr MyMPFSRead(short refNum,
                         long * count,
                         void * buffPtr)
{
   FSParamRec fsrr;   // make the record on the stack 
                      // no worries since it is a blocking call

   fsrr.refNum   = refNum;
   fsrr.count    = *count;
   fsrr.buffPtr  = buffPtr;
   fsrr.result   = paramErr;   //preset in case 
                               //anything goes wrong
   
   (void) _MPRPC(FSReadCallBack,&fsrr); 
   //ignore what is returned


   *count = fsrr.count;   //return the new count

   return fsrr.result;    //return the result
}

First we allocate a FSReadRec on the stack that gets passed to MPRPC. We fill out the fields in the struct with what was passed into us, call MPRPC and wait for the result. Then return the result to the caller.

That's it. You can now call FSRead from an MP Task. Using the same basic techniques you can implement all the file manager calls you need to get data in and out of your tasks. Now lets see how the task calls the new routines.

The MP Task itself is pretty straight forward as a result of the blocking I/O calls since there are no flags or spin loops to worry about.

static long MyMPTask(void * param)
{
   FSSpecPtr   fsp;
   Boolean     done = false;
   OSStatus    status;
   OSErr       err;
   MPQueueID   mpq = (MPQueueID) param;

   
   // don't start until we get the message
   status = MPWaitOnQueue(mpq,&fsp,nil,nil, kDurationForever);
   // the message is the file spec
   if (fsp) {   
      short          refNum;
      long          count = 1024; //read 1k of data
      
      err = MyMPFSpOpenDF(fsp,fsRdPerm,&refNum);
      if (!err) {
         err = MyMPSetFPos(refNum,fsFromStart,0);
         if (!err) {
   
#if qUseAsyncRead         
                err = MyMPFSReadAsync(refNum,&count,gBuffer);
#else
                err = MyMPFSRead(refNum,&count,gBuffer);
#endif            
            // we got some data. you could compress it
            // do FFT's on it or whatever.
            // In our case we just set the flag that we got
            // the data and tell the processors to sync up
            
            if (count > 0) {   
               gCount = count; // signal that we got some text   
               __eieio();      // sync processors
            } else {
               gCount = -1;    // signal that we got an error   
               __eieio();      // sync processors
            }
         } 
         err = MyMPFSClose(refNum);
      }
   }
}

In our task we immediately block (as every task should) on MPWaitOnQueue waiting for the FSSpecPtr from the application. When MPWaitOnQueue returns, we check the file spec pointer to make sure it is not nil and precedes to open the file. We set the file position to the beginning of the file and start the read operation. Notice that for either the async or non async case the code is still the same. The only difference is to the application since the task is blocked until the read completes. After the read completes, this is where you would do some serious processing on the data. It is very important that you do a lot of processing to minimize the overhead of the blocking I/O calls. The demo doesn't do any processing so the next thing to do is to set the gCount variable indicating we got the data making sure the write get synchronized with the other processors. We close the file and return. Returning from the task kills the task. You might want the task to hang around and be ready to process another file. In that case set up a while loop on MPWaitOnQueue. You can set the exit termination condition to be a nil FSSpecPtr.

Adding More Features

The FSRead technique is good at getting data in and out of your task but you basically block the entire application while it waits for the FSRead to complete. We can improve this by using asynchronous file manager calls to keep from blocking the main application task while executing a Read call.

We need a different structure to do an async read. I wrap the new struct around a ParamBlockRec to contain the flag needed to signal the completion of the read call.

typedef struct FSReadAsyncRec { ParamBlockRec pb; // standard paramblock Boolean callPending; // our pending flag } FSReadAsyncRec, *FSReadAsyncRecPtr;

The MyMPFSReadAsync code is a little more complicated but it saves having to have another task running just to call MPYield() since this routine spins on MPYield waiting for the PBRead to complete.

static pascal OSErr MyMPFSReadAsync(short refNum,
                               long * count,
                               void * buffPtr)
{
   FSReadAsyncRec       fsrar;      // make the rec on the stack 
   // Build a rountine descriptor by hand since we can't call
   // NewIOCompletionProc(userRoutine)
   RoutineDescriptor    ioCompProc = 
                  BUILD_ROUTINE_DESCRIPTOR(uppIOCompletionProcInfo,
                                          MyReadCompletion);

   ClearBlock(&fsrar,sizeof(fsrar)); 
   
   fsrar.pb.ioParam.ioRefNum     =   refNum;
   fsrar.pb.ioParam.ioReqCount   =   *count;
   fsrar.pb.ioParam.ioBuffer     =   buffPtr;
   fsrar.pb.ioParam.ioCompletion =   &ioCompProc;
   fsrar.callPending             = true;
   __eieio();                  //ensure that callPending gets set
                               //before we call MPRPC
   
   (void) _MPRPC(FSReadAsyncCallBack,&fsrar); //ignore what is 

   // spin waiting for flag to be set in completionRoutine

   while ( fsrar.callPending ) { //Spin waiting for completion
      MPYield();
   }

   *count = fsrar.pb.ioParam.ioActCount;   
                  //return the new count

   return fsrar.pb.ioParam.ioResult;   //return the result
}

MyMPFSReadAsync sets up the parameter block, builds a completion routine descriptor on the fly, calls MPRPC and then spins in a tight loop calling MPYield until the callPending flag is cleared.

The FSReadAsyncCallBack routine is very simple.

static void * FSReadAsyncCallBack( void * parameter)
{
   FSReadAsyncRecPtr fsr = (FSReadAsyncRecPtr) parameter;
   OSErr err;
   
   if (fsr != nil) {   
      err = PBReadAsync((ParmBlkPtr)fsr);   
                           //just call PBRead and return
   }                      // completion routine sets the flag   
   return fsr;
}

FSReadAsyncCallBack just calls PBReadAsync and returns. Below is the completion routine that tells the task the read has completed.

static void MyReadCompletion(ParmBlkPtr pb)
{
   FSReadAsyncRecPtr fs = (FSReadAsyncRecPtr)pb; 
   
   fs->callPending = false;  // set flag
   __eieio();                   // make sure it sticks
}

It just sets the callPending flag, signals the processors to sync up and returns. We can't set a MPQueue or a MPSemapore in here (which would be the better way to do it) because MP Library calls can't be called at interrupt time.

Handling asynchronous routines gets a little more complicated but it saves having to make sure other tasks are running just to call MPYield(). Now you might be thinking why are we using a flag when you could just spin on ioResult? Read on to see why this is not good idea...

Gotchas

When working with multiple processors some conventional Mac programming wisdom goes out the window. A good case in point is when ioResult is set. Normally ioResult is set to 1 to indicate a call is pending. The last thing the file manager does before calling the ioCompletion routine is to set ioResult to the error result from the parameter block call. None of this really changes when multiple processors are involved but the non-main processors are not bound by the 68k enable/disable interrupt tricks. So if your MP Task spins on ioResult waiting to see when the read is complete (ioResult != 1) your task starts to execute before the file manager is done with the parameter block. After the file manager sets the ioResult field, it gets the ioCompletion routine's address from the parameter block and jumps to it.

In our case the parameter block in on the stack and when the task unblocks, the stack is released and your task crunches merrily along where a parameter block used to be (and is still in use by the file manager). The second processor could be a 200 MHz CPU and in the time the file manager has set ioResult and jumps to the completion routine, your task could be millions of instructions away using the memory where the parameter block used to be.

The same is true for many of parts of the Mac OS Toolbox. The critical region technique of disabling interrupts does not work well when multiple processors are involved. So be careful and always use MPQueues, MPSemaphores and MPCriticalRegions to coordinate your various tasks.

Another gotcha may be in your thought process. You might be thinking that it would be cool to use the same techniques mentioned in the article to make every Toolbox call available from MP tasks. While this is possible, and would make your task code a lot easier to write, it is not a good idea. The benefits of multiprocessing only come from careful algorithm design, implementation, and profiling. Guideline #1 mentioned above says that your task should do a considerable amount of work to gain any performance improvements. Having your task block, waiting on a bunch of toolbox calls is not going to improve performance. On the other hand having to load all the data you need into memory before your task can start running may not be feasible either. This is where a careful balance of having main processor moving data in and out of your task while processors n+1 crunch along can really pay off.

More MP Information

Hopefully, this article piqued your interest in Multiprocessing. If you want more information there are a number of documents and resources to help you get the most out of MP. An introduction to MP systems was printed in MacTech March '96, TechNote 1071 on Multiprocessing is on the web http://www.apple.com/developer/ and I have set up a MP mailing list where developers can ask questions on MP programming issues. The list includes folks like the senior engineer who wrote the MP Library as well as Chris Cooksey and myself. For subscription information you can go to my web site http://www.best.com/~mxmora/mxm.html. Also, don't forget Apple Developer Technical Support is there for information about MP's past, present and future.

Summary

I hope this article shows how easy it is to get data into and out of your MP tasks. Use this information wisely and you should see some real improvements in your applications performance. You can use these techniques to work with other I/O technologies like networking, graphics and sound. I have created a MP File Library that you may want to use based on some of the techniques used in this article. It uses a slightly more complicated model for better performance. You can download a copy of my MP File Library from my web site at http://www.best.com/~mxmora/software.html. Good luck, and happy multiprocessing.


Matthew Xavier Mora was the engineer responsible for answering questions on Multitasking support in Apple's Developer Technical Support. As a self proclaimed evangelist for the Multi-processing API library he was instrumental in convincing both third-party developers and Apple engineers to implement MP support in their software. If you were ever thinking about moving into the Silicon Valley, consider that this article was written while Matt was sitting all night outside a school building waiting to register his son for pre-school. When Matt is not out doing crazy things like that you can reach him at mxmora@best.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

How to get a high score in every level o...
Sky Charms is an adorable match three puzzler that provides a decent challenge thanks to its creative level design. It regularly presents something new, forcing you to think on your feet. [Read more] | Read more »
Apestorm: Full Bananas (Games)
Apestorm: Full Bananas 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: ***Launch sale – limited time only!*** Fugitive Apes have taken to the skies in search of revenge after humans have... | Read more »
How to create bigger words in Spellspire
Words have power. At least they do in Spellspire,a game about blasting out magical attacks by making words out of a jumble of letters. And it's a lot of fun. But if you want to be the best, you're going to have to think tactically when you start... | Read more »
Steel Media and DeePoon have partnered f...
Virtual reality is the next big thing, and 148Apps's publisher,Steel Media, wants to know what the hottest upcoming games are. [Read more] | Read more »
Airline Director 2 - Tycoon Game (Games...
Airline Director 2 - Tycoon Game 1.2.1 Device: iOS Universal Category: Games Price: $2.99, Version: 1.2.1 (iTunes) Description: Airline Director 2 is a management game set in the challenging field of commercial aviation. As the... | Read more »
Dog Mendonca (Games)
Dog Mendonca 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: [ Solve a criminal case beyond believe in this supernatural adventure game based on the popular graphic novel trilogy published... | Read more »
Amidakuji Knight (Games)
Amidakuji Knight 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: Ghost leg rules meets RPG!Select the best path and save a princess! A long long time ago, there was a beautiful and peaceful... | Read more »
The 5 best mobile games like Game of Thr...
Everyone's favourite weekly dosage of medieval depression, Game of Thrones, is back for its sixth season. An excellent time for the bloodthirsty! [Read more] | Read more »
How to approach quests in Fallen London
Sitting at over 1.5 million words, Fallen London is filled to the brim with intriguing tales and mysterious characters. From the start, you’ll find a slew of quests ripe for the taking. [Read more] | Read more »
How to survive in LOUD on Planet X
LOUD on Planet X is a hybrid of a tower defense and rhythm game that pits famous indie acts against invading aliens. You need timing and strategy in this game in order to succeed, things can get pretty overwhelming pretty quickly. Here are some... | Read more »

Price Scanner via MacPrices.net

Aleratec Releases Mac Software Upgrade for 1...
California based Aleratec Inc., designer, developer and manufacturer of Portable Device Management (PDM) charge/sync products for mobile devices and professional-grade duplicators for hard disk... Read more
Sale! Amazon offers 27-inch iMac, 13-inch 2.9...
Amazon has the 27″ 3.2GHz 5K iMac and the 13″ 3.9GHz Retina MacBook Pro on sale for $300 off MSRP, each including free shipping, for a limited time: - 27″ 3.2GHz/1TB HD 5K iMac (model MK462LL/A): $... Read more
Apple refurbished 13-inch Retina MacBook Pros...
Apple has Certified Refurbished 13″ Retina MacBook Pros available for up to $270 off the cost of new models. An Apple one-year warranty is included with each model, and shipping is free: - 13″ 2.7GHz... Read more
13-inch 2.7GHz/128GB Retina MacBook Pro on sa...
Take $200 off MSRP on the price of a new 13″ 2.7GHz/128GB Retina MacBook Pro (model MF839LL/A) at Amazon. Shipping is free: - 13″ 2.7GHz/128GB Retina MacBook Pro: $1099.99 $200 off MSRP Act now if... Read more
Apple refurbished clearance 15-inch Retina Ma...
Apple has Certified Refurbished 2014 15″ 2.2GHz Retina MacBook Pros available for $1609, $390 off original MSRP. Apple’s one-year warranty is included, and shipping is free. They have refurbished 15... Read more
27-inch 5K iMacs on sale for up to $150 off M...
B&H Photo has 27″ 5K iMacs on sale for up to $150 off MSRP including free shipping plus NY sales tax only: - 27″ 3.3GHz iMac 5K: $2199 $100 off MSRP - 27″ 3.2GHz/1TB Fusion iMac 5K: $1849.99 $150... Read more
What Does The Refreshed 12-Inch MacBook Tell...
A lot of commentators are complaining that Apple’s update of the 12-Inch MacBook last week is a bit of a damp squib. I don’t know what they were expecting, since it would be very unlike Apple to do a... Read more
Free Wittify Keyboard Now Available On The Ap...
A team of Harvard Business School students have announced that the Wittify Keyboard, a new app utility for iOS devices, is now available on the Apple App Store. The Wittify keyboard and application... Read more
Apple Reports First Year-Over-Year Quarterly...
Apple on TUesday announced financial results for its fiscal 2016 second quarter ending March 26, 2016. The Company posted quarterly revenue of $50.6 billion and quarterly net income of $10.5 billion... Read more
13-inch 2.7GHz Retina MacBook Pros on sale fo...
Take $130-$150 off MSRP on the price of a new 13″ 2.7GHz Retina MacBook Pro at Amazon. Shipping is free: - 13″ 2.7GHz/128GB Retina MacBook Pro: $1169 $130 off MSRP - 13″ 2.7GHz/256GB Retina MacBook... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Restaurant Manager (Neighborhood Captain) - A...
…in every aspect of daily operation. WHY YOU'LL LIKE IT: You'll be the Big Apple . You'll solve problems. You'll get to show your ability to handle the stress and Read more
*Apple* Solutions Consultant - … (United Sta...
Job Summary As an Apple Solutions Consultant, you'll be the link between our future customers and our products. You'll showcase your entrepreneurial spirit as you Read more
Restaurant Manager (Neighborhood Captain) - A...
…in every aspect of daily operation. WHY YOU'LL LIKE IT: You'll be the Big Apple . You'll solve problems. You'll get to show your ability to handle the stress and Read more
Restaurant Manager (Neighborhood Captain) - A...
…in every aspect of daily operation. WHY YOU'LL LIKE IT: You'll be the Big Apple . You'll solve problems. You'll get to show your ability to handle the stress and Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.