TweetFollow Us on Twitter

Using File Manager From MP Tasks

Volume Number: 14 (1998)
Issue Number: 8
Column Tag: Toolbox Techniques

Using the File Manager from MP Tasks

by by Matthew Xavier Mora
Edited by Peter N Lewis

How to get data in and out of your MP Task

One of the most common complaints I received while supporting MP Library in Developer Technical Support was that you could not call the toolbox from an MP Task. Multiple preemptive tasks are not much use if you cannot get data into and out of them efficiently. This article shows one way to get data into and out of an MP Task using the file manager, however the techniques used here can be modified for other I/O operations (like audio, video or networking). "But, I thought you couldn't call the file manager from MP Tasks?" Well, you thought wrong. :-) Read on...

Background

In the early version of the MP library there was no easy way to call the toolbox because the MP Library was designed to be compatible with Copland's kernel tasking model. Since the Mac OS toolbox wasn't going to be available from Copland's kernel tasks, the same was done for the Mac OS version of the MP Library. After the Copland project was canceled it was decided to publish a few previously undocumented routines that let you work with the Mac OS toolbox from a task. One of the routines published is MPRPC. MPRPC is a remote procedure mechanism that lets you specify a routine to execute at a time when it is safe to make toolbox calls. It does this by suspending the task and then executing the supplied routine during SystemTask() time. The task is suspended until MPYield is called or until any toolbox routine calls SystemTask(). MPRPC is used internally in the MP Library to implement calls such as MPAllocate and MPAllocateSys (which is why these are blocking calls).

The code in this article is based on the MP File Library that I wrote before the MPRPC call was published. The MP File Library used MPQueues to communicate with the main task and have it execute toolbox commands.

Review

Let's review some of the MP programming guidelines and how adding blocking calls can change some of these guidelines.

  1. Your tasks should do a considerable amount of work. If not, the benefits of using MP will be lost in the overhead of the scheduler and task switching. Adding blocking calls to your tasks adds additional overhead. The main benefit here is that by being able to call the toolbox from an MP Task your task can run autonomously from the main application thread. This results in a better user interface response from the application since the application can off load a time consuming task and call the main event loop more often giving the blocking calls more time to execute the toolbox calls.
  2. You should allocate no more than (MPProcessors() - 1) number of tasks. While it is important to keep the number of tasks low so that task switching does not impact performance, adding blocking calls to a task will also hurt performance if nothing calls MPYield(). "Wait, I thought MP Tasks were preemptive?" Yes they are but if the task is blocked waiting on a resource, the resource can't be released until the main thread calls WaitNextEvent() or another task calls MPYield(). That being the case, if you use MPRPC calls it is a good idea to bend the n-1 rule and create an extra task that can help unblock any waiting tasks.
  3. You should use MPQueues or MPSemaphores when communicating with MP Tasks. This does not change if you are using MPRPC so you should heed this warning.

Get On With It

OK, so how do I call the File Manager? For this simple example I will implement five MP calls that duplicate FSpOpenDF, FSClose, FSRead, FSWrite and SetFPos. Those are all the calls we need for a simple demo. We'll start with a FSRead type call.

First lets define a structure to pass to the MPRPC callback routines that will hold the values that we need to handle all the File Manager calls.

typedef struct FSParamRec{
   short        refNum;        // file ref num
   long         count;         // for read
   Ptr          buffPtr;       // for read
   FSSpecPtr    spec;          // for open
   short        permission;    // for open
   short        posMode;       // for setfpos
   long         posOff;        // for setfpos   
   OSErr        result;        // error result
} FSParamRec,*FSParamRecPtr;

Now lets implement the callback routine that gets called at main application time. This routine will be executing at SystemTask time which means you can call any toolbox routine except for any routines that might call SystemTask() again.

static void * FSReadCallBack( void * parameter)
{
   FSParamRecPtr fsprp = (FSParamRecPtr)parameter;
   
   if (fsprp != nil) {
      fsprp->result = FSRead(fsprp->refNum,
                   &fsprp->count,
                   fsprp->buffPtr);
   }
   
   return fsprp;
}

First we check to make sure the parameter that was passed in is not nil then we simply call the File manager's FSRead call. When FSRead returns, we put the result into the result field and then return the pointer to the struct that was passed in.

All that is left is to do is to implement the new MyMPFSRead call.

pascal OSErr MyMPFSRead(short refNum,
                         long * count,
                         void * buffPtr)
{
   FSParamRec fsrr;   // make the record on the stack 
                      // no worries since it is a blocking call

   fsrr.refNum   = refNum;
   fsrr.count    = *count;
   fsrr.buffPtr  = buffPtr;
   fsrr.result   = paramErr;   //preset in case 
                               //anything goes wrong
   
   (void) _MPRPC(FSReadCallBack,&fsrr); 
   //ignore what is returned


   *count = fsrr.count;   //return the new count

   return fsrr.result;    //return the result
}

First we allocate a FSReadRec on the stack that gets passed to MPRPC. We fill out the fields in the struct with what was passed into us, call MPRPC and wait for the result. Then return the result to the caller.

That's it. You can now call FSRead from an MP Task. Using the same basic techniques you can implement all the file manager calls you need to get data in and out of your tasks. Now lets see how the task calls the new routines.

The MP Task itself is pretty straight forward as a result of the blocking I/O calls since there are no flags or spin loops to worry about.

static long MyMPTask(void * param)
{
   FSSpecPtr   fsp;
   Boolean     done = false;
   OSStatus    status;
   OSErr       err;
   MPQueueID   mpq = (MPQueueID) param;

   
   // don't start until we get the message
   status = MPWaitOnQueue(mpq,&fsp,nil,nil, kDurationForever);
   // the message is the file spec
   if (fsp) {   
      short          refNum;
      long          count = 1024; //read 1k of data
      
      err = MyMPFSpOpenDF(fsp,fsRdPerm,&refNum);
      if (!err) {
         err = MyMPSetFPos(refNum,fsFromStart,0);
         if (!err) {
   
#if qUseAsyncRead         
                err = MyMPFSReadAsync(refNum,&count,gBuffer);
#else
                err = MyMPFSRead(refNum,&count,gBuffer);
#endif            
            // we got some data. you could compress it
            // do FFT's on it or whatever.
            // In our case we just set the flag that we got
            // the data and tell the processors to sync up
            
            if (count > 0) {   
               gCount = count; // signal that we got some text   
               __eieio();      // sync processors
            } else {
               gCount = -1;    // signal that we got an error   
               __eieio();      // sync processors
            }
         } 
         err = MyMPFSClose(refNum);
      }
   }
}

In our task we immediately block (as every task should) on MPWaitOnQueue waiting for the FSSpecPtr from the application. When MPWaitOnQueue returns, we check the file spec pointer to make sure it is not nil and precedes to open the file. We set the file position to the beginning of the file and start the read operation. Notice that for either the async or non async case the code is still the same. The only difference is to the application since the task is blocked until the read completes. After the read completes, this is where you would do some serious processing on the data. It is very important that you do a lot of processing to minimize the overhead of the blocking I/O calls. The demo doesn't do any processing so the next thing to do is to set the gCount variable indicating we got the data making sure the write get synchronized with the other processors. We close the file and return. Returning from the task kills the task. You might want the task to hang around and be ready to process another file. In that case set up a while loop on MPWaitOnQueue. You can set the exit termination condition to be a nil FSSpecPtr.

Adding More Features

The FSRead technique is good at getting data in and out of your task but you basically block the entire application while it waits for the FSRead to complete. We can improve this by using asynchronous file manager calls to keep from blocking the main application task while executing a Read call.

We need a different structure to do an async read. I wrap the new struct around a ParamBlockRec to contain the flag needed to signal the completion of the read call.

typedef struct FSReadAsyncRec { ParamBlockRec pb; // standard paramblock Boolean callPending; // our pending flag } FSReadAsyncRec, *FSReadAsyncRecPtr;

The MyMPFSReadAsync code is a little more complicated but it saves having to have another task running just to call MPYield() since this routine spins on MPYield waiting for the PBRead to complete.

static pascal OSErr MyMPFSReadAsync(short refNum,
                               long * count,
                               void * buffPtr)
{
   FSReadAsyncRec       fsrar;      // make the rec on the stack 
   // Build a rountine descriptor by hand since we can't call
   // NewIOCompletionProc(userRoutine)
   RoutineDescriptor    ioCompProc = 
                  BUILD_ROUTINE_DESCRIPTOR(uppIOCompletionProcInfo,
                                          MyReadCompletion);

   ClearBlock(&fsrar,sizeof(fsrar)); 
   
   fsrar.pb.ioParam.ioRefNum     =   refNum;
   fsrar.pb.ioParam.ioReqCount   =   *count;
   fsrar.pb.ioParam.ioBuffer     =   buffPtr;
   fsrar.pb.ioParam.ioCompletion =   &ioCompProc;
   fsrar.callPending             = true;
   __eieio();                  //ensure that callPending gets set
                               //before we call MPRPC
   
   (void) _MPRPC(FSReadAsyncCallBack,&fsrar); //ignore what is 

   // spin waiting for flag to be set in completionRoutine

   while ( fsrar.callPending ) { //Spin waiting for completion
      MPYield();
   }

   *count = fsrar.pb.ioParam.ioActCount;   
                  //return the new count

   return fsrar.pb.ioParam.ioResult;   //return the result
}

MyMPFSReadAsync sets up the parameter block, builds a completion routine descriptor on the fly, calls MPRPC and then spins in a tight loop calling MPYield until the callPending flag is cleared.

The FSReadAsyncCallBack routine is very simple.

static void * FSReadAsyncCallBack( void * parameter)
{
   FSReadAsyncRecPtr fsr = (FSReadAsyncRecPtr) parameter;
   OSErr err;
   
   if (fsr != nil) {   
      err = PBReadAsync((ParmBlkPtr)fsr);   
                           //just call PBRead and return
   }                      // completion routine sets the flag   
   return fsr;
}

FSReadAsyncCallBack just calls PBReadAsync and returns. Below is the completion routine that tells the task the read has completed.

static void MyReadCompletion(ParmBlkPtr pb)
{
   FSReadAsyncRecPtr fs = (FSReadAsyncRecPtr)pb; 
   
   fs->callPending = false;  // set flag
   __eieio();                   // make sure it sticks
}

It just sets the callPending flag, signals the processors to sync up and returns. We can't set a MPQueue or a MPSemapore in here (which would be the better way to do it) because MP Library calls can't be called at interrupt time.

Handling asynchronous routines gets a little more complicated but it saves having to make sure other tasks are running just to call MPYield(). Now you might be thinking why are we using a flag when you could just spin on ioResult? Read on to see why this is not good idea...

Gotchas

When working with multiple processors some conventional Mac programming wisdom goes out the window. A good case in point is when ioResult is set. Normally ioResult is set to 1 to indicate a call is pending. The last thing the file manager does before calling the ioCompletion routine is to set ioResult to the error result from the parameter block call. None of this really changes when multiple processors are involved but the non-main processors are not bound by the 68k enable/disable interrupt tricks. So if your MP Task spins on ioResult waiting to see when the read is complete (ioResult != 1) your task starts to execute before the file manager is done with the parameter block. After the file manager sets the ioResult field, it gets the ioCompletion routine's address from the parameter block and jumps to it.

In our case the parameter block in on the stack and when the task unblocks, the stack is released and your task crunches merrily along where a parameter block used to be (and is still in use by the file manager). The second processor could be a 200 MHz CPU and in the time the file manager has set ioResult and jumps to the completion routine, your task could be millions of instructions away using the memory where the parameter block used to be.

The same is true for many of parts of the Mac OS Toolbox. The critical region technique of disabling interrupts does not work well when multiple processors are involved. So be careful and always use MPQueues, MPSemaphores and MPCriticalRegions to coordinate your various tasks.

Another gotcha may be in your thought process. You might be thinking that it would be cool to use the same techniques mentioned in the article to make every Toolbox call available from MP tasks. While this is possible, and would make your task code a lot easier to write, it is not a good idea. The benefits of multiprocessing only come from careful algorithm design, implementation, and profiling. Guideline #1 mentioned above says that your task should do a considerable amount of work to gain any performance improvements. Having your task block, waiting on a bunch of toolbox calls is not going to improve performance. On the other hand having to load all the data you need into memory before your task can start running may not be feasible either. This is where a careful balance of having main processor moving data in and out of your task while processors n+1 crunch along can really pay off.

More MP Information

Hopefully, this article piqued your interest in Multiprocessing. If you want more information there are a number of documents and resources to help you get the most out of MP. An introduction to MP systems was printed in MacTech March '96, TechNote 1071 on Multiprocessing is on the web http://www.apple.com/developer/ and I have set up a MP mailing list where developers can ask questions on MP programming issues. The list includes folks like the senior engineer who wrote the MP Library as well as Chris Cooksey and myself. For subscription information you can go to my web site http://www.best.com/~mxmora/mxm.html. Also, don't forget Apple Developer Technical Support is there for information about MP's past, present and future.

Summary

I hope this article shows how easy it is to get data into and out of your MP tasks. Use this information wisely and you should see some real improvements in your applications performance. You can use these techniques to work with other I/O technologies like networking, graphics and sound. I have created a MP File Library that you may want to use based on some of the techniques used in this article. It uses a slightly more complicated model for better performance. You can download a copy of my MP File Library from my web site at http://www.best.com/~mxmora/software.html. Good luck, and happy multiprocessing.


Matthew Xavier Mora was the engineer responsible for answering questions on Multitasking support in Apple's Developer Technical Support. As a self proclaimed evangelist for the Multi-processing API library he was instrumental in convincing both third-party developers and Apple engineers to implement MP support in their software. If you were ever thinking about moving into the Silicon Valley, consider that this article was written while Matt was sitting all night outside a school building waiting to register his son for pre-school. When Matt is not out doing crazy things like that you can reach him at mxmora@best.com.

 
AAPL
$112.14
Apple Inc.
+2.73
MSFT
$47.25
Microsoft Corpora
+1.51
GOOG
$507.67
Google Inc.
+2.78

MacTech Search:
Community Search:

Software Updates via MacUpdate

BBEdit 11.0.2 - Powerful text and HTML e...
BBEdit is the leading professional HTML and text editor for the Mac. Specifically crafted in response to the needs of Web authors and software developers, this award-winning product provides a... Read more
ExpanDrive 4.2.1 - Access cloud storage...
ExpanDrive builds cloud storage in every application, acts just like a USB drive plugged into your Mac. With ExpanDrive, you can securely access any remote file server directly from the Finder or... Read more
Adobe After Effects CC 2014 13.2 - Creat...
After Effects CC 2014 is available as part of Adobe Creative Cloud for as little as $19.99/month (or $9.99/month if you're a previous After Effects customer). After Effects CS6 is still available... Read more
Command-C 1.1.7 - Clipboard sharing tool...
Command-C is a revolutionary app which makes easy to share your clipboard between iOS and OS X using your local WiFi network, even if the app is not currently opened. Copy anything (text, pictures,... Read more
Tidy Up 4.0.2 - Find duplicate files and...
Tidy Up is a complete duplicate finder and disk-tidiness utility. With Tidy Up you can search for duplicate files and packages by the owner application, content, type, creator, extension, time... Read more
Typinator 6.3 - Speedy and reliable text...
Typinator turbo-charges your typing productivity. Type a little. Typinator does the rest. We've all faced projects that require repetitive typing tasks. With Typinator, you can store commonly used... Read more
GraphicConverter 9.5 - Graphics editor w...
GraphicConverter is an all-purpose image-editing program that can import 200 different graphic-based formats, edit the image, and export it to any of 80 available file formats. The high-end editing... Read more
Toast Titanium 12.0.1 - The ultimate med...
Toast Titanium goes way beyond the very basic burning in the Mac OS and iLife software, and sets the standard for burning CDs, DVDs, and now Blu-ray discs on the Mac. Create superior sounding audio... Read more
QuickBooks 2015 16.0.2.1422 R3 - Financi...
Save 20% on QuickBooks Pro for Mac today through this special discount link QuickBooks Pro 2013 helps you manage your business easily and efficiently. Organize your finances all in one place, track... Read more
Remotix 3.0.6 - Access all your computer...
Remotix is a fast and powerful application to easily access multiple Macs (and PCs) from your own Mac. Features: Complete Apple Screen Sharing support - including Mac OS X login, clipboard... Read more

Latest Forum Discussions

See All

Game of the Year 2014 – 148Apps Staff Pi...
The end of 2014 is almost here, which can only mean one thing. Okay it can mean a lot of things, but in this specific context it means Game of the Year lists! Which is why the 148Apps staff have all picked their favorites from the past year. And why... | Read more »
Dragon Quest III Review
Dragon Quest III Review By Jennifer Allen on December 18th, 2014 Our Rating: :: DATED BUT NOT WITHOUT MERITUniversal App - Designed for iPhone and iPad A walk down memory lane isn’t foolproof for Dragon Quest III, but it has its... | Read more »
8 KEMCO JRPGs Are Just $0.99 in Celebrat...
8 KEMCO JRPGs Are Just $0.99 in Celebration of the Holidays Posted by Jessica Fisher on December 18th, 2014 [ permalink ] KEMCO has announc | Read more »
Brothers in Arms 3: Sons of War Review
Brothers in Arms 3: Sons of War Review By Jennifer Allen on December 18th, 2014 Our Rating: :: FUN BUT PUSHYUniversal App - Designed for iPhone and iPad Brothers in Arms 3: Sons of War could be great fun, but its plethora of... | Read more »
The Snow is Falling on the French Countr...
The Snow is Falling on the French Countryside of Carcassonne’s Newest Update Posted by Jessica Fisher on December 18th, 2014 [ permalink ] < | Read more »
Creatures Such As We Review
Creatures Such As We Review By Jennifer Allen on December 18th, 2014 Our Rating: :: INTERESTING DISCUSSIONSUniversal App - Designed for iPhone and iPad Taking a more sedate approach to interactive fiction, Creatures Such As We... | Read more »
Stair: Slide the Blocks to Ascend (Game...
Stair: Slide the Blocks to Ascend 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Slide the blocks to ascend! Stair: Slide the Blocks to Ascend | Read more »
Give It Up! (Games)
Give It Up! 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: GIVE IT UP is a rather hard game where you have to assist this cheerful, singing Blob in jumping through 9 different tracks.So far... | Read more »
The Drive : Devil's Run (Games)
The Drive : Devil's Run 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: ON THE ROAD AGAIN! The Drive - Devil’s Run is classic point to point style racing game that pays homage to the classics... | Read more »
Procreate Pocket (Entertainment)
Procreate Pocket 1.01 Device: iOS iPhone Category: Entertainment Price: $2.99, Version: 1.01 (iTunes) Description: Create - anytime, anywhere. Made by the developers of the award-winning Procreate® for iPad®, Procreate Pocket™ allows... | Read more »

Price Scanner via MacPrices.net

iOS and Android OS Targeted by Man-in-the-Mid...
Cloud services security provider Akamai Technologies, Inc. has released, through the company’s Prolexic Security Engineering & Research Team (PLXsert), a new cybersecurity threat advisory. The... Read more
KMI MIDI K-Board Great Gift for Amateur &...
The K-Board is a MIDI Nano keyboard for music creation for iPad, Android, And computers; the easiest way to make music with iPads & Android tablets, and Mac, Windows, or Linux computers. Ultra-... Read more
Amazon offers 15-inch 2.2GHz Retina MacBook P...
 Amazon.com has the 15″ 2.2GHz Retina MacBook Pro on sale for $1699 including free shipping. Their price is $300 off MSRP. Stock is limited, so act now if you’re interested. Read more
Holiday sales continue: MacBook Pros for up t...
 B&H Photo has new MacBook Pros on sale for up to $300 off MSRP as part of their Holiday pricing. Shipping is free, and B&H charges NY sales tax only: - 15″ 2.2GHz Retina MacBook Pro: $1699... Read more
Google Search App For iOS Gets A Major Makeov...
Google has given iOS users an early Christmas present with a substantial update of it’s not-very-often-upgraded Google Search app. Google Search has been my go-to tool for Web searches since it was... Read more
ShopKeep Apple Pay And Chip Card Reader Avail...
ShopKeep, a cloud-based technology provider to more than 10,000 small business owners to manage retail shops and restaurants with iPads, has released its new Apple Pay and chip card reader. This... Read more
Holiday sale! 27-inch 5K iMac for $2299, save...
 B&H Photo has the 27″ 3.5GHz 5K iMac in stock today and on sale for $2299 including free shipping plus NY sales tax only. Their price is $200 off MSRP, and it’s the lowest price available for... Read more
Holiday Sale! 3.7GHz Quad Core Mac Pro availa...
 B&H Photo has the 3.7GHz Quad Core Mac Pro on sale for $2599 including free shipping plus NY sales tax only. Their price is $400 off MSRP, and it’s the lowest price for this model from any... Read more
iPhone 6 Number 3 Canadian Google Search Of 2...
CTVNews.ca reports that Apple’s iPhone 6 was the third highest-trending Google Canada search topic of 2014, exceeded only by Robin Williams largely after his death by suicide in August, and the FIFA... Read more
New iPad mini 3 Counter-Top & Wall Mount...
newMacgadgets has announced new secure all-acrylic displays for the iPad mini 3 (also works fine with the mini 2, last year’s iPad mini With Retina Display, and the original iPad mini). The new iPad... Read more

Jobs Board

*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Project Manager / Business Analyst, WW *Appl...
…a senior project manager / business analyst to work within our Worldwide Apple Fulfillment Operations and the Business Process Re-engineering team. This role will work Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.