TweetFollow Us on Twitter

Using File Manager From MP Tasks

Volume Number: 14 (1998)
Issue Number: 8
Column Tag: Toolbox Techniques

Using the File Manager from MP Tasks

by by Matthew Xavier Mora
Edited by Peter N Lewis

How to get data in and out of your MP Task

One of the most common complaints I received while supporting MP Library in Developer Technical Support was that you could not call the toolbox from an MP Task. Multiple preemptive tasks are not much use if you cannot get data into and out of them efficiently. This article shows one way to get data into and out of an MP Task using the file manager, however the techniques used here can be modified for other I/O operations (like audio, video or networking). "But, I thought you couldn't call the file manager from MP Tasks?" Well, you thought wrong. :-) Read on...

Background

In the early version of the MP library there was no easy way to call the toolbox because the MP Library was designed to be compatible with Copland's kernel tasking model. Since the Mac OS toolbox wasn't going to be available from Copland's kernel tasks, the same was done for the Mac OS version of the MP Library. After the Copland project was canceled it was decided to publish a few previously undocumented routines that let you work with the Mac OS toolbox from a task. One of the routines published is MPRPC. MPRPC is a remote procedure mechanism that lets you specify a routine to execute at a time when it is safe to make toolbox calls. It does this by suspending the task and then executing the supplied routine during SystemTask() time. The task is suspended until MPYield is called or until any toolbox routine calls SystemTask(). MPRPC is used internally in the MP Library to implement calls such as MPAllocate and MPAllocateSys (which is why these are blocking calls).

The code in this article is based on the MP File Library that I wrote before the MPRPC call was published. The MP File Library used MPQueues to communicate with the main task and have it execute toolbox commands.

Review

Let's review some of the MP programming guidelines and how adding blocking calls can change some of these guidelines.

  1. Your tasks should do a considerable amount of work. If not, the benefits of using MP will be lost in the overhead of the scheduler and task switching. Adding blocking calls to your tasks adds additional overhead. The main benefit here is that by being able to call the toolbox from an MP Task your task can run autonomously from the main application thread. This results in a better user interface response from the application since the application can off load a time consuming task and call the main event loop more often giving the blocking calls more time to execute the toolbox calls.
  2. You should allocate no more than (MPProcessors() - 1) number of tasks. While it is important to keep the number of tasks low so that task switching does not impact performance, adding blocking calls to a task will also hurt performance if nothing calls MPYield(). "Wait, I thought MP Tasks were preemptive?" Yes they are but if the task is blocked waiting on a resource, the resource can't be released until the main thread calls WaitNextEvent() or another task calls MPYield(). That being the case, if you use MPRPC calls it is a good idea to bend the n-1 rule and create an extra task that can help unblock any waiting tasks.
  3. You should use MPQueues or MPSemaphores when communicating with MP Tasks. This does not change if you are using MPRPC so you should heed this warning.

Get On With It

OK, so how do I call the File Manager? For this simple example I will implement five MP calls that duplicate FSpOpenDF, FSClose, FSRead, FSWrite and SetFPos. Those are all the calls we need for a simple demo. We'll start with a FSRead type call.

First lets define a structure to pass to the MPRPC callback routines that will hold the values that we need to handle all the File Manager calls.

typedef struct FSParamRec{
   short        refNum;        // file ref num
   long         count;         // for read
   Ptr          buffPtr;       // for read
   FSSpecPtr    spec;          // for open
   short        permission;    // for open
   short        posMode;       // for setfpos
   long         posOff;        // for setfpos   
   OSErr        result;        // error result
} FSParamRec,*FSParamRecPtr;

Now lets implement the callback routine that gets called at main application time. This routine will be executing at SystemTask time which means you can call any toolbox routine except for any routines that might call SystemTask() again.

static void * FSReadCallBack( void * parameter)
{
   FSParamRecPtr fsprp = (FSParamRecPtr)parameter;
   
   if (fsprp != nil) {
      fsprp->result = FSRead(fsprp->refNum,
                   &fsprp->count,
                   fsprp->buffPtr);
   }
   
   return fsprp;
}

First we check to make sure the parameter that was passed in is not nil then we simply call the File manager's FSRead call. When FSRead returns, we put the result into the result field and then return the pointer to the struct that was passed in.

All that is left is to do is to implement the new MyMPFSRead call.

pascal OSErr MyMPFSRead(short refNum,
                         long * count,
                         void * buffPtr)
{
   FSParamRec fsrr;   // make the record on the stack 
                      // no worries since it is a blocking call

   fsrr.refNum   = refNum;
   fsrr.count    = *count;
   fsrr.buffPtr  = buffPtr;
   fsrr.result   = paramErr;   //preset in case 
                               //anything goes wrong
   
   (void) _MPRPC(FSReadCallBack,&fsrr); 
   //ignore what is returned


   *count = fsrr.count;   //return the new count

   return fsrr.result;    //return the result
}

First we allocate a FSReadRec on the stack that gets passed to MPRPC. We fill out the fields in the struct with what was passed into us, call MPRPC and wait for the result. Then return the result to the caller.

That's it. You can now call FSRead from an MP Task. Using the same basic techniques you can implement all the file manager calls you need to get data in and out of your tasks. Now lets see how the task calls the new routines.

The MP Task itself is pretty straight forward as a result of the blocking I/O calls since there are no flags or spin loops to worry about.

static long MyMPTask(void * param)
{
   FSSpecPtr   fsp;
   Boolean     done = false;
   OSStatus    status;
   OSErr       err;
   MPQueueID   mpq = (MPQueueID) param;

   
   // don't start until we get the message
   status = MPWaitOnQueue(mpq,&fsp,nil,nil, kDurationForever);
   // the message is the file spec
   if (fsp) {   
      short          refNum;
      long          count = 1024; //read 1k of data
      
      err = MyMPFSpOpenDF(fsp,fsRdPerm,&refNum);
      if (!err) {
         err = MyMPSetFPos(refNum,fsFromStart,0);
         if (!err) {
   
#if qUseAsyncRead         
                err = MyMPFSReadAsync(refNum,&count,gBuffer);
#else
                err = MyMPFSRead(refNum,&count,gBuffer);
#endif            
            // we got some data. you could compress it
            // do FFT's on it or whatever.
            // In our case we just set the flag that we got
            // the data and tell the processors to sync up
            
            if (count > 0) {   
               gCount = count; // signal that we got some text   
               __eieio();      // sync processors
            } else {
               gCount = -1;    // signal that we got an error   
               __eieio();      // sync processors
            }
         } 
         err = MyMPFSClose(refNum);
      }
   }
}

In our task we immediately block (as every task should) on MPWaitOnQueue waiting for the FSSpecPtr from the application. When MPWaitOnQueue returns, we check the file spec pointer to make sure it is not nil and precedes to open the file. We set the file position to the beginning of the file and start the read operation. Notice that for either the async or non async case the code is still the same. The only difference is to the application since the task is blocked until the read completes. After the read completes, this is where you would do some serious processing on the data. It is very important that you do a lot of processing to minimize the overhead of the blocking I/O calls. The demo doesn't do any processing so the next thing to do is to set the gCount variable indicating we got the data making sure the write get synchronized with the other processors. We close the file and return. Returning from the task kills the task. You might want the task to hang around and be ready to process another file. In that case set up a while loop on MPWaitOnQueue. You can set the exit termination condition to be a nil FSSpecPtr.

Adding More Features

The FSRead technique is good at getting data in and out of your task but you basically block the entire application while it waits for the FSRead to complete. We can improve this by using asynchronous file manager calls to keep from blocking the main application task while executing a Read call.

We need a different structure to do an async read. I wrap the new struct around a ParamBlockRec to contain the flag needed to signal the completion of the read call.

typedef struct FSReadAsyncRec { ParamBlockRec pb; // standard paramblock Boolean callPending; // our pending flag } FSReadAsyncRec, *FSReadAsyncRecPtr;

The MyMPFSReadAsync code is a little more complicated but it saves having to have another task running just to call MPYield() since this routine spins on MPYield waiting for the PBRead to complete.

static pascal OSErr MyMPFSReadAsync(short refNum,
                               long * count,
                               void * buffPtr)
{
   FSReadAsyncRec       fsrar;      // make the rec on the stack 
   // Build a rountine descriptor by hand since we can't call
   // NewIOCompletionProc(userRoutine)
   RoutineDescriptor    ioCompProc = 
                  BUILD_ROUTINE_DESCRIPTOR(uppIOCompletionProcInfo,
                                          MyReadCompletion);

   ClearBlock(&fsrar,sizeof(fsrar)); 
   
   fsrar.pb.ioParam.ioRefNum     =   refNum;
   fsrar.pb.ioParam.ioReqCount   =   *count;
   fsrar.pb.ioParam.ioBuffer     =   buffPtr;
   fsrar.pb.ioParam.ioCompletion =   &ioCompProc;
   fsrar.callPending             = true;
   __eieio();                  //ensure that callPending gets set
                               //before we call MPRPC
   
   (void) _MPRPC(FSReadAsyncCallBack,&fsrar); //ignore what is 

   // spin waiting for flag to be set in completionRoutine

   while ( fsrar.callPending ) { //Spin waiting for completion
      MPYield();
   }

   *count = fsrar.pb.ioParam.ioActCount;   
                  //return the new count

   return fsrar.pb.ioParam.ioResult;   //return the result
}

MyMPFSReadAsync sets up the parameter block, builds a completion routine descriptor on the fly, calls MPRPC and then spins in a tight loop calling MPYield until the callPending flag is cleared.

The FSReadAsyncCallBack routine is very simple.

static void * FSReadAsyncCallBack( void * parameter)
{
   FSReadAsyncRecPtr fsr = (FSReadAsyncRecPtr) parameter;
   OSErr err;
   
   if (fsr != nil) {   
      err = PBReadAsync((ParmBlkPtr)fsr);   
                           //just call PBRead and return
   }                      // completion routine sets the flag   
   return fsr;
}

FSReadAsyncCallBack just calls PBReadAsync and returns. Below is the completion routine that tells the task the read has completed.

static void MyReadCompletion(ParmBlkPtr pb)
{
   FSReadAsyncRecPtr fs = (FSReadAsyncRecPtr)pb; 
   
   fs->callPending = false;  // set flag
   __eieio();                   // make sure it sticks
}

It just sets the callPending flag, signals the processors to sync up and returns. We can't set a MPQueue or a MPSemapore in here (which would be the better way to do it) because MP Library calls can't be called at interrupt time.

Handling asynchronous routines gets a little more complicated but it saves having to make sure other tasks are running just to call MPYield(). Now you might be thinking why are we using a flag when you could just spin on ioResult? Read on to see why this is not good idea...

Gotchas

When working with multiple processors some conventional Mac programming wisdom goes out the window. A good case in point is when ioResult is set. Normally ioResult is set to 1 to indicate a call is pending. The last thing the file manager does before calling the ioCompletion routine is to set ioResult to the error result from the parameter block call. None of this really changes when multiple processors are involved but the non-main processors are not bound by the 68k enable/disable interrupt tricks. So if your MP Task spins on ioResult waiting to see when the read is complete (ioResult != 1) your task starts to execute before the file manager is done with the parameter block. After the file manager sets the ioResult field, it gets the ioCompletion routine's address from the parameter block and jumps to it.

In our case the parameter block in on the stack and when the task unblocks, the stack is released and your task crunches merrily along where a parameter block used to be (and is still in use by the file manager). The second processor could be a 200 MHz CPU and in the time the file manager has set ioResult and jumps to the completion routine, your task could be millions of instructions away using the memory where the parameter block used to be.

The same is true for many of parts of the Mac OS Toolbox. The critical region technique of disabling interrupts does not work well when multiple processors are involved. So be careful and always use MPQueues, MPSemaphores and MPCriticalRegions to coordinate your various tasks.

Another gotcha may be in your thought process. You might be thinking that it would be cool to use the same techniques mentioned in the article to make every Toolbox call available from MP tasks. While this is possible, and would make your task code a lot easier to write, it is not a good idea. The benefits of multiprocessing only come from careful algorithm design, implementation, and profiling. Guideline #1 mentioned above says that your task should do a considerable amount of work to gain any performance improvements. Having your task block, waiting on a bunch of toolbox calls is not going to improve performance. On the other hand having to load all the data you need into memory before your task can start running may not be feasible either. This is where a careful balance of having main processor moving data in and out of your task while processors n+1 crunch along can really pay off.

More MP Information

Hopefully, this article piqued your interest in Multiprocessing. If you want more information there are a number of documents and resources to help you get the most out of MP. An introduction to MP systems was printed in MacTech March '96, TechNote 1071 on Multiprocessing is on the web http://www.apple.com/developer/ and I have set up a MP mailing list where developers can ask questions on MP programming issues. The list includes folks like the senior engineer who wrote the MP Library as well as Chris Cooksey and myself. For subscription information you can go to my web site http://www.best.com/~mxmora/mxm.html. Also, don't forget Apple Developer Technical Support is there for information about MP's past, present and future.

Summary

I hope this article shows how easy it is to get data into and out of your MP tasks. Use this information wisely and you should see some real improvements in your applications performance. You can use these techniques to work with other I/O technologies like networking, graphics and sound. I have created a MP File Library that you may want to use based on some of the techniques used in this article. It uses a slightly more complicated model for better performance. You can download a copy of my MP File Library from my web site at http://www.best.com/~mxmora/software.html. Good luck, and happy multiprocessing.


Matthew Xavier Mora was the engineer responsible for answering questions on Multitasking support in Apple's Developer Technical Support. As a self proclaimed evangelist for the Multi-processing API library he was instrumental in convincing both third-party developers and Apple engineers to implement MP support in their software. If you were ever thinking about moving into the Silicon Valley, consider that this article was written while Matt was sitting all night outside a school building waiting to register his son for pre-school. When Matt is not out doing crazy things like that you can reach him at mxmora@best.com.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

DiskCatalogMaker 6.6 - Catalog your disk...
DiskCatalogMaker is a simple disk management tool which catalogs disks. Simple, light-weight, and fast Finder-like intuitive look and feel Super-fast search algorithm Can compress catalog data for... Read more
RapidWeaver 7.3 - Create template-based...
RapidWeaver is a next-generation Web design application to help you easily create professional-looking Web sites in minutes. No knowledge of complex code is required, RapidWeaver will take care of... Read more
iFFmpeg 6.2.6 - Convert multimedia files...
iFFmpeg is a comprehensive media tool to convert movie, audio and media files between formats. The FFmpeg command line instructions can be very hard to master/understand, so iFFmpeg does all the hard... Read more
Amazon Chime 4.0.5540 - Amazon-based com...
Amazon Chime is a communications service that transforms online meetings with a secure, easy-to-use application that you can trust. Amazon Chime works seamlessly across your devices so that you can... Read more
OnyX 3.2.4 - Maintenance and optimizatio...
OnyX is a multifunction utility that you can use to verify the startup disk and the structure of its system files, to run miscellaneous maintenance and cleaning tasks, to configure parameters in the... Read more
Opera 43.0.2442.991 - High-performance W...
Opera is a fast and secure browser trusted by millions of users. With the intuitive interface, Speed Dial and visual bookmarks for organizing favorite sites, news feature with fresh, relevant content... Read more
VueScan 9.5.71 - Scanner software with a...
VueScan is a scanning program that works with most high-quality flatbed and film scanners to produce scans that have excellent color fidelity and color balance. VueScan is easy to use, and has... Read more
SpamSieve 2.9.28 - Robust spam filter fo...
SpamSieve is a robust spam filter for major email clients that uses powerful Bayesian spam filtering. SpamSieve understands what your spam looks like in order to block it all, but also learns what... Read more
GarageSale 7.0.7 - Create outstanding eB...
GarageSale is a slick, full-featured client application for the eBay online auction system. Create and manage your auctions with ease. With GarageSale, you can create, edit, track, and manage... Read more
Thunderbird 45.7.1 - Email client from M...
As of July 2012, Thunderbird has transitioned to a new governance model, with new features being developed by the broader free software and open source community, and security fixes and improvements... Read more

Last week on Pocket Gamer
If you’re wondering what’s going on in the wider world of portable gaming, our sister site PocketGamer has you covered. Each week we like to check in on the PG team and see what they’ve been preoccupied with. From the latest on the Nintendo Switch... | Read more »
Mudd Masher arrives this week
Atooi Games, the minds behind Totes the Goat and Mutant Mudds, have a new game in the works -- Mudd Masher. The game, a hybrid of the independent studio's first two titles, is expected to launch this week on March 2. [Read more] | Read more »
The best sales on the App Store this wee...
The App Store has quite an exciting lineup of discount games this week that range across a variety of genres. It's a great opportunity to catch up on some of the premium games you may have been holding off on -- and some you can even grab for free... | Read more »
The best new games we played this week
Ah, here we are again at the close of another busy week. Don't rest too easy, though. We had a lot of great new releases in mobile games this week, and now you're going to have to spend all weekend playing them. That shouldn't be too much of a... | Read more »
Rollercoaster Tycoon Touch Guide: How to...
| Read more »
Rabbids Crazy Rush Guide: How to unlock...
The Rabbids are back in a new endless running adventure, Rabbids Crazy Rush. It's more ridiculous cartoon craziness as you help the little furballs gather enough fuel (soda) to get to the moon. Sure, it's a silly idea, but everyone has dreams --... | Read more »
Tavern Guardians (Games)
Tavern Guardians 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: Tavern Guardians is a Hack-and-Slash action game played in the style of a match-three. You can experience high pace action... | Read more »
Slay your way to glory in idle RPG Endle...
It’s a golden age for idle games on the mobile market, and those addictive little clickers have a new best friend. South Korean developer Ekkorr released Endless Frontier last year, and players have been idling away the hours in the company of its... | Read more »
Tiny Striker: World Football Guide - How...
| Read more »
Good news everyone! Futurama: Worlds of...
Futurama is finding a new home on mobile in TinyCo and Fox Interactive's new game, Futurama: Worlds of Tomorrow. They're really doing it up, bringing on board Futurama creator Matt Groening along with the original cast and writers. TinyCo wants... | Read more »

Price Scanner via MacPrices.net

SanDisk Introduces 256GB Storage Expansion Fo...
Western Digital Corporation at Mobile World Congress in Barcelona, Spain on Tuesday announced that it has increased the capacity of its line of iOS mobile flash drives with the introduction of its... Read more
FINSiX DART World’s Smallest and Lightest Lap...
Up to 4x smaller and lighter than today’s typical 65W laptop chargers, and with a built-in Smallest and Lightest Laptop Charger, DART allows you to charge all your devices from a single outlet.... Read more
New iPads May Not Arrive Until May or June –...
Digitimes’ Siu Han and Steve Shen say their sources in the iPad upstream supply chain are anticipating disappointing calendar first-quarter 2017 sales. The same sources at upstream suppliers had... Read more
Olay Unveils Skin Advisor App To Help Women B...
Olay celebrated its Mobile World Congress debut with the global launch of Olay Skin Advisor, a new app designed to help women better understand their skin and find the products best-suited to their... Read more
12-inch 1.1GHz Retina MacBooks on sale for $1...
B&H has 12″ 1.1GHz Retina MacBooks on sale for $150 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 12″ 1.1GHz Space Gray Retina MacBook: $1149 $150 off MSRP - 12″ 1.1GHz... Read more
Save up to $600 with Apple refurbished Mac Pr...
Apple has Certified Refurbished Mac Pros available for up to $600 off the cost of new models. An Apple one-year warranty is included with each Mac Pro, and shipping is free. The following... Read more
13-inch 2.7GHz Retina MacBook Pro on sale for...
B&H Photo has the 2015 13″ 2.7GHz/128GB Retina Apple MacBook Pro on sale for $150 off MSRP. Shipping is free, and B&H charges NY tax only: - 13″ 2.7GHz/128GB Retina MacBook Pro (MF839LL/A): $... Read more
13-inch 1.6GHz/256GB MacBook Air on sale for...
Newegg has the 13″ 1.6GHz/256GB MacBook Air (MMGG2LL/A) on sale for $1029.99 including free shipping. Their price is $170 off MSRP, and it’s the lowest price available for this model. Choose Newegg... Read more
Apple refurbished Apple TVs available for up...
Apple has Certified Refurbished 32GB and 64GB Apple TVs available for up to $30 off the cost of new models. Apple’s standard one-year warranty is included with each model, and shipping is free: -... Read more
27-inch 3.3GHz 5K iMac on sale for $2099, sav...
B&H Photo has the 27″ 3.3GHz 5K Apple iMac on sale for $2099.99 including free shipping plus NY sales tax only. Their price is $200 off MSRP. Amazon also has the 27″ 3.3GHz 5K iMac on sale for $... Read more

Jobs Board

*Apple* Solutions Consultant - Apple (United...
# Apple Solutions Consultant Job Number: 55676865 Los Angeles, California, United States Posted: Feb. 22, 2017 Weekly Hours: 40.00 **Job Summary** As an Apple Read more
Programmer/Editor *Apple* Music Dance - App...
# Programmer/Editor Apple Music Dance Job Number: 55565967 Culver City, California, United States Posted: Feb. 23, 2017 Weekly Hours: **Job Summary** Apple Music Read more
Digital Marketing Specialist - *Apple* iClo...
# Digital Marketing Specialist - Apple iCloud Job Number: 54729233 Culver City, California, United States Posted: Feb. 22, 2017 Weekly Hours: 40.00 **Job Summary** Read more
Marketing Specialist, iTunes & *Apple*...
# Marketing Specialist, iTunes & Apple Music Job Number: 55704205 Culver City, California, United States Posted: Feb. 23, 2017 Weekly Hours: 40.00 **Job Summary** Read more
*Apple* Wireless Lead - T-ROC - The Retail O...
…of knowledge in wireless sales and activations to the Beautiful and NEW APPLE Experiencestore within MACYS. THIS role, APPLE Wireless Lead, isbrandnewas MACYS Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.