Volume Number: 25
Issue Number: 03
Column Tag: Care Data

Core Data Versioning

How to migrate your Core Data persistent store

by Marcus S. Zarra

Core Data allows developers the ability to rapidly develop their applications with little worry about object persistence. By utilizing Core Data, developers can "future proof" their applications by taking advantage of the data migration functionality built into Core Data.

Core Data provides an infrastructure for object graph management and persistence. The basis for this infrastructure is often referred to as the Core Data "Stack". While the Xcode templates build this stack for us, it is important to understand how it works.

The Core Data stack is composed of three elements: a persistent store coordinator, a managed object model and a managed object context. In a non-document based application design, this stack is routinely initialized either as part of the start up of the application or lazily upon first request.

It is unusual for an application to access the NSManagedObjectModel directly. Normally it is built upon initialization of the stack and only referenced via the NSManagedObjectContext. Generally, there are two ways to build the managed object model.

managedObjectModel
Return the existing managedObjectModel if it exists or create a new one.
- (NSManagedObjectModel*)managedObjectModel
{
   if (managedObjectModel) return managedObjectModel;
   NSString *path = [[NSBundle mainBundle] pathForResource:@"Model" 
      ofType:@"mom"];
   NSURL *url = [NSURL fileURLWithPath:path];
   managedObjectModel = [[NSManagedObjectModel alloc] 
      initWithContentsOfURL:url];
   return managedObjectModel;
}

In this first example, the model is built using the initWithContentsOfURL: initalizer of NSManagedObjectModel. This has the advantage of only loading the models that are appropriate. This is useful in many situations such as where it is possible that multiple data models may be in the application bundle. Having an export data model in the bundle is an example of this situation.

managedObjectModel

Return the existing managedObjectModel if it exists or create a new one.
- (NSManagedObjectModel*)managedObjectModel
{
   if (managedObjectModel) return managedObjectModel;
   managedObjectModel = [[NSManagedObjectModel 
      mergedModelFromBundles:nil] retain];    
   return managedObjectModel;
}

This second example does not require knowledge of what models are available in the application bundle as it will grab every model available and load it into a single merged NSManagedObjectModel object. This choice allows the developer to easily add and remove models during development.

The persistent store is a representation of the model stored on the file system. Depending on the options chosen when the store was built, it can be in a SQLite database, XML, binary or a custom file format. The persistent store is responsible for serializing all access to the data stored on the file system. Therefore only one persistent store should ever be initialized per file.

When the stack is being initialized, the store accepts a NSManagedObjectModel and a url for the location of the store on the filesystem.

persistentStoreCoordinator

Return the existing NSPersistentStoreCoordinator. If there is not one then initialize a new one with the NSManagedObjectModel from the -managedObjectModel method and adding a xml based persistent store to it.

- (NSPersistentStoreCoordinator*)persistentStoreCoordinator 
{
   NSError *error = nil;
   NSFileManager *fileManager = [NSFileManager defaultManager];
   NSString *path = [self applicationSupportFolder];
   if (![fileManager fileExistsAtPath:path isDirectory:NULL] ) {
      if (![fileManager createDirectoryAtPath:path attributes:nil]) {
         return nil;
      }
   }
   path = [path stringByAppendingPathComponent:@"Example.xml"];
   
   NSURL *url = [NSURL fileURLWithPath:path];
   NSManagedObjectModel *model = [self managedObjectModel];
   if (!model) return nil;
   id psc = [[NSPersistentStoreCoordinator alloc] 
      initWithManagedObjectModel:model];
   if (![psc addPersistentStoreWithType:NSXMLStoreType configuration:nil 
      URL:url options:nil error:&error]) {
      [[NSApplication sharedApplication] presentError:error];
      return nil;
   }       
   return [psc autorelease];
}

Those familiar with building Core Data document based applications will notice quite a bit of a difference with the construction of the NSPersistentStoreCoordinator. Instead of accepting a path coming into the method, the path is constructed from the application support directory. The application support directory is the recommended location for storing application specific data for non-document based applications.

After the NSPersistentStoreCoordinator is initialized, the next step is to add the persistent store to the coordinator. It should be noted that it is possible to add more than one persistent store to the coordinator. Finally the newly created NSPersistentStoreCoordinator is returned to the calling method.

The last step in building the stack, and the first step from the calling methods point of view is the NSManagedObjectContext.

managedObjectContext

Return the existing NSManagedObjectContext. If one does not exist, then create it using the persistent store from the -persistentStoreCoordinator method.

- (NSManagedObjectContext*)managedObjectContext 
{
   if (managedObjectContext) return managedObjectContext;
   NSPersistentStoreCoordinator *psc = [self persistentStoreCoordinator];
   if (!psc) return nil;
   managedObjectContext = [[NSManagedObjectContext alloc] init];
   [managedObjectContext setPersistentStoreCoordinator:psc];
   return managedObjectContext;
}

This method simply retrieves a reference to the NSPersistentStoreCoordinator, initializes a new NSManagedObjectContext and sets the retrieved NSPersistentStoreCoordiantor on the NSManagedObjectContext.

One of the biggest challenges with Core Data is the migration of the store from one version to another. As applications evolve, the data model invariably changes. Prior to OS X 10.5 Leopard, the developer was required to handle these migrations manually. However, starting with Leopard, migration is handled directly by Core Data.

The first time that the data model needs to be revised it needs to be converted from a "xcdatamodel" file to a "xcdatamodeld" bundle. This is done by selecting Add Model Version with the current model selected.

Figure 1: Adding a Model Version

This will create a xcdatamodeld bundle with the same name as the data file and it will create a copy of the original data model. Both the original data model and the copy will be inserted into the xcdatamodeld bundle. It is recommended to rename the model files for clarity.

Figure 2: The xcdatamodeld structure

Once this is completed, both xcdatamodel files will be added to the application bundle. After the application has been complied there will be a new directory under Resources called "Example_DataModel.momd" which includes both of the complied mom files.

For a data migration to occur, a NSMappingModel object is required. This object describes how the data is migrated from one model to another. This object is normally constructed from within XCode with the mapping model editor.

In the current implementation of Core Data, one mapping model is required for each combination of source and destination models. Therefore careful consideration should be used when creating a new version of a data model.

While the NSMappingModel and its editor can handle a wide range of data migration needs, it does not cover every possible situation. When a migration goes beyond the abilities of the NSMappingModel, it is possible to extend it with custom subclasses. This is normally accomplished by subclassing the NSEntityMigrationPolicy object and using that subclass for one or more objects in the NSMappingModel.

The ToDo list will have two objects. The first object will be a list of todo items that are to be displayed to the user. The second object is a list of possible categories for the todo items. The resulting model is as follows:

Figure 3: The data model

Since this application will want to access the core data stack immediately to display the ToDo items, I am going to pre-load the ToDo items from the persistent store. To accomplish this, I have added a -applicationDidFinishLaunching: method as follows.

applicationDidFinishLaunching

Delegate method that is called by the NSApplication once the application has finished start up. Loads the core data objects into memory by doing a retrieval on startup.

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
{
   NSManagedObjectContext *moc = [self managedObjectContext];
   NSFetchRequest *request = [[NSFetchRequest alloc] init];
   [request setEntity:[NSEntityDescription entityForName:@"ToDo" 
      inManagedObjectContext:moc]];
      
   NSError *error = nil;
   [moc executeFetchRequest:request error:&error];
   if (error) {
      NSLog(@"%@:%s Error retrieving ToDo items:\n%@", [self class], _cmd, 
         error);
   }
}

In this method, I am forcing the core data stack to be initialized and then I am retrieving all of the ToDo items. This will cause them to be in memory and available for the user interface when it is presented to the user.

Since this application has only one core data stack it is unnecessary to force the user to save their changes manually. Therefore the application itself needs to handle saving automatically on exit. To do this, the application delegate needs to implement -applicationShouldTerminate: and handle saving.

applicationShouldTerminate

Called by the NSApplication just before it is going to quit. This will attempt to save the NSManagedObjectContext or will abort the termination if an error occurred.

- (NSApplicationTerminateReply)applicationShouldTerminate:(id)sender
{
   if (!managedObjectContext) return NSTerminateNow;
   if (![managedObjectContext commitEditing]) return NSTerminateCancel;
   if (![managedObjectContext hasChanges]) return NSTerminateNow;
      
   NSError *error = nil;
   if ([managedObjectContext save:&error]) return NSTerminateNow;
      
   if ([[NSApplication sharedApplication] presentError:error]) {
      return NSTerminateCancel;
   }
   int alertReturn = NSRunAlertPanel(nil, 
      @"Could not save changes while quitting. Quit anyway?" , 
      @"Quit anyway", @"Cancel", nil);
   if (alertReturn == NSAlertAlertnateReturn) {
      return NSTerminateCancel;
   }
   return NSTerminateNow;
}

When the application is told to quit it will attempt to save the Core Data stack to disk if at all possible and will warn the user if a failure occurred.

The user interface for this application should present the user a list of ToDo items that can be selected, edited, and deleted. While categories are available, there is no need to present those as editable in the main window. This will help to focus the user's attention on the ToDo list itself rather than the details of the relationship between Categories and ToDo items. However, the user will need to edit the categories, therefore I will add a panel that can be accessed from the window menu.

The main window should be a simple master/detail view. Using the latest version of Interface Builder, drag the Core Data Entity object from the library onto the main window. From there select the ToDo entity from the list and select the master/detail option. Once the UI has been created, change the plain NSTextField for the date to a Date Picker and removed the category from the table view. The resulting UI can be seen in Figure 4.

Figure 4: The Main Window

The design of the category panel is almost identical to the main window. To start with add a panel from the library in Interface Builder. Then add a menu item to the window menu called Categories. Next, bind the menu item to the makeKeyAndOrderFront: method on the category panel.

Again using the Core Data library item but selecting Category instead of ToDo in the entity list, the resulting panel is shown in Figure 5.

Figure 5: The Category Edit Window

Although the Core Data object does all of the bindings for us, it is important to understand how all of this is connected. Looking at the objects in the MainMenu.xib file, you can see that the Core Data library object added two Array Controllers: one for ToDo and one for the Categories.

Figure 6: MainMenu.xib

Selecting either one of these NSArrayControllers and inspecting its bindings will reveal that it is bound to the NSManagedObjectContext inside of the Application's delegate object.

Looking at the attributes tab of the inspector for either of these Array Controllers will reveal that they are defined to display a specific Core Data Entity. This information is used to populate the array directly from the NSManagedObjectContext when the xib is loaded and initialized.

A ToDo list application is not very useful unless it can alert the user to a pending ToDo item. Therefore we want to wake up once per minute and look for any pending items and let the user know. However we do not want to notify users of items that are far into the future so it will need to filter the results.

The first change is in the -applicationDidFinishLaunching: method to start a timer that will fire every minute.

[NSTimer timerWithTimeInterval:]

Schedule a timer to fire every 60 seconds and call the -minuteTimerFired: method.

minuteTimer = [NSTimer timerWithTimeInterval:60.0 target:self  selector:@selector(minuteTimerFired:)  userInfo:nil repeats:YES];

This timer will cause the -minuteTimerFired: method to be called every minute. We should also invalidate this timer in the -dealloc method.

[NSTimer invalidate]

Invalidate the NSTimer so that it will no longer fire and will release itself from memory.

[minuteTimer invalidate], minuteTimer = nil;

Calling -invalidate on the timer will cause it to stop firing and release itself. To be safe from future edits set the reference to the timer to nil.

When the -minuteTimerFired: is called, all ToDo items that are due in the next five minutes need to be located. If one or more ToDo items that match that criteria are found, you want to display them to the user.

minuteTimerFired:

Called by the NSTimer every 60 seconds, will check for any ToDo items that are scheduled within the next five minutes.

- (void)minuteTimerFired:(id)sender
{
   NSDate *now = [NSDate date];
   NSDate *later = [now addTimeInterval:(60.0 * 5)];
   NSManagedObjectContext *moc = [self managedObjectContext];
   NSFetchRequest *request = [[[NSFetchRequest alloc] init] autorelease];
   [request setEntity:[NSEntityDescription entityForName:@"ToDo" 
      inManagedObjectContext:moc]];
   NSString *ps = @"dueDate >= %@ AND dueDate <= %@";
   NSPredicate *pred = [NSPredicate predicateWithFormat:pred, 
      now, later];
   [request setPredicate:pred];
   NSError *error = nil;
   NSArray *itemsThatAreDue = [moc executeFetchRequest:request 
      error:&error];
   if (error) { //Error retrieving the todo list
      [[NSApplication sharedApplication] presentError:error];
      return;
   }
   if ([itemsThatAreDue count] == 0) return; //No due items
   [alertItems setContent:itemsThatAreDue];
   [alertWindow makeKeyAndOrderFront:self];
   NSApplication *sa = [NSApplication sharedApplication];
   [sa requestUserAttention:NSInformationalRequest];
}

When the timer fires, you want to first get the current date and then create a second date object set to 5 minutes from now. With those two dates initialized it is time to find all of the matching ToDo items.

Using a NSFetchRequest item, set the entity and, add a NSPredicate to the NSFetchRequest. The predicate will filter the ToDo items down to all of the items that have a dueDate after now and before 5 minutes from now.

Using the completed NSFetchRequest, retrieve all of the ToDo items from the NSManagedObjectContext and set the resulting NSArray as the content for the NSArrayController that handles the alert window and call -makeKeyAndOrderFront: on the alert window. To let the user know that the window has changed, a call to [[NSApplication sharedApplication] requestUserAttention:NSInformationalRequest] will cause the application's dock icon to bounce for 1 second.

Using this application for a few minutes will quickly expose a few flaws. First, for a ToDo item that is due in 5 minutes the user will be alerted 5 times. In addition, there is no way to tell the application to avoid alerting for a particular ToDo item. To correct these issues an update to the data model is required.

Figure 7: Version 2 Data Model

The new data model introduces two new attributes on the ToDo entity: lastAlertDate and doNotAlert.

Once the new model is built, the current data model version needs to be set in XCode. This is done by selecting the version 2 model and choosing Set Current Version under the design menu.

To get core data to migrate the data, two changes are required. First, the persistent store needs to be told to attempt automatic migration. This is an option that is added when the persistent store is added to the coordinator.

persistentStoreCoordinator

Updated section of the persistentStoreCoordinator method that includes a NSDictionary instructing Core Data to attempt an automatic data migration.

id psc = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:model];
    
NSMutableDictionary *options = [NSMutableDictionary dictionary];
[options setValue:@"YES" forKey:NSMigratePersistentStoresAutomaticallyOption];
    
if (![psc addPersistentStoreWithType:NSXMLStoreType configuration:nil URL:url options:options error:&error]) {
   [[NSApplication sharedApplication] presentError:error];
   return nil;
}

With this change, Core Data will attempt to find a source model that matches the persistent store and a mapping model that will describe how to migrate the data from the source model to the current model.

The next step is to build the mapping model that will describe the migration. To do this, create a new file in Xcode and selected the Mapping Model template. Then set the Version 1 model as the source and the Version 2 model as the destination for the map. Upon creation, Xcode makes an effort to create the mapping model. For this migration the default is sufficient so no additional work is required.

To expose the Do Not Alert option to the user, add another checkbox to the main window next to the completed checkbox and bind it to the doNotAlert attribute. Lastly, update the -minuteTimerFired: method to use these new attributes.

minuteTimerFired

Modification to the method to set the last alert time on a ToDo item to avoid repeated alerts.

- (void)minuteTimerFired:(id)sender
{
   NSDate *now = [NSDate date];
   NSDate *later = [now addTimeInterval:(60.0 * 5)];
   NSDate *before = [now addTimeInterval:0 - (60.0 * 5)];
   NSManagedObjectContext *moc = [self managedObjectContext];
   NSFetchRequest *request = [[[NSFetchRequest alloc] init] autorelease];
   [request setEntity:[NSEntityDescription entityForName:@"ToDo" 
      inManagedObjectContext:moc]];
   [request setPredicate:[NSPredicate predicateWithFormat:@"dueDate >= %@ 
      AND dueDate <= %@ AND doNotAlert == NO AND (lastAlertDate == nil || 
      lastAlertDate < %@", now, later, before]];
   NSError *error = nil;
   NSArray *itemsThatAreDue = [moc executeFetchRequest:request 
      error:&error];
   if (error) { //Error retrieving the todo list
      [[NSApplication sharedApplication] presentError:error];
      return;
   }
   if ([itemsThatAreDue count] == 0) return; //No due items
   [alertItems setContent:itemsThatAreDue];
   [alertWindow makeKeyAndOrderFront:self];
   [[NSApplication sharedApplication] 
      requestUserAttention:NSInformationalRequest];
   [itemsThatAreDue setValue:now forKey:@"lastAlertDate"];
}

A common situation is where the migration of data from one model to another is beyond the scope of the standard NSMappingModel. For example, setting the doNotAlert bit to YES on all ToDo items that start with A. When the migration requirements stray outside of the default mapping, it is possible to build a custom migration policy. This requires subclassing NSEntityMigrationPolicy and implementing one of several methods that will be called during the migration processing. For this example, only one of the NSEntityMigrationPolicy object's methods need to be overridden.

createDestinationInstancesForSourceInstances

Called during the migration process and adds custom logic to the migration so that ToDo items that start with the letter A are flagged as DoNotAlert.

- (BOOL)createDestinationInstancesForSourceInstance:(NSManagedObject*)src ¬
entityMapping:(NSEntityMapping*)map manager:(NSMigrationManager*)mgr error:(NSError**)error
{
  NSManagedObjectContext *destMOC = [mgr destinationContext];
  NSString *destEntityName = [map destinationEntityName];
  
  NSString *name = [source valueForKey:@"name"];
  NSManagedObject *dest = [NSEntityDescription insertNewObjectForEntityForName:destEntityName ¬
inManagedObjectContext:destMOC];
  
  NSArray *keys = [[[source entity] attributesByName] allKeys];
  NSDictionary *values = [source valuesForKeys:keys];
  
  [dest setValuesForKeysWithDictionary:values];
  if ([[name lowercaseString] hasPrefix:@"a"]) {
    [dest setValue:@"YES" forKey:@"doNotAlert"];
  }
  [manager associateSourceInstance:src withDestinationInstance:dest forEntityMapping:map];
  
  return YES;
}

From the passed in NSMigrationManager it is possible to get a reference to the destination NSManagedObjectContext and then construct the destination NSManagedObject. In this example the change from the source object are trivial so you can copy the attributes directly from the source. The two new attributes, doNotAlert and lastAlertDate are either optional or contain default values so there is no need to alter them. After the destination object has been populated comes the custom logic that this subclass was built for. Using the name from the source, check to see if it starts with the letter a and if it does, then set the doNotAlert attribute to YES.

Since the Category object retains a reference to the ToDo lists, it is not necessary to manage the relationship between the two objects in this policy. However, if it were needed, then you would need to implement the -createRelationshipsForDestinationInstance: entityMapping: manager: error: method.

Since the introduction of Core Data in OS X 10.4 Tiger, data migration within Core Data has come a long way. In one OS X update we have moved from migration being an afterthought that developers tacked on when it was needed to migration becoming a first class citizen with a robust real world solution.

I would highly recommend that anyone who is now targeting OS X 10.5 Leopard for their applications to start using the build in migration tools of Core Data as opposed to a home grown solution.

Marcus S. Zarra is the owner of Zarra Studios, based out of Colorado Springs, Colorado. He has been developing Cocoa software since 2003, Java software since 1996, and has been in the industry since 1985. In addition to writing software, he assists other developers by blogging about development (www.cimgf.com) and supplying code samples. Currently, Marcus is in the process of writing a book on Core Data for The Pragmatic Programmer.