TweetFollow Us on Twitter

Entourage 2004 Spotlight Support

Volume Number: 22 (2006)
Issue Number: 5
Column Tag: Programming

MICROSOFT | MAC IN THE ENTERPRISE

Entourage 2004 Spotlight Support

by Brian Johnson and Andy Ruff

An IT Perspective: How Microsoft Entourage 2004 now takes advantage of Spotlight

Introduction

In update 11.2.3, Microsoft added support for Spotlight and Sync Services to Microsoft Entourage 2004 running on Mac OS X 10.4. These two features allow users to search Entourage e-mail stores and to synchronize Entourage data with any software or hardware that takes advantage of sync services in the OS.

It's important for system administrators who need to plan deployment of this technology on Macs, and may need to consider configurations with many users, and with limited disk space, to understand how this all works. In this article, we'll focus on the Spotlight support added to Entourage. We'll tell you about how Spotlight support works in this update. Specifically, we'll also address Spotlight support considerations for multi-identity installations of Entourage.

An Overview of Spotlight in Entourage

From the user perspective, Spotlight search in Entourage provides a mechanism that allows for the full text search of items in the Entourage database. Spotlight uses file based metadata and a constantly updating index to return results to queries passed through the Spotlight search interface in the operating system. Results return quickly because the index is updated based on messages coming from the file system. Once the initial indexing is complete on a set of data, additional data is indexed automatically as files update on the system.

One of the difficulties in making Spotlight work with Entourage had to do with how Entourage stores its data. All Entourage data is stored within a single database file per user identity. Entourage was designed to be multi-user at the application level. This was to allow multiple family members to have their own identities in the application in the home environment. Entourage stores its data in a single database for each user identity created in Entourage. When a user first sets up Entourage the Identity they get is named "Main Identity".

In order to support Spotlight searching, we had to develop a mechanism for providing Entourage's database content to Spotlight's file-oriented indexing process. We settled upon a solution that "mirrors" the essential item content and metadata to a series of cache files. As a new message arrives, we store the message within our database and spawn a cache file representing the message. When a user modifies a contact's phone number or changes the dates on an event, we update our database and the contents of each item's cache file. When Spotlight indexes Entourage, it is actually indexing the contents of each cache file rather than the Entourage database. This approach allows Spotlight's indexing process to work it's magic on file change notifications, while not requiring a large overhaul of Entourage's data access architecture.

As an Entourage user's database potentially holds years of e-mail messages, the creation of cache files chances consuming large amounts of disk space for essentially redundant data. When we were considering this design, we found that through optimizations such as writing only plain text content rather than HTML and ignoring e-mail attachments, we were able to generate a cache roughly 20% of the original Entourage database's size. We also decided that the feature would be optional, allowing any user to simply disable the creation of the cache within their Entourage preferences.



Figure 1: Spotlight is enabled by default for the first identity opened after update 11.2.3 is installed.

The Spotlight preference pane in Entourage allows the user to both toggle the feature and rebuild the contents of the cache. On a moderately sized database of 200 MB, the creation of the cache file takes only a few minutes and happens in the background. The Rebuild button simply deletes all existing cache files, crawls the Entourage database, and generates a set of new cache files. A user would only need to rebuild if problems arise, as Entourage will continue to create, update, and delete cache files with each action performed on the Entourage database.

Once the cache files are created, Entourage's role in the indexing process is complete. Spotlight chooses when to index the cache files and how the results are displayed in the Spotlight Search Window, Smart Folders, and the Finder's Find functionality. As indexing progresses, the index in Spotlight is updated and queries containing the information the user is searching for begins to show up in the Search window. If you search for a set of words and Spotlight indexes an Entourage mail message with a matching phrase, the message will suddenly appear within the Spotlight Search Results Window. Figure 2 shows the results of a typical Spotlight search with Spotlight enabled in Entourage. The returned Entourage items can include mail, appointment, contact, task, and notes data.



Figure 2: Spotlight search results with Entourage items returned.

Double clicking on a returned item in Spotlight works as expected. You see the Entourage item open, just as if you had clicked on it in Entourage. So what's going on under the covers? Let's use some command line tools and take a look.

Query with Command Line Tools

There are a number of command line utilities that we can use to query the Spotlight database. We can use these tools to see where Entourage is storing the Spotlight metadata that it's creating and we can also see what the metadata files themselves look like.

The first tool to look at is mdfind. mdfind queries the metadata store and returns the results of our query. This tool takes three parameters. The -live parameter will continuously scan the database for results and you'll see items added as they come into Entourage. The -onlyin parameter allows us to specify a particular folder for the search. Finally, the query parameter, a string representing the information that we're searching for. Apple's developer documentation provides more details on the syntax of Spotlight queries. Let's see if we can use this tool to find an Entourage item and see where the metadata is being stored:

Running the command "mdfind welcomee@microsoft.com" on my machine returns a result with the path:

/Users/Brianjo/Library/Caches/Metadata/Microsoft/Entourage/2004/Main 
Identity/Messages/0T/0B/0M/0K/1.vRgeMessage

The .vRgeMessage file is an Entourage mail message's cache file. When you perform a Spotlight search, the results always return cache files. As mentioned previously, cache files are merely file-based mirrors of Entourage database records with the metadata and content necessary for Spotlight indexing. The name of the cache file is the record ID for the corresponding database record. When a user opens the cache file from a Spotlight result, Entourage reads the filename, looks up the record ID within the database, and shows the item directly from the database.

The mdls command line utility allows you to see the metadata Spotlight has indexed for any given file. By passing the path to the 1.vRgeMessage cache file from our mdfind result to mdls, we can see Spotlight knows the following about the e-mail message:

/Users/Brianjo/Library/Caches/Metadata/Microsoft/Entourage/2004/Main Identity/Messages/0T/0B/0M/0K/1.vRgeMessage -------------

com_microsoft_entourage_folderID         = 1
com_microsoft_entourage_messageSent      = 2006-03-21 00:23:21 -0800
com_microsoft_entourage_recordID         = 1
com_microsoft_entourage_size             = 37783
kMDItemAttributeChangeDate               = 2006-03-21 21:55:25 -0800
kMDItemAuthors                           = ("The Microsoft Mac Team <WelcomeE@microsoft.com>")
kMDItemContentCreationDate               = 2006-03-21 00:23:21 -0800
kMDItemContentModificationDate           = 2006-03-21 21:55:24 -0800
kMDItemContentType                       = "com.microsoft.entourage.virtual.message"
kMDItemContentTypeTree                   = (
    "com.microsoft.entourage.virtual.message", 
    "public.message", 
    "public.data", 
    "public.item"
)
kMDItemCoverage                          = "Inbox"
kMDItemDisplayName                       = "Welcome to Microsoft Entourage 2004 for Macintosh"
kMDItemFSContentChangeDate               = 2006-03-21 21:55:24 -0800
kMDItemFSCreationDate                    = 2006-03-21 21:55:24 -0800
kMDItemFSCreatorCode                     = 0
kMDItemFSFinderFlags                     = 0
kMDItemFSInvisible                       = 0
kMDItemFSIsExtensionHidden               = 0
kMDItemFSLabel                           = 0
kMDItemFSName                            = "1.vRgeMessage"
kMDItemFSNodeCount                       = 0
kMDItemFSOwnerGroupID                    = 501
kMDItemFSOwnerUserID                     = 501
kMDItemFSSize                            = 6584
kMDItemFSTypeCode                        = 0
kMDItemID                                = 4306567
kMDItemKind                              = "Microsoft Entourage message pointer"
kMDItemLastUsedDate                      = 2006-03-21 00:23:21 -0800
kMDItemRecipients                        = ("New Microsoft Entourage User ")
kMDItemTitle                             = "Welcome to Microsoft Entourage 2004 for Macintosh"

Whenever possible, metadata provided by Entourage is designed so that attribute names and values match those used by an analogous Apple application (e.g. message title). We hope that this will allow anyone who builds a solution on top of Spotlight may easily support Entourage alongside Apple's applications. We only deviated by adding additional attributes - nearly all properties available in our AppleScript dictionary are available as metadata. The design is intended so that scripters can use Spotlight as a quick way to query information and round-trip interactions with results through AppleScript. It also makes it possible to create useful queries such as "all unread messages today"( com_microsoft_entourage_unread == 1 && kMDItemContentCreationDate >= $time.today ).

Importing Metadata

Spotlight is designed such that it does not need to know about the file format of each file in order to index the file's contents. Instead, a developer provides a plug-in for Spotlight that handles both reading a file and returning metadata to the Spotlight indexing engine. These plugins are known as Metadata Importers and may be found \Library\Spotlight or are sometimes located within an application's bundle.



Figure 3: The Microsoft Entourage.mdimporter plug-in

The metadata importer plug-in that's used with Entourage is called Microsoft Entourage.mdimporter. When Spotlight comes across an Entourage-generated cache file, Spotlight passes the path of the file to the Entourage metadata importer, the importer reads the file, and then passes the metadata back to Spotlight. You can see the info for this plug-in in Figure 3. Notice that this plug-in is a universal binary and that it runs natively on an Intel-based Mac.

Multiple Identities?

There are a few things that system administrators should understand if they are going to use Spotlight search in multi-user scenarios. Given that Entourage can work as a multi-identity application, one thing you'll probably wonder about is, how does Spotlight know about the currently active identity in Entourage? The answer is that it doesn't. While we only automatically enable Spotlight indexing for the first identity launched after the update is applied, a user may turn on indexing of additional identities by enabling the Spotlight preference in Entourage. If the user then double clicks on an item, that item is only opened if its associated identity is currently active. Entourage actually uses the folder path to determine the identity of a result. If the identity is not currently active then the user will get the message shown in Figure 4.



Figure 4: Trying to open an item associated with an inactive identity

As you can imagine, in scenarios where many people use the same account on a Mac and then differentiate identities in Entourage, Spotlight could become pretty useless when trying to find specific e-mail items as search results intermix results across multiple identities. For that reason, Microsoft recommends that in situations where multiple users will want to use Spotlight search with Entourage, users should have their own user accounts set up on the Macs.

Removing Spotlight from Entourage

There are a number of reasons a system administrator might want to completely disable Spotlight searching in Entourage. First, multiple user accounts on a machine are not always practical. In some cases, schoolrooms use a single account per classroom and kids are able to check their e-mail by simply switching identities in Entourage. On a machine with many dozens of identities, using Spotlight to find anything could be pretty difficult. Second, the cache used for Entourage content does take disk space. In a scenario where a user has a large entourage database, or there are multiple accounts on the machine with large databases, disk space can potentially become an issue. Finally there are privacy considerations around using Spotlight searches on Entourage content, especially if multiple identities are used on the same user account. Even if the searcher can't see the e-mail that's returned, they might be able to get more information than the user wants them to have about items returned in a search.

To completely disable Spotlight in Entourage, simply remove the Microsoft Entourage.mdimporter plug-in from the /Library/Spotlight folder and restart Entourage. When you look at the Entourage preferences you'll see as in Figure 5, that the Spotlight preference is no longer available.



Figure 5: Spotlight removed from Entourage preferences

Finally, if you completely disable Spotlight in entourage, remember to go to the /Users/<username>/Library/Caches/Metadata/Microsoft/Entourage/2004/ folder and delete any folders there that you no longer want to be available in Spotlight searches.

Conclusion

In this article, we've described how Spotlight works with Entourage 2004. The Spotlight search functionality added in update 11.2.3 fundamentally changes the way that you can work with e-mail, contacts, calendar items, and notes in Entourage, allowing you to instantly find data you need using the tools built right into Tiger. This update makes Entourage a true, first class citizen in the OS and it makes working with Entourage data on the Mac easier than ever.

Bibliography and References

Apple Computer. "Working with Spotlight". http://developer.apple.com/macosx/spotlight.html.

M. Amir Haque. "Spotlight Support in Entourage 2004". The Entourage Blog, http://blogs.msdn.com/entourage/archive/2006/03/17/553801.aspx


Brian Johnson is the Microsoft Entourage Product Manager. You can contact Brian by e-mail at brianjo@microsoft.com, or you can read his blog at http://bufferoverrun.net.

Andy Ruff is an Entourage Program Manager, and is the author of The Entourage Blog (http://blogs.msdn.com/Entourage).

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Civilization VI 1.0.1 - Next iteration o...
Sid Meier’s Civilization VI is the next entry in the popular Civilization franchise. Originally created by legendary game designer Sid Meier, Civilization is a strategy game in which you attempt to... Read more
Chromium 55.0.2883.75 - Fast and stable...
Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all Internet users to experience the web. Version 55.0.2883.75: Security fixes: High CVE-2016... Read more
Google Chrome 55.0.2883.75 - Modern and...
Google Chrome is a Web browser by Google, created to be a modern platform for Web pages and applications. It utilizes very fast loading of Web pages and has a V8 engine, which is a custom built... Read more
VirtualBox 5.1.10 - x86 virtualization s...
VirtualBox is a family of powerful x86 virtualization products for enterprise as well as home use. Not only is VirtualBox an extremely feature rich, high performance product for enterprise customers... Read more
Pixa 1.1.9 - Quickly and easily organize...
Pixa is an image-organizing application. The new app functions well, is easy to use, and helps people organize their images quickly and easily on their computers. For those who prefer not to use the... Read more
VirtualBox 5.1.10 - x86 virtualization s...
VirtualBox is a family of powerful x86 virtualization products for enterprise as well as home use. Not only is VirtualBox an extremely feature rich, high performance product for enterprise customers... Read more
Pixa 1.1.9 - Quickly and easily organize...
Pixa is an image-organizing application. The new app functions well, is easy to use, and helps people organize their images quickly and easily on their computers. For those who prefer not to use the... Read more
Civilization VI 1.0.1 - Next iteration o...
Sid Meier’s Civilization VI is the next entry in the popular Civilization franchise. Originally created by legendary game designer Sid Meier, Civilization is a strategy game in which you attempt to... Read more
Google Chrome 55.0.2883.75 - Modern and...
Google Chrome is a Web browser by Google, created to be a modern platform for Web pages and applications. It utilizes very fast loading of Web pages and has a V8 engine, which is a custom built... Read more
Chromium 55.0.2883.75 - Fast and stable...
Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all Internet users to experience the web. Version 55.0.2883.75: Security fixes: High CVE-2016... Read more

Latest Forum Discussions

See All

Amateur Surgeon 4 Guide: Become the worl...
It's time to wield your trusty pizza cutter again, as Amateur Surgeon has returned with a whole fresh set of challenges (and some old, familiar ones, too). Starting anew isn't easy, especially when all you have at your disposal is a lighter, the... | Read more »
Le Parker: Sous Chef Extraordinaire (Ga...
Le Parker: Sous Chef Extraordinaire 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: | Read more »
Telltale Games really is working on a Gu...
Telltale Games' next episodic adventure is indeed Guardians of the Galaxy. A document tied to the voice actors strike suggested that the project was in the work, but now we have direct confirmation following an announcement at the Game Awards that... | Read more »
Amateur Surgeon returns to iOS and Andro...
Amateur Surgeon and its two sequels disappeared from the App Store some time and it was sad days for all. But now, just in time for the holidays, the Adult Swim favorite makes its joyous return in the shape of Amateur Surgeon 4, a remake with... | Read more »
The best board games on mobile
Sometimes you need to ditch all of the high speed, high action games in favor of something a little more traditional. If you don't feel like parting ways from your mobile device, though, there are still plenty of ways to get that old-school fix.... | Read more »
The best Facebook Messenger Instant Game...
Facebook's new Instant Games is now here, meaning you can play games with your friends directly via Facebook. It's a fun new way to connect with friends, of course, but it's also proving to be a solid gaming experience in its own right, with a... | Read more »
You can now play game's on Facebook...
Facebook launched its new Instant Games platform in an exciting new attempt to engage its user base. As a result, you can now play a number of different games directly through Facebook Messenger. All of these games run with HTML5, meaning you play... | Read more »
Apollo Justice Ace Attorney (Games)
Apollo Justice Ace Attorney 1.00.00 Device: iOS Universal Category: Games Price: $.99, Version: 1.00.00 (iTunes) Description: Court Is Back In Session Star as rookie defense attorney, Apollo Justice, as he visits crime scenes,... | Read more »
KORG iWAVESTATION (Music)
KORG iWAVESTATION 1.0 Device: iOS Universal Category: Music Price: $19.99, Version: 1.0 (iTunes) Description: A revolutionary new world of sound.The Wave Sequence Synthesizer for iPad - KORG iWAVESTATION | Read more »
Don't Grind Guide: Tips for becomin...
Don’t Grind is a surprising, derpy little one touch game with fun hand-drawn graphics. The goal is simple -- get the high score without being chopped to bits. That can be tough when you’re not used to the game, and that’s compounded by the fact... | Read more »

Price Scanner via MacPrices.net

13-inch Silver Touch Bar MacBook Pro in stock...
Amazon has the new 2016 13″ 2.9GHz/256GB Silver Touch Bar MacBook Pro (MLVP2LL/A) in stock today and on sale for $1749 including free shipping. That’s $50 off MSRP, and it’s the lowest price... Read more
Parallels Toolbox 1.3 for Mac Offers 25 Singl...
Parallels has launched Parallels Toolbox 1.3 for Mac, an upgrade that adds five new utilities to the stand-alone application which was released in August and is available exclusively online at http... Read more
OWC Mercury Elite Pro Dual mini Ultra-Portabl...
OWC has introduced the new OWC Mercury Elite Pro Dual mini, a powerful yet ultra-portable dual-drive RAID solution. The new Mercury Elite Pro Dual mini packs phenomenal performance into a small... Read more
Clearance 13-inch Retina MacBook Pros availab...
B&H Photo has clearance 2015 13″ Retina Apple MacBook Pros available for up to $200 off original MSRP. Shipping is free, and B&H charges NY tax only: - 13″ 2.7GHz/128GB Retina MacBook Pro: $... Read more
Roundup of 2016 13-inch 2.0GHz MacBook Pro sa...
B&H has the non-Touch Bar 13″ MacBook Pros in stock today for $50-$100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.0GHz MacBook Pro Space Gray (MLL42LL/A): $1449 $... Read more
New 13-inch 2.0GHz Space Gray MacBook Pro in...
Adorama has the new 13″ 2.0GHz Space Gray MacBook Pro (non-Touch Bar, MLL42LL/A) in stock for $1499 including a free 3-year AppleCare Protection Plan. Shipping is free, and Adorama charges sales tax... Read more
Finnair Adopts iOS Enterprise iPad Apps from...
Finnair and IBM have announced a first-of-its-kind agreement to utilize iOS enterprise apps from IBM to support the airline’s overall digital transformation. Finnair is focused on Asia-Europe traffic... Read more
Tech21 Launches Evo Go iPhone 7 Case Availabl...
Tech21 has announced the launch of the Evo Go case for Apple iPhone 7 and iPhone 7 Plus, exclusively at T-Mobile. Available online and at participating T-Mobile stores nationwide, Evo Go cases start... Read more
Apple Turns (RED) with More Ways to Join the...
In recognition of World AIDS Day, Apple is offering more ways than ever for customers to join (RED) in its mission to create an AIDS-free generation. Apple is the worlds largest corporate contributor... Read more
Deals on new 15-inch Touch Bar MacBook Pros,...
B&H Photo has new 2016 Apple 15″ Touch Bar MacBook Pro models in stock today with some available for $50 off MSRP, each including free shipping plus NY sales tax only: - 15″ 2.7GHz Touch Bar... Read more

Jobs Board

*Apple* Brand Ambassador (Macy's) - The...
…(T-ROC), is proud of its unprecedented relationship with our partner and client, APPLE ,in bringing amazing" APPLE ADVOCATES"to "non" Apple store locations. Read more
US- *Apple* Store Leader Program - Apple (Un...
…Summary Learn and grow as you explore the art of leadership at the Apple Store. You'll master our retail business inside and out through training, hands-on Read more
*Apple* Retail - Multiple Positions- White P...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Automotive Detailer - *Apple* Used Autos -...
We are currently conductinginterviews and will be accepting applications for a part-time detailer. Apple Used Autos is a great place to work andstart a career. We Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.