TweetFollow Us on Twitter

Office 2004 Benchmarks on Intel-based Macs

Volume Number: 22 (2006)
Issue Number: 5
Column Tag: MacTech Labs

Office 2004 Benchmarks on Intel-based Macs

How well does Office 2004 run on Rosetta technology?

by the MacTech Editorial Staff

The Big Question

If you are a Microsoft Office user on the Mac, there's likely a question on your mind if you are considering purchasing a new Mac based on the Intel processor. You see, currently, Microsoft Office is not "Universal." In other words, it runs on top of Rosetta (Apple's technology to dynamically translate PowerPC-based applications to work on Intel-based Macs). Now, Microsoft has announced that the next version of Office will be Universal, but no one expects this imminently. (It's a big job, Apple released their Intel-based machines earlier than expected, and Office isn't borne from Xcode.)

The big question is this: Does Microsoft Office 2004 run well enough on the new Intel-based Macs? Or should you delay your purchase of these machines?

The Test Bench

We chose three machines to compare. Our baseline machine is a PowerBook G4 15-inch, running a 1.5 GHz PowerPC G4, with 2 GB RAM, and an 80 GB/5400 rpm hard disk.

We compared a MacBook Pro 15-inch, running a 1.83 GHz Intel Core Duo, with 2 GB RAM, and a 120GB/5400 rpm hard disk.

And finally, we compared an iMac 20-inch, running a 2.0 GHz Intel Core Duo, 1.5 GB RAM, and a 250GB 7200-rpm Serial ATA hard disk.

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, measured in seconds, see Appendix B. Both appendices are available on the MacTech web site.

Overview

We won't keep you in suspense. In general, Office 2004 under Rosetta works "well enough" to "very well," and in some cases, it's even faster than on the PowerPC machine.

To determine this, MacTech ran over a thousand tests across three models of Macs, and the four major Office applications: Word, Excel, PowerPoint and Entourage. And, since graphics code is shared between Office applications, we ran a suite of graphics tests as well. These are each covered in more detail below.

In one of the most critical set of tests, we specifically looked at whether the user could type or interact faster than Office could keep up, and even in the slowest of scenarios, we never found the user waiting for typing, or other interactions like selecting menus. Even when typing at over 100 wpm, Word was able to stay ahead of the user.

Of the four applications, PowerPoint, is the one that struggled the most. It appears this is due to Office graphics engine shared by all of the Office applications.

At the other end of the spectrum, Entourage was not only on par, it was faster in many cases than our PowerPC baseline. In fact, with the exception of launching the application, Entourage was faster across the board on the Intel iMac, while the MacBook Pro was about on par with the PowerBook G4 (slightly faster in some cases, slightly slower in others).

What was interesting is that even on those tests that were slow, they were still acceptable. For example, the test with the worst performance was inserting a large JPG (over 10mb) into a document was considerably slower. In most cases, a 10 megabyte JPEG would be larger than an 8.5x11 page. How often do users insert large JPEGs in a single sitting? And, when they do, does it matter when that something takes 10 seconds more? Not in our opinion ... and we use Word a lot at MacTech.

It appeared to us that the more modern the application, the better it did under Rosetta. Furthermore, the more that something used the underlying Mac OS, the better it did as well. For example, Entourage did very well with networking related items. Raw imports and opening did significantly better especially when a faster disk was involved.

And, the more times you did a function, the better it performed especially on the second iteration of a command. While it's difficult to confirm, this is due to a combination of code working smarter with caches, both in Rosetta as well as within the Office code base. An obvious benefit for those tasks that are most sensitive to time: the repetitive ones.

The Test Suite and Results

The tests used were selected specifically to give a real-world view of what Microsoft Office 2004 is like to run. We eliminated those tests that we ran that were so short a time frame (e.g., fast) that we could not create statistically significant results, or that had imperceivable differences.

We did one test suite for each of the four major applications in Office, as well as a series of tests focused on the graphics libraries. As you may or may not know, Office 2004 uses a common set of graphics libraries across the applications in Office. With that in mind, we considered a set of tests that would test these technologies, and they are a good representation across all the applications in Office. For example, when you import a graphic, it should be the same across the Office suite.

To give users a good idea of what it's like to use each of these apps, we came up with a list of tests that represented what we felt were the most relevant to regular use. For example, we ran tests on application launches, but did not focus on them since it's something that people tend to do only a couple of times a day. But, we did include the most repetitive of tasks within an application, as those affect productivity most.

The one thing that we did see across the board is that the Intel iMac is consistently faster than the MacBook Pro. Since the iMac had slightly less RAM, and the processor and front side bus speeds are the same, we looked further inside. The iMac has some pretty serious sub-systems design to make it a screaming machine (see <http://developer.apple.com/documentation/HardwareDrivers/Conceptual/iMac_06Jan/>. When you take into account that an iMac has SATA (instead of ATA), a faster hard drive (a 7200 vs. the MacBook Pro's 5400 rpm), and a variety of optimized sub-systems that would be more difficult to implement on a laptop, it's easy to see why the iMac is faster.

It's important to realize that many of the actions that users do when using these applications are so fast already, that even a degradation of 50% may not even be noticeable for most tasks. And in the non-repetitive tasks, they are nearly irrelevant. For example, if when launching an application, you have to wait several seconds, many users will notice, but it won't matter to their overall productivity.

Other areas, like repetitive tasks or editing actions, are far more important, and speed makes a great deal more difference in not only perception, but in productivity.

An overview of tests that best represents productivity is displayed in Figure 1. These metrics include a variety of tests normalized to give you an overview. Smaller bars are faster (better). Information on what is included in each of these overview metrics, as well as additional testing information and graphics are shown later for each application.




Figure 1: Office 2004, Average User Tests


Word

Of all the applications, Word is probably the most widely used by people. The most relevant tests to most Word users are those that are repeated throughout the use of Word. For productivity tests, we selected a variety of scrolling tests, saving, word count, find & replace, opening files, pasting and printing.

The end result is that, for productivity tasks, Word on MacBook Pro performed at 82% of the speed of a PowerBook G4, and on an Intel iMac at 92%. In a real life scenario, these numbers put Word at working pretty well. There's not much a perceptual difference for most tasks.

Our top level grouping included a application launching, variety of scrolling tests, saving, word counts, find & replace, opening documents, pasting and printing. The results are in Figure 2.




Figure 2: Word Benchmark Results


But, of course, the most important of the tests - typing - were so fast that we couldn't measure them. No matter what the computer model, Word was ahead of the typist (even at 100 wpm).

The areas that Word performed "well enough" in are the initial launch of the application launching and printing. Fortunately, these are things that most users do relatively infrequently and don't materially affect productivity.

The balance of the tests ran on par, or in some cases faster on the Intel iMac. As is consistent throughout, the Intel iMac was faster than the MacBook Pro, and in some cases (most notably saving and pasting), the iMac was faster than the PowerBook baseline.

Our one and only crash during the entire Office testing set was during a find and replace in Word. A quick re-launch, and everything proceeded as expected.

The end result is that while there are a couple of areas that a user may notice it's slower, Word works well on a MacBook Pro; and, on the Intel iMac, it works very well, under Rosetta.


Excel

For Excel, we wanted to focus again on the most repetitive tasks. For productivity tests, we selected fill range, scrolling vertically, zooming out, arranging windows, subtotals, auto-formatting, and editing in-cell. On average for the productivity tests, Excel performed at 83% of baseline for the MacBook Pro, and 91% for the Intel iMac.

Most users use relatively small spreadsheets, but it's important to take a look at what a more experienced spreadsheeter may experience. As a general rule, editing and calculations were very fast and in some cases, faster than on the baseline PowerPC.

For our top level testing, we looked at the common commands including fill range, vertical scroll, zoom out, arranging windows, subtotals, auto-format, and editing in-cell. The results are displayed in Figure 3.




Figure 3: Excel Benchmark Results


Scrolling did leave a bit to be desired, but it definitely worked well enough, even on the slower MacBook Pro. In our tests, we used a very large Excel document, and even so, the test ranges were 6.18 to 10.15 seconds. Certainly enough to perceive, but given the very large size of the test document, under normal circumstances, it's not enough to affect productivity.

The end result is that for Excel on a MacBook Pro, it works very well, and on the Intel iMac, even better. Users may notice slower scrolling, but that's about it as far as the most common tasks.


PowerPoint

For PowerPoint, we chose a variety of tests to represent productivity including opening, saving, scrolling slides in the sorter view, adding slides, viewing slide shows, slide transitions, text animations, printing, inserting images from the ClipArt Gallery, changing color schemes and applying templates. As we already mentioned, PowerPoint was the slowest of the Office suite, but in our most indicative tests for productivity, it did perform at 77% of the speed for the MacBook Pro, and nearly 90% for the Intel iMac.

In our top level testing, we took a look at launching, opening files, saving, scrolling slides in sorter view, adding slides, viewing slide transitions, viewing text animations, printing, changing color schemes and applying templates. The results are shown in Figure 4.




Figure 4: PowerPoint Benchmark Results


The areas PowerPoint was weakest were with complex files, printing and applying templates. That said, PowerPoint did pretty well with view slides and transitions, animations, and scrolling in the sorter view.

The bottom line is that even if you use PowerPoint regularly, it definitely works well enough for most people, and in the most important areas ... the presentation itself ... it works very well. If you are a heavy PowerPoint user, you will notice sluggishness, and you should take a closer look.


Entourage

For Entourage, the productivity tests included IMAP account sync, empty deleted items, opening messages, pop message download, opening folders, sending messages via SMTP, sorting and grouping 2000 messages, and searching the address book, mail and tasks. For the productivity tests, Entourage shined with the MacBook Pro performing at 92% of the baseline speed, and the Intel iMac performing 14% faster than the baseline PowerPC.

In our top level testing, we took a look at launch time, empty deleted items, POP message download, sending SMTP messages, sorting and grouping thousands of messages, and searching the address book, mail and tasks. Results are shown in Figure 5.




Figure 5: Entourage Benchmark Results


Like Word, the most important of the tests, typing, were so fast that we couldn't measure them regardless of which test machine.

Boot time was clearly slower - about double in fact. But in a game of seconds, and for something that you generally only do a couple of times a day, it's not particularly relevant.

Entourage is clearly the best performer in the suite, presumably because Entourage is the most modern code base of the Office 2004 suite, and because it relies the most on Mac OS X technologies that have already been made Universal.

The end result is that, across the board, Entourage under Rosetta performed terrifically. The MacBook Pro usually performed as good or better than the PowerBook. The Intel iMac was faster in almost every test.


Microsoft Office Graphics

The weakest part of Office under Rosetta is the graphics code shared across Microsoft Office. PowerPoint shows this most given its nature.

In reality, however, this affects the overall Office experience relatively little. Furthermore, for the graphics libraries, our tests were designed to point out weaknesses and give us measurable results. But let's be real: how often do you import a 10-megabyte JPEG? Most JPEG's that size would be much larger than a full page.

That said, we focused on the tests on inserting large graphics, opening, saving, printing, working with the WordArt features, and the ClipArt library.

The end result is that inserting JPEG's are slow, and we saw the impact across all the Office applications. Fortunately, most users don't tend to do this repetitively. Other formats like inserting EPS and PSD files did much better. In other words, you'll have no issues with your logos and template graphics.

Opening graphics files was ok, but fortunately, saving files was significantly faster (a good thing since users typically save much more often than open files). The ClipArt interfaces work very well including the importing and searching.




Figure 6: Office Graphics Benchmark Results


Conclusion

As we stated at the beginning of this article, in general, Office 2004 under Rosetta works "well enough" to "very well," and in some cases, it's even faster than on the PowerPC baseline machine.

Given the amount we were pushing these apps, we were thoroughly pleased with the stability of the entire Office suite of applications. Considering how complex Rosetta is, and how big a code base Office is, it's entirely remarkable how stable we found it. Furthermore, we expect to see Apple further optimize Rosetta, particularly in launch times.

While Microsoft has already announced that it will be making the next version of Office a "Universal" application, Mac users wishing to dive into Intel-based Macs now can rest assured in knowing they can move forward and be patient for the Universal version.





Appendix A: Benchmarking Methodology

Introduction

The purpose of this appendix is to outline the basic parameters for how MacTech Magazine performed benchmarking tests on Microsoft Office 2004 for Intel iMacs and the MacBook Pro compared to a PowerBook G4.

Benchmarking Methodology


Consistency

Since the tests involve both multiple machines and multiple pieces of software, the focus was on creating as much consistency across the tests as possible. MacTech accomplished this in several ways.

First, all timing of tests were performed by a single MacTech staff member so as to eliminate any of the natural inconsistencies that often occur across individuals.

All of the tests were performed on the same version of the Mac operating system across the different hardware. At the time of the tests, this was Mac OS X 10.4.5 and included the most up-to-date versions of Apple patches as prescribed through "Software Update" in Mac OS X.

All of the tests were done on "virgin" systems, i.e., freshly wiped hard disks, with fresh Mac OS X and Office installations, with no third party software installed beyond the standard Apple and Microsoft Office installations.

All of the tests were performed with the most up to date set of patches for Microsoft Office as prescribed by Microsoft's Auto-updater. At the time of the test, this included the Microsoft Office 2004 for Mac 11.2.3 Update.


Avoiding Interactions

While the tests covered the bulk of the Microsoft Office suite, all tests (where appropriate) were performed with only that single application within Microsoft Office open. In other words, to the extent possible, no other applications will be running. (Obviously excluding background and OS tasks that are part of a standard install of Mac OS X or Office.)

To avoid issues with a noisy network, the test machines were installed on what was considered a "quiet" network with minimal traffic. MacTech monitored the use of the network to make sure that the machine does have network access, but is not impacted by the network.


Measurements, Testing and Outliers

For timed tests with results under 15 seconds, tests were measured to within 1/10th of a second. For those over 15 seconds, tests were measured to within a second.

Most tests were performed at least three times per test and per machine, and up to 5 times depending on the test. On average, tests were conducted three to four times per machine per test case. Outliers indicating a testing anomaly were retested as appropriate.

In most cases, tester used successive tests, not "Adam" or "first" tests to better emulate the daily use of Office products.

Those tests that could be impacted by the size of the window, were tested with the same window size under all scenarios.

Some tests were eliminated because the machines simply performed too fast to get an accurate measurement. For example, typing always displayed faster on the machine than the tester could type (even at 100 wpm).





Appendix B: Testing Results

Introduction

The below chart contains the raw data for the tests selected to judge Office 2004 applications. All results are in seconds, and represent the best time for each test on each machine.

                                   PowerBook G4   MacBook Pro   Intel iMac
Word:
Scroll Real World Doc                      8.12          9.47         8.34
Scroll Doc with Tables                     3.47          4.13         3.91
Scroll Doc with Graphics                   5.50          7.22         6.72
Save As Word Doc                           1.19          1.22         1.12
Word Count                                 1.69          2.22         1.84
Find & Replace                             0.85          0.93         0.88
Open Word Doc                              1.28          1.59         1.34
Paste                                      1.84          1.94         1.75
Print a Word Doc                           7.24         11.00        10.10

Excel:
Fill Range                                 1.41          1.66         1.68
Scroll Vertical                            6.18         10.15         8.97
Zoom Out                                   1.25          1.56         1.47
Window Arrange                             2.25          2.56         2.15
Subtotals                                  1.44          1.47         1.16
AutoFormat                                 0.94          1.06         1.00
Edit in-cell                               1.44          1.57         1.47

PowerPoint:
Open typical simple file                   1.47          1.78         1.56
Save complex file                          1.40          2.10         1.78
Scroll slide in sorter view                2.72          3.25         3.00
Add slides in sorter view                  2.16          2.56         2.47
View slide show                            1.19          1.12         1.16
View slide transitions                     0.82          0.85         0.88
View text animations                       0.81          0.85         0.66
Print simple file                          1.53          2.25         1.94
Insert from Clip Art Gallery               1.34          1.94         1.50
Change color scheme                        0.88          1.13         0.72
Apply Template                             2.59          4.88         4.38

Entourage:
IMAP Account Sync                          1.84          2.31         1.57
Empty Deleted Items                        5.25          6.50         4.06
Open Messages                              3.03          4.46         3.65
POP Message Download                      63.00         73.00        57.00
Open Folder                                0.69          0.69         0.56
Sending Messages SMTP                      1.35          1.19         1.16
Sorting 2000 messages                      0.81          0.81         0.78
Grouping 2000 messages                     1.78          1.65         1.50
Address Book Search                        1.06          1.06         0.85
Mail Search                                0.56          0.59         0.53
Task Search                                0.68          0.62         0.47

Graphics:
Insert Large JPG                           2.40         12.22        11.12
Insert Large EPS                          25.56         37.81        21.87
Open Document                              1.19          1.94         1.68
Save Document                              2.81          3.59         2.07
Print Document                            55.00        100.00        91.00
WordArt: Insert                            0.84          1.16         1.06
ClipArt: Import                           98.00         89.00        49.00
ClipArt: Search                            0.50          0.53         0.44




 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Ys Chronicles II (Games)
Ys Chronicles II 1.0.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0.0 (iTunes) Description: After a hard fight to recover the six sacred books in Ys Chronicles I, Adol is back for a sequel! | Read more »
FINAL FANTASY Ⅸ (Games)
FINAL FANTASY Ⅸ 1.0.4 Device: iOS Universal Category: Games Price: $16.99, Version: 1.0.4 (iTunes) Description: ==========●Special sale price for the FINAL FANTASY IX release! ●20% off from February 10 to February 21, 2016... | Read more »
Tennis Club Story (Games)
Tennis Club Story 1.03 Device: iOS Universal Category: Games Price: $4.99, Version: 1.03 (iTunes) Description: Aim for the ace position of tennis club prestige in this simulation! Your leadership decides if players make it to the big... | Read more »
Juggernaut Wars guide - How to use skill...
Juggernaut Warsis a brand new auto-RPG on iOS and Android that challenges you to build a team of heroes, send them out into various different missions to defeat waves of heroes, and level them up to increase their power. The actual combat itself... | Read more »
Check out the new Pirate Attack update i...
Love pirates and board games? Well, you'll love the new Pirate Attack themed update that just launched in Game of Dice. It adds a bunch of new content themed around pirates, like an all new event map based on a pirate ship which revamps the toll... | Read more »
Splash Cars guide - How to paint the tow...
Splash Cars is an arcade driving game that feels like a hybrid between Dawn of the Plow and Splatoon. In it, you'll need to drive a car around to repaint areas of a town that have lost all of their color. Check out these tips to help you perform... | Read more »
The best video player on mobile
We all know the stock video player on iOS is not particularly convenient, primarily because it asks us to hook a device up to iTunes to sync video in a world that has things like Netflix. [Read more] | Read more »
Four apps to help improve your Super Bow...
Super Bowl Sunday is upon us, and whether you’re a Panthers or a Broncos fan you’re no doubt gearing up for it. [Read more] | Read more »
LooperSonic (Music)
LooperSonic 1.0 Device: iOS Universal Category: Music Price: $4.99, Version: 1.0 (iTunes) Description: LooperSonic is a multi-track audio looper and recorder that will take your loops to the next level. Use it like a loop pedal to... | Read more »
Space Grunts guide - How to survive
Space Grunts is a fast-paced roguelike from popular iOS developer, Orange Pixel. While it taps into many of the typical roguelike sensibilities, you might still find yourself caught out by a few things. We delved further to find you some helpful... | Read more »

Price Scanner via MacPrices.net

Apple restocks Certified Refurbished Mac mini...
Apple has restocked Certified Refurbished 2014 Mac minis, with models available starting at $419. Apple’s one-year warranty is included with each mini, and shipping is free: - 1.4GHz Mac mini: $419 $... Read more
What iPad Pro Still Needs To Make It Truly Pr...
I love my iPad Air 2. So much that I’m grudgingly willing to put up with its compromises and limitations as a production tool in order to take advantage of its virtues. However, since a computer for... Read more
21-inch 3.1GHz 4K on sale for $1399, $100 off...
B&H Photo has the 21″ 3.1GHz 4K iMac on sale $1399 for a limited time. Shipping is free, and B&H charges NY sales tax only. Their price is $100 off MSRP: - 21″ 3.1GHz 4K iMac (MK452LL/A): $... Read more
Apple price trackers, updated continuously
Scan our Apple Price Trackers for the latest information on sales, bundles, and availability on systems from Apple’s authorized internet/catalog resellers. We update the trackers continuously: - 15″... Read more
Save up to $240 with Apple Certified Refurbis...
Apple is now offering Certified Refurbished 12″ Retina MacBooks for up to $240 off the cost of new models. Apple will include a standard one-year warranty with each MacBook, and shipping is free. The... Read more
Apple refurbished 13-inch Retina MacBook Pros...
Apple has Certified Refurbished 13″ Retina MacBook Pros available for up to $270 off the cost of new models. An Apple one-year warranty is included with each model, and shipping is free: - 13″ 2.7GHz... Read more
Apple refurbished Time Capsules available for...
Apple has certified refurbished Time Capsules available for $120 off MSRP. Apple’s one-year warranty is included with each Time Capsule, and shipping is free: - 2TB Time Capsule: $179, $120 off - 3TB... Read more
13-inch 2.5GHz MacBook Pro (refurbished) avai...
Apple has Certified Refurbished 13″ 2.5GHz MacBook Pros available for $829, or $270 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 13″ 2.5GHz MacBook Pros... Read more
Apple refurbished 15-inch Retina MacBook Pros...
Apple has Certified Refurbished 2015 15″ Retina MacBook Pros available for up to $380 off the cost of new models. An Apple one-year warranty is included with each model, and shipping is free: - 15″ 2... Read more
New Liquid Crystal Technology Prevents Automo...
Researchers at the University of Central Florida have developed three new liquid crystal mixtures which will allow automobile displays to operate at unprecedented high and low temperatures In... Read more

Jobs Board

Lead Engineer - *Apple* OSX & Hardware...
Lead Engineer - Apple OSX & Hardware **Job ID:** 3125919 **Full/Part\-Time:** Full\-time **Regular/Temporary:** Regular **Listed:** 2016\-02\-10 **Location:** Cary, Read more
*Apple* System Analyst - ATOS IT Services...
Apple System AnalystReference no.198783CountryUSARegionUS - CALIFORNIACityUS - CALIFORNIA - BURBANKPosition TypeProfessionalJob AreaIT SupportJob TypeFull Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions (US) - A...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Subject Matter Expert - Experis (Uni...
This position is for an Apple Subject Matter Expert to assist in developing the architecture, support and services for integration of Apple devices into the domain. Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.