TweetFollow Us on Twitter

Office 2004 Benchmarks on Intel-based Macs

Volume Number: 22 (2006)
Issue Number: 5
Column Tag: MacTech Labs

Office 2004 Benchmarks on Intel-based Macs

How well does Office 2004 run on Rosetta technology?

by the MacTech Editorial Staff

The Big Question

If you are a Microsoft Office user on the Mac, there's likely a question on your mind if you are considering purchasing a new Mac based on the Intel processor. You see, currently, Microsoft Office is not "Universal." In other words, it runs on top of Rosetta (Apple's technology to dynamically translate PowerPC-based applications to work on Intel-based Macs). Now, Microsoft has announced that the next version of Office will be Universal, but no one expects this imminently. (It's a big job, Apple released their Intel-based machines earlier than expected, and Office isn't borne from Xcode.)

The big question is this: Does Microsoft Office 2004 run well enough on the new Intel-based Macs? Or should you delay your purchase of these machines?

The Test Bench

We chose three machines to compare. Our baseline machine is a PowerBook G4 15-inch, running a 1.5 GHz PowerPC G4, with 2 GB RAM, and an 80 GB/5400 rpm hard disk.

We compared a MacBook Pro 15-inch, running a 1.83 GHz Intel Core Duo, with 2 GB RAM, and a 120GB/5400 rpm hard disk.

And finally, we compared an iMac 20-inch, running a 2.0 GHz Intel Core Duo, 1.5 GB RAM, and a 250GB 7200-rpm Serial ATA hard disk.

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, measured in seconds, see Appendix B. Both appendices are available on the MacTech web site.

Overview

We won't keep you in suspense. In general, Office 2004 under Rosetta works "well enough" to "very well," and in some cases, it's even faster than on the PowerPC machine.

To determine this, MacTech ran over a thousand tests across three models of Macs, and the four major Office applications: Word, Excel, PowerPoint and Entourage. And, since graphics code is shared between Office applications, we ran a suite of graphics tests as well. These are each covered in more detail below.

In one of the most critical set of tests, we specifically looked at whether the user could type or interact faster than Office could keep up, and even in the slowest of scenarios, we never found the user waiting for typing, or other interactions like selecting menus. Even when typing at over 100 wpm, Word was able to stay ahead of the user.

Of the four applications, PowerPoint, is the one that struggled the most. It appears this is due to Office graphics engine shared by all of the Office applications.

At the other end of the spectrum, Entourage was not only on par, it was faster in many cases than our PowerPC baseline. In fact, with the exception of launching the application, Entourage was faster across the board on the Intel iMac, while the MacBook Pro was about on par with the PowerBook G4 (slightly faster in some cases, slightly slower in others).

What was interesting is that even on those tests that were slow, they were still acceptable. For example, the test with the worst performance was inserting a large JPG (over 10mb) into a document was considerably slower. In most cases, a 10 megabyte JPEG would be larger than an 8.5x11 page. How often do users insert large JPEGs in a single sitting? And, when they do, does it matter when that something takes 10 seconds more? Not in our opinion ... and we use Word a lot at MacTech.

It appeared to us that the more modern the application, the better it did under Rosetta. Furthermore, the more that something used the underlying Mac OS, the better it did as well. For example, Entourage did very well with networking related items. Raw imports and opening did significantly better especially when a faster disk was involved.

And, the more times you did a function, the better it performed especially on the second iteration of a command. While it's difficult to confirm, this is due to a combination of code working smarter with caches, both in Rosetta as well as within the Office code base. An obvious benefit for those tasks that are most sensitive to time: the repetitive ones.

The Test Suite and Results

The tests used were selected specifically to give a real-world view of what Microsoft Office 2004 is like to run. We eliminated those tests that we ran that were so short a time frame (e.g., fast) that we could not create statistically significant results, or that had imperceivable differences.

We did one test suite for each of the four major applications in Office, as well as a series of tests focused on the graphics libraries. As you may or may not know, Office 2004 uses a common set of graphics libraries across the applications in Office. With that in mind, we considered a set of tests that would test these technologies, and they are a good representation across all the applications in Office. For example, when you import a graphic, it should be the same across the Office suite.

To give users a good idea of what it's like to use each of these apps, we came up with a list of tests that represented what we felt were the most relevant to regular use. For example, we ran tests on application launches, but did not focus on them since it's something that people tend to do only a couple of times a day. But, we did include the most repetitive of tasks within an application, as those affect productivity most.

The one thing that we did see across the board is that the Intel iMac is consistently faster than the MacBook Pro. Since the iMac had slightly less RAM, and the processor and front side bus speeds are the same, we looked further inside. The iMac has some pretty serious sub-systems design to make it a screaming machine (see <http://developer.apple.com/documentation/HardwareDrivers/Conceptual/iMac_06Jan/>. When you take into account that an iMac has SATA (instead of ATA), a faster hard drive (a 7200 vs. the MacBook Pro's 5400 rpm), and a variety of optimized sub-systems that would be more difficult to implement on a laptop, it's easy to see why the iMac is faster.

It's important to realize that many of the actions that users do when using these applications are so fast already, that even a degradation of 50% may not even be noticeable for most tasks. And in the non-repetitive tasks, they are nearly irrelevant. For example, if when launching an application, you have to wait several seconds, many users will notice, but it won't matter to their overall productivity.

Other areas, like repetitive tasks or editing actions, are far more important, and speed makes a great deal more difference in not only perception, but in productivity.

An overview of tests that best represents productivity is displayed in Figure 1. These metrics include a variety of tests normalized to give you an overview. Smaller bars are faster (better). Information on what is included in each of these overview metrics, as well as additional testing information and graphics are shown later for each application.




Figure 1: Office 2004, Average User Tests


Word

Of all the applications, Word is probably the most widely used by people. The most relevant tests to most Word users are those that are repeated throughout the use of Word. For productivity tests, we selected a variety of scrolling tests, saving, word count, find & replace, opening files, pasting and printing.

The end result is that, for productivity tasks, Word on MacBook Pro performed at 82% of the speed of a PowerBook G4, and on an Intel iMac at 92%. In a real life scenario, these numbers put Word at working pretty well. There's not much a perceptual difference for most tasks.

Our top level grouping included a application launching, variety of scrolling tests, saving, word counts, find & replace, opening documents, pasting and printing. The results are in Figure 2.




Figure 2: Word Benchmark Results


But, of course, the most important of the tests - typing - were so fast that we couldn't measure them. No matter what the computer model, Word was ahead of the typist (even at 100 wpm).

The areas that Word performed "well enough" in are the initial launch of the application launching and printing. Fortunately, these are things that most users do relatively infrequently and don't materially affect productivity.

The balance of the tests ran on par, or in some cases faster on the Intel iMac. As is consistent throughout, the Intel iMac was faster than the MacBook Pro, and in some cases (most notably saving and pasting), the iMac was faster than the PowerBook baseline.

Our one and only crash during the entire Office testing set was during a find and replace in Word. A quick re-launch, and everything proceeded as expected.

The end result is that while there are a couple of areas that a user may notice it's slower, Word works well on a MacBook Pro; and, on the Intel iMac, it works very well, under Rosetta.


Excel

For Excel, we wanted to focus again on the most repetitive tasks. For productivity tests, we selected fill range, scrolling vertically, zooming out, arranging windows, subtotals, auto-formatting, and editing in-cell. On average for the productivity tests, Excel performed at 83% of baseline for the MacBook Pro, and 91% for the Intel iMac.

Most users use relatively small spreadsheets, but it's important to take a look at what a more experienced spreadsheeter may experience. As a general rule, editing and calculations were very fast and in some cases, faster than on the baseline PowerPC.

For our top level testing, we looked at the common commands including fill range, vertical scroll, zoom out, arranging windows, subtotals, auto-format, and editing in-cell. The results are displayed in Figure 3.




Figure 3: Excel Benchmark Results


Scrolling did leave a bit to be desired, but it definitely worked well enough, even on the slower MacBook Pro. In our tests, we used a very large Excel document, and even so, the test ranges were 6.18 to 10.15 seconds. Certainly enough to perceive, but given the very large size of the test document, under normal circumstances, it's not enough to affect productivity.

The end result is that for Excel on a MacBook Pro, it works very well, and on the Intel iMac, even better. Users may notice slower scrolling, but that's about it as far as the most common tasks.


PowerPoint

For PowerPoint, we chose a variety of tests to represent productivity including opening, saving, scrolling slides in the sorter view, adding slides, viewing slide shows, slide transitions, text animations, printing, inserting images from the ClipArt Gallery, changing color schemes and applying templates. As we already mentioned, PowerPoint was the slowest of the Office suite, but in our most indicative tests for productivity, it did perform at 77% of the speed for the MacBook Pro, and nearly 90% for the Intel iMac.

In our top level testing, we took a look at launching, opening files, saving, scrolling slides in sorter view, adding slides, viewing slide transitions, viewing text animations, printing, changing color schemes and applying templates. The results are shown in Figure 4.




Figure 4: PowerPoint Benchmark Results


The areas PowerPoint was weakest were with complex files, printing and applying templates. That said, PowerPoint did pretty well with view slides and transitions, animations, and scrolling in the sorter view.

The bottom line is that even if you use PowerPoint regularly, it definitely works well enough for most people, and in the most important areas ... the presentation itself ... it works very well. If you are a heavy PowerPoint user, you will notice sluggishness, and you should take a closer look.


Entourage

For Entourage, the productivity tests included IMAP account sync, empty deleted items, opening messages, pop message download, opening folders, sending messages via SMTP, sorting and grouping 2000 messages, and searching the address book, mail and tasks. For the productivity tests, Entourage shined with the MacBook Pro performing at 92% of the baseline speed, and the Intel iMac performing 14% faster than the baseline PowerPC.

In our top level testing, we took a look at launch time, empty deleted items, POP message download, sending SMTP messages, sorting and grouping thousands of messages, and searching the address book, mail and tasks. Results are shown in Figure 5.




Figure 5: Entourage Benchmark Results


Like Word, the most important of the tests, typing, were so fast that we couldn't measure them regardless of which test machine.

Boot time was clearly slower - about double in fact. But in a game of seconds, and for something that you generally only do a couple of times a day, it's not particularly relevant.

Entourage is clearly the best performer in the suite, presumably because Entourage is the most modern code base of the Office 2004 suite, and because it relies the most on Mac OS X technologies that have already been made Universal.

The end result is that, across the board, Entourage under Rosetta performed terrifically. The MacBook Pro usually performed as good or better than the PowerBook. The Intel iMac was faster in almost every test.


Microsoft Office Graphics

The weakest part of Office under Rosetta is the graphics code shared across Microsoft Office. PowerPoint shows this most given its nature.

In reality, however, this affects the overall Office experience relatively little. Furthermore, for the graphics libraries, our tests were designed to point out weaknesses and give us measurable results. But let's be real: how often do you import a 10-megabyte JPEG? Most JPEG's that size would be much larger than a full page.

That said, we focused on the tests on inserting large graphics, opening, saving, printing, working with the WordArt features, and the ClipArt library.

The end result is that inserting JPEG's are slow, and we saw the impact across all the Office applications. Fortunately, most users don't tend to do this repetitively. Other formats like inserting EPS and PSD files did much better. In other words, you'll have no issues with your logos and template graphics.

Opening graphics files was ok, but fortunately, saving files was significantly faster (a good thing since users typically save much more often than open files). The ClipArt interfaces work very well including the importing and searching.




Figure 6: Office Graphics Benchmark Results


Conclusion

As we stated at the beginning of this article, in general, Office 2004 under Rosetta works "well enough" to "very well," and in some cases, it's even faster than on the PowerPC baseline machine.

Given the amount we were pushing these apps, we were thoroughly pleased with the stability of the entire Office suite of applications. Considering how complex Rosetta is, and how big a code base Office is, it's entirely remarkable how stable we found it. Furthermore, we expect to see Apple further optimize Rosetta, particularly in launch times.

While Microsoft has already announced that it will be making the next version of Office a "Universal" application, Mac users wishing to dive into Intel-based Macs now can rest assured in knowing they can move forward and be patient for the Universal version.





Appendix A: Benchmarking Methodology

Introduction

The purpose of this appendix is to outline the basic parameters for how MacTech Magazine performed benchmarking tests on Microsoft Office 2004 for Intel iMacs and the MacBook Pro compared to a PowerBook G4.

Benchmarking Methodology


Consistency

Since the tests involve both multiple machines and multiple pieces of software, the focus was on creating as much consistency across the tests as possible. MacTech accomplished this in several ways.

First, all timing of tests were performed by a single MacTech staff member so as to eliminate any of the natural inconsistencies that often occur across individuals.

All of the tests were performed on the same version of the Mac operating system across the different hardware. At the time of the tests, this was Mac OS X 10.4.5 and included the most up-to-date versions of Apple patches as prescribed through "Software Update" in Mac OS X.

All of the tests were done on "virgin" systems, i.e., freshly wiped hard disks, with fresh Mac OS X and Office installations, with no third party software installed beyond the standard Apple and Microsoft Office installations.

All of the tests were performed with the most up to date set of patches for Microsoft Office as prescribed by Microsoft's Auto-updater. At the time of the test, this included the Microsoft Office 2004 for Mac 11.2.3 Update.


Avoiding Interactions

While the tests covered the bulk of the Microsoft Office suite, all tests (where appropriate) were performed with only that single application within Microsoft Office open. In other words, to the extent possible, no other applications will be running. (Obviously excluding background and OS tasks that are part of a standard install of Mac OS X or Office.)

To avoid issues with a noisy network, the test machines were installed on what was considered a "quiet" network with minimal traffic. MacTech monitored the use of the network to make sure that the machine does have network access, but is not impacted by the network.


Measurements, Testing and Outliers

For timed tests with results under 15 seconds, tests were measured to within 1/10th of a second. For those over 15 seconds, tests were measured to within a second.

Most tests were performed at least three times per test and per machine, and up to 5 times depending on the test. On average, tests were conducted three to four times per machine per test case. Outliers indicating a testing anomaly were retested as appropriate.

In most cases, tester used successive tests, not "Adam" or "first" tests to better emulate the daily use of Office products.

Those tests that could be impacted by the size of the window, were tested with the same window size under all scenarios.

Some tests were eliminated because the machines simply performed too fast to get an accurate measurement. For example, typing always displayed faster on the machine than the tester could type (even at 100 wpm).





Appendix B: Testing Results

Introduction

The below chart contains the raw data for the tests selected to judge Office 2004 applications. All results are in seconds, and represent the best time for each test on each machine.

                                   PowerBook G4   MacBook Pro   Intel iMac
Word:
Scroll Real World Doc                      8.12          9.47         8.34
Scroll Doc with Tables                     3.47          4.13         3.91
Scroll Doc with Graphics                   5.50          7.22         6.72
Save As Word Doc                           1.19          1.22         1.12
Word Count                                 1.69          2.22         1.84
Find & Replace                             0.85          0.93         0.88
Open Word Doc                              1.28          1.59         1.34
Paste                                      1.84          1.94         1.75
Print a Word Doc                           7.24         11.00        10.10

Excel:
Fill Range                                 1.41          1.66         1.68
Scroll Vertical                            6.18         10.15         8.97
Zoom Out                                   1.25          1.56         1.47
Window Arrange                             2.25          2.56         2.15
Subtotals                                  1.44          1.47         1.16
AutoFormat                                 0.94          1.06         1.00
Edit in-cell                               1.44          1.57         1.47

PowerPoint:
Open typical simple file                   1.47          1.78         1.56
Save complex file                          1.40          2.10         1.78
Scroll slide in sorter view                2.72          3.25         3.00
Add slides in sorter view                  2.16          2.56         2.47
View slide show                            1.19          1.12         1.16
View slide transitions                     0.82          0.85         0.88
View text animations                       0.81          0.85         0.66
Print simple file                          1.53          2.25         1.94
Insert from Clip Art Gallery               1.34          1.94         1.50
Change color scheme                        0.88          1.13         0.72
Apply Template                             2.59          4.88         4.38

Entourage:
IMAP Account Sync                          1.84          2.31         1.57
Empty Deleted Items                        5.25          6.50         4.06
Open Messages                              3.03          4.46         3.65
POP Message Download                      63.00         73.00        57.00
Open Folder                                0.69          0.69         0.56
Sending Messages SMTP                      1.35          1.19         1.16
Sorting 2000 messages                      0.81          0.81         0.78
Grouping 2000 messages                     1.78          1.65         1.50
Address Book Search                        1.06          1.06         0.85
Mail Search                                0.56          0.59         0.53
Task Search                                0.68          0.62         0.47

Graphics:
Insert Large JPG                           2.40         12.22        11.12
Insert Large EPS                          25.56         37.81        21.87
Open Document                              1.19          1.94         1.68
Save Document                              2.81          3.59         2.07
Print Document                            55.00        100.00        91.00
WordArt: Insert                            0.84          1.16         1.06
ClipArt: Import                           98.00         89.00        49.00
ClipArt: Search                            0.50          0.53         0.44




 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Is GO Gear the Pokemon GO map app you...
Now that we've settled into something of a Pokemon GO status quo, the number one desire of most players can best be summed out by modifying a quote from Rod Tidwell of Jerry Maguire: "Show me the Pokemon!" [Read more] | Read more »
Rodeo Stampede update: Mountains, new an...
The Savannah and Jungle were just the beginning in Rodeo Stampede. Get ready to head for the Mountains. I think I heard that in a beer ad once. [Read more] | Read more »
COSMOS RINGS (Games)
COSMOS RINGS 1.0.0 Device: iOS iPhone Category: Games Price: $5.99, Version: 1.0.0 (iTunes) Description: This game cannot be played without the Apple Watch.Released anniversary sale until August 31,2016 PST! A tragic tale of time's... | Read more »
How to get started selling on Mercari
As far as ecommerce has come over the last decade or so, there's still a tremendous opportunity to make it easier for people to buy and sell goods. That's especially true when it comes to shopping apps, which should only continue to increase in... | Read more »
Human Anatomy Atlas 2017 Edition - Compl...
Human Anatomy Atlas 2017 Edition - Complete 3D Human Body 1.0.24 Device: iOS iPhone Category: Medical Price: $24.99, Version: 1.0.24 (iTunes) Description: | Read more »
Heroes of Normandie (Games)
Heroes of Normandie 1.5 Device: iOS Universal Category: Games Price: $14.99, Version: 1.5 (iTunes) Description: The game does not support iPhone 4s and below | Read more »
Why you should never power up Pokemon in...
There's no question that candy is dandy in Pokemon GO. You need big quantities of it to evolve your Pokemon, and when combined with stardust, it can be used to power up your favorite pocket monsters as well, making them more formidable for the gym... | Read more »
Webzen launches 3D MMORPG MU Origin on i...
Mu Origin is featured time and time again at the very top of App Stores in China, and within the top five worldwide top-grossing charts on Google Play.Its popularity in Korea and China, featuring more than 120 registered players in China and 6... | Read more »
Severed (Games)
Severed 1.0 Device: iOS Universal Category: Games Price: $5.99, Version: 1.0 (iTunes) Description: LAUNCH DISCOUNT ON NOW!! ENDS AUGUST 4! ==== Take control of a one-armed warrior named Sasha, wielding a living sword on her journey... | Read more »
CSR Racing 2: How to master Live Races
Getting better at racing the AI in CSR Racing 2 is a pretty straightforward process. You run enough races with a particular car to get its shift timing down, continually improve it by buying upgrades whenever you can, and add any fusion parts you... | Read more »

Price Scanner via MacPrices.net

Second-Quarter Tablet Shipments Fell 4.8% –...
The latest report from the global market research firm TrendForce finds that worldwide tablet shipments for this second quarter totaled 33.54 million units, representing a quarterly drop of 4.8% and... Read more
Global Smartphone Sales Volumes Mark Second S...
According to preliminary results from the International Data Corporation (IDC) Worldwide Quarterly Mobile Phone Tracker, vendors shipped a total of 343.3 million smartphones worldwide in the second... Read more
Apple TVs on sale for $20-$40 off MSRP
Best Buy has 32GB and 64GB Apple TVs on sale for $20-$40 off MSRP on their online store. Choose free shipping or free local store pickup (if available). Sale prices for online orders only, in-store... Read more
Mac minis on sale for $50-$100 off MSRP
B&H Photo has Mac minis on sale for $50 off MSRP including free shipping plus NY sales tax only: - 1.4GHz Mac mini: $449 $50 off MSRP - 2.6GHz Mac mini: $649 $50 off MSRP - 2.8GHz Mac mini: $949... Read more
Clearance 2015 13-inch MacBook Airs available...
B&H Photo has clearance 2015 13″ MacBook Airs available for $300 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 1.6GHz/4GB/128GB MacBook Air (MJVE2LL/A): $799... Read more
Apple certified refurbished iPad mini 4s avai...
Apple has certified refurbished iPad mini 4s now available for up to $120 off the cost of new models. An Apple one-year warranty is included with each iPad, and shipping is free. The following models... Read more
Notebook Makers In No Rush To Adopt USB-C – R...
Digitimes’ Cage Chao and Joseph Tsai note that while the USB Type-C interface is enjoying growing popularity among smartphones and tablet makers, notebook and all-in-one (AIO) PC vendors (other than... Read more
iMacs on sale for up to $250 off MSRP
B&H Photo has 21″ and 27″ Apple iMacs on sale for up to $250 off MSRP including free shipping plus NY sales tax only: - 27″ 3.3GHz iMac 5K: $2049 $250 off MSRP - 27″ 3.2GHz/1TB Fusion iMac 5K: $... Read more
12-inch 1.1GHz Retina MacBooks on sale for up...
Amazon has 2016 12″ 1.1GHz/256GB Retina MacBooks on sale for up to $100 off MSRP including free shipping: - 12″ 1.1GHz Space Gray Retina MacBook: $1199 $100 off MSRP - 12″ 1.1GHz Silver Retina... Read more
Bare Bones Software Releases Free TextWrangle...
Bare Bones Software has announced the release and immediate availability of TextWrangler 5.5, a significant update to its powerful, free, general purpose text editor for Mac OS X. TextWrangler is a... Read more

Jobs Board

*Apple* Solutions Consultant - APPLE (United...
Job Summary As an Apple Solutions Consultant, you'll be the link between our future customers and our products. You'll showcase your entrepreneurial spirit as you Read more
Lead *Apple* Solutions Consultant - APPLE (...
Job Summary The Lead ASC is an Apple employee who serves as the Apple business manager and influencer across a number of Reseller locations. The Lead ASC's role Read more
*Apple* Retail - Multiple Positions, Charles...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Retail - Multiple Positions, Willow...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Evangelist - JAMF Software (United S...
The Apple Evangelist is responsible for building and cultivating strategic relationships with Apple 's small and mid-market business development field teams. This Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.