TweetFollow Us on Twitter

Office 2004 Benchmarks on Intel-based Macs

Volume Number: 22 (2006)
Issue Number: 5
Column Tag: MacTech Labs

Office 2004 Benchmarks on Intel-based Macs

How well does Office 2004 run on Rosetta technology?

by the MacTech Editorial Staff

The Big Question

If you are a Microsoft Office user on the Mac, there's likely a question on your mind if you are considering purchasing a new Mac based on the Intel processor. You see, currently, Microsoft Office is not "Universal." In other words, it runs on top of Rosetta (Apple's technology to dynamically translate PowerPC-based applications to work on Intel-based Macs). Now, Microsoft has announced that the next version of Office will be Universal, but no one expects this imminently. (It's a big job, Apple released their Intel-based machines earlier than expected, and Office isn't borne from Xcode.)

The big question is this: Does Microsoft Office 2004 run well enough on the new Intel-based Macs? Or should you delay your purchase of these machines?

The Test Bench

We chose three machines to compare. Our baseline machine is a PowerBook G4 15-inch, running a 1.5 GHz PowerPC G4, with 2 GB RAM, and an 80 GB/5400 rpm hard disk.

We compared a MacBook Pro 15-inch, running a 1.83 GHz Intel Core Duo, with 2 GB RAM, and a 120GB/5400 rpm hard disk.

And finally, we compared an iMac 20-inch, running a 2.0 GHz Intel Core Duo, 1.5 GB RAM, and a 250GB 7200-rpm Serial ATA hard disk.

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, measured in seconds, see Appendix B. Both appendices are available on the MacTech web site.

Overview

We won't keep you in suspense. In general, Office 2004 under Rosetta works "well enough" to "very well," and in some cases, it's even faster than on the PowerPC machine.

To determine this, MacTech ran over a thousand tests across three models of Macs, and the four major Office applications: Word, Excel, PowerPoint and Entourage. And, since graphics code is shared between Office applications, we ran a suite of graphics tests as well. These are each covered in more detail below.

In one of the most critical set of tests, we specifically looked at whether the user could type or interact faster than Office could keep up, and even in the slowest of scenarios, we never found the user waiting for typing, or other interactions like selecting menus. Even when typing at over 100 wpm, Word was able to stay ahead of the user.

Of the four applications, PowerPoint, is the one that struggled the most. It appears this is due to Office graphics engine shared by all of the Office applications.

At the other end of the spectrum, Entourage was not only on par, it was faster in many cases than our PowerPC baseline. In fact, with the exception of launching the application, Entourage was faster across the board on the Intel iMac, while the MacBook Pro was about on par with the PowerBook G4 (slightly faster in some cases, slightly slower in others).

What was interesting is that even on those tests that were slow, they were still acceptable. For example, the test with the worst performance was inserting a large JPG (over 10mb) into a document was considerably slower. In most cases, a 10 megabyte JPEG would be larger than an 8.5x11 page. How often do users insert large JPEGs in a single sitting? And, when they do, does it matter when that something takes 10 seconds more? Not in our opinion ... and we use Word a lot at MacTech.

It appeared to us that the more modern the application, the better it did under Rosetta. Furthermore, the more that something used the underlying Mac OS, the better it did as well. For example, Entourage did very well with networking related items. Raw imports and opening did significantly better especially when a faster disk was involved.

And, the more times you did a function, the better it performed especially on the second iteration of a command. While it's difficult to confirm, this is due to a combination of code working smarter with caches, both in Rosetta as well as within the Office code base. An obvious benefit for those tasks that are most sensitive to time: the repetitive ones.

The Test Suite and Results

The tests used were selected specifically to give a real-world view of what Microsoft Office 2004 is like to run. We eliminated those tests that we ran that were so short a time frame (e.g., fast) that we could not create statistically significant results, or that had imperceivable differences.

We did one test suite for each of the four major applications in Office, as well as a series of tests focused on the graphics libraries. As you may or may not know, Office 2004 uses a common set of graphics libraries across the applications in Office. With that in mind, we considered a set of tests that would test these technologies, and they are a good representation across all the applications in Office. For example, when you import a graphic, it should be the same across the Office suite.

To give users a good idea of what it's like to use each of these apps, we came up with a list of tests that represented what we felt were the most relevant to regular use. For example, we ran tests on application launches, but did not focus on them since it's something that people tend to do only a couple of times a day. But, we did include the most repetitive of tasks within an application, as those affect productivity most.

The one thing that we did see across the board is that the Intel iMac is consistently faster than the MacBook Pro. Since the iMac had slightly less RAM, and the processor and front side bus speeds are the same, we looked further inside. The iMac has some pretty serious sub-systems design to make it a screaming machine (see <http://developer.apple.com/documentation/HardwareDrivers/Conceptual/iMac_06Jan/>. When you take into account that an iMac has SATA (instead of ATA), a faster hard drive (a 7200 vs. the MacBook Pro's 5400 rpm), and a variety of optimized sub-systems that would be more difficult to implement on a laptop, it's easy to see why the iMac is faster.

It's important to realize that many of the actions that users do when using these applications are so fast already, that even a degradation of 50% may not even be noticeable for most tasks. And in the non-repetitive tasks, they are nearly irrelevant. For example, if when launching an application, you have to wait several seconds, many users will notice, but it won't matter to their overall productivity.

Other areas, like repetitive tasks or editing actions, are far more important, and speed makes a great deal more difference in not only perception, but in productivity.

An overview of tests that best represents productivity is displayed in Figure 1. These metrics include a variety of tests normalized to give you an overview. Smaller bars are faster (better). Information on what is included in each of these overview metrics, as well as additional testing information and graphics are shown later for each application.




Figure 1: Office 2004, Average User Tests


Word

Of all the applications, Word is probably the most widely used by people. The most relevant tests to most Word users are those that are repeated throughout the use of Word. For productivity tests, we selected a variety of scrolling tests, saving, word count, find & replace, opening files, pasting and printing.

The end result is that, for productivity tasks, Word on MacBook Pro performed at 82% of the speed of a PowerBook G4, and on an Intel iMac at 92%. In a real life scenario, these numbers put Word at working pretty well. There's not much a perceptual difference for most tasks.

Our top level grouping included a application launching, variety of scrolling tests, saving, word counts, find & replace, opening documents, pasting and printing. The results are in Figure 2.




Figure 2: Word Benchmark Results


But, of course, the most important of the tests - typing - were so fast that we couldn't measure them. No matter what the computer model, Word was ahead of the typist (even at 100 wpm).

The areas that Word performed "well enough" in are the initial launch of the application launching and printing. Fortunately, these are things that most users do relatively infrequently and don't materially affect productivity.

The balance of the tests ran on par, or in some cases faster on the Intel iMac. As is consistent throughout, the Intel iMac was faster than the MacBook Pro, and in some cases (most notably saving and pasting), the iMac was faster than the PowerBook baseline.

Our one and only crash during the entire Office testing set was during a find and replace in Word. A quick re-launch, and everything proceeded as expected.

The end result is that while there are a couple of areas that a user may notice it's slower, Word works well on a MacBook Pro; and, on the Intel iMac, it works very well, under Rosetta.


Excel

For Excel, we wanted to focus again on the most repetitive tasks. For productivity tests, we selected fill range, scrolling vertically, zooming out, arranging windows, subtotals, auto-formatting, and editing in-cell. On average for the productivity tests, Excel performed at 83% of baseline for the MacBook Pro, and 91% for the Intel iMac.

Most users use relatively small spreadsheets, but it's important to take a look at what a more experienced spreadsheeter may experience. As a general rule, editing and calculations were very fast and in some cases, faster than on the baseline PowerPC.

For our top level testing, we looked at the common commands including fill range, vertical scroll, zoom out, arranging windows, subtotals, auto-format, and editing in-cell. The results are displayed in Figure 3.




Figure 3: Excel Benchmark Results


Scrolling did leave a bit to be desired, but it definitely worked well enough, even on the slower MacBook Pro. In our tests, we used a very large Excel document, and even so, the test ranges were 6.18 to 10.15 seconds. Certainly enough to perceive, but given the very large size of the test document, under normal circumstances, it's not enough to affect productivity.

The end result is that for Excel on a MacBook Pro, it works very well, and on the Intel iMac, even better. Users may notice slower scrolling, but that's about it as far as the most common tasks.


PowerPoint

For PowerPoint, we chose a variety of tests to represent productivity including opening, saving, scrolling slides in the sorter view, adding slides, viewing slide shows, slide transitions, text animations, printing, inserting images from the ClipArt Gallery, changing color schemes and applying templates. As we already mentioned, PowerPoint was the slowest of the Office suite, but in our most indicative tests for productivity, it did perform at 77% of the speed for the MacBook Pro, and nearly 90% for the Intel iMac.

In our top level testing, we took a look at launching, opening files, saving, scrolling slides in sorter view, adding slides, viewing slide transitions, viewing text animations, printing, changing color schemes and applying templates. The results are shown in Figure 4.




Figure 4: PowerPoint Benchmark Results


The areas PowerPoint was weakest were with complex files, printing and applying templates. That said, PowerPoint did pretty well with view slides and transitions, animations, and scrolling in the sorter view.

The bottom line is that even if you use PowerPoint regularly, it definitely works well enough for most people, and in the most important areas ... the presentation itself ... it works very well. If you are a heavy PowerPoint user, you will notice sluggishness, and you should take a closer look.


Entourage

For Entourage, the productivity tests included IMAP account sync, empty deleted items, opening messages, pop message download, opening folders, sending messages via SMTP, sorting and grouping 2000 messages, and searching the address book, mail and tasks. For the productivity tests, Entourage shined with the MacBook Pro performing at 92% of the baseline speed, and the Intel iMac performing 14% faster than the baseline PowerPC.

In our top level testing, we took a look at launch time, empty deleted items, POP message download, sending SMTP messages, sorting and grouping thousands of messages, and searching the address book, mail and tasks. Results are shown in Figure 5.




Figure 5: Entourage Benchmark Results


Like Word, the most important of the tests, typing, were so fast that we couldn't measure them regardless of which test machine.

Boot time was clearly slower - about double in fact. But in a game of seconds, and for something that you generally only do a couple of times a day, it's not particularly relevant.

Entourage is clearly the best performer in the suite, presumably because Entourage is the most modern code base of the Office 2004 suite, and because it relies the most on Mac OS X technologies that have already been made Universal.

The end result is that, across the board, Entourage under Rosetta performed terrifically. The MacBook Pro usually performed as good or better than the PowerBook. The Intel iMac was faster in almost every test.


Microsoft Office Graphics

The weakest part of Office under Rosetta is the graphics code shared across Microsoft Office. PowerPoint shows this most given its nature.

In reality, however, this affects the overall Office experience relatively little. Furthermore, for the graphics libraries, our tests were designed to point out weaknesses and give us measurable results. But let's be real: how often do you import a 10-megabyte JPEG? Most JPEG's that size would be much larger than a full page.

That said, we focused on the tests on inserting large graphics, opening, saving, printing, working with the WordArt features, and the ClipArt library.

The end result is that inserting JPEG's are slow, and we saw the impact across all the Office applications. Fortunately, most users don't tend to do this repetitively. Other formats like inserting EPS and PSD files did much better. In other words, you'll have no issues with your logos and template graphics.

Opening graphics files was ok, but fortunately, saving files was significantly faster (a good thing since users typically save much more often than open files). The ClipArt interfaces work very well including the importing and searching.




Figure 6: Office Graphics Benchmark Results


Conclusion

As we stated at the beginning of this article, in general, Office 2004 under Rosetta works "well enough" to "very well," and in some cases, it's even faster than on the PowerPC baseline machine.

Given the amount we were pushing these apps, we were thoroughly pleased with the stability of the entire Office suite of applications. Considering how complex Rosetta is, and how big a code base Office is, it's entirely remarkable how stable we found it. Furthermore, we expect to see Apple further optimize Rosetta, particularly in launch times.

While Microsoft has already announced that it will be making the next version of Office a "Universal" application, Mac users wishing to dive into Intel-based Macs now can rest assured in knowing they can move forward and be patient for the Universal version.





Appendix A: Benchmarking Methodology

Introduction

The purpose of this appendix is to outline the basic parameters for how MacTech Magazine performed benchmarking tests on Microsoft Office 2004 for Intel iMacs and the MacBook Pro compared to a PowerBook G4.

Benchmarking Methodology


Consistency

Since the tests involve both multiple machines and multiple pieces of software, the focus was on creating as much consistency across the tests as possible. MacTech accomplished this in several ways.

First, all timing of tests were performed by a single MacTech staff member so as to eliminate any of the natural inconsistencies that often occur across individuals.

All of the tests were performed on the same version of the Mac operating system across the different hardware. At the time of the tests, this was Mac OS X 10.4.5 and included the most up-to-date versions of Apple patches as prescribed through "Software Update" in Mac OS X.

All of the tests were done on "virgin" systems, i.e., freshly wiped hard disks, with fresh Mac OS X and Office installations, with no third party software installed beyond the standard Apple and Microsoft Office installations.

All of the tests were performed with the most up to date set of patches for Microsoft Office as prescribed by Microsoft's Auto-updater. At the time of the test, this included the Microsoft Office 2004 for Mac 11.2.3 Update.


Avoiding Interactions

While the tests covered the bulk of the Microsoft Office suite, all tests (where appropriate) were performed with only that single application within Microsoft Office open. In other words, to the extent possible, no other applications will be running. (Obviously excluding background and OS tasks that are part of a standard install of Mac OS X or Office.)

To avoid issues with a noisy network, the test machines were installed on what was considered a "quiet" network with minimal traffic. MacTech monitored the use of the network to make sure that the machine does have network access, but is not impacted by the network.


Measurements, Testing and Outliers

For timed tests with results under 15 seconds, tests were measured to within 1/10th of a second. For those over 15 seconds, tests were measured to within a second.

Most tests were performed at least three times per test and per machine, and up to 5 times depending on the test. On average, tests were conducted three to four times per machine per test case. Outliers indicating a testing anomaly were retested as appropriate.

In most cases, tester used successive tests, not "Adam" or "first" tests to better emulate the daily use of Office products.

Those tests that could be impacted by the size of the window, were tested with the same window size under all scenarios.

Some tests were eliminated because the machines simply performed too fast to get an accurate measurement. For example, typing always displayed faster on the machine than the tester could type (even at 100 wpm).





Appendix B: Testing Results

Introduction

The below chart contains the raw data for the tests selected to judge Office 2004 applications. All results are in seconds, and represent the best time for each test on each machine.

                                   PowerBook G4   MacBook Pro   Intel iMac
Word:
Scroll Real World Doc                      8.12          9.47         8.34
Scroll Doc with Tables                     3.47          4.13         3.91
Scroll Doc with Graphics                   5.50          7.22         6.72
Save As Word Doc                           1.19          1.22         1.12
Word Count                                 1.69          2.22         1.84
Find & Replace                             0.85          0.93         0.88
Open Word Doc                              1.28          1.59         1.34
Paste                                      1.84          1.94         1.75
Print a Word Doc                           7.24         11.00        10.10

Excel:
Fill Range                                 1.41          1.66         1.68
Scroll Vertical                            6.18         10.15         8.97
Zoom Out                                   1.25          1.56         1.47
Window Arrange                             2.25          2.56         2.15
Subtotals                                  1.44          1.47         1.16
AutoFormat                                 0.94          1.06         1.00
Edit in-cell                               1.44          1.57         1.47

PowerPoint:
Open typical simple file                   1.47          1.78         1.56
Save complex file                          1.40          2.10         1.78
Scroll slide in sorter view                2.72          3.25         3.00
Add slides in sorter view                  2.16          2.56         2.47
View slide show                            1.19          1.12         1.16
View slide transitions                     0.82          0.85         0.88
View text animations                       0.81          0.85         0.66
Print simple file                          1.53          2.25         1.94
Insert from Clip Art Gallery               1.34          1.94         1.50
Change color scheme                        0.88          1.13         0.72
Apply Template                             2.59          4.88         4.38

Entourage:
IMAP Account Sync                          1.84          2.31         1.57
Empty Deleted Items                        5.25          6.50         4.06
Open Messages                              3.03          4.46         3.65
POP Message Download                      63.00         73.00        57.00
Open Folder                                0.69          0.69         0.56
Sending Messages SMTP                      1.35          1.19         1.16
Sorting 2000 messages                      0.81          0.81         0.78
Grouping 2000 messages                     1.78          1.65         1.50
Address Book Search                        1.06          1.06         0.85
Mail Search                                0.56          0.59         0.53
Task Search                                0.68          0.62         0.47

Graphics:
Insert Large JPG                           2.40         12.22        11.12
Insert Large EPS                          25.56         37.81        21.87
Open Document                              1.19          1.94         1.68
Save Document                              2.81          3.59         2.07
Print Document                            55.00        100.00        91.00
WordArt: Insert                            0.84          1.16         1.06
ClipArt: Import                           98.00         89.00        49.00
ClipArt: Search                            0.50          0.53         0.44




 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Whitethorn Games combines two completely...
If you have ever gone fishing then you know that it is a lesson in patience, sitting around waiting for a bite that may never come. Well, that's because you have been doing it wrong, since as Whitehorn Games now demonstrates in new release Skate... | Read more »
Call of Duty Warzone is a Waiting Simula...
It's always fun when a splashy multiplayer game comes to mobile because they are few and far between, so I was excited to see the notification about Call of Duty: Warzone Mobile (finally) launching last week and wanted to try it out. As someone who... | Read more »
Albion Online introduces some massive ne...
Sandbox Interactive has announced an upcoming update to its flagship MMORPG Albion Online, containing massive updates to its existing guild Vs guild systems. Someone clearly rewatched the Helms Deep battle in Lord of the Rings and spent the next... | Read more »
Chucklefish announces launch date of the...
Chucklefish, the indie London-based team we probably all know from developing Terraria or their stint publishing Stardew Valley, has revealed the mobile release date for roguelike deck-builder Wildfrost. Developed by Gaziter and Deadpan Games, the... | Read more »
Netmarble opens pre-registration for act...
It has been close to three years since Netmarble announced they would be adapting the smash series Solo Leveling into a video game, and at last, they have announced the opening of pre-orders for Solo Leveling: Arise. [Read more] | Read more »
PUBG Mobile celebrates sixth anniversary...
For the past six years, PUBG Mobile has been one of the most popular shooters you can play in the palm of your hand, and Krafton is celebrating this milestone and many years of ups by teaming up with hit music man JVKE to create a special song for... | Read more »
ASTRA: Knights of Veda refuse to pump th...
In perhaps the most recent example of being incredibly eager, ASTRA: Knights of Veda has dropped its second collaboration with South Korean boyband Seventeen, named so as it consists of exactly thirteen members and a video collaboration with Lee... | Read more »
Collect all your cats and caterpillars a...
If you are growing tired of trying to build a town with your phone by using it as a tiny, ineffectual shover then fear no longer, as Independent Arts Software has announced the upcoming release of Construction Simulator 4, from the critically... | Read more »
Backbone complete its lineup of 2nd Gene...
With all the ports of big AAA games that have been coming to mobile, it is becoming more convenient than ever to own a good controller, and to help with this Backbone has announced the completion of their 2nd generation product lineup with their... | Read more »
Zenless Zone Zero opens entries for its...
miHoYo, aka HoYoverse, has become such a big name in mobile gaming that it's hard to believe that arguably their flagship title, Genshin Impact, is only three and a half years old. Now, they continue the road to the next title in their world, with... | Read more »

Price Scanner via MacPrices.net

B&H has Apple’s 13-inch M2 MacBook Airs o...
B&H Photo has 13″ MacBook Airs with M2 CPUs and 256GB of storage in stock and on sale for up to $150 off Apple’s new MSRP, starting at only $849. Free 1-2 day delivery is available to most US... Read more
M2 Mac minis on sale for $100-$200 off MSRP,...
B&H Photo has Apple’s M2-powered Mac minis back in stock and on sale today for $100-$200 off MSRP. Free 1-2 day shipping is available for most US addresses: – Mac mini M2/256GB SSD: $499, save $... Read more
Mac Studios with M2 Max and M2 Ultra CPUs on...
B&H Photo has standard-configuration Mac Studios with Apple’s M2 Max & Ultra CPUs in stock today and on Easter sale for $200 off MSRP. Their prices are the lowest available for these models... Read more
Deal Alert! B&H Photo has Apple’s 14-inch...
B&H Photo has new Gray and Black 14″ M3, M3 Pro, and M3 Max MacBook Pros on sale for $200-$300 off MSRP, starting at only $1399. B&H offers free 1-2 day delivery to most US addresses: – 14″ 8... Read more
Department Of Justice Sets Sights On Apple In...
NEWS – The ball has finally dropped on the big Apple. The ball (metaphorically speaking) — an antitrust lawsuit filed in the U.S. on March 21 by the Department of Justice (DOJ) — came down following... Read more
New 13-inch M3 MacBook Air on sale for $999,...
Amazon has Apple’s new 13″ M3 MacBook Air on sale for $100 off MSRP for the first time, now just $999 shipped. Shipping is free: – 13″ MacBook Air (8GB RAM/256GB SSD/Space Gray): $999 $100 off MSRP... Read more
Amazon has Apple’s 9th-generation WiFi iPads...
Amazon has Apple’s 9th generation 10.2″ WiFi iPads on sale for $80-$100 off MSRP, starting only $249. Their prices are the lowest available for new iPads anywhere: – 10″ 64GB WiFi iPad (Space Gray or... Read more
Discounted 14-inch M3 MacBook Pros with 16GB...
Apple retailer Expercom has 14″ MacBook Pros with M3 CPUs and 16GB of standard memory discounted by up to $120 off Apple’s MSRP: – 14″ M3 MacBook Pro (16GB RAM/256GB SSD): $1691.06 $108 off MSRP – 14... Read more
Clearance 15-inch M2 MacBook Airs on sale for...
B&H Photo has Apple’s 15″ MacBook Airs with M2 CPUs (8GB RAM/256GB SSD) in stock today and on clearance sale for $999 in all four colors. Free 1-2 delivery is available to most US addresses.... Read more
Clearance 13-inch M1 MacBook Airs drop to onl...
B&H has Apple’s base 13″ M1 MacBook Air (Space Gray, Silver, & Gold) in stock and on clearance sale today for $300 off MSRP, only $699. Free 1-2 day shipping is available to most addresses in... Read more

Jobs Board

Medical Assistant - Surgical Oncology- *Apple...
Medical Assistant - Surgical Oncology- Apple Hill Location: WellSpan Medical Group, York, PA Schedule: Full Time Sign-On Bonus Eligible Remote/Hybrid Regular Apply Read more
Omnichannel Associate - *Apple* Blossom Mal...
Omnichannel Associate - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
Cashier - *Apple* Blossom Mall - JCPenney (...
Cashier - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Blossom Mall Read more
Operations Associate - *Apple* Blossom Mall...
Operations Associate - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
Business Analyst | *Apple* Pay - Banco Popu...
Business Analyst | Apple PayApply now " Apply now + Apply Now + Start applying with LinkedIn Start + Please wait Date:Mar 19, 2024 Location: San Juan-Cupey, PR Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.