04-Test Suite and Results
TweetFollow Us on Twitter

04-Test Suite and Results

The Test Suite and Results

In the sections below, we'll walk you through what we tested, and the results for each. These tests are designed to arm you with the information so you can make the best decision for your type of use.

For each set of results, you can see the analysis for each model of computer for XP, and for Windows 7. If you want to see more detail for single vs. multiple processors, or on an individual Mac model, you may want to review the spreadsheet for those details.

As you look through the charts below, take note on whether faster is represented by longer or shorter bars (see the lower left corner of each chart).

For the launch tests (launching the VM, Windows, and Applications), we had the option of an "Adam" test, and a "Successive" test. Adam tests are when the computer has been completely restarted (hence avoiding both host and guest OS caching). Successive tests are repeated tests without restarting the machine in between tests, and can benefit from caching. Both mimic real use situations.

The tests used were selected specifically to give a real-world view of what VMware Fusion and Parallels Desktop are like to run for many users. We eliminated those tests that we ran which were so short in time frame (e.g., fast) that we could not create statistically significant results, or that had unperceivable differences.

For some of the analysis, we "normalized" results by dividing the result by the fastest result for that test across all the machine configurations. We did this specifically so that we could make comparisons across different groups, and to be able to give you overview results combining a series of types of tests, and computer models.

Instead of a plain "average" or "mean", overall conclusions are done using a "geomean", which is a specific type of average that focuses on the central results and minimizes outliers. Geomean is the same averaging methodology used by SPEC tests, PCMark, Unixbench, and others, and it helps prevent against minor result skewing. (If you are interested in how it differs from a mean, instead of adding the set of numbers and then dividing the sum by the count of numbers in the set, n, the numbers are multiplied and then the nth root of the resulting product is taken).

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, see Appendix B. Both appendices are available on the MacTech web site.

Launch and CPU Tests

There are three situations in which users commonly launch a virtual machine:

  • Launch the virtual machine from "off" mode, including a full Windows boot.
  • Launch the virtual machine from a suspended state, and resume from suspend (Adam).
  • Launch the virtual machine from a suspended state, and resume from suspend (Successive).

For the first test, we started at the Finder and launched the virtualization application, which were set up to immediately launch the virtual machine. The visual feedback is fairly different between Parallels Desktop and VMware Fusion when Windows first starts up. Windows actually does its startup for quite some time after reaching the desktop. In some cases, it can take quite some time for Windows to complete its boot process. Most users don't care if things continue, so long as they aren't held up.

As a result, we focused on timing to the point of actually accomplishing something. In this case, we hovered over the Task Bar icons and launched the Windows Security Center window (XP), or the networking interface icon (Windows 7). The test ended when the window started to render. This gave us a real world scenario of being able to actually do something as opposed to Windows just looking like it was booted.

The primary difference between the last two types of VM launch test is that the computer is fully rebooted (both the virtual machine as well as Mac OS X) in between the "Adam" tests. The successive tests are launching the virtual machines and restoring them without restarting the Mac in between.

Successive tests benefit from both Mac OS X and possibly virtual machine caching, and are significantly faster. However, you may only see these types of situations if you are constantly switching in and out of your virtual machine.

As with all of our tests, we perform these tests multiple times to handle the variability that can occur. Of these, we took the best results for each product.

Description: VMPerf
Virtual Machine Performance

Clearly, machines with more memory take longer to restore and that accounts for some of the differences (remember, we selected RAM configurations based on VMware Fusion's default which varied by OS and Mac model). One thing to remember as a virtualization user is that if you are switching in and out of a VM often, you may want to think about using less RAM, not more. In fact, you should just use as little as you need for the best experience under either virtualized environment. (We suggest 512MB to 1GB for most people. 512MB to 768MB on 2GB MacBooks).

Most benchmarking suites measure CPU performance using file compression as at least one part of their analysis. We did the same. As a matter of interest, we used compression instead of decompression, because with today's fast computers, decompression is actually much closer to a file copy than it is to CPU work. Compression requires the system to perform analysis to do the compression, and is therefore a better measurement of CPU.

VMware Fusion performed better in the compression test under Windows 7, and Parallels Desktop did better in the same test under Windows XP. For tests related to Windows boot time, suspend and resume, Parallels Desktop was noticeably quicker than VMware Fusion.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 
AAPL
$101.32
Apple Inc.
+0.74
MSFT
$45.15
Microsoft Corpora
-0.07
GOOG
$582.56
Google Inc.
-0.81

MacTech Search:
Community Search:

Software Updates via MacUpdate

calibre 2.0.0 - Complete e-library manag...
Calibre is a complete e-book library manager. Organize your collection, convert your books to multiple formats, and sync with all of your devices. Let Calibre be your multi-tasking digital... Read more
Apple iMovie 10.0.5 - Edit personal vide...
With an all-new design, Apple iMovie lets you enjoy your videos like never before. Browse your clips more easily, instantly share your favorite moments, and create beautiful HD movies and Hollywood-... Read more
Apple Keynote 6.2.2 - Apple's prese...
Apple Keynote makes it simple to create and deliver beautiful presentations. Powerful tools and dazzling effects bring your ideas to life. You can work seamlessly between Mac and iOS devices. And... Read more
Apple Numbers 3.2.2 - Apple's sprea...
With Apple Numbers, sophisticated spreadsheets are just the start. The whole sheet is your canvas. Just add dramatic interactive charts, tables, and images that paint a revealing picture of your data... Read more
OpenOffice 4.1.1 - Free and open-source...
OpenOffice.org is both an Open Source product and a project. The product is a multi-platform office productivity suite. It includes the key desktop applications, such as a word processor,... Read more
Pages 5.2.2 - Apple's word processo...
Apple Pages is a powerful word processor that gives you everything you need to create documents that look beautiful. And read beautifully. It lets you work seamlessly between Mac and iOS devices. And... Read more
Quicken 2015 2.0.1 - Complete personal f...
The new Quicken 2015 helps you manage all your personal finances in one place, so you can see where you're spending and where you can save. Quicken automatically categorizes your financial... Read more
CleanMyMac 2.2.7 - Delete files that was...
CleanMyMac makes space for the things you love. Sporting a range of ingenious new features, CleanMyMac 2 lets you safely and intelligently scan and clean your entire system, delete large, unused... Read more
MacFamilyTree 7.2.4 - Create and explore...
MacFamilyTree gives genealogy a facelift: it's modern, interactive, incredibly fast, and easy to use. We're convinced that generations of chroniclers would have loved to trade in their genealogy... Read more
Videobox 4.1.1 - Download Flash video th...
Videobox allows you to quickly and easily download Flash video from most all of the popular video sites on the internet. Videobox will convert the video into a native Quicktime format so it's ready... Read more

Latest Forum Discussions

See All

Trolls vs Vikings Update Adds Over One H...
Trolls vs Vikings Update Adds Over One Hundred Levels, Reduces Item Cost, and More Posted by Ellis Spice on August 22nd, 2014 [ permalink ] | Read more »
SNK Celebrates the 20th Anniversary of T...
SNK Celebrates the 20th Anniversary of The King of Fighters With a Big Sale Posted by Ellis Spice on August 22nd, 2014 [ permalink ] | Read more »
It Came From Canada: Star Wars: Commande...
With a brand new Star Wars trilogy on the horizon, prepare yourselves for Disney and George Lucas’s space fantasy throwback to be more omnipresent than ever before. So it should come as no surprise that new adventures in that galaxy far, far away... | Read more »
Swing Copters Review
Swing Copters Review By Jordan Minor on August 22nd, 2014 Our Rating: :: DIE TRYINGUniversal App - Designed for iPhone and iPad The creator of Flappy Bird is back with a vengeance.   | Read more »
Beam Me an Update Scotty – Star Trek Tre...
Beam Me an Update Scotty – Star Trek Trexels Receives its Biggest Update Yet Posted by Jessica Fisher on August 22nd, 2014 [ permalink ] | Read more »
The Outcast Review
The Outcast Review By Nadia Oxford on August 22nd, 2014 Our Rating: :: HANDS OFF. WAY OFF.Universal App - Designed for iPhone and iPad It’s easy to see what The Outcast is trying for, but its execution needs a lot of work.   | Read more »
HeroCraft Unveils New iOS Game, Marble D...
HeroCraft Unveils New iOS Game, Marble Duel Posted by Jessica Fisher on August 22nd, 2014 [ permalink ] HeroCraft is developing a new chain popper game called Marble Duel, wh | Read more »
Brain+ Review
Brain+ Review By Nadia Oxford on August 22nd, 2014 Our Rating: :: DIM BULBUniversal App - Designed for iPhone and iPad Brain+ is just another entry in an over-saturated brain-training marketplace – and not a particularly fun entry... | Read more »
The Witcher Battle Arena – New Gameplay...
The Witcher Battle Arena – New Gameplay Trailer Revealed Posted by Jessica Fisher on August 22nd, 2014 [ permalink ] Based in the Witcher universe, | Read more »
Max Gentlemen Review
Max Gentlemen Review By Jennifer Allen on August 22nd, 2014 Our Rating: :: OUTSTAYING ITS WELCOMEiPhone App - Designed for the iPhone, compatible with the iPad Max Gentlemen seems pretty quirky initially but that appeal wears thin... | Read more »

Price Scanner via MacPrices.net

Leftover 15-inch 2.0GHz Retina MacBook Pros a...
B&H Photo has leftover previous-generation 15″ 2.0GHz Retina MacBook Pros now available for $1599 including free shipping plus NY sales tax only. Their price is $400 off original MSRP. B&H... Read more
Pro.Calendar – New Productivity App for iPad...
Austin, Texas based mobile business and productivity app developer LightArrow, Inc. has announced Pro.Calendar, a powerful and intuitive calendar app with eight versatile calendar options including... Read more
SanDisk Ultra II SSD — Supercharge Your Syste...
SanDisk Corporation has announced the new SanDisk Ultra II SSD with enhanced SSD Dashboard. The new drive is designed to deliver a cost-effective and easy upgrade solution for PC owners looking to... Read more
Samsung and Barnes & Noble Introduce New...
Samsung Electronics America and NOOK Media, a subsidiary of Barnes & Noble, Inc. have announced the introduction of the new Samsung Galaxy Tab 4 NOOK, a 7-inch tablet combining Samsung’s leading... Read more
21-inch iMacs on sale for up to $150 off MSRP
B&H Photo has 21″ iMacs on sale for up to $150 off MSRP including free shipping plus NY sales tax only. B&H will also include a free copy of Parallels Desktop software: - 21″ 2.7GHz iMac: $... Read more
27-inch 3.2GHz iMac on sale for $1698, save $...
Abt has the 27″ 3.2GHz iMac on sale for $1698 including free shipping. Their price is $101 off MSRP. Read more
Mac Backup Guru 2.0 Drive Backup/Cloneing Uti...
Mac Backup Guru developer MacDaddy has released Mac Backup Guru 2.0, offering new and enhanced advanced features, such as bootable backups, synchronised volumes and folders, and a Snapshot mode that... Read more
Operate GE’s New Free-Standing KItchen Range...
Think you accidentally left the oven on? Switch it off while on the go. The new free-standing Profile™ Series gas and electric ranges are GE’s second cooking appliances, following their wall oven, to... Read more
Parallels Announces Parallels Desktop 10 for...
The no. 1-selling software for running Windows applications on a Mac becomes an even easier choice for millions of consumers and IT professionals worldwide with the launch of the most powerful... Read more
Apple now offering certified refurbished 2014...
 The Apple Store is now offering Apple Certified Refurbished 2014 MacBook Airs for up to $180 off the cost of new models. An Apple one-year warranty is included with each MacBook, and shipping is... Read more

Jobs Board

Position Opening at *Apple* - Apple (United...
**Job Summary** At the Apple Store, you connect business professionals and entrepreneurs with the tools they need in order to put Apple solutions to work in their Read more
Project Manager / Business Analyst, WW *Appl...
…a senior project manager / business analyst to work within our Worldwide Apple Fulfillment Operations and the Business Process Re-engineering team. This role will work Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Position Opening at *Apple* - Apple (United...
**Job Summary** As more and more people discover Apple , they visit our stores seeking ways to incorporate our products into their lives. It's your job, as a Store Read more
Position Opening at *Apple* - Apple (United...
…Summary** As a Specialist, you help create the energy and excitement around Apple products, providing the right solutions and getting products into customers' hands. You Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.