04-Test Suite and Results
TweetFollow Us on Twitter

04-Test Suite and Results

The Test Suite and Results

In the sections below, we'll walk you through what we tested, and the results for each. These tests are designed to arm you with the information so you can make the best decision for your type of use.

For each set of results, you can see the analysis for each model of computer for XP, and for Windows 7. If you want to see more detail for single vs. multiple processors, or on an individual Mac model, you may want to review the spreadsheet for those details.

As you look through the charts below, take note on whether faster is represented by longer or shorter bars (see the lower left corner of each chart).

For the launch tests (launching the VM, Windows, and Applications), we had the option of an "Adam" test, and a "Successive" test. Adam tests are when the computer has been completely restarted (hence avoiding both host and guest OS caching). Successive tests are repeated tests without restarting the machine in between tests, and can benefit from caching. Both mimic real use situations.

The tests used were selected specifically to give a real-world view of what VMware Fusion and Parallels Desktop are like to run for many users. We eliminated those tests that we ran which were so short in time frame (e.g., fast) that we could not create statistically significant results, or that had unperceivable differences.

For some of the analysis, we "normalized" results by dividing the result by the fastest result for that test across all the machine configurations. We did this specifically so that we could make comparisons across different groups, and to be able to give you overview results combining a series of types of tests, and computer models.

Instead of a plain "average" or "mean", overall conclusions are done using a "geomean", which is a specific type of average that focuses on the central results and minimizes outliers. Geomean is the same averaging methodology used by SPEC tests, PCMark, Unixbench, and others, and it helps prevent against minor result skewing. (If you are interested in how it differs from a mean, instead of adding the set of numbers and then dividing the sum by the count of numbers in the set, n, the numbers are multiplied and then the nth root of the resulting product is taken).

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, see Appendix B. Both appendices are available on the MacTech web site.

Launch and CPU Tests

There are three situations in which users commonly launch a virtual machine:

  • Launch the virtual machine from "off" mode, including a full Windows boot.
  • Launch the virtual machine from a suspended state, and resume from suspend (Adam).
  • Launch the virtual machine from a suspended state, and resume from suspend (Successive).

For the first test, we started at the Finder and launched the virtualization application, which were set up to immediately launch the virtual machine. The visual feedback is fairly different between Parallels Desktop and VMware Fusion when Windows first starts up. Windows actually does its startup for quite some time after reaching the desktop. In some cases, it can take quite some time for Windows to complete its boot process. Most users don't care if things continue, so long as they aren't held up.

As a result, we focused on timing to the point of actually accomplishing something. In this case, we hovered over the Task Bar icons and launched the Windows Security Center window (XP), or the networking interface icon (Windows 7). The test ended when the window started to render. This gave us a real world scenario of being able to actually do something as opposed to Windows just looking like it was booted.

The primary difference between the last two types of VM launch test is that the computer is fully rebooted (both the virtual machine as well as Mac OS X) in between the "Adam" tests. The successive tests are launching the virtual machines and restoring them without restarting the Mac in between.

Successive tests benefit from both Mac OS X and possibly virtual machine caching, and are significantly faster. However, you may only see these types of situations if you are constantly switching in and out of your virtual machine.

As with all of our tests, we perform these tests multiple times to handle the variability that can occur. Of these, we took the best results for each product.

Description: VMPerf
Virtual Machine Performance

Clearly, machines with more memory take longer to restore and that accounts for some of the differences (remember, we selected RAM configurations based on VMware Fusion's default which varied by OS and Mac model). One thing to remember as a virtualization user is that if you are switching in and out of a VM often, you may want to think about using less RAM, not more. In fact, you should just use as little as you need for the best experience under either virtualized environment. (We suggest 512MB to 1GB for most people. 512MB to 768MB on 2GB MacBooks).

Most benchmarking suites measure CPU performance using file compression as at least one part of their analysis. We did the same. As a matter of interest, we used compression instead of decompression, because with today's fast computers, decompression is actually much closer to a file copy than it is to CPU work. Compression requires the system to perform analysis to do the compression, and is therefore a better measurement of CPU.

VMware Fusion performed better in the compression test under Windows 7, and Parallels Desktop did better in the same test under Windows XP. For tests related to Windows boot time, suspend and resume, Parallels Desktop was noticeably quicker than VMware Fusion.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 
AAPL
$108.00
Apple Inc.
+1.02
MSFT
$46.95
Microsoft Corpora
+0.90
GOOG
$559.08
Google Inc.
+8.77

MacTech Search:
Community Search:

Software Updates via MacUpdate

Vitamin-R 2.20b1 - Personal productivity...
Vitamin-R creates the optimal conditions for your brain to work at its best by structuring your work into short bursts of distraction-free, highly focused activity alternating with opportunities for... Read more
Dropbox 2.10.44 - Cloud synchronization...
Dropbox is an application that creates a special Finder folder that automatically syncs online and between your computers. It allows you to both backup files and keep them up-to-date between systems... Read more
Sandvox 2.9.2 - Easily build eye-catchin...
Sandvox is for Mac users who want to create a professional looking website quickly and easily. With Sandvox, you don't need to be a Web genius to build a stylish, feature-rich, standards-compliant... Read more
Cocktail 8.0.1 - General maintenance and...
Cocktail is a general purpose utility for OS X that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more
LibreOffice 4.3.3.2 - Free Open Source o...
LibreOffice is an office suite (word processor, spreadsheet, presentations, drawing tool) compatible with other major office suites. The Document Foundation is coordinating development and... Read more
VMware Fusion 7.0.1 - Run Windows apps a...
VMware Fusion allows you to create a Virtual Machine on your Mac and run Windows (including Windows 8.1) and Windows software on your Mac. Run your favorite Windows applications alongside Mac... Read more
OneNote 15.3.2 - Free digital notebook f...
OneNote is your very own digital notebook. With OneNote, you can capture that flash of genius, that moment of inspiration, or that list of errands that's too important to forget. Whether you're at... Read more
Audio Hijack Pro 2.11.4 - Record and enh...
Audio Hijack Pro drastically changes the way you use audio on your computer, giving you the freedom to listen to audio when you want and how you want. Record and enhance any audio with Audio Hijack... Read more
Iridient Developer 3.0.0 beta 3 - Powerf...
Iridient Developer (was RAW Developer) is a powerful image conversion application designed specifically for OS X. Iridient Developer gives advanced photographers total control over every aspect of... Read more
TextWrangler 4.5.11 - Free general purpo...
TextWrangler is the powerful general purpose text editor, and Unix and server administrator's tool. Oh, and also, like the best things in life, it's free. TextWrangler is the "little brother" to... Read more

Latest Forum Discussions

See All

Monster Flash Review
Monster Flash Review By Jordan Minor on October 31st, 2014 Our Rating: :: ALONE IN THE DARKUniversal App - Designed for iPhone and iPad Solid shooting and a surprising amount of spooky tension make Monster Flash a great portable... | Read more »
Retry Review
Retry Review By Rob Thomas on October 31st, 2014 Our Rating: :: SOARING HIGHUniversal App - Designed for iPhone and iPad Flappy who? Let Retry wash all those bad bird-related memories away on a cool retro-flavored flight… right... | Read more »
Dementia: Book of the Dead Review
Dementia: Book of the Dead Review By Lee Hamlet on October 31st, 2014 Our Rating: :: A TOUGH READUniversal App - Designed for iPhone and iPad A witch hunter is sent after a demonic book in the spooky but short-lived Dementia: Book... | Read more »
Card Dungeon, the Semi-Board Game Roguel...
Card Dungeon, the Semi-Board Game Roguelike, Has Been Renovated Posted by Jessica Fisher on October 31st, 2014 [ permalink ] | Read more »
Logitech Protection + Power iPhone5/5S C...
Made by: Logitech Price: $99.99 Hardware/iOS Integration Rating: 3 out of 5 stars Usability Rating: 0.5 out of 5 stars Reuse Value Rating: 0.75 out of 5 stars Build Quality Rating: 0.75 out of 5 stars Overall Rating: 1.25 out of 5 stars | Read more »
This Is Not a Test Goes Free, Permanentl...
This Is Not a Test Goes Free, Permanently Posted by Jessica Fisher on October 31st, 2014 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Swap Heroes Review
Swap Heroes Review By Campbell Bird on October 31st, 2014 Our Rating: :: STRATEGIC SWAPPINGUniversal App - Designed for iPhone and iPad Rotate a cast of heroes to fend of waves of monsters in this difficult, puzzle rpg.   | Read more »
Night Sky Pro™ (Reference)
Night Sky Pro™ 3.0.1 Device: iOS Universal Category: Reference Price: $2.99, Version: 3.0.1 (iTunes) Description: Night Sky Pro™Wonder No More™ Night Sky Pro™ is the ultimate stargazing experience. From the creators of the original... | Read more »
Audio Defence : Zombie Arena (Games)
Audio Defence : Zombie Arena 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: A zombie shooter audio game. Made from gut-wrenching 3D binaural sound, for a new kind of weird immersion. You... | Read more »
RPG Asdivine Hearts (Games)
RPG Asdivine Hearts 1.1.0 Device: iOS Universal Category: Games Price: $3.99, Version: 1.1.0 (iTunes) Description: SPECIAL PRICE50% OFF (USD 7.99 -> USD 3.99)!!! Travel alongside four companions and a cat in the adventure of a... | Read more »

Price Scanner via MacPrices.net

Tablets Ascendent Again; Global Tablet Market...
The worldwide tablet grew 11.5% year over year in the third quarter of 2014 (3Q14) with shipments reaching 53.8 million units according to preliminary data from the International Data Corporation (... Read more
Apple now offering refurbished 2014 13-inch R...
The Apple Store is now offering Apple Certified Refurbished 2014 13″ Retina MacBook Pros for up to $270 off the cost of new models. An Apple one-year warranty is included with each model, and... Read more
Apple Regains Momentum As Windows Stutters An...
The latest smartphone sales data from Kantar Worldpanel ComTech, for the three months to March 2014, shows Apple performing strongly in the first quarter of the year, with sales bouncing back in... Read more
Worldwide Smartphone Shipments Increase 25.2%...
New smartphone releases and an increased emphasis on emerging markets drove global smartphone shipments above 300 million units for the second consecutive quarter, according to preliminary data from... Read more
Apple now offering refurbished 2014 15-inch M...
The Apple Store is now offering Apple Certified Refurbished 2014 15″ Retina MacBook Pros for up to $400 off the cost of new models. An Apple one-year warranty is included with each model, and... Read more
Apple drops prices on refurbished 2013 Retina...
The Apple Store has dropped prices on 2013 Apple Certified Refurbished 13″ and 15″ Retina MacBook Pros, with Retina models now available starting at $999. Apple’s one-year warranty is standard, and... Read more
New 2.8GHz Mac mini on sale for $949, save $5...
Abt Electronics has the new 2.8GHz Mac mini in stock and on sale for $949.05 including free shipping. Their price is $50 off MSRP, and it’s the lowest price available for this model from any reseller... Read more
Sale! 3.7GHz Quad Core Mac Pro available for...
 B&H Photo has the 3.7GHz Quad Core Mac Pro on sale for $2649 including free shipping plus NY sales tax only. Their price is $350 off MSRP, and it’s the lowest price for this model from any... Read more
Mujjo Steps Up The Game With Refined Touchscr...
Netherlands based Mujjo have just launched their Refined Touchscreen Gloves, stepping up their game. The gloves feature a updated elegant design that takes these knitted gloves to the next level. A... Read more
Sale! Preorder the new 27-inch 5K iMac for $2...
 Abt Electronics has the new 27″ 3.5GHz 5K iMac on sale and available for preorder for $2374.05 including free shipping. Their price is $125 off MSRP, and it’s the lowest price available for this... Read more

Jobs Board

Position Opening at *Apple* - Apple (United...
…Summary** As a Specialist, you help create the energy and excitement around Apple products, providing the right solutions and getting products into customers' hands. You Read more
Position Opening at *Apple* - Apple (United...
**Job Summary** Being a Business Manager at an Apple Store means you're the catalyst for businesses to discover and leverage the power, ease, and flexibility of Apple Read more
Position Opening at *Apple* - Apple (United...
**Job Summary** As more and more people discover Apple , they visit our stores seeking ways to incorporate our products into their lives. It's your job, as a Store Read more
Position Opening at *Apple* - Apple (United...
**Job Summary** At the Apple Store, you connect business professionals and entrepreneurs with the tools they need in order to put Apple solutions to work in their Read more
Solutions Specialist with *Apple* Knowledge...
Company Description: We are an Apple Authorized Sales and Service Provider. We have been selling and servicing Apple computers in the Fairfield County area for over Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.