04-Test Suite and Results
TweetFollow Us on Twitter

04-Test Suite and Results

The Test Suite and Results

In the sections below, we'll walk you through what we tested, and the results for each. These tests are designed to arm you with the information so you can make the best decision for your type of use.

For each set of results, you can see the analysis for each model of computer for XP, and for Windows 7. If you want to see more detail for single vs. multiple processors, or on an individual Mac model, you may want to review the spreadsheet for those details.

As you look through the charts below, take note on whether faster is represented by longer or shorter bars (see the lower left corner of each chart).

For the launch tests (launching the VM, Windows, and Applications), we had the option of an "Adam" test, and a "Successive" test. Adam tests are when the computer has been completely restarted (hence avoiding both host and guest OS caching). Successive tests are repeated tests without restarting the machine in between tests, and can benefit from caching. Both mimic real use situations.

The tests used were selected specifically to give a real-world view of what VMware Fusion and Parallels Desktop are like to run for many users. We eliminated those tests that we ran which were so short in time frame (e.g., fast) that we could not create statistically significant results, or that had unperceivable differences.

For some of the analysis, we "normalized" results by dividing the result by the fastest result for that test across all the machine configurations. We did this specifically so that we could make comparisons across different groups, and to be able to give you overview results combining a series of types of tests, and computer models.

Instead of a plain "average" or "mean", overall conclusions are done using a "geomean", which is a specific type of average that focuses on the central results and minimizes outliers. Geomean is the same averaging methodology used by SPEC tests, PCMark, Unixbench, and others, and it helps prevent against minor result skewing. (If you are interested in how it differs from a mean, instead of adding the set of numbers and then dividing the sum by the count of numbers in the set, n, the numbers are multiplied and then the nth root of the resulting product is taken).

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, see Appendix B. Both appendices are available on the MacTech web site.

Launch and CPU Tests

There are three situations in which users commonly launch a virtual machine:

  • Launch the virtual machine from "off" mode, including a full Windows boot.
  • Launch the virtual machine from a suspended state, and resume from suspend (Adam).
  • Launch the virtual machine from a suspended state, and resume from suspend (Successive).

For the first test, we started at the Finder and launched the virtualization application, which were set up to immediately launch the virtual machine. The visual feedback is fairly different between Parallels Desktop and VMware Fusion when Windows first starts up. Windows actually does its startup for quite some time after reaching the desktop. In some cases, it can take quite some time for Windows to complete its boot process. Most users don't care if things continue, so long as they aren't held up.

As a result, we focused on timing to the point of actually accomplishing something. In this case, we hovered over the Task Bar icons and launched the Windows Security Center window (XP), or the networking interface icon (Windows 7). The test ended when the window started to render. This gave us a real world scenario of being able to actually do something as opposed to Windows just looking like it was booted.

The primary difference between the last two types of VM launch test is that the computer is fully rebooted (both the virtual machine as well as Mac OS X) in between the "Adam" tests. The successive tests are launching the virtual machines and restoring them without restarting the Mac in between.

Successive tests benefit from both Mac OS X and possibly virtual machine caching, and are significantly faster. However, you may only see these types of situations if you are constantly switching in and out of your virtual machine.

As with all of our tests, we perform these tests multiple times to handle the variability that can occur. Of these, we took the best results for each product.

Description: VMPerf
Virtual Machine Performance

Clearly, machines with more memory take longer to restore and that accounts for some of the differences (remember, we selected RAM configurations based on VMware Fusion's default which varied by OS and Mac model). One thing to remember as a virtualization user is that if you are switching in and out of a VM often, you may want to think about using less RAM, not more. In fact, you should just use as little as you need for the best experience under either virtualized environment. (We suggest 512MB to 1GB for most people. 512MB to 768MB on 2GB MacBooks).

Most benchmarking suites measure CPU performance using file compression as at least one part of their analysis. We did the same. As a matter of interest, we used compression instead of decompression, because with today's fast computers, decompression is actually much closer to a file copy than it is to CPU work. Compression requires the system to perform analysis to do the compression, and is therefore a better measurement of CPU.

VMware Fusion performed better in the compression test under Windows 7, and Parallels Desktop did better in the same test under Windows XP. For tests related to Windows boot time, suspend and resume, Parallels Desktop was noticeably quicker than VMware Fusion.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Soft Drummer (Music)
Soft Drummer 1.0 Device: iOS Universal Category: Music Price: $14.99, Version: 1.0 (iTunes) Description: Soft Drummer is the closest thing to a pro session subtle drummer in your pocket. Easy to use and fast, it's much more than a... | Read more »
Is GO Gear the Pokemon GO map app you...
Now that we've settled into something of a Pokemon GO status quo, the number one desire of most players can best be summed out by modifying a quote from Rod Tidwell of Jerry Maguire: "Show me the Pokemon!" [Read more] | Read more »
Rodeo Stampede update: Mountains, new an...
The Savannah and Jungle were just the beginning in Rodeo Stampede. Get ready to head for the Mountains. I think I heard that in a beer ad once. [Read more] | Read more »
COSMOS RINGS (Games)
COSMOS RINGS 1.0.0 Device: iOS iPhone Category: Games Price: $5.99, Version: 1.0.0 (iTunes) Description: This game cannot be played without the Apple Watch.Released anniversary sale until August 31,2016 PST! A tragic tale of time's... | Read more »
How to get started selling on Mercari
As far as ecommerce has come over the last decade or so, there's still a tremendous opportunity to make it easier for people to buy and sell goods. That's especially true when it comes to shopping apps, which should only continue to increase in... | Read more »
Human Anatomy Atlas 2017 Edition - Compl...
Human Anatomy Atlas 2017 Edition - Complete 3D Human Body 1.0.24 Device: iOS iPhone Category: Medical Price: $24.99, Version: 1.0.24 (iTunes) Description: | Read more »
Heroes of Normandie (Games)
Heroes of Normandie 1.5 Device: iOS Universal Category: Games Price: $14.99, Version: 1.5 (iTunes) Description: The game does not support iPhone 4s and below | Read more »
Why you should never power up Pokemon in...
There's no question that candy is dandy in Pokemon GO. You need big quantities of it to evolve your Pokemon, and when combined with stardust, it can be used to power up your favorite pocket monsters as well, making them more formidable for the gym... | Read more »
Webzen launches 3D MMORPG MU Origin on i...
Mu Origin is featured time and time again at the very top of App Stores in China, and within the top five worldwide top-grossing charts on Google Play.Its popularity in Korea and China, featuring more than 120 registered players in China and 6... | Read more »
Severed (Games)
Severed 1.0 Device: iOS Universal Category: Games Price: $5.99, Version: 1.0 (iTunes) Description: LAUNCH DISCOUNT ON NOW!! ENDS AUGUST 4! ==== Take control of a one-armed warrior named Sasha, wielding a living sword on her journey... | Read more »

Price Scanner via MacPrices.net

9-inch 32GB Space Gray iPad Pro on sale for $...
B&H Photo has the 9″ 32GB WiFi Space Gray Apple iPad Pro on sale for $50 off MSRP including free shipping. B&H charges sales tax in NY only: - 9″ Space Gray 32GB WiFi iPad Pro: $549 $50 off... Read more
15-inch Retina MacBook Pros on sale for up to...
B&H Photo has 15″ Retina MacBook Pros on sale for up to $200 off MSRP. Shipping is free, and B&H charges NY tax only: - 15″ 2.2GHz Retina MacBook Pro: $1849 $150 off MSRP - 15″ 2.5GHz Retina... Read more
Second-Quarter Tablet Shipments Fell 4.8% –...
The latest report from the global market research firm TrendForce finds that worldwide tablet shipments for this second quarter totaled 33.54 million units, representing a quarterly drop of 4.8% and... Read more
Global Smartphone Sales Volumes Mark Second S...
According to preliminary results from the International Data Corporation (IDC) Worldwide Quarterly Mobile Phone Tracker, vendors shipped a total of 343.3 million smartphones worldwide in the second... Read more
Apple TVs on sale for $20-$40 off MSRP
Best Buy has 32GB and 64GB Apple TVs on sale for $20-$40 off MSRP on their online store. Choose free shipping or free local store pickup (if available). Sale prices for online orders only, in-store... Read more
Mac minis on sale for $50-$100 off MSRP
B&H Photo has Mac minis on sale for $50 off MSRP including free shipping plus NY sales tax only: - 1.4GHz Mac mini: $449 $50 off MSRP - 2.6GHz Mac mini: $649 $50 off MSRP - 2.8GHz Mac mini: $949... Read more
Clearance 2015 13-inch MacBook Airs available...
B&H Photo has clearance 2015 13″ MacBook Airs available for $300 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 1.6GHz/4GB/128GB MacBook Air (MJVE2LL/A): $799... Read more
Apple certified refurbished iPad mini 4s avai...
Apple has certified refurbished iPad mini 4s now available for up to $120 off the cost of new models. An Apple one-year warranty is included with each iPad, and shipping is free. The following models... Read more
Notebook Makers In No Rush To Adopt USB-C – R...
Digitimes’ Cage Chao and Joseph Tsai note that while the USB Type-C interface is enjoying growing popularity among smartphones and tablet makers, notebook and all-in-one (AIO) PC vendors (other than... Read more
iMacs on sale for up to $250 off MSRP
B&H Photo has 21″ and 27″ Apple iMacs on sale for up to $250 off MSRP including free shipping plus NY sales tax only: - 27″ 3.3GHz iMac 5K: $2049 $250 off MSRP - 27″ 3.2GHz/1TB Fusion iMac 5K: $... Read more

Jobs Board

*Apple* Retail - Bilingual - Multiple Positi...
…speaking a plus Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the Read more
Simply Mac *Apple* Specialist- Repair Techn...
…The Technician is a master at working with our customers to diagnose and repair Apple devices in a manner that exceeds the expectations set forth by Apple Read more
*Apple* Mobile Master - Best Buy (United Sta...
What does a Best Buy Apple Mobile Master do? At Best Buy, our mission is to leverage the unique talents and passions of our employees to inspire, delight, and enrich Read more
Best Buy *Apple* Computing Master - Best Bu...
What does a Best Buy Apple Computing Master do? At Best Buy our mission is to leverage the unique talents and passions of our employees to inspire, delight, and Read more
*Apple* Valley, CA School Speech Therapy Ope...
Apple Valley, CA School Speech Therapy Openings + Job Location: Apple Valley, CA + Category: Schools - SLP - CFY + Apply Now! + Back to Results Speech Language Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.