04-Test Suite and Results
TweetFollow Us on Twitter

04-Test Suite and Results

The Test Suite and Results

In the sections below, we'll walk you through what we tested, and the results for each. These tests are designed to arm you with the information so you can make the best decision for your type of use.

For each set of results, you can see the analysis for each model of computer for XP, and for Windows 7. If you want to see more detail for single vs. multiple processors, or on an individual Mac model, you may want to review the spreadsheet for those details.

As you look through the charts below, take note on whether faster is represented by longer or shorter bars (see the lower left corner of each chart).

For the launch tests (launching the VM, Windows, and Applications), we had the option of an "Adam" test, and a "Successive" test. Adam tests are when the computer has been completely restarted (hence avoiding both host and guest OS caching). Successive tests are repeated tests without restarting the machine in between tests, and can benefit from caching. Both mimic real use situations.

The tests used were selected specifically to give a real-world view of what VMware Fusion and Parallels Desktop are like to run for many users. We eliminated those tests that we ran which were so short in time frame (e.g., fast) that we could not create statistically significant results, or that had unperceivable differences.

For some of the analysis, we "normalized" results by dividing the result by the fastest result for that test across all the machine configurations. We did this specifically so that we could make comparisons across different groups, and to be able to give you overview results combining a series of types of tests, and computer models.

Instead of a plain "average" or "mean", overall conclusions are done using a "geomean", which is a specific type of average that focuses on the central results and minimizes outliers. Geomean is the same averaging methodology used by SPEC tests, PCMark, Unixbench, and others, and it helps prevent against minor result skewing. (If you are interested in how it differs from a mean, instead of adding the set of numbers and then dividing the sum by the count of numbers in the set, n, the numbers are multiplied and then the nth root of the resulting product is taken).

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, see Appendix B. Both appendices are available on the MacTech web site.

Launch and CPU Tests

There are three situations in which users commonly launch a virtual machine:

  • Launch the virtual machine from "off" mode, including a full Windows boot.
  • Launch the virtual machine from a suspended state, and resume from suspend (Adam).
  • Launch the virtual machine from a suspended state, and resume from suspend (Successive).

For the first test, we started at the Finder and launched the virtualization application, which were set up to immediately launch the virtual machine. The visual feedback is fairly different between Parallels Desktop and VMware Fusion when Windows first starts up. Windows actually does its startup for quite some time after reaching the desktop. In some cases, it can take quite some time for Windows to complete its boot process. Most users don't care if things continue, so long as they aren't held up.

As a result, we focused on timing to the point of actually accomplishing something. In this case, we hovered over the Task Bar icons and launched the Windows Security Center window (XP), or the networking interface icon (Windows 7). The test ended when the window started to render. This gave us a real world scenario of being able to actually do something as opposed to Windows just looking like it was booted.

The primary difference between the last two types of VM launch test is that the computer is fully rebooted (both the virtual machine as well as Mac OS X) in between the "Adam" tests. The successive tests are launching the virtual machines and restoring them without restarting the Mac in between.

Successive tests benefit from both Mac OS X and possibly virtual machine caching, and are significantly faster. However, you may only see these types of situations if you are constantly switching in and out of your virtual machine.

As with all of our tests, we perform these tests multiple times to handle the variability that can occur. Of these, we took the best results for each product.

Description: VMPerf
Virtual Machine Performance

Clearly, machines with more memory take longer to restore and that accounts for some of the differences (remember, we selected RAM configurations based on VMware Fusion's default which varied by OS and Mac model). One thing to remember as a virtualization user is that if you are switching in and out of a VM often, you may want to think about using less RAM, not more. In fact, you should just use as little as you need for the best experience under either virtualized environment. (We suggest 512MB to 1GB for most people. 512MB to 768MB on 2GB MacBooks).

Most benchmarking suites measure CPU performance using file compression as at least one part of their analysis. We did the same. As a matter of interest, we used compression instead of decompression, because with today's fast computers, decompression is actually much closer to a file copy than it is to CPU work. Compression requires the system to perform analysis to do the compression, and is therefore a better measurement of CPU.

VMware Fusion performed better in the compression test under Windows 7, and Parallels Desktop did better in the same test under Windows XP. For tests related to Windows boot time, suspend and resume, Parallels Desktop was noticeably quicker than VMware Fusion.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 
AAPL
$104.83
Apple Inc.
+1.84
MSFT
$45.02
Microsoft Corpora
+0.64
GOOG
$543.98
Google Inc.
+11.27

MacTech Search:
Community Search:

Software Updates via MacUpdate

Delicious Library 3.3.2 - Import, browse...
Delicious Library allows you to import, browse, and share all your books, movies, music, and video games with Delicious Library. Run your very own library from your home or office using our... Read more
Art Text 2.4.8 - Create high quality hea...
Art Text is an OS X application for creating high quality textual graphics, headings, logos, icons, Web site elements, and buttons. Thanks to multi-layer support, creating complex graphics is no... Read more
Live Interior 3D Pro 2.9.6 - Powerful an...
Live Interior 3D Pro is a powerful yet very intuitive interior designing application. View Video Tutorials It has every feature of Live Interior 3D Standard, plus some exclusive ones: Create multi... Read more
The Hit List 1.1.7 - Advanced reminder a...
The Hit List manages the daily chaos of your modern life. It's easy to learn - it's as easy as making lists. And it's powerful enough to let you plan, then forget, then act when the time is right.... Read more
jAlbum Pro 12.2.4 - Organize your digita...
jAlbum Pro has all the features you love in jAlbum, but comes with a commercial license. With jAlbum, you can create gorgeous custom photo galleries for the Web without writing a line of code!... Read more
jAlbum 12.2.4 - Create custom photo gall...
With jAlbum, you can create gorgeous custom photo galleries for the Web without writing a line of code! Beginner-friendly, with pro results Simply drag and drop photos into groups, choose a design... Read more
ExpanDrive 4.1.7 - Access remote files o...
ExpanDrive builds cloud storage in every application, acts just like a USB drive plugged into your Mac. With ExpanDrive, you can securely access any remote file server directly from the Finder or... Read more
OmniOutliner Pro 4.1.3 - Pro version of...
OmniOutliner Pro is a flexible program for creating, collecting, and organizing information. Give your creativity a kick start by using an application that's actually designed to help you think. It'... Read more
Evernote 5.6.2 - Create searchable notes...
Evernote allows you to easily capture information in any environment using whatever device or platform you find most convenient, and makes this information accessible and searchable at anytime, from... Read more
OmniOutliner 4.1.3 - Organize your ideas...
OmniOutliner is a flexible program for creating, collecting, and organizing information. Give your creativity a kick start by using an application that's actually designed to help you think. It's... Read more

Latest Forum Discussions

See All

A Few Days Left (Games)
A Few Days Left 1.01 Device: iOS Universal Category: Games Price: $3.99, Version: 1.01 (iTunes) Description: Screenshots are in compliance to App Store's 4+ age rating! Please see App Preview for real game play! **Important: Make... | Read more »
Toca Boo (Education)
Toca Boo 1.0.2 Device: iOS Universal Category: Education Price: $2.99, Version: 1.0.2 (iTunes) Description: BOO! Did I scare you!? My name is Bonnie and my family loves to spook! Do you want to scare them back? Follow me and I'll... | Read more »
Intuon (Games)
Intuon 1.1 Device: iOS Universal Category: Games Price: $.99, Version: 1.1 (iTunes) Description: Join the battle with your intuition in a new hardcore game Intuon! How well do you trust your intuition? Can you find a needle in a... | Read more »
Ravenous Rampage (Games)
Ravenous Rampage 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: | Read more »
Partia 2 (Games)
Partia 2 1.0.1 Device: iOS Universal Category: Games Price: $5.99, Version: 1.0.1 (iTunes) Description: Partia 2 is a SRPG (Strategy Role-playing) video game inspired by Fire Emblem and Tear Ring Saga series. In a high fantasy... | Read more »
Puzzle to the Center of the Earth Review
Puzzle to the Center of the Earth Review By Campbell Bird on October 23rd, 2014 Our Rating: :: SPELUNKING PUZZLESUniversal App - Designed for iPhone and iPad Do some puzzles to make some platforms in this smart and fun free-to-play... | Read more »
Sleep Attack TD Review
Sleep Attack TD Review By Jennifer Allen on October 23rd, 2014 Our Rating: :: A TRUE TWISTUniversal App - Designed for iPhone and iPad Sleep Attack TD is a tower defense game with a difference – you can rotate the layout – and it’s... | Read more »
Mecanic (Education)
Mecanic 1.0 Device: iOS Universal Category: Education Price: $1.99, Version: 1.0 (iTunes) Description: Plates, screws, wheels ... Everything you need to achieve whatever you want... MECHANICWith 'MECANIC' kids will have fun... | Read more »
Earn Your Master Camper Badge in Camp Po...
Earn Your Master Camper Badge in Camp Pokemon Posted by Jessica Fisher on October 23rd, 2014 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Garruk Gets His Revenge in a New Magic 2...
Garruk Gets His Revenge in a New Magic 2015 Expansion, Coming This November Posted by Jessica Fisher on October 23rd, 2014 [ permalink ] | Read more »

Price Scanner via MacPrices.net

13-inch 2.5GHz MacBook Pro on sale for $949,...
Best Buy has the 13″ 2.5GHz MacBook Pro available for $949.99 on their online store. Choose free shipping or free instant local store pickup (if available). Their price is $150 off MSRP. Price is... Read more
Save up to $125 on Retina MacBook Pros
B&H Photo has the new 2014 13″ and 15″ Retina MacBook Pros on sale for up to $125 off MSRP. Shipping is free, and B&H charges NY sales tax only. They’ll also include free copies of Parallels... Read more
Apple refurbished Time Capsules available sta...
The Apple Store has certified refurbished Time Capsules available for up to $60 off MSRP. Apple’s one-year warranty is included with each Time Capsule, and shipping is free: - 2TB Time Capsule: $255... Read more
Textilus New Word, Notes and PDF Processor fo...
Textilus is new word-crunching, notes, and PDF processor designed exclusively for the iPad. I haven’t had time to thoroughly check it out yet, but it looks great and early reviews are positive.... Read more
WD My Passport Pro Bus-Powered Thunderbolt RA...
WD’s My Passport Pro RAID solution is powered by an integrated Thunderbolt cable for true portability and speeds as high as 233 MB/s. HighlightsOverviewSpecifications Transfer, Back Up And Edit In... Read more
Save with Best Buy’s College Student Deals
Take an additional $50 off all MacBooks and iMacs at Best Buy Online with their College Students Deals Savings, valid through November 1st. Anyone with a valid .EDU email address can take advantage... Read more
iPad Air 2 & iPad mini 3 Best Tablets Yet...
The new iPads turned out to be pretty much everything I’d been hoping for and more than I’d expected.”More” particularly in terms of a drinking-from-a-firehose choice of models and configurations,... Read more
Drafts 4 Reinvents iOS Productivity App
N Richland Hills, Texas based Agile Tortoise has announced the release of Drafts 4 for iPhone and iPad. Drafts is a quick capture note taking app with flexible output actions. Drafts 4 scales from... Read more
AT&T accepting preorders for new iPads fo...
AT&T Wireless is accepting preorders for the new iPad Air 2 and iPad mini 3, cellular models, for $100 off MSRP with a 2-year service agreement: - 16GB iPad Air 2 WiFi + Cellular: $529.99 - 64GB... Read more
Apple offering refurbished Mac Pros for up to...
The Apple Store is offering Apple Certified Refurbished 2013 Mac Pros for up to $600 off the cost of new models. An Apple one-year warranty is included with each Mac Pro, and shipping is free. The... Read more

Jobs Board

*Apple* Solutions Consultant - Apple Inc. (U...
…important role that the ASC serves is that of providing an excellent Apple Customer Experience. Responsibilities include: * Promoting Apple products and solutions Read more
Senior Event Manager, *Apple* Retail Market...
…This senior level position is responsible for leading and imagining the Apple Retail Team's global event strategy. Delivering an overarching brand story; in-store, Read more
*Apple* Solutions Consultant (ASC) - Apple (...
**Job Summary** The ASC is an Apple employee who serves as an Apple brand ambassador and influencer in a Reseller's store. The ASC's role is to grow Apple Read more
Project Manager / Business Analyst, WW *Appl...
…a senior project manager / business analyst to work within our Worldwide Apple Fulfillment Operations and the Business Process Re-engineering team. This role will work Read more
*Apple* Retail - Multiple Positions (US) - A...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.