04-Test Suite and Results
TweetFollow Us on Twitter

04-Test Suite and Results

The Test Suite and Results

In the sections below, we'll walk you through what we tested, and the results for each. These tests are designed to arm you with the information so you can make the best decision for your type of use.

For each set of results, you can see the analysis for each model of computer for XP, and for Windows 7. If you want to see more detail for single vs. multiple processors, or on an individual Mac model, you may want to review the spreadsheet for those details.

As you look through the charts below, take note on whether faster is represented by longer or shorter bars (see the lower left corner of each chart).

For the launch tests (launching the VM, Windows, and Applications), we had the option of an "Adam" test, and a "Successive" test. Adam tests are when the computer has been completely restarted (hence avoiding both host and guest OS caching). Successive tests are repeated tests without restarting the machine in between tests, and can benefit from caching. Both mimic real use situations.

The tests used were selected specifically to give a real-world view of what VMware Fusion and Parallels Desktop are like to run for many users. We eliminated those tests that we ran which were so short in time frame (e.g., fast) that we could not create statistically significant results, or that had unperceivable differences.

For some of the analysis, we "normalized" results by dividing the result by the fastest result for that test across all the machine configurations. We did this specifically so that we could make comparisons across different groups, and to be able to give you overview results combining a series of types of tests, and computer models.

Instead of a plain "average" or "mean", overall conclusions are done using a "geomean", which is a specific type of average that focuses on the central results and minimizes outliers. Geomean is the same averaging methodology used by SPEC tests, PCMark, Unixbench, and others, and it helps prevent against minor result skewing. (If you are interested in how it differs from a mean, instead of adding the set of numbers and then dividing the sum by the count of numbers in the set, n, the numbers are multiplied and then the nth root of the resulting product is taken).

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, see Appendix B. Both appendices are available on the MacTech web site.

Launch and CPU Tests

There are three situations in which users commonly launch a virtual machine:

  • Launch the virtual machine from "off" mode, including a full Windows boot.
  • Launch the virtual machine from a suspended state, and resume from suspend (Adam).
  • Launch the virtual machine from a suspended state, and resume from suspend (Successive).

For the first test, we started at the Finder and launched the virtualization application, which were set up to immediately launch the virtual machine. The visual feedback is fairly different between Parallels Desktop and VMware Fusion when Windows first starts up. Windows actually does its startup for quite some time after reaching the desktop. In some cases, it can take quite some time for Windows to complete its boot process. Most users don't care if things continue, so long as they aren't held up.

As a result, we focused on timing to the point of actually accomplishing something. In this case, we hovered over the Task Bar icons and launched the Windows Security Center window (XP), or the networking interface icon (Windows 7). The test ended when the window started to render. This gave us a real world scenario of being able to actually do something as opposed to Windows just looking like it was booted.

The primary difference between the last two types of VM launch test is that the computer is fully rebooted (both the virtual machine as well as Mac OS X) in between the "Adam" tests. The successive tests are launching the virtual machines and restoring them without restarting the Mac in between.

Successive tests benefit from both Mac OS X and possibly virtual machine caching, and are significantly faster. However, you may only see these types of situations if you are constantly switching in and out of your virtual machine.

As with all of our tests, we perform these tests multiple times to handle the variability that can occur. Of these, we took the best results for each product.

Description: VMPerf
Virtual Machine Performance

Clearly, machines with more memory take longer to restore and that accounts for some of the differences (remember, we selected RAM configurations based on VMware Fusion's default which varied by OS and Mac model). One thing to remember as a virtualization user is that if you are switching in and out of a VM often, you may want to think about using less RAM, not more. In fact, you should just use as little as you need for the best experience under either virtualized environment. (We suggest 512MB to 1GB for most people. 512MB to 768MB on 2GB MacBooks).

Most benchmarking suites measure CPU performance using file compression as at least one part of their analysis. We did the same. As a matter of interest, we used compression instead of decompression, because with today's fast computers, decompression is actually much closer to a file copy than it is to CPU work. Compression requires the system to perform analysis to do the compression, and is therefore a better measurement of CPU.

VMware Fusion performed better in the compression test under Windows 7, and Parallels Desktop did better in the same test under Windows XP. For tests related to Windows boot time, suspend and resume, Parallels Desktop was noticeably quicker than VMware Fusion.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Printopia 3.0.4 - Share Mac printers wit...
Run Printopia on your Mac to share its printers to any capable iPhone, iPad, or iPod Touch. Printopia will also add virtual printers, allowing you to save print-outs to your Mac and send to apps.... Read more
Tinderbox 7.3.1 - Store and organize you...
Tinderbox is a personal content management assistant. It stores your notes, ideas, and plans. It can help you organize and understand them. And Tinderbox helps you share ideas through Web journals... Read more
ExpanDrive 6.1.6 - Access cloud storage...
ExpanDrive builds cloud storage in every application, acts just like a USB drive plugged into your Mac. With ExpanDrive, you can securely access any remote file server directly from the Finder or... Read more
VOX 3.0.1 - Music player that supports m...
VOX just sounds better! The beauty is in its simplicity, yet behind the minimal exterior lies a powerful music player with a ton of features and support for all audio formats you should ever need.... Read more
Merlin Project 4.3.3 - $289.00
Merlin Project is the leading professional project management software for OS X. If you plan complex projects on your Mac, you won’t get far with a simple list of tasks. Good planning raises... Read more
Mac DVDRipper Pro 7.1 - Copy, backup, an...
Mac DVDRipper Pro is the DVD backup solution that lets you protect your DVDs from scratches, save your batteries by reading your movies from your hard disk, manage your collection with just a few... Read more
iMazing 2.5.2 - Complete iOS device mana...
iMazing (was DiskAid) is the ultimate iOS device manager with capabilities far beyond what iTunes offers. With iMazing and your iOS device (iPhone, iPad, or iPod), you can: Copy music to and from... Read more
Pinegrow 4 - Mockup and design webpages...
Pinegrow (was Pinegrow Web Designer) is desktop app that lets you mockup and design webpages faster with multi-page editing, CSS and LESS styling, and smart components for Bootstrap, Foundation,... Read more
iExplorer 4.1.11 - View and transfer fil...
iExplorer is an iPhone browser for Mac lets you view the files on your iOS device. By using a drag and drop interface, you can quickly copy files and folders between your Mac and your iPhone or... Read more
Merlin Project 4.3.3 - $289.00
Merlin Project is the leading professional project management software for OS X. If you plan complex projects on your Mac, you won’t get far with a simple list of tasks. Good planning raises... Read more

Latest Forum Discussions

See All

The best Black Friday mobile game deals
Black Friday's upon us, and if you've happened to nab a fancy new phone during the week's big savings, you might be searching for some new games to fill up space on your new gadget. There are a lot of great games on sale right now for Black Friday... | Read more »
The best mobile games to play while your...
Thanksgiving is a time to reconnect with loved ones, eat lots of food, and all of that jazz, but once the festivities start to wind down, folks tend to head to the couch to watch whatever football is happening for Turkey Day. | Read more »
The best Black Friday deals for Apple ga...
Black Friday is hours away at this point, but many popular retailers are getting a jump on things with plenty of pre-Black Friday sales already available. Many of those early bird sales including some sharp discounts on the latest Apple phones... | Read more »
The Inner World 2 (Games)
The Inner World 2 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: Solve mind-bending puzzles in a world full of mystery and save the family of the flute-noses! Their dynasty has been... | Read more »
warbot.io wants you for the robot wars
Fans of epic gundam-style battles will find a lot to love in warbot.io, the first game for up and coming developer Wondersquad. The game saw a lot of success when it first launched for browsers and Facebook, and now even more people are getting the... | Read more »
Uncover alien mysteries in cross-genre s...
If the Alien franchise taught us anything, it’s that landing on a strange planet at the behest of a faceless corporation is probably asking for trouble. And Eldritch Game’s Deliria doesn’t prove otherwise. In 2107, Dimension LG7 is rich with... | Read more »
The best mobile games to play during dre...
| Read more »
Animal Crossing: Pocket Camp beginner...
Animal Crossing: Pocket Camp, was just announced yesterday, but it's already in soft launch in Australia. No matter where you are in the world, you can still get access to the soft launch on iOS, so we've devised a few beginner tips for folks who... | Read more »
The mobile gamer's guide to Black F...
We're starting to catch wind of some exciting deals in the mobile gaming space for Black Friday. There are big discounts on mobile phones and accessories cropping up already, so you might want to get a move on things ahead of the big day. It's... | Read more »
The best pre-Black Friday deals - Novemb...
Black Friday will soon be upon us, but online retailers are already getting a headstart on the steep discounts. Don't wait until Friday—you'll find some pretty good deals all over the internet without waiting in lines or competing with other... | Read more »

Price Scanner via MacPrices.net

Black Friday 2017: Find the best deals and lo...
Scan our exclusive price trackers for the latest Black Friday 2017 sales & deals and the lowest prices available on Apple Macs, iPads, and gear from Apple’s authorized resellers. We update the... Read more
Black Friday: 27″ 3.4GHz iMac for $1599, save...
Amazon has the 27″ 3.4GHz Apple iMac on sale for $1599.99 as part of their Black Friday sale. That’s $200 off MSRP, and shipping is free. Their price is currently the lowest price available for this... Read more
Black Friday: 13″ 2.3GHz/256GB MacBook Pro fo...
Amazon has the 13″ 2.3GHz/256GB Apple MacBook Pro on sale for $1299.99 as part of their Black Friday sale. Shipping is free: – 13-inch 2.3GHz/256GB Space Gray MacBook Pro (MPXT2LL/A): $1299.99 $200... Read more
Black Friday: 15″ 2.9GHz MacBook Pros for $25...
Amazon has lowered prices on Silver and Gray 15″ 2.9GHz MacBook Pros to $2549.99. That’s $250 off MSRP, and shipping is free. Their prices are the lowest available for these models from any reseller... Read more
Lowest Black Friday prices on Apple MacBooks:...
Save $150-$420 on the purchase of a MacBook Pro, MacBook, or MacBook Air this Black Friday and Holiday weekend with Certified Refurbished models at Apple. In many cases, Apple’s refurbished prices... Read more
Black Friday: Apple Watch Series 1 for $70 of...
Macy’s has discounted Series 1 Apple Watches by $70 on their online store as part of their Black Friday sale: – 38mm Series 1 Apple Watch: $179, $70 off – 42mm Series 1 Apple Watch: $209, $70 off... Read more
Apple offers 2016 13-inch MacBook Airs, certi...
Apple has Certified Refurbished 2016 13″ MacBook Airs available starting at $809. An Apple one-year warranty is included with each MacBook, and shipping is free: – 13″ 1.6GHz/8GB/128GB MacBook Air: $... Read more
Black Friday sale: Mac minis for $100 off MSR...
B&H Photo has Mac minis on sale for up to $100 off MSRP as part of their Black Friday sale, each including free shipping plus NY & NJ sales tax only: – 1.4GHz Mac mini: $399 $100 off MSRP – 2... Read more
Use your Apple Education discount to save up...
Purchase a new Mac using Apple’s Education discount, and take up to $300 off MSRP. All teachers, students, and staff of any educational institution with a .edu email address qualify for the discount... Read more
Adorama posts Black Friday deals on Apple Mac...
Adorama has posted Black Friday sale prices on many Macs, with MacBooks and iMacs available for up to $200 off MSRP. Shipping is free, and Adorama charges sales tax in NJ and NY only: MacBook Pros... Read more

Jobs Board

Business Development Manager, *Apple* Pay -...
# Business Development Manager, Apple Pay Job Number: 112919084 Santa Clara Valley, California, United States Posted: 18-Aug-2017 Weekly Hours: 40.00 **Job Summary** Read more
Digital Marketing Media Planner, *Apple* Se...
# Digital Marketing Media Planner, Apple Services Job Number: 113080212 Culver City, California, United States Posted: 03-Oct-2017 Weekly Hours: **Job Summary** Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Business Development Manager, *Apple* Pay -...
# Business Development Manager, Apple Pay Job Number: 112919084 Santa Clara Valley, California, United States Posted: 18-Aug-2017 Weekly Hours: 40.00 **Job Summary** Read more
*Apple* Solutions Consultant - Apple (United...
# Apple Solutions Consultant Job Number: 56553863 North Wales, Pennsylvania, United States Posted: 17-Jun-2017 Weekly Hours: 40.00 **Job Summary** Are you passionate Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.