02-Test Types and Overview
TweetFollow Us on Twitter

02-Test Types and Overview

Test Types

There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, WorldBench, Performance Test, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.

When testing virtual machines, however, using outside utilities within a virtualized environment can provide strange and incorrect results. For example, we saw one testing tool that reported graphics running twice as fast when you could clearly see on the screen that it was running at half the speed. Other tests use older versions of applications. Clearly, if one is going to use testing tools, they need to be carefully picked: the results need to reflect what the user sees in real world conditions.

Instead, as we have in the past, MacTech chose to create a suite of tests that would mimic what many users would do in normal use, but stick to those actions that were repeatable, and measurable with a stopwatch. After all, if you couldn't perceive a difference with a stopwatch, the user is not likely to perceive it at all.

There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results. This time, we had the opportunity to use a tool that measures frames per second to verify the numbers reported by 3DMark06.

To that end, there are several kinds of tests that we ran: Virtual Machine Launch/Suspend, CPU tests, File and Network IO, CPU Usage Footprint, Application Performance, 3D and verification of real world game play.

Overview

We won't keep you in suspense. When we look at the "big picture" of all the testing, Parallels is the clear winner. If you count up the general tests (including one 3D graphics score), Parallels won 61% of the tests by 10% or more, and was also a bit faster on an additional 23% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 84% of the general tests we ran.

Description: Tally-Gen
Test Tally: General Virtualization Tests

If you focus exclusively on 3D graphics, as measured by 3DMark06 version 1.2, Parallels won by an even larger margin. Specifically, Parallels won 73% of the tests by 10% or more, and was also a bit faster on an additional 19% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 92% of the 3D graphics tests we ran.

When VMware Fusion was faster in 3D graphics, it was typically on the HDR/SM3.0 Score and the Batch Size tests (e.g., triangles). But, as you can see from both the test tally, and the colored cell worksheet overview, Parallels Desktop was overwhelmingly faster in graphics, and in working with real games, it was easy to see confirmation of the 3DMark06 scores.

Description: Tally-3D
Test Tally: 3D Graphics Tests

There are a handful of places that VMware Fusion consistently was faster than Parallels Desktop. In Microsoft Word 2010, VMware launched (initial launch after boot, aka "Adam" launch) faster, and was faster on large, global search and replaces. It was also consistently faster under Windows 7 doing file compression, and loading SSL pages.

Overall, VMware Fusion won 6% of the tests by at least 10%, and was also a bit faster on an additional 4% more of the general tests. For the 3D tests, VMware Fusion won 3% of the tests by at least 10%, and was also a bit faster on an additional 3% more for a total of 6%.

One of the best ways to visualize the huge amount of data points is through MacTech's well-known "Colored Cell Worksheet Overview." In these worksheets, a cell represents the best result for each test for each version of Windows for each virtualization product. These are then colored according to which product was faster.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Soft Drummer (Music)
Soft Drummer 1.0 Device: iOS Universal Category: Music Price: $14.99, Version: 1.0 (iTunes) Description: Soft Drummer is the closest thing to a pro session subtle drummer in your pocket. Easy to use and fast, it's much more than a... | Read more »
Is GO Gear the Pokemon GO map app you...
Now that we've settled into something of a Pokemon GO status quo, the number one desire of most players can best be summed out by modifying a quote from Rod Tidwell of Jerry Maguire: "Show me the Pokemon!" [Read more] | Read more »
Rodeo Stampede update: Mountains, new an...
The Savannah and Jungle were just the beginning in Rodeo Stampede. Get ready to head for the Mountains. I think I heard that in a beer ad once. [Read more] | Read more »
COSMOS RINGS (Games)
COSMOS RINGS 1.0.0 Device: iOS iPhone Category: Games Price: $5.99, Version: 1.0.0 (iTunes) Description: This game cannot be played without the Apple Watch.Released anniversary sale until August 31,2016 PST! A tragic tale of time's... | Read more »
How to get started selling on Mercari
As far as ecommerce has come over the last decade or so, there's still a tremendous opportunity to make it easier for people to buy and sell goods. That's especially true when it comes to shopping apps, which should only continue to increase in... | Read more »
Human Anatomy Atlas 2017 Edition - Compl...
Human Anatomy Atlas 2017 Edition - Complete 3D Human Body 1.0.24 Device: iOS iPhone Category: Medical Price: $24.99, Version: 1.0.24 (iTunes) Description: | Read more »
Heroes of Normandie (Games)
Heroes of Normandie 1.5 Device: iOS Universal Category: Games Price: $14.99, Version: 1.5 (iTunes) Description: The game does not support iPhone 4s and below | Read more »
Why you should never power up Pokemon in...
There's no question that candy is dandy in Pokemon GO. You need big quantities of it to evolve your Pokemon, and when combined with stardust, it can be used to power up your favorite pocket monsters as well, making them more formidable for the gym... | Read more »
Webzen launches 3D MMORPG MU Origin on i...
Mu Origin is featured time and time again at the very top of App Stores in China, and within the top five worldwide top-grossing charts on Google Play.Its popularity in Korea and China, featuring more than 120 registered players in China and 6... | Read more »
Severed (Games)
Severed 1.0 Device: iOS Universal Category: Games Price: $5.99, Version: 1.0 (iTunes) Description: LAUNCH DISCOUNT ON NOW!! ENDS AUGUST 4! ==== Take control of a one-armed warrior named Sasha, wielding a living sword on her journey... | Read more »

Price Scanner via MacPrices.net

9-inch 32GB Space Gray iPad Pro on sale for $...
B&H Photo has the 9″ 32GB WiFi Space Gray Apple iPad Pro on sale for $50 off MSRP including free shipping. B&H charges sales tax in NY only: - 9″ Space Gray 32GB WiFi iPad Pro: $549 $50 off... Read more
15-inch Retina MacBook Pros on sale for up to...
B&H Photo has 15″ Retina MacBook Pros on sale for up to $200 off MSRP. Shipping is free, and B&H charges NY tax only: - 15″ 2.2GHz Retina MacBook Pro: $1849 $150 off MSRP - 15″ 2.5GHz Retina... Read more
Second-Quarter Tablet Shipments Fell 4.8% –...
The latest report from the global market research firm TrendForce finds that worldwide tablet shipments for this second quarter totaled 33.54 million units, representing a quarterly drop of 4.8% and... Read more
Global Smartphone Sales Volumes Mark Second S...
According to preliminary results from the International Data Corporation (IDC) Worldwide Quarterly Mobile Phone Tracker, vendors shipped a total of 343.3 million smartphones worldwide in the second... Read more
Apple TVs on sale for $20-$40 off MSRP
Best Buy has 32GB and 64GB Apple TVs on sale for $20-$40 off MSRP on their online store. Choose free shipping or free local store pickup (if available). Sale prices for online orders only, in-store... Read more
Mac minis on sale for $50-$100 off MSRP
B&H Photo has Mac minis on sale for $50 off MSRP including free shipping plus NY sales tax only: - 1.4GHz Mac mini: $449 $50 off MSRP - 2.6GHz Mac mini: $649 $50 off MSRP - 2.8GHz Mac mini: $949... Read more
Clearance 2015 13-inch MacBook Airs available...
B&H Photo has clearance 2015 13″ MacBook Airs available for $300 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 1.6GHz/4GB/128GB MacBook Air (MJVE2LL/A): $799... Read more
Apple certified refurbished iPad mini 4s avai...
Apple has certified refurbished iPad mini 4s now available for up to $120 off the cost of new models. An Apple one-year warranty is included with each iPad, and shipping is free. The following models... Read more
Notebook Makers In No Rush To Adopt USB-C – R...
Digitimes’ Cage Chao and Joseph Tsai note that while the USB Type-C interface is enjoying growing popularity among smartphones and tablet makers, notebook and all-in-one (AIO) PC vendors (other than... Read more
iMacs on sale for up to $250 off MSRP
B&H Photo has 21″ and 27″ Apple iMacs on sale for up to $250 off MSRP including free shipping plus NY sales tax only: - 27″ 3.3GHz iMac 5K: $2049 $250 off MSRP - 27″ 3.2GHz/1TB Fusion iMac 5K: $... Read more

Jobs Board

*Apple* Retail - Bilingual - Multiple Positi...
…speaking a plus Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the Read more
Simply Mac *Apple* Specialist- Repair Techn...
…The Technician is a master at working with our customers to diagnose and repair Apple devices in a manner that exceeds the expectations set forth by Apple Read more
*Apple* Mobile Master - Best Buy (United Sta...
What does a Best Buy Apple Mobile Master do? At Best Buy, our mission is to leverage the unique talents and passions of our employees to inspire, delight, and enrich Read more
Best Buy *Apple* Computing Master - Best Bu...
What does a Best Buy Apple Computing Master do? At Best Buy our mission is to leverage the unique talents and passions of our employees to inspire, delight, and Read more
*Apple* Valley, CA School Speech Therapy Ope...
Apple Valley, CA School Speech Therapy Openings + Job Location: Apple Valley, CA + Category: Schools - SLP - CFY + Apply Now! + Back to Results Speech Language Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.