02-Test Types and Overview
TweetFollow Us on Twitter

02-Test Types and Overview

Test Types

There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, WorldBench, Performance Test, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.

When testing virtual machines, however, using outside utilities within a virtualized environment can provide strange and incorrect results. For example, we saw one testing tool that reported graphics running twice as fast when you could clearly see on the screen that it was running at half the speed. Other tests use older versions of applications. Clearly, if one is going to use testing tools, they need to be carefully picked: the results need to reflect what the user sees in real world conditions.

Instead, as we have in the past, MacTech chose to create a suite of tests that would mimic what many users would do in normal use, but stick to those actions that were repeatable, and measurable with a stopwatch. After all, if you couldn't perceive a difference with a stopwatch, the user is not likely to perceive it at all.

There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results. This time, we had the opportunity to use a tool that measures frames per second to verify the numbers reported by 3DMark06.

To that end, there are several kinds of tests that we ran: Virtual Machine Launch/Suspend, CPU tests, File and Network IO, CPU Usage Footprint, Application Performance, 3D and verification of real world game play.


We won't keep you in suspense. When we look at the "big picture" of all the testing, Parallels is the clear winner. If you count up the general tests (including one 3D graphics score), Parallels won 61% of the tests by 10% or more, and was also a bit faster on an additional 23% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 84% of the general tests we ran.

Description: Tally-Gen
Test Tally: General Virtualization Tests

If you focus exclusively on 3D graphics, as measured by 3DMark06 version 1.2, Parallels won by an even larger margin. Specifically, Parallels won 73% of the tests by 10% or more, and was also a bit faster on an additional 19% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 92% of the 3D graphics tests we ran.

When VMware Fusion was faster in 3D graphics, it was typically on the HDR/SM3.0 Score and the Batch Size tests (e.g., triangles). But, as you can see from both the test tally, and the colored cell worksheet overview, Parallels Desktop was overwhelmingly faster in graphics, and in working with real games, it was easy to see confirmation of the 3DMark06 scores.

Description: Tally-3D
Test Tally: 3D Graphics Tests

There are a handful of places that VMware Fusion consistently was faster than Parallels Desktop. In Microsoft Word 2010, VMware launched (initial launch after boot, aka "Adam" launch) faster, and was faster on large, global search and replaces. It was also consistently faster under Windows 7 doing file compression, and loading SSL pages.

Overall, VMware Fusion won 6% of the tests by at least 10%, and was also a bit faster on an additional 4% more of the general tests. For the 3D tests, VMware Fusion won 3% of the tests by at least 10%, and was also a bit faster on an additional 3% more for a total of 6%.

One of the best ways to visualize the huge amount of data points is through MacTech's well-known "Colored Cell Worksheet Overview." In these worksheets, a cell represents the best result for each test for each version of Windows for each virtualization product. These are then colored according to which product was faster.

Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.

Community Search:
MacTech Search:

Software Updates via MacUpdate

World of Tanks Generals guide - Tips and...
World of Tanks Generals is a brand new card game by the developer behind the World of Tanks shooter franchise. It plays like a cross between chess and your typical card game. You have to keep in consideration where you place your tanks on the board... | Read more »
TruckSimulation 16 guide: How to succeed...
Remember those strangely enjoyable truck missions in Grand Theft Auto V whereit was a disturbing amount of fun to deliver cargo? TruckSimulation 16 is reminiscent of that, and has you play the role of a truck driver who has to deliver various... | Read more »
The best GIF making apps
Animated GIFs have exploded in popularity recently which is likely thanks to a combination of Tumblr, our shorter attention spans, and the simple fact they’re a lot of fun. [Read more] | Read more »
The best remote desktop apps for iOS
We've been sifting through the App Store to find the best ways to do computer tasks on a tablet. That gave us a thought - what if we could just do computer tasks from our tablets? Here's a list of the best remote desktop apps to help you use your... | Read more »
Warhammer 40,000: Freeblade guide - How...
Warhammer 40,000: Freebladejust launched in the App Store and it lets you live your childhood dream of blowing up and slashing a bunch of enemies as a massive, hulking Space Marine. It's not easy being a Space Marine though - and particularly if... | Read more »
Gopogo guide - How to bounce like the be...
Nitrome just launched a new game and, as to be expected, it's a lot of addictive fun. It's called Gopogo, and it challenges you to hoparound a bunch of platforms, avoiding enemies and picking up shiny stuff. It's not easy though - just like the... | Read more »
Sago Mini Superhero (Education)
Sago Mini Superhero 1.0 Device: iOS Universal Category: Education Price: $2.99, Version: 1.0 (iTunes) Description: KAPOW! Jack the rabbit bursts into the sky as the Sago Mini Superhero! Fly with Jack as he lifts impossible weights,... | Read more »
Star Wars: Galaxy of Heroes guide - How...
Star Wars: Galaxy of Heroes is all about collecting heroes, powering them up, and using them together to defeat your foes. It's pretty straightforward stuff for the most part, but increasing your characters' stats can be a bit confusing because it... | Read more »
The best cooking apps (just in time for...
It’s that time of year again, where you’ll be gathering around the dinner table with your family and a huge feast in front of you. [Read more] | Read more »
Square Rave guide - How to grab those te...
Square Rave is an awesome little music-oriented puzzle game that smacks of games like Lumines, but with its own unique sense of gameplay. To help wrap your head around the game, keep the following tips and tricks in mind. [Read more] | Read more »

Price Scanner via MacPrices.net

Holiday weekend: Apple Watch on sale for $50-...
B&H Photo has the Apple Watch on sale today for $50-$100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - Apple Watch Sport: $50 off - Apple Watch: $50-$100 off Read more
Holiday weekend: iPad Air 2s on sale for up t...
B&H Photo has iPad Air 2s on sale for up to $80 off MSRP including free shipping plus NY sales tax only: - 16GB iPad Air 2 WiFi: $459 $40 off - 64GB iPad Air 2 WiFi: $569 $30 off - 128GB iPad Air... Read more
Holiday weekend Mac sales roundup: B&H Ph...
B&H Photo continues to have all new Macs on sale for up to $500 off MSRP as part of their Black Friday/Holiday weekend sale. Shipping is free, and B&H charges NY tax only: - 15″ 2.2GHz Retina... Read more
iMobie Releases its Ace iOS Cleaner PhoneClea...
iMobie Inc. has announced the new update of PhoneClean 4, its iOS cleaner designed to reclaim wasted space on iPhone/iPad for use and keep the device fast. Alongside, iMobie hosts a 3-day giveaway of... Read more
U.S. Cellular Offering iPad Pro
U.S. Cellular today announced that it is offering the new iPad Pro with Wi-Fi + Cellular, featuring a 12.9-inch Retina display with 5.6 million pixels — the most ever in an iOS device. U.S. Cellular... Read more
Newegg Canada Unveils Black Friday Deals for...
Newegg Canada is offering more than 1,000 deep discounts to Canadian customers this Black Friday, available now through Cyber Monday, with new deals posted throughout the week. “Black Friday is... Read more
Black Friday: Macs on sale for up to $500 off...
BLACK FRIDAY B&H Photo has all new Macs on sale for up to $500 off MSRP as part of their early Black Friday sale including free shipping plus NY sales tax only: - 15″ 2.2GHz Retina MacBook Pro: $... Read more
Black Friday: Up to $125 off iPad Air 2s at B...
BLACK FRIDAY Walmart has the 16GB iPad Air 2 WiFi on sale for $100 off MSRP on their online store. Choose free shipping or free local store pickup (if available): - 16GB iPad Air 2 WiFi: $399, save $... Read more
Black Friday: iPad mini 4s on sale for $100 o...
BLACK FRIDAY Best Buy has iPad mini 4s on sale for $100 off MSRP on their online store for Black Friday. Choose free shipping or free local store pickup (if available): - 16GB iPad mini 4 WiFi: $299.... Read more
Black Friday: Apple Watch for up to $100 off...
BLACK FRIDAY Apple resellers are offering discounts and bundles with the purchase of an Apple Watch this Black Friday. Below is a roundup of the deals being offered by authorized Watch resellers:... Read more

Jobs Board

Specialist *Apple* /Mac Desktop - University...
…technical support, expertise and user training for a variety of Apple /Macintosh hardware, software and devices.Researches, analyzes and resolves complex Apple Read more
*Apple* Site Security Manager - Apple (Unite...
# Apple Site Security Manager Job Number: 42975010 Culver City, Califo ia, United States Posted: Oct. 2, 2015 Weekly Hours: 40.00 **Job Summary** The Apple Site Read more
WiSE *Apple* Pay Quality Engineer - Apple (...
# WiSE Apple Pay Quality Engineer Job Number: 44313381 Santa Clara Valley, Califo ia, United States Posted: Nov. 13, 2015 Weekly Hours: 40.00 **Job Summary** Join our Read more
Holiday Retail Associate with *Apple* Knowl...
…and assertive.Someone who can troubleshoot iOS devices (iPhone and iPad) and Apple Mail issues.Someone who can offer solutions.Someone who can work weekends.Someone with Read more
*Apple* Systems Engineer (Mclean, VA and NYC...
Summary:Assist in providing strategic direction and technical leadership within the Apple portfolio, including desktops, laptops, and printing environment. This person Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.