02-Test Types and Overview
TweetFollow Us on Twitter

02-Test Types and Overview

Test Types

There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, WorldBench, Performance Test, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.

When testing virtual machines, however, using outside utilities within a virtualized environment can provide strange and incorrect results. For example, we saw one testing tool that reported graphics running twice as fast when you could clearly see on the screen that it was running at half the speed. Other tests use older versions of applications. Clearly, if one is going to use testing tools, they need to be carefully picked: the results need to reflect what the user sees in real world conditions.

Instead, as we have in the past, MacTech chose to create a suite of tests that would mimic what many users would do in normal use, but stick to those actions that were repeatable, and measurable with a stopwatch. After all, if you couldn't perceive a difference with a stopwatch, the user is not likely to perceive it at all.

There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results. This time, we had the opportunity to use a tool that measures frames per second to verify the numbers reported by 3DMark06.

To that end, there are several kinds of tests that we ran: Virtual Machine Launch/Suspend, CPU tests, File and Network IO, CPU Usage Footprint, Application Performance, 3D and verification of real world game play.

Overview

We won't keep you in suspense. When we look at the "big picture" of all the testing, Parallels is the clear winner. If you count up the general tests (including one 3D graphics score), Parallels won 61% of the tests by 10% or more, and was also a bit faster on an additional 23% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 84% of the general tests we ran.

Description: Tally-Gen
Test Tally: General Virtualization Tests

If you focus exclusively on 3D graphics, as measured by 3DMark06 version 1.2, Parallels won by an even larger margin. Specifically, Parallels won 73% of the tests by 10% or more, and was also a bit faster on an additional 19% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 92% of the 3D graphics tests we ran.

When VMware Fusion was faster in 3D graphics, it was typically on the HDR/SM3.0 Score and the Batch Size tests (e.g., triangles). But, as you can see from both the test tally, and the colored cell worksheet overview, Parallels Desktop was overwhelmingly faster in graphics, and in working with real games, it was easy to see confirmation of the 3DMark06 scores.

Description: Tally-3D
Test Tally: 3D Graphics Tests

There are a handful of places that VMware Fusion consistently was faster than Parallels Desktop. In Microsoft Word 2010, VMware launched (initial launch after boot, aka "Adam" launch) faster, and was faster on large, global search and replaces. It was also consistently faster under Windows 7 doing file compression, and loading SSL pages.

Overall, VMware Fusion won 6% of the tests by at least 10%, and was also a bit faster on an additional 4% more of the general tests. For the 3D tests, VMware Fusion won 3% of the tests by at least 10%, and was also a bit faster on an additional 3% more for a total of 6%.

One of the best ways to visualize the huge amount of data points is through MacTech's well-known "Colored Cell Worksheet Overview." In these worksheets, a cell represents the best result for each test for each version of Windows for each virtualization product. These are then colored according to which product was faster.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 

Community Search:
MacTech Search:

Software Updates via MacUpdate

How to win big in Slots Deluxe
Cheating while gambling is illegal and morally wrong, and in some parts of the world it leads to men with names like Vinnie "Six Knuckles" Manchenzo beating you to a pulp in a dark alley. [Read more] | Read more »
How to take over the world in Dictator 2
Running a country isn't easy - especially when you're a dictator who wants to take over the world and crush everyone in your path while you do it. [Read more] | Read more »
Tips and tricks to get a higher score in...
Tank.iois - you guessed it! - another multiplayer arena battler likeAgar.io and Slither.io. It does differentiate itself by putting you in a tiny tank though, so it's not exactly the same. To help you get that all-important high score, we've got a... | Read more »
How to unlock characters in One Tap Tenn...
As the title suggests, One Tap Tennis requires only a single tap to play its particular brand of tennis, and rewards you with a ton of unlockable characters if you perform well. Fortunately for you, we at 148Apps have got a few tips and tricks to... | Read more »
Grab it now: Game Craft’s Legend of War...
The real time strategy game is now available for you to sink your teeth into, through the App Store and Google Play. Combining elements of skill, strategy and empire building, Legend of War is a real gamers’ game. [Read more] | Read more »
Skateboard Party 3 ft. Greg Lutzka (Gam...
Skateboard Party 3 ft. Greg Lutzka 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Skateboard Party is back! This third edition of the popular sports franchise features professional skater... | Read more »
Cubious (Games)
Cubious 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: Cubious – How smart are you? How high is your IQube? Solve the impossible puzzles to find out, and help a lost little cube find his... | Read more »
Goat Simulator Waste of Space (Games)
Goat Simulator Waste of Space 1.1 Device: iOS Universal Category: Games Price: $4.99, Version: 1.1 (iTunes) Description: ** IMPORTANT - SUPPORTED DEVICESiPhone 4S, iPad 2, iPod Touch 5 or better.** | Read more »
Wildfulness - Unwind in nature and calm...
Wildfulness - Unwind in nature and calm your mind with nature sounds and illustrations 1.0 Device: iOS Universal Category: Healthcare & Fitness Price: $1.99, Version: 1.0 (iTunes) Description: Spending time in nature helps you to... | Read more »
Dr. Panda Racers (Education)
Dr. Panda Racers 1.0 Device: iOS Universal Category: Education Price: $2.99, Version: 1.0 (iTunes) Description: STEP ON THE GAS, RACE AND WIN!Fasten your seat belts and get ready to race! Speed your way to the finish line while doing... | Read more »

Price Scanner via MacPrices.net

Enterprise Workers Pick Technology Over Perks...
New Adobe study shows surprising attitudes about office jobs and where the future of work is heading. Adobe has released survey findings revealing that a surprising 70 percent of U.S. office workers... Read more
Goal Zero and OtterBox Partner to Expand iPh...
Goal Zero, specialists in portable power, have announced a partnership with OtterBox, brand smartphone case protection, to offer the Slide and Slide Plus Batteries as modules compatible with the new... Read more
15-inch Retina MacBook Pros on sale for up to...
B&H Photo has 15″ Retina MacBook Pros on sale for up to $210 off MSRP. Shipping is free, and B&H charges NY tax only: - 15″ 2.2GHz Retina MacBook Pro: $1799 $200 off MSRP - 15″ 2.5GHz Retina... Read more
Clearance 2015 13-inch MacBook Airs available...
B&H Photo has clearance 2015 13″ MacBook Airs available for $250 off original MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 1.6GHz/4GB/128GB MacBook Air (MJVE2LL/A): $799... Read more
Apple refurbished Apple TVs available for up...
Apple has Certified Refurbished 32GB and 64GB Apple TVs available for up to $30 off the cost of new models. Apple’s standard one-year warranty is included with each model, and shipping is free: -... Read more
21-inch iMacs on sale for up to $120 off MSRP
B&H Photo has 21″ iMacs on sale for up to $120 off MSRP including free shipping plus NY sales tax only: - 21″ 3.1GHz iMac 4K: $1379.99 $120 off MSRP - 21″ 2.8GHz iMac: $1189 $110 off MSRP - 21″ 1... Read more
Kanex Introduces GoPower USB-C Rechargeable B...
Kanex has announced its GoPower USB-C portable battery for the USB-C MacBook, featuring the new industry standard connector and cable used for connectivity and power. Providing users with a new... Read more
Convertible and Detachable Devices Winning Ov...
According to the latest figures published by International Data Corporation (IDC), Western European shipments of ultraslim convertibles and detachables posted positive growth (44.7%) to account for... Read more
New MacBook Pros And Will MacBook Air Be Upgr...
With my mid-2013 13-inch MacBook Air closing on its third anniversary come November, I’m in system upgrade mode. Actually the Haswell CPU equipped Air is still doing a fine job, but my good wife is... Read more
Apple’s Education discount saves up to $300 o...
Purchase a new Mac or iPad using Apple’s Education Store and take up to $300 off MSRP. All teachers, students, and staff of any educational institution qualify for the discount. Shipping is free, and... Read more

Jobs Board

*Apple* Project Engineer - Smart Source Inc...
SmartSource is in need of an Apple Project Engineer for a 12 month contract opportunity in Pittsburg, PA. Role: Apple Project Engineer Location: Pittsburg, PA Read more
Automotive Sales Consultant - Apple Ford Linc...
…you. The best candidates are smart, technologically savvy and are customer focused. Apple Ford Lincoln Apple Valley is different, because: $30,000 annual salary Read more
Service Assistant - *Apple* Chevrolet *App...
Apple Automotive is one of the fastest growing dealer...and it shows. Consider making the switch to the Apple Automotive Group today! At Apple Automotive, we Read more
Editor, *Apple* News - APPLE (United States...
Job Summary The Apple News team is looking for a passionate and knowledgeable editor with experience covering entertainment/pop culture and experience running social Read more
*Apple* Nissan Service Technicians - Apple A...
Apple Automotive is one of the fastest growing dealer...and it shows. Consider making the switch to the Apple Automotive Group today! At Apple Automotive , Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.