02-Test Types and Overview
TweetFollow Us on Twitter

02-Test Types and Overview

Test Types

There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, WorldBench, Performance Test, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.

When testing virtual machines, however, using outside utilities within a virtualized environment can provide strange and incorrect results. For example, we saw one testing tool that reported graphics running twice as fast when you could clearly see on the screen that it was running at half the speed. Other tests use older versions of applications. Clearly, if one is going to use testing tools, they need to be carefully picked: the results need to reflect what the user sees in real world conditions.

Instead, as we have in the past, MacTech chose to create a suite of tests that would mimic what many users would do in normal use, but stick to those actions that were repeatable, and measurable with a stopwatch. After all, if you couldn't perceive a difference with a stopwatch, the user is not likely to perceive it at all.

There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results. This time, we had the opportunity to use a tool that measures frames per second to verify the numbers reported by 3DMark06.

To that end, there are several kinds of tests that we ran: Virtual Machine Launch/Suspend, CPU tests, File and Network IO, CPU Usage Footprint, Application Performance, 3D and verification of real world game play.

Overview

We won't keep you in suspense. When we look at the "big picture" of all the testing, Parallels is the clear winner. If you count up the general tests (including one 3D graphics score), Parallels won 61% of the tests by 10% or more, and was also a bit faster on an additional 23% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 84% of the general tests we ran.

Description: Tally-Gen
Test Tally: General Virtualization Tests

If you focus exclusively on 3D graphics, as measured by 3DMark06 version 1.2, Parallels won by an even larger margin. Specifically, Parallels won 73% of the tests by 10% or more, and was also a bit faster on an additional 19% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 92% of the 3D graphics tests we ran.

When VMware Fusion was faster in 3D graphics, it was typically on the HDR/SM3.0 Score and the Batch Size tests (e.g., triangles). But, as you can see from both the test tally, and the colored cell worksheet overview, Parallels Desktop was overwhelmingly faster in graphics, and in working with real games, it was easy to see confirmation of the 3DMark06 scores.

Description: Tally-3D
Test Tally: 3D Graphics Tests

There are a handful of places that VMware Fusion consistently was faster than Parallels Desktop. In Microsoft Word 2010, VMware launched (initial launch after boot, aka "Adam" launch) faster, and was faster on large, global search and replaces. It was also consistently faster under Windows 7 doing file compression, and loading SSL pages.

Overall, VMware Fusion won 6% of the tests by at least 10%, and was also a bit faster on an additional 4% more of the general tests. For the 3D tests, VMware Fusion won 3% of the tests by at least 10%, and was also a bit faster on an additional 3% more for a total of 6%.

One of the best ways to visualize the huge amount of data points is through MacTech's well-known "Colored Cell Worksheet Overview." In these worksheets, a cell represents the best result for each test for each version of Windows for each virtualization product. These are then colored according to which product was faster.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 

Community Search:
MacTech Search:

Software Updates via MacUpdate

CSR Racing 2: Your guide to what's...
CSR Racing 2, or CSR2, as it likes to call itself, has finally arrived. The follow-up to the immensely popular drag racing game CSR Racing is the first release from NaturalMotion since the studio's acquisition by Zynga in early 2014. [Read more] | Read more »
Nanuleu (Games)
Nanuleu 1.1 Device: iOS Universal Category: Games Price: $2.99, Version: 1.1 (iTunes) Description: Nanuleu is a strategy game where you take control of ancient magical trees that protect the land from an invading dark force. A... | Read more »
The Slaughter: Act One (Games)
The Slaughter: Act One 1.0.323 Device: iOS Universal Category: Games Price: $3.99, Version: 1.0.323 (iTunes) Description: “The game mixes realism and surrealism to create a story that can cause just as much laughter as fear. A-” -... | Read more »
NEO TURF MASTERS (Games)
NEO TURF MASTERS 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: NEOGEO’s legendary golf game is back, in a brand-new mobile version with touch controls! NEO TURF MASTERS (also known as “BIG... | Read more »
How to send money to a friend with the F...
Facebook is already a lot of things to a lot of people. It's a social network, sure, but also a tool for catologing memories, chatting with friends, and much more. [Read more] | Read more »
Branch out into the Chinese market with...
Created a game that you’re super enthused about, only to find yourself struggling to get it the exposure that you believe it duly deserves? With the time, energy and money that goes into development, it’s only right that games should be given the... | Read more »
Pocket Arcade Story (Games)
Pocket Arcade Story 1.00 Device: iOS Universal Category: Games Price: $4.99, Version: 1.00 (iTunes) Description: Here comes a new challenger: it's the arcade simulation game you've been waiting for! Build your very own gaming... | Read more »
How to get coins faster in Rodeo Stamped...
There comes a time in a cowboy or cowgirl's life when all the riding and lassoing skills in the world aren't enough. You're going to need some cold, hard cash to keep your sky zoo expanding in Rodeo Stampede. [Read more] | Read more »
How to out-do Cam Newton in Can You Dab?
The thing about dance crazes is that you're never really sure when they've run their course. Take the Dab, for instance. Propelled by its adoption as the touchdown celebration of choice for Carolina Panthers quarterback Cam Newton, the Dab seemed... | Read more »
Artik Games releases Splashy Cats for An...
Splashy Cats had us hooked from the title alone, and when we found out the game was literally just zig-zagging one of our favourite pop-culture references, guised as a playable cat character, down a river – our appetites were whetted to say the... | Read more »

Price Scanner via MacPrices.net

9-inch 128GB Silver iPad Pro on sale for $50...
B&H Photo has the 9.7″ 128GB Silver Apple iPad Pro on sale for $699 including free shipping plus NY tax only. Their price is $50 off MSRP. Read more
Why Use Indie Opera And Vivaldi Instead Of Sa...
For many years my web browser workhorses were various permutations and spinoffs of the Netscape/Mozilla/Firefox Open Source platform, and the Norwegian indie browser Opera, which I took a shine to... Read more
Western Digital Launches Worlds Fastest 256GB...
At the Mobile World Congress in Shanghai Western Digital Corporation this week introduced a new suite of 256 gigabyte (GB) microSD cards, which includes the new 256GB SanDisk Extreme microSDXC UHS-I... Read more
KeyCue 8.1 Integrates With Typinator To Displ...
Ergonis Software has released KeyCue 8.1, a new version of the company’s keyboard shortcut cheat sheet. KeyCue 8 introduced a new way to define a wide variety of triggers, which can be used to... Read more
Save up to $600 with Apple refurbished Mac Pr...
Apple has Certified Refurbished Mac Pros available for up to $600 off the cost of new models. An Apple one-year warranty is included with each Mac Pro, and shipping is free. The following... Read more
21-inch 2.8GHz iMac on sale for $1199, save $...
Amazon has the 21″ 2.8GHz iMac (model #MK442LL/A) on sale for $1199.99 including free shipping. Their price is $100 off MSRP, and it’s the lowest price available for this model. Read more
13-inch 2.5GHz MacBook Pro (Apple refurbished...
Apple has Certified Refurbished 13″ 2.5GHz MacBook Pros available for $829, or $270 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 13″ 2.5GHz MacBook Pros... Read more
MacBook Airs on sale for up to $50-$100 off M...
B&H Photo has 13″ and 11″ MacBook Airs on sale for up to $100 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 11″ 1.6GHz/128GB MacBook Air: $849 $50 off list price - 11″ 1.... Read more
Brexit Vote Result Forecast To Slash UK 2016...
Uncertainty and economic volatility can be expected to increase over the next nine months, as the Brexit and concerns over the future of the EU hit IT investment, say Canalys market analysts, with... Read more
13-inch 256GB MacBook Air on sale for $1149,...
Amazon has the 2016 13″ 1.6GHz/256GB MacBook Air (model MMGG2LL/A) on sale for $1149.99 including free shipping. Their price is also $50 off MSRP. Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* Retail - Multiple Positions, Willow...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.