02-Test Types and Overview
TweetFollow Us on Twitter

02-Test Types and Overview

Test Types

There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, WorldBench, Performance Test, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.

When testing virtual machines, however, using outside utilities within a virtualized environment can provide strange and incorrect results. For example, we saw one testing tool that reported graphics running twice as fast when you could clearly see on the screen that it was running at half the speed. Other tests use older versions of applications. Clearly, if one is going to use testing tools, they need to be carefully picked: the results need to reflect what the user sees in real world conditions.

Instead, as we have in the past, MacTech chose to create a suite of tests that would mimic what many users would do in normal use, but stick to those actions that were repeatable, and measurable with a stopwatch. After all, if you couldn't perceive a difference with a stopwatch, the user is not likely to perceive it at all.

There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results. This time, we had the opportunity to use a tool that measures frames per second to verify the numbers reported by 3DMark06.

To that end, there are several kinds of tests that we ran: Virtual Machine Launch/Suspend, CPU tests, File and Network IO, CPU Usage Footprint, Application Performance, 3D and verification of real world game play.

Overview

We won't keep you in suspense. When we look at the "big picture" of all the testing, Parallels is the clear winner. If you count up the general tests (including one 3D graphics score), Parallels won 61% of the tests by 10% or more, and was also a bit faster on an additional 23% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 84% of the general tests we ran.

Description: Tally-Gen
Test Tally: General Virtualization Tests

If you focus exclusively on 3D graphics, as measured by 3DMark06 version 1.2, Parallels won by an even larger margin. Specifically, Parallels won 73% of the tests by 10% or more, and was also a bit faster on an additional 19% more of the tests. In other words, Parallels Desktop 6 beat VMware Fusion 3.1 in 92% of the 3D graphics tests we ran.

When VMware Fusion was faster in 3D graphics, it was typically on the HDR/SM3.0 Score and the Batch Size tests (e.g., triangles). But, as you can see from both the test tally, and the colored cell worksheet overview, Parallels Desktop was overwhelmingly faster in graphics, and in working with real games, it was easy to see confirmation of the 3DMark06 scores.

Description: Tally-3D
Test Tally: 3D Graphics Tests

There are a handful of places that VMware Fusion consistently was faster than Parallels Desktop. In Microsoft Word 2010, VMware launched (initial launch after boot, aka "Adam" launch) faster, and was faster on large, global search and replaces. It was also consistently faster under Windows 7 doing file compression, and loading SSL pages.

Overall, VMware Fusion won 6% of the tests by at least 10%, and was also a bit faster on an additional 4% more of the general tests. For the 3D tests, VMware Fusion won 3% of the tests by at least 10%, and was also a bit faster on an additional 3% more for a total of 6%.

One of the best ways to visualize the huge amount of data points is through MacTech's well-known "Colored Cell Worksheet Overview." In these worksheets, a cell represents the best result for each test for each version of Windows for each virtualization product. These are then colored according to which product was faster.


Virtualization is one of the topics at the next MacTech Boot Camp
Packed schedule. Learn from the best.
Meet new people. Network with peers.

One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.
 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Live Home 3D Pro 3.2.2 - $69.99
Live Home 3D Pro, a successor of Live Interior 3D, is the powerful yet intuitive home design software that lets you build the house of your dreams right on your Mac. It has every feature of Live Home... Read more
Live Home 3D Pro 3.2.2 - $69.99
Live Home 3D Pro, a successor of Live Interior 3D, is the powerful yet intuitive home design software that lets you build the house of your dreams right on your Mac. It has every feature of Live Home... Read more
FileZilla 3.27.0.1 - Fast and reliable F...
FileZilla (ported from Windows) is a fast and reliable FTP client and server with lots of useful features and an intuitive interface. Version 3.27.0.1: MSW: Add misssing file to .zip binary package... Read more
Spotify 1.0.59.395. - Stream music, crea...
Spotify is a streaming music service that gives you on-demand access to millions of songs. Whether you like driving rock, silky R&B, or grandiose classical music, Spotify's massive catalogue puts... Read more
Sierra Cache Cleaner 11.0.6 - Clear cach...
Sierra Cache Cleaner is an award-winning general purpose tool for macOS X. SCC makes system maintenance simple with an easy point-and-click interface to many macOS X functions. Novice and expert... Read more
DiskCatalogMaker 7.1.2 - Catalog your di...
DiskCatalogMaker is a simple disk management tool which catalogs disks. Simple, light-weight, and fast Finder-like intuitive look and feel Super-fast search algorithm Can compress catalog data for... Read more
Live Home 3D Pro 3.1.2 - $69.99
Live Home 3D Pro, a successor of Live Interior 3D, is the powerful yet intuitive home design software that lets you build the house of your dreams right on your Mac. It has every feature of Live Home... Read more
Deeper 2.2.1 - Enable hidden features in...
Deeper is a personalization utility for macOS which allows you to enable and disable the hidden functions of the Finder, Dock, QuickTime, Safari, iTunes, login window, Spotlight, and many of Apple's... Read more
Pinegrow 3.04 - Mockup and design webpag...
Pinegrow (was Pinegrow Web Designer) is desktop app that lets you mockup and design webpages faster with multi-page editing, CSS and LESS styling, and smart components for Bootstrap, Foundation,... Read more
Deeper 2.2.1 - Enable hidden features in...
Deeper is a personalization utility for macOS which allows you to enable and disable the hidden functions of the Finder, Dock, QuickTime, Safari, iTunes, login window, Spotlight, and many of Apple's... Read more

Latest Forum Discussions

See All

The best deals on the App Store this wee...
There are quite a few truly superb games on sale on the App Store this week. If you haven't played some of these, many of which are true classics, now's the time to jump on the bandwagon. Here are the deals you need to know about. [Read more] | Read more »
Realpolitiks Mobile (Games)
Realpolitiks Mobile 1.0 Device: iOS Universal Category: Games Price: $5.99, Version: 1.0 (iTunes) Description: PLEASE NOTE: The game might not work properly on discontinued 1GB of RAM devices (iPhone 5s, iPhone 6, iPhone 6 Plus, iPad... | Read more »
Layton’s Mystery Journey (Games)
Layton’s Mystery Journey 1.0.0 Device: iOS Universal Category: Games Price: $15.99, Version: 1.0.0 (iTunes) Description: THE MUCH-LOVED LAYTON SERIES IS BACK WITH A 10TH ANNIVERSARY INSTALLMENT! Developed by LEVEL-5, LAYTON’S... | Read more »
Full Throttle Remastered (Games)
Full Throttle Remastered 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: Originally released by LucasArts in 1995, Full Throttle is a classic graphic adventure game from industry legend Tim... | Read more »
Stunning shooter Morphite gets a new tra...
Morphite is officially landing on iOS in September. The game looks like the space shooter we've been needing on mobile, and we're going to see if it fits the bill quite shortly. The game's a collaborative effort between Blowfish Studios, We're Five... | Read more »
Layton's Mystery Journey arrives to...
As you might recall, Layton's Mystery Journey is headed to iOS and Android -- tomorrow! To celebrate the impending launch, Level-5's released a new trailer, complete with an adorable hamster. [Read more] | Read more »
Sidewords (Games)
Sidewords 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: Grab a cup of coffee and relax with Sidewords. Sidewords is part logic puzzle, part word game, all original. No timers. No... | Read more »
Noodlecake Games' 'Leap On!...
Noodlecake Games is always good for some light-hearted arcade fun, and its latest project, Leap On! could carry on that tradition. It's a bit like high stakes tetherball in a way. Your job is to guide a cute little blob around a series of floating... | Read more »
RuneScape goes mobile later this year
Yes, RuneScape still exists. In fact, it's coming to iOS and Android in just a few short months. Jagex, creators of the hit fantasy MMORPG of yesteryear, is releasing RuneScape Mobile and Old School RuneScape for mobile devices, complete with... | Read more »
Crash of Cars wants you to capture the c...
Crash of Cars is going full on medieval in its latest update, introducing castles and all manner of new cars and skins fresh from the Dark Ages. The update introduces a new castle-themed map (complete with catapults) and a gladiator-style battle... | Read more »

Price Scanner via MacPrices.net

Save or Share
FotoJet Designer, is a simple but powerful new graphic design apps available on both Mac and Windows. With FotoJet Designer’s 900+ templates, thousands of resources, and powerful editing tools you... Read more
Logo Maker Shop iOS App Lets Businesses Get C...
A newly released app is designed to help business owners to get creative with their branding by designing their own logos. With more than 1,000 editable templates, Logo Maker Shop 1.0 provides the... Read more
Sale! New 15-inch MacBook Pros for up to $150...
Amazon has the new 2017 15″ MacBook Pros on sale for up to $150 off MSRP including free shipping: – 15″ 2.8GHz MacBook Pro Space Gray: $2249 $150 off MSRP – 15″ 2.89Hz MacBook Pro Space Gray: $2779 $... Read more
DEVONthink To Go 2.1.7 For iOS Brings Usabili...
DEVONtechnologies has updated DEVONthink To Go, the iOS companion to DEVONthink for Mac, with enhancements and bug fixes. Version 2.1.7 adds an option to clear the Global Inbox and makes the grid... Read more
15-inch 2.2GHz Retina MacBook Pro, Apple refu...
Apple has Certified Refurbished 2015 15″ 2.2GHz Retina MacBook Pros available for $1699. That’s $300 off MSRP, and it’s the lowest price available for a 15″ MacBook Pro. An Apple one-year warranty is... Read more
13-inch 2.3GHz Silver MacBook Pro on sale for...
B&H Photo has the new 2017 13″ 2.3GHz/256GB Silver MacBook Pro (MPXU2LL/A) on sale for $1399 including free shipping plus NY & NJ sales tax only. Their price is $100 off MSRP. Read more
Apple Tackles Distracted Driving With iOS 11...
One of the most important new features coming in iOS 11 is Do Not Disturb while driving, intended to help drivers stay more focused on the road. With Do Not Disturb while driving, your iPhone can... Read more
iMazing Mini for Mac: Free Automatic and Priv...
Geneva, Switzerland-based indie developer DigiDNA has released iMazing Mini, their free macOS utility designed to automatically back up iOS devices over any local Wi-Fi network. The app offers users... Read more
Clearance 2016 13-inch MacBook Airs, Apple re...
Apple dropped prices recently on Certified Refurbished 2016 13″ MacBook Airs, with models now available starting at $809. An Apple one-year warranty is included with each MacBook, and shipping is... Read more
9.7-inch 2017 iPads available for $299, save...
B&H Photo has 2017 9.7″ 32GB WiFi iPads on sale for $30 off MSRP for a limited time. Shipping is free, and pay sales tax in NY & NJ only: – 32GB iPad WiFi: $299, $30 off Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple...
SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
*Apple* Retail - Multiple Positions - Apple...
SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
Senior Payments Architect - *Apple* Pay - A...
Changing the world is all in a day's work at Apple . If you love innovation, here's your chance to make a career of it. You'll work hard. But the job comes with more Read more
Frameworks Engineering Manager, *Apple* Wat...
Frameworks Engineering Manager, Apple Watch Job Number: 41632321 Santa Clara Valley, California, United States Posted: Jun. 15, 2017 Weekly Hours: 40.00 Job Summary Read more
Manager, *Apple* Media Products - Apple Inc...
Job Summary The Apple Media Products Discovery, Fraud and Abuse team is responsible for protecting the integrity of Apple services. As a manager of the team, you Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.