01-The Test Bench and Test Types
TweetFollow Us on Twitter

01-The Test Bench and Test Types

The Test Bench

When we were choosing computer models, we chose the current model of Macs that give a good representation of what most people may have. Certainly, the faster models of these computers will perform even better.

We chose five Mac models to compare alongside each other: MacBook Air, MacBook Pro, a tricked out MacBook Pro Retina, iMac (late 2011), and Mac Pro. Given the RAM requirements of Windows 7, the minimum configuration tested (including on the MacBook) was 4 GB.

8 GB MacBook Air 13-inch, 1.8 GHz dual-core Intel Core i5 processor
Specifically: 8 GB/256 GB, Intel HD Graphics 4000

4 GB MacBook Pro 15.4-inch, 2.3GHz quad-core Intel Core i7 processor
Specifically: 4GB 1600MHz memory/500 GB 5400-rpm
Intel HD Graphics 4000 and NVIDIA GeForce GT 650M with 512MB of GDDR5 memory

16 GB MacBook Pro with Retina 15-inch, 2.7GHz quad-core Intel Core i7 processor
Specifically: 16GB 1600MHz DDR3L SDRAM/768GB Flash Storage
Intel HD Graphics 4000 and NVIDIA GeForce GT 650M with 1GB of GDDR5 memory

4 GB iMac 27", 2.7GHz Quad-Core Intel Core i5 processor
Specifically: 4 GB/1 TB 7200-rpm
AMD Radeon HD 6770M with 512MB

6 GB Mac Pro, One 3.2GHz Quad-Core Intel Xeon processor processors
Specifically: 6 GB/1 TB 7200-rpm / ATI Radeon HD 5770 with 1GB GDDR5

Memory for virtual machines can be configured with a wide array of settings. As a rule, both VMware Fusion's and Parallels Desktop's default memory for each configuration (of physical RAM and "guest" OS) were the same, and we made sure that was the case. Windows 7 and 8 virtual machines ran with 1 GB of virtual machine RAM (except for gaming). Lion and Mountain Lion (OS X) ran with 2 GB of virtual machine RAM. For gaming, we used 1.5 GB for 4 GB hardware, and 2 GB on hardware with 6 GB or more.

Similarly, for disk allocation we set up all of the virtual machines similarly. We used default sized (64 GB on Parallels, and 60 GB on VMware) expanding disks in a single file (e.g., not 2 GB chunks), but the disks were pre-expanded so that expanding actions wouldn't affect results. The disk location of the virtual hard drive was in a similar physical location between computers, as that can make a significant difference in disk performance.

The tests compared VMware Fusion 5.0.2 with Parallels Desktop for Mac 8.0.18314.813278, running on Mac OS X 10.8.2 with all updates. All "important," but not "optional," Windows updates also installed for Windows 7 and 8. By the time we got to testing games, we saw new versions released that could have impact, and therefore (games only) were tested on VMware Fusion 5.0.2 with Parallels Desktop for Mac 8.0.18354.823166, including additional updates to OS X 10.8.2.

Test Types

There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, WorldBench, Performance Test, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.

We are sometimes asked how we select the tests we have in benchmarks. The goal is to represent the types of actions that virtualization users are doing on a regular basis. In particular, we focus more on user tasks rather than installation or configuration (because they are typically only done once or infrequently).

In the PC market, PC World (magazine) uses WorldBench as their benchmarking tool of choice. If you don't know about WorldBench, you can see more at http://www.worldbench.com. WorldBench 7 uses a combination of applications bundled together to determine benchmarks. This may be a very good way for PC World to approach things, but obviously it's not reflecting the virtualization part of the experience or weighing towards the typical types of things people do in virtualization on the Mac.

There are a variety of other benchmarks available, and we've looked at many of them as possible tools to include in our mix. Often, we find their measurements simply don't reflect a true user experience. Other times, they don't reflect how a virtualization user may be using Windows, or even they simply report erroneous results. For example, in previous attempts to use PassMark, we found their graphics scores were not at all representative of real life graphics performance in games or other graphic uses. Those tests showed items as faster when, in fact, they were much slower. Benchmarks are only useful if they reflect the real world.

Rather than use WorldBench or others, we focus on the types of measurements we believe best represent the experience we see (in terms of speed). And while it takes practice and some skill, we test virtualization operations with a stopwatch—as the closest representation of how a real user would see it. CPU crunching ability, for example, is measured through tests like zip'ing files directly in Windows. For Office users, we use the most up to date version of Microsoft Office for Windows (2010), again with updates.

There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results. In the case of graphics benchmarking, we used 3DMark06 (the industry standard for tuning gaming systems). For game playing performance, we used FRAPS to measure frames per second. Finally, for CPU utilization we used "top" combined with Apple Remote Desktop.

Remember, benchmarks are not a review: they are benchmarks. They are simply meant to just tell you which runs faster. If you are interested in a specific feature, support, the user interface, or any of the other criteria for deciding on a product, that's a different article.


Community Search:
MacTech Search:

Software Updates via MacUpdate

Google Chrome 54.0.2840.71 - Modern and...
Google Chrome is a Web browser by Google, created to be a modern platform for Web pages and applications. It utilizes very fast loading of Web pages and has a V8 engine, which is a custom built... Read more
HoudahSpot 4.2.6 - Advanced file-search...
HoudahSpot is a powerful file search tool. Use HoudahSpot to locate hard-to-find files and keep frequently used files within reach. HoudahSpot will immediately feel familiar. It works just the way... Read more
Yummy FTP Pro 1.11.11 - $29.99
Yummy FTP Pro is an advanced Mac file transfer app which provides a full-featured professional toolkit combined with blazing speeds and impeccable reliability, so whether you want to transfer a few... Read more
Shimo - VPN client – for everyon...
Shimo is the most versatile VPN client for OS X and it enables really everybody to master secure network. It supports more protocols than any other VPN application out there! CiscoVPN, AnyConnect,... Read more
Dash 3.4.0 - Instant search and offline...
Dash is an API documentation browser and code snippet manager. Dash helps you store snippets of code, as well as instantly search and browse documentation for almost any API you might use (for a full... Read more
AirRadar 3.1.9 - $9.95
With AirRadar, scanning for wireless networks is now easier and more personalized! It allows you to scan for open networks and tag them as favourites or filter them out. View detailed network... Read more
Printopia 2.1.22 - Share Mac printers wi...
Run Printopia on your Mac to share its printers to any capable iPhone, iPad or iPod Touch. Printopia will also add virtual printers, allowing you to save print-outs to your Mac and send to apps.... Read more
SteerMouse 5.0 - Powerful third-party mo...
SteerMouse is an advanced driver for USB and Bluetooth mice. It also supports Apple Mighty Mouse very well. SteerMouse can assign various functions to buttons that Apple's software does not allow,... Read more
Alarm Clock Pro 10.2.5 - $19.95
Alarm Clock Pro isn't just an ordinary alarm clock. Use it to wake you up in the morning, send and compose e-mails, remind you of appointments, randomize the iTunes selection, control an internet... Read more
Cocktail 10.1 - General maintenance and...
Cocktail is a general purpose utility for macOS that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more

Latest Forum Discussions

See All

Game of Dice is now available on Faceboo...
After celebrating its anniversary in style with a brand new update, there’s even more excitement in store for Game of Dice has after just being launched on Facebook Gameroom. A relatively new platform, Facebook Gameroom has been designed for PC... | Read more »
4 addictive clicker games like Best Fien...
Clickers are passive games that take advantage of basic human psychology to suck you in, and they're totally unashamed of that. As long as you're aware that this game has been created to take hold of your brain and leave you perfectly content to... | Read more »
Smile Inc. Guide: How not to die on the...
As if Mondays weren't bad enough, at Smile Inc. you have to deal with giant killer donuts, massive hungry staplers, and blasting zones. It's not exactly a happy, thriving work environment. In fact, you'll be lucky to survive the nine to five.... | Read more »
Oh...Sir! The Insult Simulator (Games)
Oh...Sir! The Insult Simulator 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: | Read more »
WitchSpring2 (Games)
WitchSpring2 1.27 Device: iOS Universal Category: Games Price: $3.99, Version: 1.27 (iTunes) Description: This is the story of Luna, the Moonlight Witch as she sets out into the world. This is a sequel to Witch Spring. Witch Spring 2... | Read more »
4 popular apps getting a Halloween makeo...
'Tis the season for all things spooky. So much, so, in fact, that even apps are getting into the spirt of things, dressing up in costume and spreading jack o' lanterns all about the place. These updates bring frightening new character skins, scary... | Read more »
Pokémon GO celebrates Halloween with can...
The folks behind Pokémon GO have some exciting things planned for their Halloween celebration, the first in-game event since it launched back in July. Starting October 26 and ending on November 1, trainers will be running into large numbers of... | Read more »
Best Fiends Forever Guide: How to collec...
The fiendship in Seriously's hit Best Fiends has been upgraded this time around in Best Fiends Forever. It’s a fast-paced clicker with lots of color and style--kind of reminiscent of a ‘90s animal mascot game like Crash Bandicoot. The game... | Read more »
5 apps for the budding mixologist
Creating your own cocktails is something of an art form, requiring a knack for unique tastes and devising interesting combinations. It's easy to get started right in your own kitchen, though, even if you're a complete beginner. Try using one of... | Read more »
5 mobile strategy games to try when you...
Strategy enthusiasts everywhere are celebrating the release of Civilization VI this week, and so far everyone seems pretty satisfied with the first full release in the series since 2010. The series has always been about ultra-addictive gameplay... | Read more »

Price Scanner via MacPrices.net

13-inch 2.5GHz MacBook Pro (Apple refurbished...
Apple has Certified Refurbished 13″ 2.5GHz MacBook Pros available for $829, or $270 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 13″ 2.5GHz MacBook Pros... Read more
Apple refurbished 2015 13-inch MacBook Airs a...
Apple has Certified Refurbished 2015 13″ MacBook Airs available starting at $759. An Apple one-year warranty is included with each MacBook, and shipping is free: - 2015 13″ 1.6GHz/4GB/128GB MacBook... Read more
64GB Apple TV on sale for $159, save $40
Best Buy has the 64GB Apple TV on sale for $40 off MSRP on their online store. Choose free shipping or free local store pickup (if available). Sale price for online orders only, in-store price may... Read more
EyeQue Introduces iOS And Android Based Advan...
Affordable vision technologies developers EyeQue have announced what they claim to be the world’s most advanced intelligent vision solution, pitched as enabling anyone, anywhere to easily and... Read more
Smartwatch Market Tanks, Declining 51.6% in 2...
The worldwide smartwatch market experienced a round of growing pains in the third quarter of 2016 (3Q16), resulting in a year-over-year decline in shipment volumes. According to data from the... Read more
CAZE announces Ultra Thin Glass Screen Protec...
Hong Kong based CAZE has announced its first ultra thin glass screen protector, the Glazz Pro for iPhone 7/7 Plus. Glazz Pro is made from chemically reinforced glass with an anti-fingerprint... Read more
11-inch MacBook Airs on sale for up to $120 o...
Newegg has 11″ MacBook Airs on sale for up to $120 off MSRP. Shipping is free: - 11″ 1.6GHz/128GB MacBook Air: $799.99 $100 off MSRP - 11″ 1.6GHz/256GB MacBook Air: $979 $120 off MSRP Read more
Up to $300 off Macs, $20 off iPads with Apple...
Purchase a new Mac or iPad using Apple’s Education Store and take up to $300 off MSRP. All teachers, students, and staff of any educational institution qualify for the discount. Shipping is free, and... Read more
Apple’s Thursday “Hello Again” Event A Largel...
KGI Securities analyst Ming-Chi Kuo, who has a strong record of Apple hardware prediction accuracy, forecasts in a new note to investors released late last week that a long-overdue redo of the... Read more
12-inch Retina MacBooks on sale for $100 off...
Amazon has 2016 12″ Apple Retina MacBooks on sale for $100 off MSRP. Shipping is free: - 12″ 1.1GHz Silver Retina MacBook: $1199.99 $100 off MSRP - 12″ 1.1GHz Gold Retina MacBook: $1199.99 $100 off... Read more

Jobs Board

Security Data Analyst - *Apple* Information...
…data sources need to be collected to allow Information Security to better protect Apple employees and customers from a wide range of threats.Act as the subject Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* Solutions Consultant - Apple (United...
# Apple Solutions Consultant Job Number: 52812872 Houston, Texas, United States Posted: Oct. 18, 2016 Weekly Hours: 40.00 **Job Summary** As an Apple Solutions Read more
Lead *Apple* Solutions Consultant - Apple (...
# Lead Apple Solutions Consultant Job Number: 52812906 Houston, Texas, United States Posted: Oct. 18, 2016 Weekly Hours: 40.00 **Job Summary** The Lead ASC is an Read more
*Apple* Retail - Multiple Positions- Towson,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.