01-The Test Bench and Test Types
TweetFollow Us on Twitter

01-The Test Bench and Test Types

The Test Bench

When we were choosing computer models, we chose the current model of Macs that give a good representation of what most people may have. Certainly, the faster models of these computers will perform even better.

We chose five Mac models to compare alongside each other: MacBook Air, MacBook Pro, a tricked out MacBook Pro Retina, iMac (late 2011), and Mac Pro. Given the RAM requirements of Windows 7, the minimum configuration tested (including on the MacBook) was 4 GB.

8 GB MacBook Air 13-inch, 1.8 GHz dual-core Intel Core i5 processor
Specifically: 8 GB/256 GB, Intel HD Graphics 4000

4 GB MacBook Pro 15.4-inch, 2.3GHz quad-core Intel Core i7 processor
Specifically: 4GB 1600MHz memory/500 GB 5400-rpm
Intel HD Graphics 4000 and NVIDIA GeForce GT 650M with 512MB of GDDR5 memory

16 GB MacBook Pro with Retina 15-inch, 2.7GHz quad-core Intel Core i7 processor
Specifically: 16GB 1600MHz DDR3L SDRAM/768GB Flash Storage
Intel HD Graphics 4000 and NVIDIA GeForce GT 650M with 1GB of GDDR5 memory

4 GB iMac 27", 2.7GHz Quad-Core Intel Core i5 processor
Specifically: 4 GB/1 TB 7200-rpm
AMD Radeon HD 6770M with 512MB

6 GB Mac Pro, One 3.2GHz Quad-Core Intel Xeon processor processors
Specifically: 6 GB/1 TB 7200-rpm / ATI Radeon HD 5770 with 1GB GDDR5

Memory for virtual machines can be configured with a wide array of settings. As a rule, both VMware Fusion's and Parallels Desktop's default memory for each configuration (of physical RAM and "guest" OS) were the same, and we made sure that was the case. Windows 7 and 8 virtual machines ran with 1 GB of virtual machine RAM (except for gaming). Lion and Mountain Lion (OS X) ran with 2 GB of virtual machine RAM. For gaming, we used 1.5 GB for 4 GB hardware, and 2 GB on hardware with 6 GB or more.

Similarly, for disk allocation we set up all of the virtual machines similarly. We used default sized (64 GB on Parallels, and 60 GB on VMware) expanding disks in a single file (e.g., not 2 GB chunks), but the disks were pre-expanded so that expanding actions wouldn't affect results. The disk location of the virtual hard drive was in a similar physical location between computers, as that can make a significant difference in disk performance.

The tests compared VMware Fusion 5.0.2 with Parallels Desktop for Mac 8.0.18314.813278, running on Mac OS X 10.8.2 with all updates. All "important," but not "optional," Windows updates also installed for Windows 7 and 8. By the time we got to testing games, we saw new versions released that could have impact, and therefore (games only) were tested on VMware Fusion 5.0.2 with Parallels Desktop for Mac 8.0.18354.823166, including additional updates to OS X 10.8.2.

Test Types

There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, WorldBench, Performance Test, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.

We are sometimes asked how we select the tests we have in benchmarks. The goal is to represent the types of actions that virtualization users are doing on a regular basis. In particular, we focus more on user tasks rather than installation or configuration (because they are typically only done once or infrequently).

In the PC market, PC World (magazine) uses WorldBench as their benchmarking tool of choice. If you don't know about WorldBench, you can see more at http://www.worldbench.com. WorldBench 7 uses a combination of applications bundled together to determine benchmarks. This may be a very good way for PC World to approach things, but obviously it's not reflecting the virtualization part of the experience or weighing towards the typical types of things people do in virtualization on the Mac.

There are a variety of other benchmarks available, and we've looked at many of them as possible tools to include in our mix. Often, we find their measurements simply don't reflect a true user experience. Other times, they don't reflect how a virtualization user may be using Windows, or even they simply report erroneous results. For example, in previous attempts to use PassMark, we found their graphics scores were not at all representative of real life graphics performance in games or other graphic uses. Those tests showed items as faster when, in fact, they were much slower. Benchmarks are only useful if they reflect the real world.

Rather than use WorldBench or others, we focus on the types of measurements we believe best represent the experience we see (in terms of speed). And while it takes practice and some skill, we test virtualization operations with a stopwatch—as the closest representation of how a real user would see it. CPU crunching ability, for example, is measured through tests like zip'ing files directly in Windows. For Office users, we use the most up to date version of Microsoft Office for Windows (2010), again with updates.

There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results. In the case of graphics benchmarking, we used 3DMark06 (the industry standard for tuning gaming systems). For game playing performance, we used FRAPS to measure frames per second. Finally, for CPU utilization we used "top" combined with Apple Remote Desktop.

Remember, benchmarks are not a review: they are benchmarks. They are simply meant to just tell you which runs faster. If you are interested in a specific feature, support, the user interface, or any of the other criteria for deciding on a product, that's a different article.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Apple iBooks Author 2.5 - Create and pub...
Apple iBooks Author helps you create and publish amazing Multi-Touch books for iPad. Now anyone can create stunning iBooks textbooks, cookbooks, history books, picture books, and more for iPad. All... Read more
TunnelBear 3.0.1 - Subscription-based pr...
TunnelBear is a subscription-based virtual private network (VPN) service and companion app, enabling you to browse the internet privately and securely. Features Browse privately - Secure your data... Read more
ExpanDrive 5.4.0 - Access cloud storage...
ExpanDrive builds cloud storage in every application, acts just like a USB drive plugged into your Mac. With ExpanDrive, you can securely access any remote file server directly from the Finder or... Read more
Opera 40.0.2308.62 - High-performance We...
Opera is a fast and secure browser trusted by millions of users. With the intuitive interface, Speed Dial and visual bookmarks for organizing favorite sites, news feature with fresh, relevant content... Read more
Apple iOS 10.0.2 - The latest version of...
iOS 10 is the biggest release of iOS ever. A massive update to Messages brings the power of the App Store to your conversations and makes messaging more personal than ever. Find your route with... Read more
Herald 7.0 - Notification plugin for Mai...
Note: Versions 2.1.3 (for OS X 10.7), 3.0.6 (for OS X 10.8), 4.0.8 (for OS X 10.9), 5.0.2 (for OS X 10.10), and 6.0.3 (for OS X 10.11) are no longer supported by the developer. Herald is a... Read more
Postbox 5.0.2 - Powerful and flexible em...
Postbox is a new email application that helps you organize your work life and get stuff done. It has all the elegance and simplicity of Apple Mail, but with more power and flexibility to manage even... Read more
Yummy FTP Pro 1.11.10 - $29.99
Yummy FTP Pro is an advanced Mac file transfer app which provides a full-featured professional toolkit combined with blazing speeds and impeccable reliability, so whether you want to transfer a few... Read more
Tidy Up 4.1.17 - $29.99
Tidy Up is a full-featured duplicate finder and disk-tidiness utility. Features With Tidy Up you can search for duplicate files and packages by the owner application, content, type, creator,... Read more
Logitech Control Center 3.9.5 - Keyboard...
Logitech Control Center (LCC) is designed to support OS X and allows you to take full advantage of your Logitech keyboard, mouse, or trackball. With the LCC you can: Browse the Internet using... Read more

How to build and care for your team in D...
Before you hit the trail and become a dog sledding legend, there’s actually a fair bit of prep work to be done. In Dog Sled Saga, you’re not only racing, you’re also building and caring for a team of furry friends. There’s a lot to consider—... | Read more »
How to win every race in Dog Sled Saga
If I had to guess, I’d say Dog Sled Saga is the most adorable racing game on the App Store right now. It’s a dog sled racing sim full of adorable, loyal puppies. Just look at those fluffy little tails wagging. Behind that cute, pixelated facade is... | Read more »
Let the war games commence in Gunship Ba...
Buzz Lightyear famously said, “This isn’t flying, this is falling – with style!” In the case of Gunship Battle: Second War, though, this really is flying - with style! The flight simulator app from Joycity puts you in control of 20 faithfully... | Read more »
How to get a high score in Fired Up
Fired Up is Noodlecake Games’ high score chasing, firefighting adventure. You take control of a wayward firefighter who propels himself up the side of a highrise with blasts of water. Sound silly? It is. It’s also pretty difficult. You can’t... | Read more »
NBA 2K17 (Games)
NBA 2K17 1.0 Device: iOS iPhone Category: Games Price: $7.99, Version: 1.0 (iTunes) Description: Following the record-breaking launch of NBA 2K16, the NBA 2K franchise continues to stake its claim as the most authentic sports video... | Read more »
Dog Sled Saga (Games)
Dog Sled Saga 1.0.1 Device: iOS Universal Category: Games Price: $3.99, Version: 1.0.1 (iTunes) Description: A game by Dan + Lisa As a rookie musher, foster a dogsledding team whose skills will grow if they're treated right. Week by... | Read more »
60 Seconds! Atomic Adventure (Games)
60 Seconds! Atomic Adventure 1.2 Device: iOS Universal Category: Games Price: $2.99, Version: 1.2 (iTunes) Description: 60 Seconds! is a dark comedy atomic adventure of scavenge and survival. Collect supplies and rescue your family... | Read more »
Tons of Bullets! (Games)
Tons of Bullets! 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Tons of BulletsA retro 2D pixel platformer infused with tons of bullets and tons of features!Fight as Kenji the Ninja and... | Read more »
Sorcery! 4 (Games)
Sorcery! 4 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: *** PLEASE NOTE: Sorcery! 4 requires a minimum of iPad 3 or iPhone 4 *** An epic adventure through a cursed citadel of monsters,... | Read more »
Building the perfect Animation Throwdown...
Animation Throwdown is a casual card game full of plenty of inside jokes from some of your favorite cartoons. It’s accessible as far as card games go, but that’s not to say that it hands your victories to you easily. There’s quite a bit that goes... | Read more »

Price Scanner via MacPrices.net

Duplicate Sweeper Free On Mac App Store For O...
To celebrate the launch of Apple’s latest macOS Sierra, Stafford, United Kingdom based Wide Angle Software has announced that its duplicate file finder software, Duplicate Sweeper, is now available... Read more
13-inch Retina MacBook Pros on sale for up to...
B&H Photo has 13″ Retina Apple MacBook Pros on sale for up to $150 off MSRP. Shipping is free, and B&H charges NY tax only: - 13″ 2.7GHz/128GB Retina MacBook Pro: $1174.99 $125 off MSRP - 13... Read more
Evidence Surfaces Pointing To New A10X Chip F...
Citing a job description for a Project Lead position at Apple’s Austin, Texas engineering labs, Motley Fool’s Ashraf Eassa deduces that development is progressing well on Apple’s next-generation in-... Read more
Check Print’R for macOS Allows Anyone to Easi...
Delaware-based Match Software has announced the release and immediate availability of Check Print’R 3.21, an important update to their easy-to-use check printing application for macOS. Check Print’R... Read more
Apple refurbished 11-inch MacBook Airs availa...
Apple has Certified Refurbished 11″ MacBook Airs (the latest models), available for up to $170 off the cost of new models. An Apple one-year warranty is included with each MacBook, and shipping is... Read more
Apple refurbished 15-inch Retina MacBook Pros...
Apple has Certified Refurbished 2015 15″ Retina MacBook Pros available for up to $380 off the cost of new models. An Apple one-year warranty is included with each model, and shipping is free: - 15″ 2... Read more
Major Version 5 Release Of Free myTuner Radio...
AppGeneration Software has announced the release of myTuner Radio 5.0.1, an important update to their live radio app for iOS and Android platforms. With myTuner Radio, you can listen to more than 30,... Read more
TheAppStore Searches And Filters iOS And macO...
Boston based InchWest has announced the official launch of TheAppStore, a free web based service that lets users search iOS and Mac apps on the web without iTunes or the App store. TheAppStore... Read more
2TB Time Capsule on sale for $199, save $100
Amazon has 2TB Apple Time Capsules on sale for $199 including free shipping. Their price is $100 off MSRP. Choose Amazon as the seller when making your purchase, rather than a third party seller. Read more
12-inch 1.2GHz Rose Gold Retina MacBook on sa...
Amazon.com has the 2016 12″ 1.2GHz Rose Gold Retina MacBook on sale for $1254.02 including free shipping. Their price is $345 off MSRP, and it’s the lowest price available for this model from any... Read more

Jobs Board

Senior *Apple* Administrator - Pratt Instit...
POSITION SUMMARY: Directs the coordination and standardization of campus-wide Apple systems, including planning, analysis and implementation of Apple -related Read more
Restaurant Manager (Neighborhood Captain) - A...
…in every aspect of daily operation. WHY YOU'LL LIKE IT: You'll be the Big Apple . You'll solve problems. You'll get to show your ability to handle the stress and Read more
*Apple* Retail - Multiple Positions- Napervi...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
US- *Apple* Store Leader Program - Apple (Un...
…Summary Learn and grow as you explore the art of leadership at the Apple Store. You'll master our retail business inside and out through training, hands-on Read more
*Apple* Retail - Multiple Positions (US) - A...
Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, you're also the Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.