Volume Number: 26
Issue Number: 01
Column Tag: Virtualization
By Neil Ticktin, Editor-in-Chief/Publisher
Start | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 Next Page>
A few years back, Apple switched the Mac platform from PowerPC to Intel processors. This introduced some interesting opportunities for the Mac, including the ability to run operating systems other than Mac OS X on a Mac. This includes not only your everyday Windows XP, but also other Windows OSes, including 64-bit versions, and a wide variety of Linux and other OSes.
You may already understand your options for running Windows on a Mac, but in case you don't, your first choice is to decide between Apple's Boot Camp, and a virtualization product like VMware Fusion, or Parallels Desktop for Mac. With Apple's Boot Camp, you can run Windows natively, but you have to reboot every time you want to switch between Mac OS and Windows. In addition, you can only use one OS at a time. That's not particularly convenient. For that reason, MacTech recommends a virtualization product for most users.
Virtualization gives "switchers" (those switching from Windows to Mac) comfort because they can use their old applications, more easily get to old data, and in many cases, it allows them to use some critical piece of software not available on the Mac. For example, your job may require you to run Internet Explorer, or Outlook, if that's all that your corporate systems support. Of course, you may simply want to play a Windows game not available on the Mac.
With virtualization, like you'll see in Parallels Desktop for Mac or in VMware Fusion, you can run Windows alongside the Mac OS, getting the best of both worlds. For many, this may mean running Windows in a "window" on your Mac. Both VMware Fusion and Parallels Desktop also have the ability to run Windows applications even more transparently, but we'll leave that for another discussion.
So which virtualization product do you go with? Which solution is faster? Should you run Windows XP or 7? 32-bit or 64-bit? One virtual processor or more? In short, there are different answers for different people: it all depends on your needs.
To tackle this problem, MacTech undertook a huge benchmarking project starting a couple months ago. This is similar to the virtualization benchmarking project that MacTech undertook in the past. In both cases, as with other MacTech benchmarks, we tested performance of the types of things that everyday users typically do. In this case, it was not just testing the Windows OS performance, but also commonly used Windows applications. This time, we paid even more attention to 3D graphics and gaming.
The goal was to see how VMware Fusion and Parallels Desktop performed, covering both Windows XP and Windows 7. Furthermore, we wanted to see some of the differences with different Mac models, multiple processors and 64-bit versions of Windows.
Sounds simple enough, right? But when you start to realize that there were four different models of Macs, two virtualized environments, XP and Windows 7, some of which were 64-bit and multiprocessor while most were 32-bit and single virtual CPU, along with a whole slew of tests, each needing to be run several times, there were thousands of tests to be completed ... many with a stopwatch.
In fact, the testing took long enough that during the course of preparing these tests, both Parallels and VMware released minor updates, and those are the versions that we tested with.
To be clear, this article is not a product review; it's a benchmarking analysis (although we were morally obligated to comment on some of the 3D graphics and games <g>). The article's purpose is to assess performance (including issues we found if something didn't work right), and not product features, user interface, etc... You should use feature and support information in conjunction with the below benchmarking results to make your product choice.
Before you ask why other solutions and products were not included, let's answer that. In short, my response is "give me a break." This benchmark project is already huge with several thousand collected data points crossing two guest OSes (Windows XP and 7), four models of computers, and over 50 tests each run 3-5 times for most of the environments. And that's before a single word of this article is written! To add even one more product would increase this huge test matrix by 50%. As a result, we focused on the two leading commercial virtualization products. These two products also have free trial versions that you can compare against other solutions ... so you can try them for yourself and compare alongside any other solution you want at any time including before you buy.
One thing to note, however, is that the open source and other products may not be for everyone, especially non-technical users. For many, these offerings can be difficult to understand and install, and they do not have technical or product support behind them. Obviously, users can reach out in forums and the community, but if you don't understand something, you won't be able to pick up the phone and call support the way you can with a commercial product. If that works for you, great. If not, as is the case for most users, then a commercial product is likely your solution.
The Test Bench
When we were choosing computer models, we set out to choose the current models of Macs giving a good representation of what most people may have. Certainly, the faster models of these computers will perform even faster.
We chose four Mac models to compare alongside each other: the "White" MacBook, MacBook Pro, iMac, and Mac Pro.
Memory for virtual machines can be configured with a wide array of settings. As a general rule, VMware Fusion's default memory for each configuration (of physical RAM and "guest" OS) was the same or lower than Parallels Desktop (although sometimes it was higher). As a result, we let VMware's default guide the way, and set Parallels Desktop to the same as whatever VMware Fusion defaulted to. In most cases, this was 512MB for Windows XP, and 1GB for Windows 7.
Similarly, for disk allocation, we defaulted to the sizes set up by VMware Fusion (40GB) and allowed for expanding disks. We took special care to make sure that the disk location of the virtual hard drive was in a similar physical location as that can make a significant difference in and of itself in disk performance.
The tests compared VMware Fusion 3.0.1, with Parallels Desktop for Mac 5.0 (build 9308), running Mac OS X 10.6.2. All required Windows updates were installed for each version of XP and Windows 7.
There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.
When testing virtual machines, however, using outside utilities within a virtualized environment often provides strange and incorrect results. For example, there are many issues with internal timers not displaying measurements accurately, and some suites don't work at all in a virtual machine environment. Instead, as we have in the past, MacTech chose to create a suite of tests that would mimic what many users would do in normal use, but stick to those actions that were repeatable, and measurable with a stopwatch. After all, if you couldn't perceive a difference with a stopwatch, the user is not likely to perceive it at all.
There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results.
To that end, there are several kinds of tests that we ran: Virtual Machine Launch/Suspend, CPU tests, File and Network IO, CPU Usage Footprint, Application Performance, and 3D and HD Graphics.
