Volume Number: 25 (2009)
Issue Number: 04
Column Tag: Virtualization

Head-to-Head: Parallels Desktop for Mac vs. VMware Fusion (cont.)

How do VMware Fusion and Parallels Desktop for Mac stack up?

by Neil Ticktin, Editor-in-Chief/Publisher

When we first started out to test 3D and HD Graphics, we were looking for ways to measure in a benchmark setting. What we found were a couple of things. First, in some cases, the performance on both platforms was so good; there was nothing that we could measure. Second, the common metric, frames per second, is a poor method for measuring this. Let's look at each case to explain.

One thing to note about the differences between BootCamp and virtualization products: According to Apple's tech note, when it comes to the new MacBook Pros with multiple graphics processors, there are issues with graphics support when running Apple's BootCamp. Specifically, "You may notice that the built-in NVIDIA GeForce 9400M graphics subsystem does not appear in the Device Manager and cannot be used with Microsoft Windows XP or Vista." (See http://support.apple.com/kb/TS2457 ). This is not the case for the virtualization products mentioned here, where it works fine.

To test HD Video, we used the movies on Microsoft's web site at www.microsoft.com/windows/windowsmedia/musicandvideo/hdvideo/contentshowcase.aspx. We found that 720p movies played quickly and smoothly in both environments under XP.

For Vista, however, it was entirely differently. Parallels Desktop was unable to play it at all. VMware Fusion stuttered on the MacBook, MacBook Pro, and iMac, but was able to play it fine on the Mac Pro.

The 1080p videos also came from the above Microsoft site. They played well under both Parallels Desktop and VMware Fusion running XP. Given the 720p results, we didn't bother attempting them under Vista.

One of the most frequent questions that we were asked after our last virtualization benchmarks was "What about games?" Choosing games to test in virtualized environments is extremely difficult. After all, you'd want to look at games that run on both platforms (even though both virtualization products support slightly different graphics capabilities). And, you'd want to test a game that isn't available on the Mac (otherwise, why wouldn't you just run it there?) And, of course, you want to test games that people are eager to play.

There are very few games that fit all of these parameters, but we were able to find two.

This is a relatively new game built on the well-established Civilization IV engine. Sid Meier's Civilization IV: Colonization is the third offering in the Civilization IV series. It's essentially a modern re-make (Firaxis calls it a re-imagining) of the classic Colonization game Sid Meier created in 1994, Sid Meier's Civilization IV: Colonization is a conversion of the Civilization IV engine into a game experience in which players lead a European nation on their quest to colonize and thrive in the "New World." Players are challenged to guide their people from the oppressive motherland, discover a New World, negotiate, trade and fight as they acquire great power and battle for their freedom and independence. As you can see from the overview screen shot, the graphics provide an amazing experience.

Figure 12: Civilization IV: Colonization, Overview of game running on PC

To benchmark, we loaded the same Civilization IV: Colonization scenario on each machine, and used the built-in tools of the game to measure frames per second. The results showed Parallels Desktop winning the FPS test across the board, but this was not representative of the situation. Yes, Parallels Desktop had a faster FPS (see spreadsheet if you are really interested), but the richness of the VMware Fusion graphics was significantly more detailed and better. For example, instead of just water, VMware Fusion had many more fish jumping out of the water, more realistic textures, really nice reflections, detail on land, etc. At the same time, Parallels Desktop was more fluid (i.e., it worked better) on the slower machines. FPS was clearly not a good test to judge these environments by.

In general, the game ran very well in both environments. VMware Fusion had difficulties showing the startup video (it would often not play the video, just the sound), but if you hit the escape key, and took it full screen, it worked fine after that. Parallels Desktop didn't have those issues, but as we mentioned, the graphics were not as rich. You can get an idea comparing the two screen shots here, but it's even more apparent with the graphics in motion.

Figure 13: Civilization IV: Colonization under Parallels Desktop

Figure 14: Civilization IV: Colonization under VMware Fusion

VMware Fusion had issues running it on the MacBook, presumably because of the lighter graphics capabilities of that model. Under Parallels Desktop on a MacBook, it was playable, but not a great experience.

Either way, if you have an interest in this period of history, they both played well in virtualized environments, and you should check them out. http://www.firaxis.com/

Portal is an incredibly enticing "action/puzzle" game developed by Valve Software (http://www.valvesoftware.com). As described by Valve, "The game is designed to change the way players approach, manipulate, and surmise the possibilities in a given environment: players must solve physical puzzles and challenges by opening portals to maneuvering objects, and themselves, through space."

Figure 15: Portal under Parallels Desktop

After installing the game through Steam (http://store.steampowered.com/), we tested the game by measuring frames per second while running a consistent demo mode. Here, there were mixed results. When looking at frames per second in Portal, all the machines did fairly well, except we were unable to get the game to run at all on the MacBook running VMware Fusion. No idea what that's about other than possibly, again, the light graphics capabilities of a MacBook. There were some additional comments on the VMware community board on problems with Portal for some running XP SP3. Parallels Desktop was a bit choppier on the MacBook, but it did run, and was completely playable.

For the other Mac models, VMware Fusion was much faster when looking at the frame rate measurements, but we didn't really see a difference in smoothness. One issue, however, with VMware Fusion is that we found that the graphics on all screens was much lighter (it made us feel like we were outside on a bright sunny day without sunglasses). Parallels Desktop felt much more comfortable to look at, and the detail showed better with the proper contrast/brightness. Compare the two figures for yourself to see.

Figure 16: Portal under VMware Fusion

First, while we were eager to see the frames per second measurements, and we've left them in the spreadsheet for you to see if you want to, we find this measurement a poor way to judge things. Clearly, the graphics for VMware Fusion were much richer in Civilization IV: Colonization, despite a lower FPS. And, similarly, while Parallels Desktop's FPS was lower for Portal, it was a better experience because of the "brightness" of the graphics. [Ed. Note: Parallels has told MacTech that Parallels Desktop is designed to work with vsync enabled in a synchronous mode with the monitor. In other words, they intentionally limit Frames Per Second (FPS) to not exceed the limits of the monitor, and that 60 FPS is standard for the majority of LCD monitors. Apparently, Apple's guidelines recommend this to save battery life and minimize fan operation.]

Our advice? If gaming is your primary reason for a virtual machine, then figure out what games you want to play, and see what the online discussions are about them. If gaming is just an added benefit for you, then choose the games that work best for your VM. Either way, we were pleasantly surprised at how well both these games played in a virtualized environment. And highly recommend both of the games we played with ... er, I mean, looked at, here. :)

The graphics standards can get confusing quickly. To clarify, DirectX 9.0 Shader Model 2 standard has two parts: Pixel and Vertex Shaders 2.0.

Both VMware Fusion 2.0.1 and 2.0.2 supports DirectX 9.0 with full Shader Model 2 graphics. That's why reflections in water and such look so good above.

Parallels Desktop 4.0 for Mac build 3540 (the version tested) supported Pixel Shaders 2.0 only and that made a visible difference as you saw in Civilization IV: Colonization. But, in build 3810 they added support for Vertex Shaders 2.0. This addition means that build 3810 and beyond supports DirectX 9.0 the full Shader Model 2 like VMware Fusion. [Ed. Note: Parallels reports to MacTech that this new version makes a visible difference in the quality of the graphics in Civilization IV: Colonization.]

There are additional conclusions that we can extract from the results as well. Specifically, we looked at the differences between XP and Vista, as well as multiple virtual processors and 64-bit Vista.

In our last virtualization benchmarking article, there was a significant difference between running Microsoft XP vs. Microsoft Vista. Now, the difference is much less significant, in fact, while XP continues to be faster for most things, it's probably not enough to matter. You should make your choice based on which OS you want. That said, anecdotally, we find XP under virtualization to be far more stable, and definitely less annoying to run.

Figure 17: Windows XP vs. Windows Vista Performance

There's a big push right now for multiple virtual processors and 64-bit Windows. And, while there are times that you may need them, most people will not.

Multiple virtual processors are helpful for when you have a computationally intensive application, and you need to split the work. The types of applications that you normally would need this for include video, Photoshop, CAD, etc... Frankly, if speed is that important to you, you should be asking yourself about whether to run the app native on your Mac instead of in a virtual machine. Sometimes, like for CAD, you may not have an option. Take note, however, Microsoft has limitations in their user license on the number of processors you are running. While Parallels Desktop supports multiple cores up to two quad core virtual processors, VMware Fusion supports only multiple virtual processors, not multiple virtual cores. As a result, we could only test both up to two virtual CPUs under Windows.

64-bit is another issue. Primarily, your big benefit here is that you can address a whole lot more memory. For most virtualization users, this is likely not relevant. Many of the Windows applications and drivers still are not 64-bit compatible, and there are all kinds of reports of issues. This is the future of where Windows is going, but it's not necessary for most users.

That said, we wanted to give you a look at what performance looked like for both of these for what most people probably run: Windows, Microsoft Office, and Internet Explorer. As you can see, while there's some difference, you have to judge if it's enough to be worthwhile. For most, 32-bit XP is likely adequate.

Figure 18: Multiple Virtual Processors, 64-bit Performance