Footprint Tests

One of the most interesting things in the virtualization market is how little overhead virtualization takes today compared to what it used to. To assess this, we measured in a variety of ways. Specifically, we focused on CPU usage (overall for the Mac), which has an impact not only on usability but also, on laptops, how long the battery would last. CPU usage was measured using "top" (a command line tool that’s part of UNIX with a minimum of 50 continuous samples averaged for the result). For all tests, we removed the high and low use outliers to give a more typical impact.

We ran two CPU usage tests. The first test was to boot Windows and let it sit for a few minutes to finish up its startup activities. Once done, CPU use was measured while both the Mac OS X and Windows were sitting idle. The next test studied the same thing, except both Microsoft Word and Microsoft Outlook were launched and left sitting idle.

In this case, both products win. CPU idle was as low as 0.3% to as much as 8% (on a MacBook). Sometimes, one product won, and then the other. Overall, the CPU usage was so low for both products they should be applauded. If battery usage is your concern, Parallels had slightly lower CPU utilization on the laptops, and you would see slightly better battery duration using Parallels Desktop.

File and Network IO Tests

One of the common problems with File and Network IO Tests is caching. In fact, it’s common for benchmarkers to think they are avoiding caching when in fact, they aren’t. In the case of File and Network IO tests, there are two types of caching: at the host OS, and the guest OS. Furthermore, sometimes the caching can appear to happen even after restarts of both the host and guest OS.

In our tests, we used four different types of data sets to give the most representative results, and used a methodology to avoid caching. The three file sets were:


  • 1GB file with randomized data. To generate the file of exactly 1GB, we used the command “dd if=/dev/random” from the command line.
  • 100MB version of randomized data (again, generated from the command line)
  • 450MB of small files in our "document set" consisting of:

    • Office Files: 1000+ .xls, 200 .ppt, and 500+ .doc files.
    • Text Files: 125+ rtf, and 25+ text files.
    • Graphics Files: 1000+ gif, and 250+ jpgs.
    • Web Page Files: 6500 htm, and 25+ swf files.
    • Compressed Files: 5 approximately ten MB rar, and 5 approximately ten MB zip files.

  • 250 MBs of jpgs for photo imports, 50+ 3-5MB files.

The idea behind the "document set" is to test a number of smaller files to test more real world scenarios related to items written to the local drive. It turns out there’s a fair amount of overhead and a significant difference in these scenarios, so we assembled a set of documents of varying types to represent what users may move around in Windows environment.

For the USB testing, we used a set of JPEG files to simulate "photo imports" that are common for people to do.

FileIO
Virtual Machine File I/O Performance

For File I/O, we found Parallels to be slightly faster than VMware Fusion in copy tests to and from the USB flash drive. Using the "document set" described above to test copying a number of different types and kinds of files, Parallels was more than double the speed when copying files to and from the local hard drive. In a test of the virtual hard drive alone, Parallels was somewhat faster than VMware Fusion.

Networking was done via the default networking setup in both virtualized environments, and the defaults for disk setup for each application were used as well. WiFi was set up on the host OS (e.g., the Mac’s WiFi) and because the Windows "client" OS was just sharing the Mac’s network connection; no changes were necessary within Windows (other than a reboot).

Network.pdf
Virtual Machine Network I/O Performance

In terms of network performance, Parallels Desktop was consistently faster in both wired and wireless copying to and from the server on our test LAN, and most of the time, the difference was significant.


Virtualization is one of the topics at the next MacTech Boot Camp

Packed schedule. Learn from the best.
Meet new people. Network with peers.
One day semimar for techs and consultants who support home users, small office, and small to medium size business. Invest in yourself. Click here for more info.
Space is limited. Register today as early bird and save $100.