TweetFollow Us on Twitter

Major Releases, Benchmarks, and Feedback

Volume Number: 24 (2008)
Issue Number: 08
Column Tag: Industry

Major Releases, Benchmarks, and Feedback

Benchmarks and how the community contributes

By Neil Ticktin, Editor-in-Chief/Publisher

Complexities

Software is a complex beast these days, and it has been for a while. Just take a look at the storage on your local machine, and you will likely see hundreds of thousands of files and folders, if not a million or more.

OS X, like other OSes, has many different configurations - more than is realistically possible for anyone to test on even for the biggest of companies. As a result, developers have gone into the mode of trying to write the best quality software that they can, do realistic testing, and then focus enormous efforts on listening for problems, and reacting to them. Both Microsoft and Apple are great examples of having done this on the Mac.

But with this approach comes a lot of responsibility on the community, especially those that are technical in the market. As a MacTech reader you are at the core of this market). Like it or not, this responsibility lands largely on your shoulders.

Leopard and Microsoft Office 2008 are both giving us good examples of what to look for, what to expect, and what our responsibilities are.

Benchmarks

MacTech strives to give you quality benchmarks. We've done it for virtualization environments, Rosetta, hard drives, and Microsoft Office. In all of these cases, like anyone else that does high end testing, we do this in "virgin" or "clean" environments. It's the only way to do the testing, as you cannot penalize a product by putting it on a machine that needs to have a disk utility run on it, or has system problems, or a congested network.

Benchmarks are a great tool — they allow you to see, in an "apples to apples" comparison, how well a product performs. As the pros in the industry, you take those benchmarks and start to apply them to your real world setting, as only you know the setups you have. When major releases come out, the distance between a clean setup and yours may be that much more important, and here, your involvement grows.

Let's take Leopard and Microsoft Office 2008 as examples. Both of these are herculean efforts on behalf of their developers, and are massive releases. As anyone experienced will tell you, releases of this size are never perfect. There's no way for the developer to test all permutations of hardware, OS release and other software installed on the machine.

Community Feedback

So, what's a geek to do? Well, that's a good question. When major new releases come out, you should get a copy in your hands as fast as possible, and start to use it yourself. What you should not to is put yourself in a situation where you are supporting others using it, and furthermore, you should not yourself be in a position where you are dependent on it working.

As you are able to experiment with these new versions, you want to understand what, if any, pitfalls you fall into. You should be looking for bugs, incompatibilities and more.

Now here's the important part ...

you need to report them.

This is so much more important today than it was in the past. As the technical community, it's become our responsibility to report the problems so that they can be fixed. If we don't report them, they are not likely to be fixed. Griping about issues on public mailing lists and in blogs doesn't count. Even more important, the bug tracking and reporting tools have become sophisticated enough that they not only provide information on what you are experiencing, but give statistical information to help rank the importance of the problems within the community.

Apple and Mac OS X / Mac OS X Server

As many in the industry will tell you, Apple versions tend to stabilize on the .3 version. For example: Leopard will likely stabilize quite a bit on 10.5.3. It just takes that many interactions of versions and general usage to get the major kinks worked out. That's not to say that there won't be bugs with a .3 version; Historically, the .3 version is the one that generally runs well, and then has incremental improvements from there.

Every time you have an application that unexpectedly quits, a dialog appears asking for permission to send information to Apple. You should use this as much as you can, as it's not only fast, but it gives very specific machine generated information that is very helpful to Apple in understanding the situation that just happened.

In looking through Apple's online support, you'll see a way to give Apple feedback on whether an answer was helpful to you or not. Take the few seconds to let them know if it was helpful or not— over time, it makes a difference.

If you are looking for help on an issue, there are a couple of things that you can do. First, you can use MacTech's "Community Search Engine." Here, we've selected the best web sites in the community, including Apple's support sections, to be a part of the search. So, when you are looking for "rebuild degraded RAID," you'll get only those search results relevant to the Mac and OS X, and not the hundreds of thousands of answers you might see with a general Google search. See the left navigation bar on MacTech.com or http://www.mactech.com/advancedsearch.html — either way don't forget to press the "Mac Community" radio button for that type of search.

Apple uses their "Bug Reporter" system at http://bugreporter.apple.com/ as the primary way to report bugs to Apple. ADC Membership (free registration) is required to use this tool. And, there's a great article on AFP548.com that talks about the best ways to use Apple's system — see http://www.afp548.com/article.php?story=20051102133234150

Microsoft Office

Microsoft also uses a variety of ways to have people report in information. In fact, they appear to have broader methods of interacting than Apple does.

Microsoft has a "Send Feedback" command in the Help menu that redirects users to a page on the Mactopia web site. This information is generally used to plan future products for Microsoft.

See http://www.microsoft.com/mac/suggestions.mspx

Like Apple, the online Help sections on the Microsoft web site also asks, "Was this helpful?" Clicking the "No" or "Somewhat" buttons prompts for slightly more detail. When the community tells them why a topic isn't helpful, they can use that feedback to change the topic online, without having to wait for the next major release for offline content updates. Microsoft calls this "continuous publishing."

If you are looking for help on any of the Microsoft Office products, you should look to the newsgroup forums. Here, there are MVPs (not Microsoft employees) that know the products in depth, and will give you very candid answers. They are particularly good at answering "how to" questions. In addition, Microsoft read the posts looking for problem reports, and if appropriate will compare them to reported issues, or create a new one. Here are the URLs for the web interface into the newsgroup forums:

http://www.officeformac.com/ProductForums/Word/

http://www.officeformac.com/ProductForums/Excel/

http://www.officeformac.com/ProductForums/PowerPoint/

http://www.officeformac.com/ProductForums/Entourage/

http://www.officeformac.com/ProductForums/MacMessenger/

When you first install Office, it asks if you want to be part of the Customer Experience Improvement Program (aka CEIP) which gives Office permission to periodically send statistical data back to Microsoft about how the products are used. No personal information is collected. This is very helpful to Microsoft, if you are game. (The default is off, so you would need to turn it on.)

Finally, Microsoft uses Microsoft Error Reporting (MERP on the Mac, and Watson for Windows), which is for real time error reporting. Like Apple's reporter for Mac OS X, this is an excellent way to quickly send information that is machine generated about the problem that you are experiencing.

Statistical Benefits

What all the larger organizations have in their error reporting is the ability to gather significance statistically from the error reports. For example, there are patterns to problems, and the bug reports are done in a way that Microsoft (and presumably Apple), can compare two bug reports and see if they are the same issue or not. Once that happens, they can determine whether an issue has widespread impact, or is localized to just a few users. In other words, it helps them to prioritize which things need to be fixed first.

Another example is that they can see that certain problems may be on specific OS versions, categories of machines, or even specific models of machines. This not only helps in understanding how to fix the bug, but is also may give insight that it's not a vendors bug.

For example, in the next maintenance release of Leopard, Apple is rumored to be fixing a large number of items. Microsoft may be looking at bug reports for an issue, and may very well find that the problem in question is not one with Office, but with the Operating System instead. Of course, the same thing could happen in the opposite direction where Apple receives bug reports, and finds that it's a third party.

Near Term Expectations

Leopard's next update will likely be the one that fixes so much stuff that those that have been waiting, should now move forward. Naturally, Microsoft is working on updates for Office 2008 as well, presumably called SP1 (Service Pack 1, if they hold true to naming conventions). These will likely to fix many issues that widespread distribution saw, but MacTech may not have with the benchmarks on clean machines.

Why is this expected? Because both these companies have mechanisms in place to listen to customer experiences, problems, and bugs, research them down through statistical and other comparisons, and rank their priority accordingly.

But, all of that doesn't work unless we, the technical community, make sure to use the reporting mechanisms as much as possible.


Neil is the Editor-in-Chief and Publisher of MacTech Magazine. Neil has been in the Mac industry since 1985, has developed software, written documentation, been heading up the magazine since 1992. When Neil writes a review, he likes to put solutions into a real-life scenario and then write about that experience from the user point of view. That said, Neil has a reputation around the office for pushing software to its limits and crashing software/finding bugs. Drop him a line at publisher@mactech.com

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Apple Remote Desktop 3.8 - Remotely cont...
Apple Remote Desktop is the best way to manage the Mac computers on your network. Distribute software, provide real-time online help to end users, create detailed software and hardware reports, and... Read more
NeoOffice 2014.7 - Mac-tailored, OpenOff...
NeoOffice is a complete office suite for OS X. With NeoOffice, users can view, edit, and save OpenOffice documents, PDF files, and most Microsoft Word, Excel, and PowerPoint documents. NeoOffice 3.x... Read more
DesktopLyrics 2.6.6 - Displays current i...
DesktopLyrics is an application that displays the lyrics of the song currently playing in "iTunes" right on your desktop. The lyrics for the song have to be set in iTunes; DesktopLyrics does nothing... Read more
Ember 1.8.3 - Versatile digital scrapboo...
Ember (formerly LittleSnapper) is your digital scrapbook of things that inspire you: websites, photos, apps or other things. Just drag in images that you want to keep, organize them into relevant... Read more
Apple iTunes 12.1 - Manage your music, m...
Apple iTunes lets you organize and play digital music and video on your computer. It can automatically download new music, app, and book purchases across all your devices and computers. And it's a... Read more
LibreOffice 4.4.3 - Free, open-source of...
LibreOffice is an office suite (word processor, spreadsheet, presentations, drawing tool) compatible with other major office suites. The Document Foundation is coordinating development and... Read more
FoldersSynchronizer 4.2.1 - Synchronize...
FoldersSynchronizer is a popular and useful utility that synchronizes and backs-up files, folders, disks and boot disks. On each session you can apply special options like Timers, Multiple Folders,... Read more
Simon 4.0.2 - Monitor changes and crashe...
Simon monitors websites and alerts you of crashes and changes. Select pages to monitor, choose your alert options, and customize your settings. Simon does the rest. Keep a watchful eye on your... Read more
Cocktail 8.1.2 - General maintenance and...
Cocktail is a general purpose utility for OS X that lets you clean, repair and optimize your Mac. It is a powerful digital toolset that helps hundreds of thousands of Mac users around the world get... Read more
Cyberduck 4.6.4 - FTP and SFTP browser....
Cyberduck is a robust FTP/FTP-TLS/SFTP browser for the Mac whose lack of visual clutter and cleverly intuitive features make it easy to use. Support for external editors and system technologies such... Read more

Playworld Superheroes Review
Playworld Superheroes Review By Tre Lawrence on January 30th, 2015 Our Rating: :: HERO CRAFTINGUniversal App - Designed for iPhone and iPad It’s all about the imagination, fighting bad creatures — and looking good while doing so.   | Read more »
Join the SpongeBob Bubble Party in this...
Join the SpongeBob Bubble Party in this New Match 3 Bubble Poppin’ Frenzy Posted by Jessica Fisher on January 30th, 2015 [ permalink ] | Read more »
Handpick Review
Handpick Review By Jennifer Allen on January 30th, 2015 Our Rating: :: TANTALIZING SUGGESTIONSiPhone App - Designed for the iPhone, compatible with the iPad Handpick will make you hungry, as well as inspire you to cook something... | Read more »
Storm the Halls of Echo Base in First St...
Storm the Halls of Echo Base in First Star Wars: Galactic Defense Event Posted by Jessica Fisher on January 30th, 2015 [ permalink ] Universal App - Designed for iPhone and iPad | Read more »
Contradiction Review
Contradiction Review By Tre Lawrence on January 30th, 2015 Our Rating: :: SPOT THE LIEiPad Only App - Designed for the iPad Contradiction is a live action point and click adventure that’s pretty engaging.   Developer: Tim Follin... | Read more »
Unlock Sunshine Girl in Ironkill with th...
Unlock Sunshine Girl in Ironkill with this special 148Apps code Posted by Rob Rich on January 29th, 2015 [ permalink ] Robo-fighter Ironkill has been out on iOS a | Read more »
Crossroad Zombies Review
Crossroad Zombies Review By Jordan Minor on January 29th, 2015 Our Rating: :: CROSSWALKING DEADiPad Only App - Designed for the iPad Crossroad Zombies is a rough draft of a cool genre mash-up.   | Read more »
Blood Brothers 2 – Tips, Cheats, and Str...
War is hell: Is it the kind of hell you want to check out? Read our Blood Brothers 2 review to find out! Blood Brothers 2, DeNA’s follow-up to the original Blood Brothers, is an intriguing card collecting / role-playing / strategy hybrid. There’s... | Read more »
Blood Brothers 2 Review
Blood Brothers 2 Review By Nadia Oxford on January 29th, 2015 Our Rating: :: AN AGGRAVATING RELATIVEUniversal App - Designed for iPhone and iPad Blood Brothers 2 is built on a simple, solid foundation, but its free-to-play system... | Read more »
I AM BREAD, the Toast of the Town, is Ro...
Have you ever dreamt of being deliciously gluten-y? Do you feel passionate about Rye and Wheat? The guys at Bossa Studios do and that is why they are bringing I AM BREAD to iOS soon. The loafy app will feature all the new content that is being... | Read more »

Price Scanner via MacPrices.net

Intel Aims to Transform Workplace With 5th-Ge...
Intel Corporation today announced the availability of its 5th generation Intel Core vPro processor family that provides cutting-edge features to enable a new and rapidly shifting workplace. To meet... Read more
iOS App Sharalike Introduces New Instant Smar...
Sharalike slideshow and photo management software for iOS, is making it easier than ever to create shareable meaningful moments with its new instant SmartShow technology. Staying organized is a goal... Read more
Apple Becomes World’s Largest Smartphone Vend...
According to the latest research data from Strategy Analytics, as global smartphone shipments grew 31 percent annually to reach a record 380 million units in the fourth quarter of 2014. Apple became... Read more
Cut the Cord: OtterBox Resurgence Power Case...
Dead batteries and broken phones are two of the biggest issues for smartphone users today. Otterbox addresses both with the new Resurgence Power Case for Apple iPhone 6, promising to make those panic... Read more
13-inch Retina MacBook Pros on sale for up to...
B&H Photo has 13″ Retina MacBook Pros on sale for $200 off MSRP. Shipping is free, and B&H charges NY sales tax only: - 13″ 2.6GHz/128GB Retina MacBook Pro: $1199.99 save $100 - 13″ 2.6GHz/... Read more
15-inch 2.5GHz Retina MacBook Pro on sale for...
 B&H Photo has the 15″ 2.5GHz Retina MacBook Pro on sale for $2319.99 including free shipping plus NY sales tax only. Their price is $180 off MSRP, and it’s the lowest price available for this... Read more
Back in stock: Refurbished iPod nanos for $99...
The Apple Store has Apple Certified Refurbished 16GB iPod nanos available for $99 including free shipping and Apple’s standard one-year warranty. That’s $50 off the cost of new nanos. Most colors are... Read more
Apple lowers price on refurbished 256GB MacBo...
The Apple Store has lowered prices on Apple Certified Refurbished 2014 MacBook Airs with 256GB SSDs, now available for up to $200 off the cost of new models. An Apple one-year warranty is included... Read more
New Good Management Suite Simplifies Enterpri...
Good Technology has announced the availability of the Good Management Suite, a comprehensive cross-platform solution for organizations getting started with mobile business initiatives. Built on the... Read more
15-inch 2.0GHz Retina MacBook Pro (refurbishe...
The Apple Store has Apple Certified Refurbished previous-generation 15″ 2.0GHz Retina MacBook Pros available for $1489 including free shipping plus Apple’s standard one-year warranty. Their price is... Read more

Jobs Board

At-Home Chat Specialist- *Apple* Online Stor...
**Job Summary** At Apple , we believe in hard work, a fun environment, and the kind of creativity and innovation that only comes about when talented people from diverse Read more
Sr. Mac Expert- *Apple* Online Store - Apple...
**Job Summary** The World Wide Apple Online Store (AOS) Sales and Service team is looking for motivated, outgoing, and tech savvy individuals who want to offer Apple Read more
*Apple* Solutions Consultant- Retail Sales (...
**Job Summary** The ASC is an Apple employee who serves as an Apple brand ambassador and influencer in a Reseller's store. The ASC's role is to grow Apple Read more
Event Director, *Apple* Retail Marketing -...
…This senior level position is responsible for leading and imagining the Apple Retail Team's global engagement strategy and team. Delivering an overarching brand Read more
At-Home Chat Specialist- *Apple* Online Stor...
**Job Summary** At Apple , we believe in hard work, a fun environment, and the kind of creativity and innovation that only comes about when talented people from diverse Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.