The Nielsen Norman Group Intranet Design Annual is a unique insight into a small number of intranets. It is such a high-profile publication that I feel it deserves two blog posts. This post covers the methodology and top-level analysis and the second post covers the intranets that appear in the 2013 report. A detailed critique of the methodology of the 2012 report and also of the current report has been published by the Intranetizen team.
It is important to understand the core methodology. The NNGroup invite companies to submit their intranets for assessment and select what it feels are the best 10 intranets in terms of design to form the basis of the report. The subtitle of “The Year’s 10 Best Intranets” is both true and also misleading. It is not even necessarily the 10 Best Intranets of those submitted because there could be problems gaining corporate approval to not only release the screen shots but also the detailed descriptions of how the intranets were built and maintained.
Now I must declare an interest here, as for several years I was a presenter on the Usability Week training courses in the USA and Europe. When you have worked with the NNGroup team you begin to appreciate the professionalism that goes into all their research and events, and the massive effort that goes into the reports. It’s just the same with the Step Two Intranet Innovation Awards. For neither company is the effort put in to the evaluation and report publication is in no way offset by the sales revenues.
That does not mean to say that this report could not be improved. The statistical analysis of team size does not match up with analysis in the Intranet Usability Guidelines, where based on a much larger sample 42% have 2-5 employees. Worryingly 32% did not have enough people to do the work that needed to be done. NNGroup may have no idea how valuable this team size information is in making a business case for resources. You may be able to identify usability issues from 5 tests but you can’t do trend analysis on 10 intranets. My version had no bookmarks but it was a review copy.
To me the value of this report is not whether every intranet meets the ultimate design benchmarks but the behind-the-scene stories of how the intranets were developed and are now managed, what governance structures are in place and how is user satisfaction being assessed. In the average intranet conference paper all this might be summarised in a single slide. These stories need to be told to encourage and warn the intranet community.
Martin White