I have written quite a detailed commentary on some of the research that has been carried out by Microsoft Research into defining, measuring and enhancing productivity. One of the problems with assessing productivity in knowledge work is that the number of items produced in a day is not necessarily a reflection of their value to the operational success of the organisation. In the current Covid situation there will be pressure on distributed employees to create as many documents as possible to show that they are ‘working hard’ even though they are not in the office. Whether they are also generating documents of information and knowledge value will only emerge in the future.

In the case of search applications in particular there has always been an unhelpful metric about the time spent searching which dates back to some comments made in a conference paper presented at the International Conference on the Social Impact of Information Technologies in St. Louis, Missouri, October 12–14, 1998. The study by Kit Sims Taylor found that knowledge workers spend more time unwittingly recreating existing knowledge than in creating new knowledge. This was a very small-scale survey and was the basis for the scenarios used in the 2001 IDC briefing paper on The High Cost of Not Finding Information. The outcomes of these scenarios have been used ever since by search vendors seeking to present a business case for their software. When they do ask them if they have actually read the 1998 paper!

I was therefore delighted to see a conference paper to the CHI (ACM Human Factors in Computing Systems) 2020 conference by Hayley Gillou and her colleagues at the University of British Columbia and the University of Zurich. In this paper the authors present the concept of Time Well Spent (TWS) as a holistic view of performance of a knowledge worker. (There are in fact two versions of the Gillou paper. The CHI conference paper can be downloaded from the University of Zurich but I would also advise you to read the MSc thesis of Hayley Gillou which at 130pp does not add to the outcomes of the CHI paper but does contain all the survey methodology.)

The concept dates back to a TEDx talk by Ted Harris in 2014 and some previous work undertaken by Young-Ho Kim and colleagues [pdf download]. They found six themes that characterize the productivity assessment—work product, time management, worker’s state, attitude toward work, impact & benefit, and compound task—and identified how participants interleaved multiple facets when assessing their productivity. This lies at the very core of the both the definition and measurement issues – ‘productivity’ has these multiple dimensions and all need to be addressed in arriving at solutions.

The Gillou study looks at the challenges of measuring Time Well Spent and reports on a small-scale investigation. The authors are careful not to overstate either the value of TWS or the scalability of the technique. In my view this is an approach that is well worth considering not just as as a potential research opportunity but as the basis for internal discussions around prodcutivity measures. The concept links well to search satisfaction, and how stopping distances for search results examination are a trade-off between value and effort. The effort is not just a time parameter but of the intellectual element of constructing and revising the query and working through perhaps poor-quality snippets of document results.

Martin White