Charlie Hull has posted an excellent analysis of the difficulty in finding experienced search specialists to support search development. Sadly the challenges he outlines are only one half of the problem. Back in 2012 I was asked by the European Commission to identify barriers to the development of a European market for enterprise search that would stimulate the demand for search software applications from EU-based companies. I identified that the major barrier was a lack of experienced search specialists not only with the skills to develop search applications but also with the skills to assess user requirements.

Since 2012 nothing has changed. With the praiseworthy exception of OSC there are no training courses for enterprise search managers and not a single Information School offers enterprise search as a Masters course. I suspect that the reason for this omission is that academics have little or no direct contact with organisations that need to have effective search. As a result even if they specify a course they have no one to teach it. This lack of academic focus on enterprise search also means that the amount of research carried out in an enterprise setting remains at around 0.1% of the total number of information retrieval/search papers.

Paul Cleverley has made the point that there is a need for a systems approach in enterprise search deployment, which I would totally support. In my view this systems approach has to take account of the reality of enterprise information management, the subject of my presentation to the Microsoft Research New Future of Work conference in early August and my paper published recently in Business Information Review.

It is also the case that over the same period of time a number of surveys have shown that the levels of satisfaction are terrifyingly low, with typically around 40% of organisations reporting that their employees are dissatisfied with their enterprise search applications. If this was the case with any other enterprise application the CIO would have been moved out but it seems that enterprise search is just not on anyone’s radar. The concomitant risks are huge.

It does not help that search vendors are intent on selling solely on technical sophistication with the implied view that the AI/ML underpinnings of their technology render as superfluous any requirement to define user requirements or to establish a search support team. Nothing could be farther from reality. To quote from Paul Cleverley’s paper (above)

“Every interview with a [major law firm] partner started with ‘hope you are going to fix search because search is rubbish, we need a better search engine’, but they had probably one of the best technically competent search engines around.”

I have had exactly the same experience probably three or four times a year for the last decade.

When a business application fails to support a business process immediate action is taken. In the case of search there are no specific business processes. Every user and every search that they make will be initiated by a different information requirement and process. One of the reasons I offer the SearchCheck service is to be able to provide a benchmark score in the space of two hours. My guess is that when you take the test the results will be well below your expectations, and that information could be exactly what is needed to establish a coherent approach to delivering search across your organisation.

Martin White