Two recent conferences neatly sum up the current situation in search innovation. Let me start with the Haystack event conceived and organized by Eric Pugh and Doug Turnbull of OpenSource Connections (OSC) in Charlottesville, South Carolina. This was a community meeting of practitioners and attracted around 100 delegates from around the world. In the words of the organisers “Haystack is the no-holds-barred conference for organizations where search, matching, and relevance really matters to the bottom line. For search managers, developers & data scientists finding ways to innovate, see past the silver bullets, and share what actually has worked well for their unique problems. Please come share and learn!”

Looking at the tweets and the blog posts from Charlie Hull and Sujit Pal the range of topics was very wide indeed and yet also very focused on delivering a significantly better search experience to external and internal users. To me the conference illustrated just how much development is taking place in achieving better search relevance. Although many of the papers were specifically about open source search code development the principles and algorithms could be applied to any search application. To quote from Charlie Hull “I think this conference may also have signalled the birth of a new profession of ‘relevance engineer’ – someone who can understand both the business and technical aspects of search relevance, work with a variety of underlying search engines and expertly use the correct tools for the job to drive a continuing process of search quality improvement.”

To switch from the USA to Australia, in February 2018 the third SWIRL conference took place in Lorne, near to Melbourne. The objective of the meeting was “to explore the long-range issues of the Information Retrieval field, to recognise challenges that are on — or even over — the horizon, to build consensus on some of the key challenges, and to disseminate the resulting information to the research community. A total of 60 information retrieval academic researchers came together in a well-structured set of workshops and plenary discussions. A draft report on the conference (download) has now been published and run to 57 pages of text. From an IR research perspective I found much of value in the outcomes of the conference. To quote from the Conclusion “Information Retrieval remains a vital and active area of research in both academia and industry. Satisfying people’s information needs is a fundamental, multi-disciplinary problem, and this report captures the many important research themes in this important research area.”

Haystack was full of practitioners and SWIRL was full of academics, and therein lies a fundamental and challenging problem. The two groups have no common platform (sadly least of all ACM SIGIR) where ideas and opportunities can be exchanged. This disconnect was a core issue in the conference that was held in Brussels in 2012 to consider the report that I wrote with Stavri Nikolov on the state of enterprise search in the EU. David Hawking spoken at length about the lack of primary research in enterprise search, and nothing has changed since that time. The common excuse is that firms are not willing to disclose confidential information, but that does not seem to have been a problem in the case of the many hundreds of papers that have been published in ERP implementation issues. It was also noted that despite many institutions involved in IR research there was no Masters in Enterprise Search available anywhere in the EU.

I agree with Charlie Hull that there is a substantial opportunity for ‘relevance engineers’ who can bring the best of IR research and search implementation together. It happens in a very limited way at the Search Solutions event run by the BCS IRSG in London in November each year and with the Industry Day at the European Conference on Information Retrieval. There is also an Industry Day at SIGIR 2018 but to say that  the topic list is ‘ad hoc’ is an understatement. Without an extended conference that is designed from first principles to bring together research, education (the Information Schools) and practice the evolution of search inside and outside the corporate firewall is never going to have the impact it needs to have in supporting innovation, productivity and business growth.

Martin White