Understanding Search Engines – Dirk Lewandowski

It is remarkable that this book is unique in its coverage of the development of the technology, business and impact of web search. The web search engines play such an important role in our lives and our business activities that we take them for granted. It’s not just a lack of books but a lack of research papers as well, other than those that look at elements of the search process. Last year I published a history of enterprise search and trying to confirm dates, vendors and technical developments was a far from easy exercise. The development of online retrieval services was expertly documented by Charles Bourne and Trudi Bellardo Hahn in 2002 and Stephen Arnold’s interviews with the pioneers of online and web search give a sense of the pace of change from 2008 -2012 but the focus is primarily on enterprise search.

The arrival of this English edition of a book originally published in German by an author with an outstanding reputation in the science and the use of web search is very timely given the launch of so many AI-Generated Content (AIGC) applications, all of them offering search-by-prompt rather than search-by-query. Dirk Lewandowski is a Professor of Department of Information, Hamburg University of Applied Sciences and from 2013-2020 was Editor-in-Chief of the Aslib Journal of Information Management.

In this book he considers five aspects of search, namely the technology, the user (and how user data is used to improve relevance), the process of web-based search, the impact of search on the economy and the impact on society, especially around issues relating to the level of trust that users can place in the results.

What immediately struck me about the book was the very wide scope presented in a very well-conceived structure. This structure is maintained in each of the sixteen chapters. Each chapter starts with a scope note, followed by text that strikes a balance between educating the reader and yet not preaching to them.  At the end of each chapter there is a summary and a bibliography, with in some cases suggestions for further reading on some of the topics covered in the chapter. It is this organisation that makes the book a pleasure to read. The references have clearly been chosen with care. The only potential omission I noticed was of the book by Langville and Meyer on the mathematics of PageRank, but that is a very tough read!

I’m not going to list out the contents of this 300pp book as these can be found on the Springer site. To highlight the breath of coverage there are chapters on Search Skills, Search Result Quality, The Deep Web, Search Engines between Bias and Neutrality and the Future of Search. The English translation is excellent and the images and references have been tailored (as much as it is reasonable to expect) to English language readers. There is a very comprehensive glossary running to ten pages. Given the very poor quality of the index it would have been good to link these terms (where possible) into the text of the book.

Whatever your role in search this book should be on either your physical or digital bookshelf. As I read it through I found myself saying “Ah, now I understand!” on a rather too frequent basis for someone who likes to think of themselves as being at least reasonably familiar with web search. It is written with authority and experience but also with a commitment to making often quite complex topics understandable to anyone who wants to get the best from the technology that is now on offer from a range of search vendors. The title says it all.

[This review originally appeared in Informer]

Martin White

28 April 2023