In 1973 Horst Wittel and Melvin Webber authored a paper entitled ‘Dilemmas in a General Theory of Planning’ (Policy Sciences 4 (1973), 155-169). In this paper they set out the basis for what they regarded as ‘wicked problems’, which were beyond the capacity of traditional methods to resolve. In particular wicked problems cannot be addressed by a linear project management methodology because of the multi-dimensional nature of the problems that need to be resolved. Over the last few years a design thinking approach has been used with some success. Design thinking in management is a creative process, in which after gathering information (often through ethnographic techniques) the manager approaches problems through imagining possible solutions, rather than analysing the existing issue reductively. A key element in resolving wicked problems is that the leader’s role is in asking questions in order to help define the complexity of the problem facing the organisation and create conditions for ‘collective responsibility’ in addressing it, rather than the traditional expectation that they will offer a solution.

All too often I find that organisations are treating enterprise search as a project. At the end of the project the team is dispersed and gradually whatever quality was there at launch gradually fades away. The complexity of the workflow between the content being indexed and then found is rarely appreciated. If it doesn’t meet requirements then it must be the technology! In my experience that is very rarely the case.

I have created a table that looks at enterprise search as a ‘wicked problem’‘  Looking at the 16 elements of a wicked problem shows that traditional waterfall or even agile project approaches are totally unsuited to enterprise search applications. The requirement is to work as a team across multiple elements of an enterprise search implementation with a leader who has the experience to challenge and then work with a team to resolve an element. Even then there is a high probability that not all the elements can be resolved, which is why enterprise search applications need to be well supported with a search team post a nominal implementation. Earlier this week I was talking with Darron Chapman at CBResourcing, one of the most experienced recruitment consultants in the information and knowledge management sectors here in the UK. We agreed that the demand for experienced search managers was well in excess of supply and that salary requirements were very much on the high side. Organisations are now recognising that enterprise search is indeed a wicked problem and there are just not enough people around to solve all the problems. That raises another problem – where can people get a thorough training in enterprise search that is vendor-neutral and covers both commercial and open source applications?

Martin White