Search and ML/AI vendors are pushing hard to exploit the apparent requirement for expertise search. I use the word ‘apparent’ because there is very little evidence about the scale of the requirement. Senior management teams may think it is a great idea because it plays to the ethos of an organisation making the best use of its talents but I have yet to come across any organisation who has carried out user research to define requirements. The impression created by vendors is that they able to use a smart vacuum cleaner to gather in the sum total of corporate expertise and make it available to employees. If only it was that simple. If you want to get a sense of the mathematics behind expertise search download this pdf of a paper by Craig MacDonald and Iahd Ounis (both at the University of Glasgow) and gaze at the equations.

I’m in the process of writing a report on people and expertise search for publication next month which will examine the different ways in which expertise can be identified and then searched for. However it transpires that the models and algorithms used to create expert profiles are not the same as those needed to search for experts. As with any technology it is a very good idea to focus on user expectations when selecting an application. It is not difficult to present at least one expert in response to a query, and so in principle satisfy the requirement. In practice a user may wonder if this expert is the only expert, the most experienced expert or an artifact of a poor search algorithm. If this expert is always presented as a result of a query for a topic they may get annoyed at constantly having to respond to requests for help.

The user may well prefer to see a list of experts, preferably ranked in order of their expertise, so that they can make a trade off between (say) expertise and the location of the expert. This requires a ranking scheme that can rank expertise, which is a very considerable challenge. The user may also want to know what elements went into the selection of the experts to list, especially if the list does not include someone they already consider to be an expert. That is another very considerable challenge.

If you are considering investing in an expertise finding solution the first question you need to answer is how will you test that it works? To follow on from the paragraph above, will you expect to see one, some, or all of the organisation’s experts in a topic listed out, and will you expect a ranked list?  It is comparatively (and I stress ‘comparatively’) easy to undertake this with a collection of documents which already exists but for experts the application first has to generate the profiles. Do you just focus in on one expertise topic and trust that it scales or test on a range of topics?  If you are going out to tender for an expertise search application how will you compare the performance of each against an internal benchmark?

Another aspect to consider is how you deliver expertise search on a mobile device? My sense is that people need to find an expert especially quickly when they are on site with a customer, and expect delivery over their smartphone so that they can go immediately from profile to call. That of course assumes the person is still with the organisation. Ask your prospective vendor how easy it will be to remove an expert from the database and how quickly the person coming in to take their place will be displayed at an equivalent position on the experts list. If you are not able to answer some of questions posed in this post perhaps you need expert help.

Martin White