Reviewing the Clearbox Consulting report that profiles in considerable (30pp per vendor) detail intranet and employee experience applications has caused me to reflect on the lack of anything similar for the enterprise search and e-commerce search sectors. The Real Story Group used to publish search vendor profiles but the last of these was V4 in 2012. Most of the vendors have now disappeared, including Autonomy IDOL, Endeca, FAST, Vivisimo, IBM Omnifind, Exalead Cloud View, ISYS, Recommind and the Google GSA. There are of course the overviews by Gartner and Forrester but time and lawyers both cause me to hold fire on what I think of these reports! They only cover a minority of the 70+ vendors even though there are many very good solutions out there from small innovative companies that often fail the revenue test.
Having been involved in the periphery of the RSG reports, and also of the outstanding report published by Ovum in 2012 which set a very high standard of analysis and insight, I am well aware of the difficulty of profiling enterprise search applications. They are multi-component applications which are customized to meet a specific client requirement, and often include modules bought in from other software companies to address specific requirements, especially with regard to connector and language technology. During 2021 project work required me to undertake very deep dives into Algolia, Bloomreach, Coveo, Elastic, LucidWorks and to a lesser extent a dozen other vendors as I developed a short list of potential vendors for consideration. As the project proceeded it was interesting to note the differences between the claims in the RFI response (which is always a sales pitch) and the RFP response (which will form the basis for a contract) where there was usually a lot more detail but paradoxically often less clarity!
Client success stories are also very absent from both marketing pitches and proposals. I was struck this morning by a LucidWorks claim that its software could generate improved search performance in a matter of minutes, despite an RFP response they provided in the above project in which the implementation schedule was in weeks, with a commensurate professional services cost. Vendors often complain to me about how long it takes potential customers to move ahead with serious discussions when the core reason for the delay is the lack of information provided by vendors that would assist the prospective customers to make a sound business case.
In every search project I have been involved with, the reticence of the vendor to disclose any pricing information is very frustrating when a client needs to have some sense of the potential budget for the project. Two of the vendors mentioned above refused, even at a close-to-contract level, to send pricing information to anyone other than the project director and the CFO. As I remarked recently pricing models for search, especially SaaS, are becoming very complicated.
Far more challenging is any attempt to dig deep into exactly how machine learning is going to be applied in the project. It would seem that the entire search industry is unaware (or scared) of the Explainable AI movement. There is never a reference to the requirements for building a suite of test collections using the levels of search traffic that are common in enterprise search applications. E-commerce is marginally less challenging.
I can’t see any immediate prospect of a consulting business offering search vendor profiles. What would be interesting would be to see a vendor (or two) breaking cover and being very open about capabilities, caveats, project schedules, and above all transparency on ML-related issues. I doubt their pre-sales team could cope with the avalanche.
Martin White