There is of course a range of text generation applications but for the purposes of this post I will use ChatGPT as a generic tag. I’m in the middle of writing a book on workarounds and am starting to see ChatGPT as a Workaround Machine. A recent online survey by Nature, the prestigious science journal, showed that these applications are being explored for a wide range of purposes, of which search is but one. The publishers of science journals are already starting to develop policies towards the use of LLMs and Nature has developed two principles
- No LLM tool will be accepted as a credited author on a research paper. That is because any attribution of authorship carries with it accountability for the work, and AI tools cannot take such responsibility.
- Researchers using LLM tools should document this use in the methods or acknowledgements sections. If a paper does not include these sections, the introduction or another appropriate section can be used to document the use of the LLM.
In my experience very few organisations have a corporate information management policy that focuses on information quality, and even fewer implement the policy with an appropriate level of governance. Whether you do or you don’t, now is the time to start to consider benefits and risks of accepting text generated by ChatGPT to be interpolated in text written by an employee or a team. Much is made of these applications as being good at providing starting points for a document but the danger is scope creep. The pressure on employees to improve their productivity is inevitably going to push them towards using these applications quite widely, perhaps (as one example) when they do not have English as their primary written language and need to create some boiler-plate text very rapidly.
One of the use cases that concerns me is document summarization. Early in my career I spent seven years writing around 10,000 abstracts of articles in metallurgical journals. I was trained not just to create ‘a summary’ but to convey to a reader anything that was of particular interest, even if it was a throwaway line in the paper. The famous 1953 paper by Watson and Crick on the structure of DNA contains just such a line “It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material”. Summaries of meetings are also going to be challenging, as so often it is not the words that are used but the way they are used and when there are telling silences.
The core question is who should own such a policy. Based on job titles you would think that the Chief Information Officer would be the ideal candidate but information quality and acceptable use are invariably absent from the role description. Certainly the risk management team should be involved to ensure that the risks associated with the occasional hallucination can be managed appropriately. One of the risks could be that as a result of using ChatGPT a decision is made which severely impacts the reputation of the organisation. If it comes down to a legal case are you willing to stand up in court and state that you used information from ChatGPT as the basis of your decision?
Martin White Principal Analyst 1 March 2023