Research on enterprise AI – June 2023

Every day I scan through around 200-300 additions to the arXiv preprint database looking for research  that has implications for  the assessment, adoption and governance of AI in the enterprise. I skip over technical developments on LLMs and research into the performance of ChatGPT et al. The benefit of the arXiv service is that it facilitates the early publication of research, The fact that it is open source means that IT managers and developers without access to the academic journal services can download the research. The downside is that sometimes subsequent peer review prior to publication results in modifications to the pre-print version.

This is the first of what I intend to be a monthly synopsis of a selection of the outcomes of my scanning. I have not tried to summarise or critique the research papers as a good abstract is only a click away. The objective is to give you a curated list of open-source research outcomes that in my opinion you should at least be aware of and perhaps download and circulate to colleagues even if you yourself do not have time to read through them.

Forgotten Knowledge: Examining the Citational Amnesia in NLP

ChatGPT is a Remarkable Tool—For Experts

When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset of 53,000+ Legal Holdings

The Two Word Test: A Semantic Benchmark for Large Language Models

AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

Lost in Translation: Large Language Models in Non-English Content Analysis

The Ethics of Algorithms: Key Problems and Solutions

Inverse Scaling: When Bigger Isn’t Better

Friend or Foe? Exploring the Implications of Large Language Models on the Science System

TRUSTGPT: A Benchmark for Trustworthy and Responsible Large Language Models

An Overview of Catastrophic AI Risks

Testing of Detection Tools for AI-Generated Text

Martin White

20 July 2023