Did ChatGPT lie to Sky News’ Sam Coates? No.

On Saturday a friend shared this video with me:

I love that this was broadcast on Sky News for two reasons:

1. It reinforces that people should not trust the output of the ‘AI’ systems that we are using. Having such a stark example of this on a mainstream TV channel that will be picked up and shared by so many people is excellent.
2. It is a fantastic, citable example of a fundamental misread of what this technology is. ChatGPT generated false statements, but it didn’t lie. It cannot. It isn’t a being. It is a non-deterministic tool that generates what the probable text output is based on its training data, and the combination of the user’s prompt and the system prompt that was submitted to it.
   

My guess is that as Sam Coates has been uploading transcripts of his previous podcast episodes and asking questions about them over a period of time, the chat history and context led ChatGPT’s statistical generation process to produce what it calculated to be the most likely response to the prompt — one that included a made-up reference to today’s episode, despite it not having been uploaded yet. It was incorrect information, but it was a plausible response to his prompt of “what date transcripts do you have access to”. It wasn’t going back and searching a database of previously submitted transcripts. It just doesn’t work like that.

This isn’t to take away from the usefulness of ChatGPT and other tools like it. I use generative AI all the time in a variety of ways, such as to proofread my writing. But the way in which we anthropomorphise these computer programs and words we use to describe what they do — learn, know, think, reason, understand, lie — make us walk into the trap of assuming that there is somehow a being, a person, on the other side of the chat box. There isn’t. This trap is only going to get worse the better these systems get.