ChatGPT, an AI that speaks very well… but for what?

ChatGPT takes center stage since its release on November 30, because of its amazing features, especially for talk and answer questionseven complex, in a natural and realistic way.

As we begin to have a little perspective on this tool, questions arise: what are the current and future limits of ChatGPT, and what are the potential markets for this type of system?

ChatGPT, a “Google killer”? Not necessarily…

ChatGPT is often described as a future competitor of Google, even as a “Google killer” for its search engine part: even if the tool sometimes produces bizarre answers, even downright false, it answers in a direct way and does not simply offer an ordered list of documents, like Google’s search engine.

There is certainly a serious potential danger for Google, which could threaten its position of virtual monopoly on search engines. Microsoft in particular (main investor in OpenAI, which also has privileged access to the technology developed) is working to integrate ChatGPT with its Bing search enginehoping to regain the edge over Google.

However, there are several uncertainties surrounding such a prospect. Search engine queries are usually made up of a few words, or even a single word, such as an event or a personality name. ChatGPT is currently arousing the curiosity of a technophile population, but this is very different from the traditional, general public use of a search engine.

We can also imagine ChatGPT accessible through a voice interface, which would avoid having to type the request. But the systems like Amazon’s Alexa have struggled to catch on, and remain confined to specific and limited uses (ask for movie times, the weather, etc.). 10 years ago, Alexa was seen as the future of the American distribution company, but today is a bit abandoned, becauseAmazon has never managed to monetize its toolthat is to say, to make it economically profitable.

Can ChatGPT succeed where Alexa partly failed?

Other frameworks of use?

Of course, the future of ChatGPT shouldn’t be all about finding information. There are a host of other situations where you need to produce text: production of standard letters, summaries, advertising texts…

ChatGPT is also a good writing aid. We already see different uses: requesting ChatGPT to start with a few paragraphs that can inspire and avoid fear of blank page ; see what points the tool puts forward on a particular question (to check if it corresponds to what we would have said ourselves or not); ask for plan suggestions on a particular issue. ChatGPT is not a magic tool and cannot know what the user has in mind, so when faced with writing a complex document, it can only be a help.

We can obviously imagine more problematic uses and many articles have already been published in the press concerning, for example, theuse of ChatGPT in education, with fears, justified or not. We can thus imagine students producing homework thanks to ChatGPT, but also teachers using the tool to write their assessments, or researchers producing scientific articles semi-automatically. There are plenty of stories about students in the press, but they won’t be the only ones making potentially problematic use of this kind of technology.

Of course there is reason to ask questions, but the technology is there and not going away. It therefore seems essential to talk about them, and to train pupils and students in these tools, to explain their interest and their limits, and to discuss the place they should have in the training.

Finally, at the extreme end of the spectrum of problematic uses, we will obviously think of the production of fake news : false information that can then be disseminated in industrial quantities.

These dangers should not be exaggerated, but they are real. Even if Text finders produced by ChatGPT begin to appearthese will necessarily be imperfect, because the texts produced are too diverse and too realistic to be 100% recognized by a system… except by the OpenAI company itself, of course!

The limits of ChatGPT: when the AI ​​”hallucinates”

The mass of interactions with ChatGPT since it opened to the general public on November 30 has already identified some of its limitations.

ChatGPT generally provides correct answers, often bluffing… but if you ask him about areas that he does not master, or even if you invent a question that appears serious but is in fact absurd (for example on facts or people who don’t exist), the system produces a seemingly equally serious response, but is in fact completely absurd or made up.

Examples on Twitter are legion: ChatGPT offers scientific references that do not exist, hazy explanations, even a demonstration where it is postulated that -4 = -5. This would be a treasure, if ChatGPT was just a tool intended to produce stories, pastiches or parodies.

But what the public expects is above all proven answers to real questions, or the absence of an answer in the contrary case (if the system cannot find the answer, or if the question is absurd). This is the main weakness of the tool, and therefore probably also the main obstacle to making it a competitor to Google’s search engine, as we have already seen.

For this reason, a conference like ICML (International Conference on Machine Learning) has already prohibited researchers from submit articles produced in part with ChatGPT. stackoverflowa platform for exchanges between IT developers, has also disallow responses generated by ChatGPTafraid of being overwhelmed by a flood of automatically generated (and partly false) answers.

This is because the system has no “world model”. In other words, he does not know what is true, he can generate nonsense, false information, invent things from scratch with the aplomb of a professional liar. This is called “hallucinations”, as if ChatGPT then sees imaginary elements (in fact, you can’t really say that the system is lying, since it has no model of truth ).

This is especially true when the question itself is not turned towards reality, in which case the system begins to invent: in this sense, GPT is neither a journalist nor a scientist, but rather a storyteller. stories.

It’s a safe bet that OpenAI will try in future versions to provide a system that avoids fabricating when the context does not lend itself to it, thanks to a detailed analysis of the question asked, or the addition of validated knowledge. (as Amazon already does with Alexa or Google with sound knowledge graphwhich is simply a database).

Google, precisely, through its branch Deepmind, is currently working on a model similar to ChatGPT called Sparrow, trying to strengthen the reliability of the system. For example, it is a question of the system providing a source list on which it relies to provide an answer.

Challenges for tomorrow

The other limit of this system is that it is based on data (roughly, all the texts available on the Internet) in mid-2021 and that his knowledge cannot be updated live. This is obviously a problem, ChatGPT cannot respond relevantly to questions about current events, while this is a particularly important aspect.

The continuous updating of the model is therefore logically one of the next goals of OpenAI, which makes no secret of it. Revising a model, retraining it “from scratch” (from scratch) is a time-consuming and expensive process, which can involve thousands of GPUs or TPUs for several weeks or several months, which is not in phase with the speed of the news. The next big innovation will therefore consist of systems capable of updating in a more localized way in real time (or almost), and this is probably soon.

But the main issue is obviously that of acceptability. As we have seen, the debate has already begun on the influence of such a system on education. More generally, if a system such as ChatGPT is for example integrated with software such as Word, the question will also arise of who controls what is produced. There is a narrow way between poorly controlled AI systems and capable of producing racist or homophobic contentand overly restricted systems that would prohibit the production of certain content.

In conclusion, and as the popular saying goes: it is difficult to make predictions, especially when they concern the future. There are many unknowns around ChatGPT-like technologies: the prospects of such tools are quite dizzying, likely to have a profound impact on society, but at the same time their real and commercial potential will have to pass the test of the real world. .

What is certain is that the current upheavals should encourage the development of institutes (within universities, but also through foundations or associations capable of reaching the general public) allowing a broad and open reflection on these technologies, involving all actors in society, because it is society as a whole that is already impacted, as evidenced by the current interest around ChatGPT.


Through Thierry PoibeauDR CNRS, Ecole Normale Supérieure (ENS) – PSL

The original version of this article was published on The Conversation.