Skepticism about Large Language Models (LLM) and ChatGPT

Abstract

Top AI scientist Yann LeCun is skeptical about Large Language Models (LLM) being the right path to achieving Artificial general intelligence (AGI) and believes they have serious limitations.

Large Language Models (LLM) will never be really smart

Top AI scientist, Yann LeCun, Chief AI Scientist at Meta, said the large language models (LLM) that power generative AI products such as ChatGPT would never achieve the ability to reason and plan like humans.

Who is Yann LeCun?

Yann LeCun is a French-American computer scientist working primarily in AI-related fields. He was a New York University (NYU) professor of Computer Science and Neural Science. He won a Turing Award for his work on neural networks and is considered to be one of the world's four largest artificial intelligence scholars. He joined Meta in 2013 and is currently the Chief AI Scientist at Meta.

What is Yann LeCun saying about LLMs?

Yann LeCun is very skeptical about LLMs. His views are,

• LLMs would never achieve the ability to reason and plan like humans.
• LLMs have a very limited understanding of logic.
• LLMs do not have persistent memory.
• LLMs can only answer prompts accurately if they have been fed the right training data and are, therefore, “intrinsically unsafe”.
• LLMs do not understand the physical world.
• The evolution of LLMs is superficial and limited, with the models learning only when human engineers intervene to train it on that information, rather than AI coming to a conclusion organically like people.
• LLMs appear to most people as reasoning — but mostly it’s exploiting accumulated knowledge from lots of training data.
• LLMs are very useful despite their limitations.
• LLMs cannot develop into superintelligence because of a lack of understanding of the physical world.
• LLMs can't plan hierarchically.
• LLMs are not smarter than a house cat.

What is Yann LeCun offering as an alternative?

Yann LeCun runs a team of about 500 staff at Meta’s Fundamental AI Research (Fair) lab. Here is his approach to AI:

• he is working to develop an entirely new generation of AI systems
• could take 10 years to achieve
• working towards creating AI that can develop common sense
• approach known as “world modeling”
• focused Fair Lab toward the longer-term goal of human-level AI
• Achieving AGI is a scientific problem
• Fair Lab is exploring building “a universal text encoding system” that would allow a system to process abstract representations of knowledge
• focusing on developing radical ways to give machines "superintelligence"

Academic research in AI is a nice topic, but big money is in question

While Yann LeCun is surely right when saying that LLMs have serious limitations, others are making millions of dollars selling AI services of, admittedly, imperfect LLM systems. And the general public is definitely impressed and interested in those services and has no problem calling them AI systems, however imperfect they might be.

Yann LeCun is definitely right in his arguments and anyone who has been using ChatGPT for a while soon becomes aware of the limitations of that system and would agree with the presented arguments. He worded his arguments very nicely, and most users just felt “something is wrong with ChatGPT”, and now we have a scientist explaining to us what the problem is.

But on the other hand, the ChatGPT-4o system is at times showing “elements of intelligent behavior”, from fluency in 50 natural languages to writing elaborate texts/essays/reports, at the level of an educated person with superhuman speed and efficiency. And, very importantly, ChatGPT services can be quite useful. ChatGPT has set the bar very high, and when other companies are going to put similar AI systems/products on the market, they will need to exceed it.