Meta’s artificial intelligence boss said the large language models that power generative AI products like ChatGPT will never achieve the ability to think and plan like humans, as he instead focuses on a radically alternative approach to creating “superintelligence” in Machines focused.
Yann LeCun, chief AI scientist at the social media giant that owns Facebook and Instagram, said LLMs have “a very limited understanding of logic.” . . They do not understand the physical world, have no permanent memory, cannot define the concept rationally, and cannot plan. . . hierarchical”.
In an interview with the Financial Times, he argued against relying on the advancement of LLMs in the search for human intelligence, as these models can only respond correctly to prompts if they have been fed the right training data, and therefore are “inherently unsafe”.
Instead, he is working to develop an entirely new generation of AI systems that he hopes will power machines with human intelligence, although he said it could take 10 years for that vision to come to fruition.
Meta has poured billions of dollars into developing its own LLMs as generative AI has exploded, aiming to catch up with rival tech giants including Microsoft-backed OpenAI and Alphabet’s Google.
LeCun leads a team of about 500 people in Meta’s Fundamental AI Research (Fair) lab. They are working to create an AI that can develop common sense and learn how the world works in a similar way to humans, in an approach known as “world modeling.”
The Meta AI boss’s experimental vision is a potentially risky and costly venture for the social media group at a time when investors are hoping for quick returns on AI investments.
Last month, Meta lost nearly $200 billion in value as Chief Executive Mark Zuckerberg vowed to increase spending and make the social media group the “leading AI company in the world.” In doing so, he spooked Wall Street investors who were concerned about rising costs and little immediate revenue potential.
“We’re at the point where we believe we’re on the cusp of perhaps the next generation of AI systems,” LeCun said.
LeCun’s comments come as Meta and its competitors continue to push improved LLMs. Figures like OpenAI boss Sam Altman believe they represent a crucial step toward creating artificial general intelligence (AGI) – the point at which machines have greater cognitive abilities than humans.
OpenAI released its new, faster GPT-4o model last week, and Google unveiled a new “multimodal” artificial intelligence agent that can answer real-time queries across video, audio and text, called Project Astra, based on an updated version of its Gemini model based.
Meta also launched its new model, Llama 3, last month. The company’s head of global affairs, Sir Nick Clegg, said its latest LLM had “significantly improved capabilities such as reasoning” – the ability to apply logic to queries. For example, the system would suspect that a person suffering from a headache, sore throat and runny nose has a cold, but could also recognise that allergies could be causing the symptoms.
However, LeCun said this development of LLMs is superficial and limited because the models only learn when human engineers intervene to train them on that information, rather than the AI coming to a conclusion organically like humans do.
“It certainly seems like logical thinking to most people – but most of the time it uses accumulated knowledge from a lot of training data,” LeCun said, but added: “[LLMs] are very useful despite their limitations.”
Google DeepMind has also spent several years pursuing alternative methods of building AGI, including methods such as reinforcement learning, in which AI agents learn from their surroundings in a game-like virtual environment.
At an event in London on Tuesday, DeepMind chief Sir Demis Hassabis said the language models lacked: “They didn’t understand the spatial context you’re in.” . That ultimately limits their usefulness.”
Meta set up its Fair Lab in 2013 to pioneer AI research and hired leading scientists in the field.
However, in early 2023, Meta formed a new GenAI team led by Chief Product Officer Chris Cox. It poached many AI researchers and engineers from Fair, led work on Llama 3, and integrated it into products such as its new AI assistants and image generation tools.
The creation of the GenAI team came as some insiders argued that an academic culture within the Fair lab was partly responsible for Meta’s late entry into the generative AI boom. Zuckerberg has pushed for more commercial applications of AI under pressure from investors.
However, according to people close to the company, LeCun remains one of Zuckerberg’s top advisors, as he is known as one of the founding fathers of AI and won a Turing Award for his work on neural networks.
“We refocused Fair on the longer-term goal of human-level AI, essentially because GenAI is now focused on the things we have a clear path to,” LeCun said.
“[Achieving AGI] “It’s not a product design problem, it’s not even a technology development problem, it’s more of a scientific problem,” he added.
LeCun first published a paper on his world modeling vision in 2022, and Meta has since published two research models based on this approach.
Today, he said Fair is testing different ideas to achieve human-level intelligence because “there’s a lot of uncertainty and exploration here. [so] We can’t say who will be successful or who will end up being picked up.”
Among other things, LeCun’s team feeds systems hours of video and deliberately omits frames to then get the AI to predict what will happen next. This is intended to emulate how children learn by passively observing the world around them.
He also said Fair is looking into building a “universal text coding system” that would allow a system to process abstract representations of knowledge in texts that can then be applied to video and audio.
Some experts doubt that LeCun’s vision is feasible.
Aron Culotta, an associate professor of computer science at Tulane University, said common sense has long been a “thorn in the side” of AI and that teaching models causality is challenging, making them “vulnerable to these unexpected errors.”
A former Meta AI employee described the world modeling initiative as “vague gobbledygook,” adding, “It feels like a lot of flag-raising.”
Another current employee said Fair has yet to emerge as a real competitor to research groups like DeepMind.
Longer term, LeCun believes the technology will power AI agents that users can interact with through wearable technologies such as augmented reality or “smart” glasses and electromyography (EMG) “bracelets.”
“[For AI agents] To be truly useful, they must have human-level intelligence,” he said.
Additional reporting by Madhumita Murgia in London