A team of researchers has developed AI-powered virtual robots that learn to understand language more efficiently when they are rewarded for exploring. The work offers a way to examine how curiosity might help children learn language.
Researchers at the Okinawa Institute of Science and Technology (OIST) found that robots trained to explore and satisfy their curiosity mastered language-based tasks in about half the time of conventionally trained robots. The robots also displayed spontaneous play-like behavior and handled unfamiliar situations better, suggesting that curiosity-driven training could make AI systems more adaptable.
"We were amazed by the play-like behavior and exception-handling performance that emerged independently during training," said Theodore Tinker, the study's first author and a PhD student in the Cognitive Neurorobotics Research Unit at OIST.
Rewarding exploration
Large language models (LLMs) learn patterns from large datasets. To generate a response, they use the text so far to predict what comes next. The researchers instead used a brain-inspired architecture known as a Predictive Coding-inspired Variational Recurrent Neural Network (PV-RNN). The model is designed to minimize a quantity called "free energy" by balancing two competing goals: improving prediction accuracy while minimizing changes to its internal beliefs. In other words, the robot aims to correctly understand the world while maintaining a stable internal model of it.
To encourage exploration, the team combined the PV-RNN with reinforcement learning. Robots received an external reward for completing language-guided tasks and an internal reward for satisfying curiosity by exploring unfamiliar situations that required updating their internal understanding. This balance between stability and exploration enabled the robots to discover new solutions to complex tasks more efficiently.
During training, the curious robots displayed unexpected play-like behavior. Even after mastering assigned tasks, they continued experimenting by knocking over objects, despite receiving no direct instruction to do so. The researchers found that this spontaneous exploration accelerated the robots' language learning. The result offers a way to explore how play might help young children learn.
"The robots are not thinking like children, and their situation is very abstract compared to the extremely dense language environments of children. But this abstract environment allows us to study the impact of curiosity within a very convincing model of how humans learn to process language," said Jun Tani, study senior author and head of the OIST unit, in a statement.
Learning new rules and exceptions
The study also sheds light on a long-standing question in linguistics known as Noam Chomsky's "Poverty of the Stimulus" problem, which asks how children rapidly learn language despite receiving incomplete and imperfect input. In the experiment, the benefits of curiosity also depended on how varied the training language was. Robots exposed to only 48 language combinations handled previously unseen instructions successfully about 25 percent of the time. Those trained with 180 combinations reached around 85 percent.
Another surprising finding was that the robots exhibited a learning pattern similar to toddlers' U-shaped learning curve. To test this, researchers deliberately reversed the meanings of two language tasks. Initially, the robots learned the exceptions correctly, but as they generalized broader language rules, their performance temporarily declined before recovering as they mastered the exceptions. This mirrors how children often say "goed" or "runned" before eventually learning irregular verbs correctly.
According to the researchers, the emergence of this behavior was entirely unexpected, as the AI model contained no explicit mechanism for handling exceptions or overgeneralization.
Beyond improving robotic language learning, the transparent design of the PV-RNN allows scientists to observe the AI's internal decision-making process in real time. With this model, researchers can directly examine how the robot updates its beliefs, something that is harder to do with massive black-box AI systems. This transparency makes the model a valuable platform for studying both human cognition and the mechanisms underlying language acquisition.