The Strange Engineering Problem of Teaching Robots Common Sense

A robot can be strong enough to lift a heavy box, fast enough to move around a warehouse and intelligent enough to understand a spoken command.

Then you put a glass of water in front of it.

Suddenly, things become much harder.

A human does not need a manual to understand that tipping the glass too far will spill the water. We instinctively know that a hot pan should not be grabbed with bare hands, that a chair is probably not a good place to put a plate of soup, and that an object partly hidden behind another object still exists.

Robots do not automatically know any of this.

That is the strange engineering problem researchers are now trying to solve: how do you give a machine something resembling common sense when the world itself is messy, unpredictable and constantly changing?

It is becoming one of the most important problems in robotics because the industry is moving beyond machines that perform one carefully controlled task.

For decades, industrial robots could be extremely good at repetitive jobs. Give a robot the same parts, the same workstation and the same instructions, and it can perform the operation thousands of times.

But move the part three centimetres.

Change the lighting.

Put another object in the way.

Give the robot a different instruction.

Suddenly, the problem becomes much harder.

Real life does not behave like a factory assembly line.

Homes are especially difficult. A kitchen might contain plates, food, towels, knives, electrical appliances, children, pets and dozens of objects that can be moved, hidden, dropped or damaged. Even something as simple as making a bed involves perception, movement, physical contact, sequencing and decisions about objects the robot may never have encountered before.

That is why today's robotics race increasingly looks less like traditional automation and more like an AI race.

Companies are building what are often called vision-language-action models, or VLAs. Instead of simply programming a robot to perform a fixed sequence, these systems connect visual information, language and physical actions.

The robot sees the environment.

The AI interprets what it sees.

A language instruction provides a goal.

The system then produces actions that move the robot toward that goal.

Google DeepMind's Gemini Robotics models, for example, are designed to connect multimodal reasoning with physical robot actions. Its newer Gemini Robotics 2 system is aimed at whole-body control, long-horizon planning and adapting to unfamiliar situations.

But understanding language is not the same thing as understanding physics.

A robot can know what the word "cup" means without knowing how much force is required to pick up a paper cup without crushing it.

It can understand "clean the table" without automatically knowing that a wet sponge should not be used to wipe an electrical outlet.

It can recognize a banana without knowing that throwing it at a wall will probably damage the fruit.

That gap between recognizing something and understanding what will happen when you interact with it is where physical common sense begins.

One approach researchers are exploring is the use of world models.

A world model attempts to give an AI system an internal representation of how its environment behaves. Instead of merely recognizing what is in front of it, the system can try to predict what will happen next.

If the robot pushes a cup, where will the cup move?

If it pulls a drawer, what objects might move with it?

If it picks up a towel, how will the flexible material behave?

If a person walks into its path, how should its movement change?

These are not ordinary image-recognition questions.

They are prediction problems.

NVIDIA's current research around world-action models describes systems that attempt to learn both how the physical world changes and how a robot's actions can influence those changes. The company's Cosmos platform is also being positioned around physical reasoning, simulation and synthetic data for robotics.

That could become a major shift.

A traditional robot might be trained to recognize a particular object.

A more general robot needs to understand what objects can do.

That distinction sounds subtle, but it changes everything.

Knowing that a chair is a chair is useful.

Knowing that a chair can be moved, sat on, pushed, blocked, tipped over or used as an obstacle is much more useful.

And teaching that knowledge is incredibly difficult.

The Internet gave modern AI an enormous amount of text and imagery to learn from. Robots have a much smaller training resource: the physical world.

There is no giant public database containing every possible way a human might fold a towel, open a cupboard, carry a glass, move around a sleeping dog or clean a kitchen.

Researchers therefore need enormous amounts of physical interaction data.

That has created an unusual new industry around collecting it.

Figure, one of the companies building general-purpose humanoid robots, recently introduced Index, a large-scale physical dataset built from human activity. The company says its system is designed to capture diverse human behavior because the data needed to generalize across physical environments cannot simply be downloaded from the Internet.

The importance of that data became particularly visible in September 2026 when Figure announced Helix 2.5.

The company tested the system in 30 homes it had not previously collected robot data from. It reported that the robots could perform tasks including tidying living rooms, folding towels and making beds without environment-specific fine-tuning or adaptation.

Figure reported zero-shot success rising from 9% to 56% when its large-scale human-behavior pretraining was used. Those are company-reported results rather than an independent verification, but they illustrate the direction the industry is pursuing: teach a robot broadly enough that it does not need to be manually programmed for every new room.

That is a radically different philosophy from traditional robotics.

Instead of saying, "Here is exactly what the robot should do," engineers are increasingly trying to give the machine enough learned knowledge to figure out what should happen.

But there is a catch.

Common sense includes knowing when you are wrong.

A human entering an unfamiliar kitchen can make a reasonable guess about what a strange object does, but can also stop and investigate if something seems dangerous.

A robot needs some equivalent of uncertainty.

It must know when it has enough information to act and when it should wait.

This is particularly important because mistakes in a chatbot are usually annoying.

Mistakes by a physical machine can be expensive or dangerous.

A language model producing the wrong answer might waste someone's time.

A robot misunderstanding a heavy object could drop it on someone's foot.

A robot misjudging a person's movement could cause a collision.

A robot grabbing a hot object could damage itself.

That is why robotics companies are increasingly treating safety as part of the intelligence problem rather than something that can simply be added afterward.

Google DeepMind's current robotics safety framework, for example, separates semantic, physical and operational safeguards. The company explicitly identifies everyday "common sense" situations—such as not handing a boiling drink to a young child or giving a very heavy box to a person—as things robotics systems need to handle safely.

There is another difficult problem hiding underneath all of this: the physical world is continuous.

A computer likes discrete answers.

A robot lives in shades of uncertainty.

The table is not simply "here" or "not here." The cup is not simply "graspable" or "not graspable." A person's movement cannot always be predicted from one image.

The robot has to constantly update its understanding as the world changes.

That makes robotics fundamentally different from many digital AI applications.

The machine is not merely producing an answer.

It is taking an action, observing the consequence and deciding what to do next.

That creates a feedback loop.

See.

Predict.

Act.

Observe.

Correct.

Repeat.

The better that loop becomes, the more capable a robot can potentially become.

Researchers are also discovering that simulation alone is not enough.

A robot can practise picking up a virtual object thousands of times, but the real world contains wrinkles, friction, slippery surfaces, broken objects, unusual lighting and tiny variations that simulations may not capture perfectly.

This is known as the sim-to-real problem.

The robot has to transfer what it learned in an artificial environment into reality without discovering that reality behaves differently at the worst possible moment.

That is why physical data is becoming as valuable to robotics companies as text data has been to language-model companies.

And the scale of the problem is enormous.

Stanford's 2026 AI Index notes that physical-world benchmarks remain particularly difficult because real environments are unpredictable and mistakes have physical consequences.

Recent robotics benchmarks are therefore becoming much more complicated than simply asking a machine to pick up an object.

The 2026 BEHAVIOR challenge, for example, includes long-horizon household tasks involving multiple rooms, changing object states and actions such as opening, pouring, wiping, spraying, attaching, cooking and slicing.

That is much closer to what humans actually mean by "common sense."

It is not one ability.

It is a huge collection of small assumptions about how the world works.

Water flows downward.

Glass can break.

Wet floors can be slippery.

People move unpredictably.

A drawer has a limited range of motion.

A towel behaves differently from a metal plate.

A box can hide something behind it.

A person asking you to clean a table probably does not want you to throw everything on the floor.

Humans acquire these concepts gradually through years of experience.

Robots have to learn them through data, simulation, demonstrations, reinforcement learning and increasingly sophisticated AI models.

That is why teaching robots common sense may ultimately be harder than teaching AI to write an essay, generate an image or answer a question.

Those tasks happen inside the digital world.

Robotics has to deal with the real one.

And the real world has no undo button.

The industry is making progress. Figure, Google DeepMind, NVIDIA and numerous robotics startups are developing systems that can perceive more, plan longer sequences and adapt to unfamiliar environments.

But a robot that can make a bed in an unfamiliar house is not the same thing as a robot with human-level common sense.

There is still a huge distance between demonstrating impressive tasks and building a machine that can safely handle the endless weirdness of everyday life.

That distance is where one of the biggest battles in robotics will be fought.

The winning technology may not simply be the robot with the strongest motors, the most cameras or the most advanced language model.

It may be the system that gets best at answering a deceptively difficult question before every action:

"What is likely to happen if I do this?"

Once a machine can answer that question reliably enough, robots stop being machines that merely follow instructions.

They start becoming machines that understand consequences.

And that could be the moment robotics finally moves from programmed automation toward something much closer to physical intelligence.