Programming robots in plain language

Programming robots in plain language means controlling a robot with AI models instead of hand-written code: a person describes the task in ordinary language, and either a language model writes a program against the robot’s interface or a vision-language-action model (VLA) turns the instruction into movements by itself. This used to take an engineer who knew C++, ROS and the workings of the particular robot.¹²

Google and Microsoft showed the first experiments in 2022–2023. By 2026 it had gone mainstream: an open robot arm for learning costs about $100–230, the Reachy Mini desktop robot from $399, and Hugging Face writes that most of the 150-plus authors of the 200-plus apps for it had never written robotics code before; one of them, 78-year-old Joel Cohen, had never programmed at all. Robot models, meanwhile, remain unreliable, and, as researchers have shown, they are easy to jailbreak with words.³⁴⁵

History

• 04.04.2022Google and Everyday Robots unveil SayCan: a language model proposes steps, and the robot’s skill estimates weed out the infeasible ones; the robot serves as the language model’s “hands and eyes”.⁶ ¶ c8epmn7fg
• 16.09.2022Google describes Code as Policies: given a natural-language command and a few examples, a language model writes a Python control program that calls the robot’s perception and motion functions.⁷ ¶ c4jvhtdzz
• 20.02.2023Microsoft shows “ChatGPT for Robotics”: robot arms and drones are controlled through a dialogue with ChatGPT; the goal is to let people interact with robots “without needing to learn complex programming languages”.¹ ¶ cicz4rr4c
• 28.07.2023Google DeepMind unveils RT-2, a vision-language-action model trained on web and robot data: in unseen scenarios it succeeds 62% of the time, against 32% for its predecessor RT-1.⁸ ¶ c8hhsyd2e
• 03.10.2023Google DeepMind and partners publish Open X-Embodiment, a dataset from 22 robot types with more than a million episodes; a model trained on it is on average 50% more successful on five commonly used robots.⁹ ¶ cpjc43y4f
• 10.2024Hugging Face and The Robot Studio release SO-100, an open robot arm “for around $100”.³ ¶ cjihw7eac
• 04.02.2025Physical Intelligence opens the code and weights of its π0 model “to enable anyone to experiment with fine-tuning” it for their own robots; according to the company, 1 to 20 hours of data was enough for various tasks.¹⁰ ¶ ctc66xfcm
• 12.03.2025Google DeepMind unveils Gemini Robotics, a vision-language-action model based on Gemini, and the embodied reasoning model Gemini Robotics-ER; in September ER 1.5 becomes available to developers through the Gemini API.²¹¹ ¶ c5f5fqjwt
• 03.06.2025Hugging Face releases SmolVLA, a 450-million-parameter model that is “Small enough to run on CPU, train on a single consumer GPU, or even a MacBook”; it was trained on 487 community-collected datasets.¹² ¶ c2b6aqcsj
• 09.07.2025Hugging Face opens orders for Reachy Mini, a $299 desktop robot programmable in Python; according to the company, $1 million worth were sold in five days.¹³¹⁴ ¶ ckhakpfjh
• 12.11.2025Anthropic publishes Project Fetch: eight employees who were not robotics specialists programmed a robot dog; on the tasks both teams completed, the team with Claude was roughly twice as fast as the team without it and wrote about 9 times more code.¹⁵ ¶ cwjvwn2s6
• 05.01.2026Jensen Huang at CES: “The ChatGPT moment for robotics is here.” NVIDIA integrates its robot models into the open LeRobot library.¹⁶ ¶ cxhfpriwa
• 06.05.2026Hugging Face opens an app store for Reachy Mini with an agent that, from a plain-language description, writes, tests and uploads code to the robot by itself; users own almost 10,000 of the robots.⁴ ¶ cfyp6q8j8
• 18.06.2026Anthropic repeats Project Fetch with Claude Opus 4.7 and no human help: on every tested task the humans had solved, the model was at least 10 times faster, and about 20 times faster than the fastest human team overall; tasks with the physical controller were left out of the comparison.¹⁷ ¶ c2mxyq77k
• 30.07.2026Google DeepMind releases Gemini Robotics 2; the embodied reasoning model ER 2 is available in Google AI Studio, while the models that control movement themselves are for partners only.¹⁸ ¶ chdbipd7p

How it works

The model writes code. A language model gets a description of the robot’s interface and a task, and writes a program: this is how Code as Policies and ChatGPT for Robotics worked, how Claude programmed the robot dog in Project Fetch, and how the Reachy Mini agent builds apps. The robot then runs ordinary code that can be read and checked.⁷¹¹⁵⁴ ¶ cp43vef38

The model produces the movements itself. Vision-language-action models (RT-2, π0, SmolVLA, Gemini Robotics) take camera images and a text instruction and output motor commands directly. They are fine-tuned on demonstrations: for its On-Device model Google cites as few as 50–100 demonstrations for a new task, Physical Intelligence 1 to 20 hours of data.⁸¹⁰¹⁹ ¶ cgrw6xq6n

Shared data. Such models live on data from many different robots: by October 2026 more than 82,000 datasets tagged LeRobot had been posted on Hugging Face. SmolVLA, pretrained on community data, succeeded at tasks on the SO-100 arm 78.3% of the time, against 51.7% without that data.²⁰¹² ¶ cddr54un2

Data

What robots and people manage

What a robot costs (as of October 2026): an SO-101 arm, about $122 for one arm according to the parts list, without the printed parts, about $230 for a pair; Reachy Mini, from $399 (from $299 at launch); the Unitree Go2 robot dog, from $1,600; the Unitree R1 humanoid robot, from $4,900, and the G1, from $13,500 ($16,000 in 2024).^2352425 ¶ cvkjsr47c

[EDITORIAL CONCLUSION] The barrier to entry has dropped from two sides: the robot has become as cheap as a smartphone, and its program is written in words. This works where a task can be broken down into calls to ready-made functions. Models that control movement themselves still make mistakes in unfamiliar conditions and need demonstrations, and Google does not open the strongest of them to the public. A schoolchild can already program a robot; a reliable robot is still engineers’ work. ¶ cu765zxai

Scale. In 2024, 542,000 industrial robots were installed worldwide, 54% of them in China.²⁶ Unitree, according to its prospectus, shipped more than 5,500 humanoid robots in 2025, and its revenue rose to 1.7 billion yuan; on 19 August 2026 the company went public in Shanghai.²⁷²⁸ In Beijing, AI has been compulsory in schools since autumn 2025: at one primary school, students work with iFlytek robots, and language models help them get executable code from instructions in Chinese.²⁹ ¶ ct6exyhxi

Criticism

Jailbreaking with words. In 2024 researchers at the University of Pennsylvania showed that robots controlled by language models are easy to make do dangerous things: their attacks often succeeded 100% of the time, and they called the jailbreak of the Unitree Go2 robot dog the first successful jailbreak of a deployed commercial robotic system. “Large language models are just not safe enough when integrated with the physical world,” said George Pappas, who led the work. The same group proposed a safeguard, RoboGuard, which cuts the share of unsafe plans executed from more than 92% to less than 3%.³⁰³¹³² ¶ c6dirm9th

Hacking without words. In September 2025 researchers found a Bluetooth vulnerability in Unitree robots (Go2, B2, G1, H1) through which an infected robot can take over other robots nearby; Unitree replied that it had completed most of the fixes and would roll out updates in the near future.³³ ¶ c89uaxk6g

A demo is not a product. The roboticist Rodney Brooks calls the expectation that humanoid robots will replace people in manual labor within the coming decades “pure fantasy thinking”, and advises people to “not come closer than 3 meters to a full size walking robot”. Andrej Karpathy speaks of a large gap between a demo and a product wherever the cost of failure is high: 90% success is just “the first nine”. The 1X NEO home robot, by the company’s own account, is guided remotely at scheduled times by a 1X expert on tasks it does not know.³⁴³⁵³⁶ ¶ c6bfp9i9v

No confirmed cases of a robot controlled by a language model harming a person had been found as of October 2026. ¶ cxpvhdfwg

Development

Safety. In 2025 Google DeepMind released ASIMOV, a set of benchmarks for the “semantic” safety of a robot’s actions, built from real hospital injury reports; the best system, with automatically generated “constitutions”, followed the rules 84.3% of the time.³⁷ ¶ c3ct3aq95

Regulation. In July 2026 the FCC added new models of mobile robots, humanoid and quadruped, produced abroad, regardless of the producer’s nationality, to its list of equipment that poses a risk to national security: in the assessment of an interagency body convened by the White House, data from them could be used for surveillance or to take over the robots remotely.³⁸ In the European Union, the AI Act’s requirements for high-risk systems in machinery and robots have been postponed until 2 August 2028.³⁹ ¶ cxgcf3z4b

[EDITORIAL CONCLUSION] A robot that a schoolchild can program can also be reprogrammed by an attacker, in words as well. For now, accessibility has outpaced protection: the tools for children and hobbyists are open, while safety benchmarks and rules are only beginning to appear. ¶ cbunz5avz