ToyBlocks: A controlled world for robot learning and evaluation
ToyBlocks: A controlled world for robot learning and evaluation
Published November 6, 2026 — Kyle Wiggers
Robots are learning to fold laundry, sort parts, and handle unfamiliar objects. But those demonstrations are often rather narrow and don't explain what a robot has learned or why it fails.
To study a complex system, scientists often begin with a small, controlled environment. Robotics needs environments that make this kind of investigation possible—researchers should be able to change the training data or model design, repeat a test, and trace the cause of a failure.
Today we're introducing a framework for creating these environments; we call them FoundationWorlds. Each pairs a controlled, simulated world targeted at studying robotics in a specific environment with tools for automatically generating tasks and training demonstrations, as well as metrics for measuring a robot's behavior over time.
Our first foundation world is ToyBlocks: an open simulation framework built around a robot arm and blocks on a tabletop. The setting is deliberately simple, but the robot must still reason about how the blocks interact and how to arrange them while pushing, flipping, and stacking.
ToyBlocks automatically generates demonstrations without requiring time-consuming teleoperation. Researchers can easily vary these training experiences. For example, a team could generate more pushing or flipping demonstrations, then evaluate performance on tasks kept out of training.
How ToyBlocks works
Building on earlier work including MolmoAct 2, MolmoBot, and MolmoSpaces, ToyBlocks is designed around four requirements: (1) realistic physics and appearance, (2) automatic generation of tasks and demonstrations at scale, (3) challenges that combine reasoning with dexterity, and (4) broad coverage of the tasks possible within a domain.
Pictures and letters on the blocks' faces allow for searching, sorting, and arranging tasks. Domain randomization – varying block size, table height, and other physical properties – tests how well models generalize as the geometry changes.
ToyBlocks uses reinforcement learning and motion planning to generate skills for short-horizon tasks in Isaac Lab. Skill generation can draw on privileged information from the simulator. A task-and-motion planner automatically selects and sequences skills; running plans in parallel on a single GPU reaches up to 24 times real-time performance.
Research building on ToyBlocks
Evaluations use ToyBlocks to study low-level VLA spatial generalization, high-level VLM planning, agentic systems via MCP tool calls (including Claude Opus 5.5), and action-conditioned world models. Real-world experiments transfer ManiFlow and Molmo 2 + ManiFlow zero-shot to a Franka FR3; ManiFlow atomic-skill success was similar in sim and real (80.0% vs 81.1%), and the hierarchical system succeeded in 55.5% of simulated and 50.0% of real episodes.
Opening the tools
Ai2 is releasing simulation assets, procedural generators, automatically generated demonstrations (initial release: 85,336 long-horizon demonstrations, nearly 1,600 hours / 144 million frames), trained models, training code, and evaluation code. A larger-scale dataset is planned in a few weeks.
https://huggingface.co/blog/allenai/toyblocksReposted. Will be removed on request. All rights belong to the original author.
More from this batch
Ramp data shows AI spending stalling just as Anthropic nears its IPO
Ramp reports token spend grew only 1.5% across August–September while usage rose, as price cuts and cost controls cap vendor revenue ahead o…
Anthropic's $1.5 trillion IPO hits resistance as investors balk at valuation
Investors push back on Anthropic's $1.5T IPO valuation amid tech selloff and missing October revenue in the S-1; a delay could shake the bro…
Frontier Radar #5:若 AGI 已近,为何大佬还在出走办「新实验室」?梳理 45 家 neolab 的六类技术赌注
专题把独立研究型 AI 公司称为 neolab,按「更好的学习方式 / 不只靠语言 / 独占真实数据 / 算力效率 / 自动化科研 / 不拼第一名也赚钱」六类赌注整理约 27 家详档加近 20 家简述,并点出英伟达几乎处处下注。适合做战略扫描:看清大厂产品化压力之外,资本在押哪条…
Stratechery Sharp Tech 上视频:美国选民如何投票「数据中心」、对大科技与 Google AI 回应意味着什么
Ben Thompson 把 Sharp Tech 首次做成视频,讨论中期选举里数据中心议题如何影响大科技叙事,以及 Google 对 AI 竞争的回应。偏宏观产业与资本开支政治风险,适合对照本周公投与抛售新闻。
After blocking Muse, Amazon now lets users text Alexa to help them shop
Amazon is rolling out texting with Alexa for Shopping so U.S. customers can search, compare and buy over SMS, after previously blocking Meta…
Visa Agentic Commerce Benchmark: almost half of AI shopping test runs fail
Visa's first agentic shopping benchmark scores 22 models at a 57.2% pass rate; 54% of failed runs still ended in a purchase.
Google Antigravity adds remote agents and voice mode
Antigravity v2.29 starts rolling out cloud remote agents for AI Pro/Ultra and Gemini Live voice mode, plus redesigned Automations and a nati…
Xiaohongshu's Mishu Personal Agent launches via Rokid AI glasses
Xiaohongshu frames Mishu as a Personal Agent and pairs it with Rokid glasses for progressive on-site sensing grounded in community notes.
China Mobile open-sources JoinAI-Agent engine
China Mobile released Jiutian Juzhi JoinAI-Agent, a multi-agent engine that extends MCP tools and A2A agents with zero-code workflows.
Physicl raises $10M to make the digital world real enough for robots
Physicl's Studio turns high-fidelity 3D pipelines into synthetic data tooling for robot and world-model post-training.
Nace.AI 开源 Drex 决策模型族:输入状态 + 题型,一次前向给出各选项概率,不对文本「生成」
改进什么:很多路由/风控/分类只要校准概率,不必生成长文。怎么做:Drex Max(27B 多模态)、Drex v2.0(35B MoE/约 3B 激活)等,兼容 TypeSafe 风格 `/v1/systemone`,可本地 Python/llama.cpp/Ollama。许可…
Underdog shows off a 32MB speech model in new demo
Underdog's open Woof TTS is ~32MB and runs offline in the browser, alongside Bark STT and a 27B on-device model.