Ai2 ToyBlocks evaluates robot learning with controlled block tasks

What is changing?

ToyBlocks pairs Molmo2-4B-tb, which reads a target block arrangement and the current camera view to choose the next action, with the ManiFlow-tb policy that executes the motion. Ai2 published model weights, a dataset and evaluation assets on Hugging Face. The model card reports 97.5% success on easy simulated tasks with a classical motion planner; the same measurement drops to 55.5% when the learned ManiFlow executor is used.

For robotics teams, the key value is not one headline score but a controlled test world that can measure high-level visual planning separately from low-level motor control. The results should be read as Ai2's own evaluation rather than independent validation or proof of real-world generalization; success needs to be measured again when hardware, cameras or the motion executor changes.