Robot Fleet Data Flywheel 2026: Diversity, Not Hours

Q: Why are hours the wrong metric for robot training data?

Because the published scaling evidence says generalisation depends on how many distinct environments and objects the data covers, not on how long the robots ran. Lin, Hu, Sheng, Wen, You and Gao collected more than 40,000 demonstrations and ran more than 15,000 real-world rollouts, and found that success in new environments and on new objects follows roughly a power law in the number of training environments and objects, while adding demonstrations inside an environment stops helping once that environment has enough. Four collectors working one afternoon across many environments and objects produced policies with about 90 percent success in novel environments with unseen objects. The deployment-side numbers agree: Physical Intelligence's pi0.5 approaches a baseline trained on the test home after about 100 training environments, with only about 400 hours of mobile-manipulation data; Figure trained Helix on about 500 hours; DROID leads with 564 scenes rather than its 350 hours. Hours in the same few facilities saturate. A fleet should count environments, objects, task variants, conditions and failure modes, with recency, and route collection to whatever is thin.

Q: What do the Tesla Optimus data numbers actually tell us?

On 25 September 2026 Electrek relayed a report from The Information that Tesla had ramped Optimus production to several hundred units per week, had accumulated more than 500,000 hours of training data with a goal of doubling it by year-end, had hired dedicated data collectors wearing camera helmets and motion-capture suits in training hubs in Colorado, Arizona and Florida, had moved much of its self-driving annotation team to the robot, and planned to lease robots to companies whose facilities resemble its own so that those deployments feed the model. The same report said the robot's AI cannot yet reliably handle a wide range of tasks, behaves unpredictably outside its training distribution, takes several days to learn a basic task, and that most units are used internally for testing and data collection. An outsider cannot say why from public reporting, and the leasing plan is explicitly a diversity strategy. What the numbers show is that an hour count three orders of magnitude larger than the labs that report generalisation did not, by itself, produce generalisation. The unit on the dashboard decides what a fleet optimises for.

Q: What should a deployed robot actually capture, and what should it discard?

Capture on triggers and sample the rest. The robot keeps a ring buffer of camera frames, proprioception and commands, and writes a full episode with pre-roll and post-roll when a trigger fires: a teleoperator intervention, a task failure, a safety stop, or a novelty detector flagging an object, scene or condition the fleet has not seen. Routine successful loops are sampled at a low rate with summary statistics kept for the rest, because a tote-picking robot produces thousands of near-duplicate episodes per shift that add nothing after the first few. The reasoning is storage and signal at once: three cameras at modest resolution compress to roughly a gigabyte or two per robot-hour, so 200 robots logging uniformly would produce on the order of 5 to 10 TB a day, almost all duplicate; and the intervention and failure episodes are a small fraction of the hours and most of the information, exactly as disengagements were in self-driving. Every captured episode is then graded, tagged with site, task, object and condition, and counted in the coverage ledger. Consent and privacy handling for faces and screens happens at the edge before upload.

Q: What is a coverage ledger and how does it drive collection?

A coverage ledger is a sparse multidimensional count of what the fleet's graded data contains: environments by objects by tasks by conditions (lighting, time of day, clutter) by failure modes, each with a recency stamp. It replaces hours as the fleet's primary data metric. It drives collection in two directions. Forward, it produces weekly directives to operations: route robots to an under-covered site or shift, stage an under-represented object class, schedule teleoperation hours on a newly observed failure mode. Backward, it receives per-cell success rates from held-out evaluation, so that a cell where the policy scores 61 percent against an 88 percent average becomes next week's collection priority. Because the ledger is queryable by the same dimensions the dataset's relational metadata carries, the training mix can be composed to target weak cells without materialising copies. The ledger is the only component that is new relative to a logging pipeline, and it is what turns a log into a flywheel: deployment decisions, including where to lease or place robots, can be made partly on what the ledger needs.

Q: How should fleet data be combined with teleoperation, synthetic and open datasets?

As a mix with a coverage target, where each source covers an axis the others cannot. In-house teleoperation (Figure's roughly 500 hours for Helix, DROID's 50 operators over 12 months) gives high-quality demonstrations of the target task on the target embodiment but saturates in the lab. The deployed fleet supplies real environments, real objects and real failures, which nothing else does, provided it is placed and sampled for them. Synthetic generation multiplies demonstrated tasks across visual and layout conditions: NVIDIA generated 780,000 synthetic trajectories, equated to 6,500 human hours, in 11 hours, and reported a 40 percent improvement for GR00T N1 when synthetic was mixed with real data, not when it replaced it. Cross-embodiment corpora such as Open X-Embodiment (60 datasets, 22 embodiments, more than a million trajectories) and web data as in pi0.5's co-training add embodiment and semantic breadth but must be re-formatted and cannot close the gap for your own robot. The fleet's episodes are typically a minority of the mix by count and the majority of its value.

Q: What dataset format should a robot fleet use?

One with three properties: many episodes packed per file, relational metadata that can be queried by the coverage ledger's dimensions, and streaming reads so that training mixes can be composed without copying. The LeRobot v3.0 format released by Hugging Face in September 2025 is the reference example: it packs multiple episodes into shared Parquet files and concatenated MP4s grouped by camera, moves per-episode metadata into chunked Parquet files, and supports a streaming dataset class that reads without downloading, because the v2 layout of one Parquet and one MP4 per episode hit filesystem limits at the scale the format now targets, described as millions of episodes and hundreds of millions to billions of frames. RLDS, used by Open X-Embodiment, is the other common choice. Whichever you adopt, version the schema and treat every hardware change (a new gripper, a moved camera) as a new dataset version, because a silent change to the observation or action space corrupts every subsequent episode. Directories of PNGs per episode will consume the data-engineering budget on filesystems rather than on the model.

Q: What is the data contract between a robot fleet and the model team?

Four clauses, with a fifth on shared floors. Schema: every episode carries the same calibrated, versioned observation and action spaces, and a hardware change triggers a dataset version rather than silent drift. Labels: every captured episode is graded as success, intervention or a named failure class within a bounded time, and ungraded episodes do not count toward coverage. Consent and privacy: the contract names what is captured at customer sites, how faces and screens are handled at the edge, where data is stored, and what the customer who leased or bought the robot has agreed to, because a fleet with unclear data rights has a flywheel that stops the first time a customer's lawyer asks. Coverage: the fleet commits to a target distribution over the ledger's dimensions and operations is measured on it, which is the clause that turns 'deploy more robots' into 'deploy robots where the model is weak'. On multi-vendor floors, episode ownership must be explicit, because the vendor that owns the data owns the flywheel. The data is a moat only to the extent it is diverse, graded, rights-clear and in a consumable format.

Q: When should a robotics company not build a data flywheel yet?

When the fleet is small enough that it does not see environments you could not stage yourself. Under roughly ten robots in two sites doing one task, you have a dataset, not a flywheel, and the better investment is a few hundred hours of deliberately diverse teleoperation planned to the scaling result (many environments and objects, a modest number of demonstrations each), open cross-embodiment corpora for pretraining breadth, and synthetic multiplication of the demonstrated tasks across visual conditions. Build the cheap parts of the pipeline first anyway: a packed, streamable dataset format, an episode grader, a coverage ledger and declared held-out sites and objects cost little and shape every later decision. Build the fleet-side capture, the collection directives and the customer data contract when deployments reach the point where robots encounter sites, objects and failures that the lab cannot reproduce. Two further honest limits: the scaling law was measured on single-arm manipulation and has not been shown for long-horizon humanoid control, and capturing failures requires a safety envelope that permits them, so a fleet that is never allowed to fail collects no failure data.

Satyam Kumar

Retour au blog

architecture

The Robot Fleet Data Flywheel in 2026: Hours Are the Wrong Unit, So Collect for Environment and Object Diversity, Grade Every Episode, and Run the Fleet as a Data Producer With a Contract

By Satyam KumarOctober 23, 202622 min read

robot data flywheel robot fleet data collection robot learning training data data scaling laws imitation learning teleoperation data collection vision-language-action training data lerobot dataset format open x-embodiment droid dataset physical intelligence pi0.5 figure helix tesla optimus training data synthetic trajectories gr00t robot fleet data contract coverage ledger robotics intervention data mining

The Robot Fleet Data Flywheel in 2026: Hours Are the Wrong Unit, So Collect for Environment and Object Diversity, Grade Every Episode, and Run the Fleet as a Data Producer With a Contract

Frequently Asked Questions

Partager l'article

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

Architecte AI & Cloud. J'aide les équipes à construire des systèmes qui scalent pour des millions.

LinkedIn GitHub

The Robot Fleet Data Flywheel in 2026: Hours Are the Wrong Unit, So Collect for Environment and Object Diversity, Grade Every Episode, and Run the Fleet as a Data Producer With a Contract

Frequently Asked Questions

Partager l'article

Comments

Leave a comment