UMI vs. Teleoperation vs. Egocentric Human Video: Robot Data Sources Compared
Robot learning in 2026 runs on three main data sources: teleoperation, where a person drives the real robot; handheld interfaces such as the Universal Manipulation Interface (UMI), where a person demonstrates with a robot-like gripper; and egocentric human video, where a person simply does the task while wearing cameras. Each trades embodiment match against scale. The best programs no longer pick one; they decide the ratio.
TL;DR: Teleoperation gives exact robot actions but scales with robot fleets and operator time. UMI gives robot-compatible gripper trajectories without a robot, at higher throughput. Egocentric human video scales furthest and captures natural behavior, but its actions must be inferred. NVIDIA's GR00T 2 pretraining mixture uses all three, with human video by far the largest share.
Key takeaways
- Teleoperation is the gold standard for embodiment match, but every hour needs a robot and an operator.
- UMI-style devices decouple data collection from robots while keeping a gripper-like action space and wrist camera view.
- Egocentric human video is the cheapest to scale and now the largest pretraining layer in several 2026 models.
- Evidence from GR00T 2, UMI-Live and Physical Intelligence points to mixtures: lots of human or handheld data plus a small amount of robot data.
- The right ratio depends on your robot's end effector, the tasks, and whether you need pretraining breadth or task-specific precision.
The three data sources in one sentence each
- Teleoperation: a human controls the actual robot (leader arms, VR, exoskeletons), and the robot's own sensors and joint commands are recorded.
- UMI and handheld interfaces: a human manipulates objects with a handheld gripper carrying a wrist camera and tracking, producing end-effector trajectories and gripper widths a robot can imitate. The original UMI paper used a GoPro with a 155° fisheye lens, side mirrors for implicit stereo, and IMU-aware visual tracking.
- Egocentric human video: a human performs everyday tasks with bare hands while wearing head, chest or wrist cameras; actions are inferred later from hand pose, motion or latent-action models.
Comparison table
| Criterion | Teleoperation | UMI / handheld gripper | Egocentric human video |
|---|---|---|---|
| Needs a robot during collection | Yes | No | No |
| Action labels | Exact robot joint or end-effector commands | Measured gripper pose and width (tracking-dependent) | Inferred: 3D hand pose, latent actions or pseudo-labels |
| Embodiment gap | None for the same robot | Small for parallel-jaw grippers; larger for dexterous hands | Largest: human hands, body and viewpoint |
| Throughput per collector | Lowest; limited by robot speed and operator skill | Higher; natural motion, no robot latency | Highest; people do tasks at normal speed |
| Environment diversity | Limited by where robots can go | Any place a person can carry a gripper | Homes, shops, factories, outdoors |
| Force and contact data | Available if the robot has sensors | Only with added tactile sensing | Generally absent unless gloves or sensors are worn |
| Best use | Post-training, evaluation, precise contact-rich tasks | Task-specific manipulation at scale; cross-embodiment gripper skills | Pretraining breadth, world models, navigation and intent |
What the 2026 evidence says
GR00T 2 trains on all three, and shows transfer from each
NVIDIA's GR00T 2 (November 2026) is pretrained on one mixture: 402,730 hours of egocentric human video, 10,184 hours of handheld UMI demonstrations across 2,897 tasks, and 23,500 hours of robot teleoperation. NVIDIA reports skills transferring from humans to a dexterous robot, from a handheld UMI gripper to a YAM arm (toy dump-truck assembly in 60% of trials, with no YAM demonstrations of that task), and from the DexUMI wearable hand to a robot hand. Increasing the share of human video in pretraining raised real-robot task progress from 50% to 70% across 72 tasks in NVIDIA's evaluation.
UMI plus a little teleoperation
The UMI-Live project (CoRL 2026 workshop) fine-tuned π0.5 on a mixture of handheld bimanual UMI demonstrations and teleoperation and reported 10 of 10 successful trials on a six-orange placement task with a 90% UMI / 10% teleoperation mixture. It is a single task with ten trials, but it illustrates the pattern: handheld data supplies volume, robot data anchors the embodiment.
Human video with robot co-training
Physical Intelligence's human-to-robot study found that transfer from egocentric human video emerges once the VLA is pretrained on diverse enough robot data, and that co-training on human video can nearly double performance on scenarios seen only in human data. Earlier, EgoMimic (2024) co-trained on Project Aria human video and robot data and reported gains over robot-only baselines.
Glove and video-only programs at companies
Industry is moving the same way. Sunday Robotics says its ACT-1 model was trained without teleoperation data, using a Skill Capture Glove that matches its robot hand's geometry. Figure's Project Go-Big trains Helix on egocentric human video and reported navigation learned from human video alone. Business Insider reported that Tesla shifted Optimus data collection toward camera rigs worn by workers in mid-2025. These are company claims, but they show where collection budgets are moving.
Strengths and weaknesses in detail
Teleoperation
Strengths: perfect embodiment match, exact actions, real contact dynamics, and data that directly reflects the robot's limits. Weaknesses: slow, robot-bound, operator-dependent, and narrow in environments. Teleoperation remains the right tool for post-training, evaluation and tasks where small errors break the task. For a deeper look see our earlier piece on teleoperation vs. human demonstration data.
UMI-style handheld collection
Strengths: robot-free, portable, natural human speed, and an action space close to a parallel-jaw gripper. Variants such as FastUMI simplify tracking with off-the-shelf modules, and FastUMI-100K reports over 100,000 trajectories across 54 tasks. Weaknesses: tracking failures, gripper mismatch with dexterous hands, and limited scene context from a wrist-only view. See our UMI variants comparison.
Egocentric human video
Strengths: the only source that plausibly scales to millions of hours and the long tail of real homes and workplaces. Weaknesses: actions are inferred, hands differ from robot end effectors, and quality varies widely without capture standards. Stereo or depth-aware capture helps; see stereo vs. mono egocentric video.
How to choose your mix
| Goal | Lead source | Supporting source |
|---|---|---|
| Foundation-model pretraining | Egocentric human video | UMI and multi-robot teleoperation for action grounding |
| Gripper manipulation skill in new environments | UMI / handheld | Small teleoperation set on the target robot |
| Dexterous hand skills | Wearable hand interfaces or gloves matched to the robot hand | Egocentric video with 3D hand pose; teleoperation for evaluation |
| Contact-rich precision (insertion, assembly) | Teleoperation | UMI for variation; real-world RL |
| Navigation and whole-home behavior | Egocentric human video | Robot data for low-level control |
Bottom line: teleoperation teaches the robot its body, UMI teaches it the task, and egocentric human video teaches it the world. Budget accordingly.
Where Field Motion fits
Field Motion collects egocentric human video (mono, stereo and multi-camera) and UMI-style handheld demonstrations through a contributor network in Latin America, Southeast Asia and the United States. Because human and handheld data are the parts of the mix that scale with people rather than robots, they are where a distributed collection network adds the most. We scope task lists, environments, capture specs and quality gates per project. Start with the Field Motion homepage or the inquiry form.
Frequently asked questions
What is the difference between UMI and teleoperation?
Teleoperation records a human controlling the real robot, so actions are exact robot commands but every hour needs a robot. UMI records a human using a handheld gripper with a wrist camera and tracking, so it needs no robot and scales faster, but the actions are measured gripper trajectories that must transfer to the robot.
Can robots learn from egocentric human video alone?
For some behaviors, companies and labs report yes. Figure reported navigation learned from human video only, and Sunday Robotics says ACT-1 used no teleoperation data. Most published results still pair human data with some robot data for low-level control and evaluation.
How much robot data do you still need if you collect UMI or human video?
It depends on the task and robot. Published examples use small robot shares: UMI-Live reported success with a 90% UMI / 10% teleoperation mixture on one task, and GR00T 2 reports adapting to new tasks with a handful of robot demonstrations after large-scale pretraining.
Which data source is cheapest per hour?
Egocentric human video is generally the cheapest to scale because it needs only wearable cameras and contributors, not robots or specialized grippers. Cost per useful hour also depends on quality checks and annotation.
Does Field Motion offer UMI and egocentric data collection?
Yes. Field Motion collects UMI-style handheld demonstrations and egocentric mono, stereo and multi-camera video through contributors in Latin America, Southeast Asia and the United States.
Need human or UMI data alongside your robot data?
We'll help you size the human-video and handheld layers around your existing teleoperation set and run a pilot first.
Talk to Field MotionRelated reading
Sources
- Chi et al. — Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots (2024)
- NVIDIA GEAR — GR00T 2: Scaling Robot Foundation Models for Generalization, Dexterity, and Adaptation (November 2026)
- UMI-Live for In-The-Wild Bimanual Robotics Data Collection (CoRL 2026 UMI Arena Workshop)
- Physical Intelligence — Emergence of Human to Robot Transfer in Vision-Language-Action Models (December 2025)
- Kareer et al. — EgoMimic: Scaling Imitation Learning via Egocentric Video (2024)
- Sunday Robotics — ACT-1: A Robot Foundation Model Trained on Zero Robot Data (November 19, 2025)
- Figure — Project Go-Big: Internet-Scale Humanoid Pretraining and Direct Human-to-Robot Transfer (September 18, 2025)
- Business Insider — Inside the Strategy Shift at Optimus, Tesla's Humanoid Robot Program (August 2025)
- FastUMI: A Scalable and Hardware-Independent Universal Manipulation Interface (2024)
- FastUMI-100K: A Large-scale UMI-style Dataset (October 2025)
Published by Field Motion on November 9, 2026. Figures are as reported by the cited organizations, many from their own evaluations. Field Motion is not affiliated with the cited projects.