ComparisonNovember 9, 20266 min readBy Oleg Shlyakhter

UMI vs. Teleoperation vs. Egocentric Human Video: Robot Data Sources Compared

Robot learning in 2026 runs on three main data sources: teleoperation, where a person drives the real robot; handheld interfaces such as the Universal Manipulation Interface (UMI), where a person demonstrates with a robot-like gripper; and egocentric human video, where a person simply does the task while wearing cameras. Each trades embodiment match against scale. The best programs no longer pick one; they decide the ratio.

TL;DR: Teleoperation gives exact robot actions but scales with robot fleets and operator time. UMI gives robot-compatible gripper trajectories without a robot, at higher throughput. Egocentric human video scales furthest and captures natural behavior, but its actions must be inferred. NVIDIA's GR00T 2 pretraining mixture uses all three, with human video by far the largest share.

Key takeaways

  • Teleoperation is the gold standard for embodiment match, but every hour needs a robot and an operator.
  • UMI-style devices decouple data collection from robots while keeping a gripper-like action space and wrist camera view.
  • Egocentric human video is the cheapest to scale and now the largest pretraining layer in several 2026 models.
  • Evidence from GR00T 2, UMI-Live and Physical Intelligence points to mixtures: lots of human or handheld data plus a small amount of robot data.
  • The right ratio depends on your robot's end effector, the tasks, and whether you need pretraining breadth or task-specific precision.

The three data sources in one sentence each

  • Teleoperation: a human controls the actual robot (leader arms, VR, exoskeletons), and the robot's own sensors and joint commands are recorded.
  • UMI and handheld interfaces: a human manipulates objects with a handheld gripper carrying a wrist camera and tracking, producing end-effector trajectories and gripper widths a robot can imitate. The original UMI paper used a GoPro with a 155° fisheye lens, side mirrors for implicit stereo, and IMU-aware visual tracking.
  • Egocentric human video: a human performs everyday tasks with bare hands while wearing head, chest or wrist cameras; actions are inferred later from hand pose, motion or latent-action models.

Comparison table

Teleoperation vs. UMI vs. egocentric human video
CriterionTeleoperationUMI / handheld gripperEgocentric human video
Needs a robot during collectionYesNoNo
Action labelsExact robot joint or end-effector commandsMeasured gripper pose and width (tracking-dependent)Inferred: 3D hand pose, latent actions or pseudo-labels
Embodiment gapNone for the same robotSmall for parallel-jaw grippers; larger for dexterous handsLargest: human hands, body and viewpoint
Throughput per collectorLowest; limited by robot speed and operator skillHigher; natural motion, no robot latencyHighest; people do tasks at normal speed
Environment diversityLimited by where robots can goAny place a person can carry a gripperHomes, shops, factories, outdoors
Force and contact dataAvailable if the robot has sensorsOnly with added tactile sensingGenerally absent unless gloves or sensors are worn
Best usePost-training, evaluation, precise contact-rich tasksTask-specific manipulation at scale; cross-embodiment gripper skillsPretraining breadth, world models, navigation and intent

What the 2026 evidence says

GR00T 2 trains on all three, and shows transfer from each

NVIDIA's GR00T 2 (November 2026) is pretrained on one mixture: 402,730 hours of egocentric human video, 10,184 hours of handheld UMI demonstrations across 2,897 tasks, and 23,500 hours of robot teleoperation. NVIDIA reports skills transferring from humans to a dexterous robot, from a handheld UMI gripper to a YAM arm (toy dump-truck assembly in 60% of trials, with no YAM demonstrations of that task), and from the DexUMI wearable hand to a robot hand. Increasing the share of human video in pretraining raised real-robot task progress from 50% to 70% across 72 tasks in NVIDIA's evaluation.

UMI plus a little teleoperation

The UMI-Live project (CoRL 2026 workshop) fine-tuned π0.5 on a mixture of handheld bimanual UMI demonstrations and teleoperation and reported 10 of 10 successful trials on a six-orange placement task with a 90% UMI / 10% teleoperation mixture. It is a single task with ten trials, but it illustrates the pattern: handheld data supplies volume, robot data anchors the embodiment.

Human video with robot co-training

Physical Intelligence's human-to-robot study found that transfer from egocentric human video emerges once the VLA is pretrained on diverse enough robot data, and that co-training on human video can nearly double performance on scenarios seen only in human data. Earlier, EgoMimic (2024) co-trained on Project Aria human video and robot data and reported gains over robot-only baselines.

Glove and video-only programs at companies

Industry is moving the same way. Sunday Robotics says its ACT-1 model was trained without teleoperation data, using a Skill Capture Glove that matches its robot hand's geometry. Figure's Project Go-Big trains Helix on egocentric human video and reported navigation learned from human video alone. Business Insider reported that Tesla shifted Optimus data collection toward camera rigs worn by workers in mid-2025. These are company claims, but they show where collection budgets are moving.

Strengths and weaknesses in detail

Teleoperation

Strengths: perfect embodiment match, exact actions, real contact dynamics, and data that directly reflects the robot's limits. Weaknesses: slow, robot-bound, operator-dependent, and narrow in environments. Teleoperation remains the right tool for post-training, evaluation and tasks where small errors break the task. For a deeper look see our earlier piece on teleoperation vs. human demonstration data.

UMI-style handheld collection

Strengths: robot-free, portable, natural human speed, and an action space close to a parallel-jaw gripper. Variants such as FastUMI simplify tracking with off-the-shelf modules, and FastUMI-100K reports over 100,000 trajectories across 54 tasks. Weaknesses: tracking failures, gripper mismatch with dexterous hands, and limited scene context from a wrist-only view. See our UMI variants comparison.

Egocentric human video

Strengths: the only source that plausibly scales to millions of hours and the long tail of real homes and workplaces. Weaknesses: actions are inferred, hands differ from robot end effectors, and quality varies widely without capture standards. Stereo or depth-aware capture helps; see stereo vs. mono egocentric video.

How to choose your mix

Starting points by goal (validate with your own pilots)
GoalLead sourceSupporting source
Foundation-model pretrainingEgocentric human videoUMI and multi-robot teleoperation for action grounding
Gripper manipulation skill in new environmentsUMI / handheldSmall teleoperation set on the target robot
Dexterous hand skillsWearable hand interfaces or gloves matched to the robot handEgocentric video with 3D hand pose; teleoperation for evaluation
Contact-rich precision (insertion, assembly)TeleoperationUMI for variation; real-world RL
Navigation and whole-home behaviorEgocentric human videoRobot data for low-level control

Bottom line: teleoperation teaches the robot its body, UMI teaches it the task, and egocentric human video teaches it the world. Budget accordingly.

Where Field Motion fits

Field Motion collects egocentric human video (mono, stereo and multi-camera) and UMI-style handheld demonstrations through a contributor network in Latin America, Southeast Asia and the United States. Because human and handheld data are the parts of the mix that scale with people rather than robots, they are where a distributed collection network adds the most. We scope task lists, environments, capture specs and quality gates per project. Start with the Field Motion homepage or the inquiry form.

Frequently asked questions

What is the difference between UMI and teleoperation?

Teleoperation records a human controlling the real robot, so actions are exact robot commands but every hour needs a robot. UMI records a human using a handheld gripper with a wrist camera and tracking, so it needs no robot and scales faster, but the actions are measured gripper trajectories that must transfer to the robot.

Can robots learn from egocentric human video alone?

For some behaviors, companies and labs report yes. Figure reported navigation learned from human video only, and Sunday Robotics says ACT-1 used no teleoperation data. Most published results still pair human data with some robot data for low-level control and evaluation.

How much robot data do you still need if you collect UMI or human video?

It depends on the task and robot. Published examples use small robot shares: UMI-Live reported success with a 90% UMI / 10% teleoperation mixture on one task, and GR00T 2 reports adapting to new tasks with a handful of robot demonstrations after large-scale pretraining.

Which data source is cheapest per hour?

Egocentric human video is generally the cheapest to scale because it needs only wearable cameras and contributors, not robots or specialized grippers. Cost per useful hour also depends on quality checks and annotation.

Does Field Motion offer UMI and egocentric data collection?

Yes. Field Motion collects UMI-style handheld demonstrations and egocentric mono, stereo and multi-camera video through contributors in Latin America, Southeast Asia and the United States.

OS
Oleg Shlyakhter
Co-Founder & CEO, Field Motion

Need human or UMI data alongside your robot data?

We'll help you size the human-video and handheld layers around your existing teleoperation set and run a pilot first.

Talk to Field Motion

Related reading