UMI Variants Compared: FastUMI, DexUMI, MV-UMI, UMI-on-Legs, UMI-Live
The Universal Manipulation Interface (UMI) turned a handheld gripper with a camera into a way to teach robots without robots. Since 2024 it has spawned a family of variants that change the tracking, the cameras, the end effector or the target robot. By late 2026, UMI data sits inside major foundation-model pretraining mixtures. Here is how the variants differ and how to choose.
TL;DR: Original UMI set the template: fisheye wrist camera, mirrors for implicit stereo, visual-inertial tracking and a parallel-jaw gripper. FastUMI simplifies tracking for scale; DexUMI moves to dexterous hands via an exoskeleton; MV-UMI adds a third-person view; UMI-on-Legs brings UMI policies to mobile quadrupeds; UMI-Live makes bimanual collection self-contained with LiDAR-inertial tracking. Pick the variant whose end effector and camera layout match your robot.
Key takeaways
- UMI's core idea is a shared wrist viewpoint and gripper action space between human demonstration and robot deployment.
- Variants mostly change one of four things: tracking, cameras, end effector or target embodiment.
- NVIDIA's GR00T 2 pretraining includes 10,184 hours of handheld UMI demonstrations across 2,897 tasks.
- Open UMI-style datasets now range from tens of hours (UMI-Live) to over 100,000 trajectories (FastUMI-100K).
- Tracking quality and gripper-width measurement are the main sources of label error in any UMI program.
What is UMI?
The Universal Manipulation Interface (Chi et al., 2024) is a data collection and policy learning framework built around a handheld gripper. Its published design uses a GoPro with a 155° fisheye lens for wide context, side mirrors that provide implicit stereo in a single image, and IMU-aware visual SLAM to recover the gripper's trajectory at metric scale. Policies learn relative end-effector trajectories and gripper widths, and the framework includes inference-time latency matching so the same policy can run on a robot with a matching wrist camera and gripper. The project page has hardware and code.
UMI variants at a glance
| System | What changes vs. UMI | Tracking | End effector | Published data / results |
|---|---|---|---|---|
| UMI (2024) | Baseline design | Visual-inertial SLAM from GoPro video and IMU | Parallel-jaw gripper | In-the-wild manipulation policies; open hardware and code |
| FastUMI (2024) | Decoupled hardware; simpler pipeline | Off-the-shelf RealSense T265 module | Adaptable fingertips for common grippers | 10,000+ trajectories across 22 tasks; FastUMI-100K adds 100,000+ trajectories across 54 tasks (~600 hours) |
| DexUMI (2025) | Human hand as the interface | Exoskeleton joint sensing plus wrist camera | Dexterous robot hands (6-DoF and 12-DoF tested) | 86% average success across four tasks; 3.2× collection efficiency vs. teleoperation reported |
| UMI-on-Legs (2024) | Mobile, legged deployment | UMI trajectories plus simulated whole-body controller | Gripper on a quadruped | Over 70% success on all evaluated tasks |
| MV-UMI (2025) | Adds a third-person camera with human masking | UMI-style SLAM | Parallel-jaw and custom three-jaw gripper | About 47% improvement on scene-dependent sub-tasks across 3 tasks |
| ActiveUMI (2025) | Adds active perception to robot-free demonstrations | See paper | Gripper | Research system |
| UMI-Live (2026) | Self-contained bimanual grippers with onboard compute | LiDAR-inertial SLAM; ArUco-based jaw opening | Two grippers | ~39.7 hours, 3,255 bimanual episodes in LeRobot format |
The variants in detail
FastUMI: built for throughput
FastUMI redesigns UMI to remove dependencies on specialized robot components and replaces the visual-inertial odometry pipeline with an off-the-shelf tracking module. Its follow-up, FastUMI-100K, reports over 100,000 demonstration trajectories collected across household environments, 54 tasks and hundreds of object types, including single-arm and dual-arm configurations with multi-view wrist fisheye images. It is a strong reference for what a structured, multi-team UMI collection operation looks like.
DexUMI: from grippers to hands
DexUMI uses the human hand itself as the interface. A wearable exoskeleton, optimized per target robot hand, constrains human finger motion to what the robot can do, and software inpaints the robot hand over the human hand in video to close the visual gap. NVIDIA's GR00T 2 reports learning bimanual dexterous tasks from DexUMI demonstrations alone.
MV-UMI: more context
Wrist-only views lose track of the wider scene. MV-UMI adds a third-person camera and removes the human demonstrator from that view with segmentation and inpainting, so training and deployment views match. The authors report gains on sub-tasks that need broad scene understanding. See our multi-camera capture guide.
UMI-on-Legs: mobile manipulation
UMI-on-Legs combines real-world UMI data for the manipulation policy with simulation-trained whole-body control on a quadruped, using end-effector trajectories in the task frame as the interface. It also reports zero-shot deployment of a manipulation checkpoint originally trained for a fixed-base arm.
UMI-Live: self-contained bimanual collection
UMI-Live (CoRL 2026 UMI Arena workshop) integrates a camera, LiDAR-inertial sensing, onboard compute and power in each gripper, needing no external tracking or environment calibration. An asynchronous pipeline reconstructs trajectories and publishes validated LeRobot datasets. Fine-tuning π0.5 on a 90% UMI / 10% teleoperation mixture produced 10 of 10 successes on a six-orange placement task.
UMI in 2026 foundation models
UMI data is no longer only a research curiosity. NVIDIA's GR00T 2 pretraining mixture contains 10,184 hours of handheld UMI gripper demonstrations spanning 2,897 tasks, alongside human video and teleoperation. NVIDIA reports that tasks demonstrated only with a handheld UMI gripper transferred to a YAM robot arm that never practiced them, including toy dump-truck assembly in 60% of trials. Public UMI-style datasets have also appeared on Hugging Face during 2026, from small samples to larger bimanual releases such as UMI-Live's.
How to choose a UMI variant
| Your robot or goal | Start with |
|---|---|
| Single parallel-jaw arm, in-the-wild tasks | UMI or FastUMI |
| Bimanual arms | UMI-Live or dual-arm FastUMI |
| Dexterous robot hands | DexUMI-style exoskeleton matched to the hand, or a glove matched to the hand |
| Tasks that need scene context | MV-UMI-style wrist plus masked third-person view |
| Legged or mobile manipulator | UMI-on-Legs approach: UMI data plus a whole-body controller |
| Foundation-model pretraining | Large, diverse UMI collection mixed with human video and some robot data |
Quality checks every UMI program needs
- Tracking validity: detect SLAM or tracker failures, relocalization jumps and scale drift per episode.
- Gripper width: verify marker detection or encoder readings, especially at full open and full close.
- Latency and sync: align camera, pose and gripper streams on one timeline.
- Viewpoint consistency: the camera mount must match the robot's wrist camera placement closely enough for transfer.
- Diversity plan: objects, environments and operators planned in advance; UMI's advantage is going where robots cannot.
- Episode metadata: task, outcome, device ID and calibration version for every episode.
Field Motion and UMI-style collection
Field Motion collects UMI-style handheld demonstrations alongside egocentric mono, stereo and multi-camera data, using contributors in Latin America, Southeast Asia and the United States. UMI's value is environment diversity, and a distributed contributor network is a direct way to get it. We define hardware, quality gates and delivery format per project. Compare UMI with other sources in UMI vs. teleoperation vs. egocentric video, or visit the Field Motion homepage.
Frequently asked questions
What is the Universal Manipulation Interface (UMI)?
UMI is a 2024 framework from Chi et al. that uses a handheld gripper with a fisheye wrist camera, side mirrors for implicit stereo and visual-inertial tracking to collect manipulation demonstrations without a robot. Policies trained on UMI data can be deployed on robots with a matching gripper and wrist camera.
What is the difference between UMI and FastUMI?
FastUMI is a redesign of UMI that decouples the hardware from specific robot components and replaces the visual-inertial odometry pipeline with an off-the-shelf tracking module, aiming for faster deployment and larger-scale collection. FastUMI-100K reports over 100,000 trajectories across 54 tasks.
Can UMI be used for dexterous robot hands?
Yes, through variants such as DexUMI, which uses a wearable hand exoskeleton matched to the target robot hand and inpaints the robot hand into the video. NVIDIA's GR00T 2 reports learning bimanual dexterous tasks from DexUMI demonstrations alone.
Is UMI data used in foundation models?
Yes. NVIDIA reports that GR00T 2 is pretrained with 10,184 hours of handheld UMI gripper demonstrations across 2,897 tasks, together with egocentric human video and robot teleoperation data.
Does Field Motion collect UMI data?
Yes. Field Motion collects UMI-style handheld demonstrations as well as egocentric mono, stereo and multi-camera data, with hardware and quality gates scoped per project.
Running a UMI collection program?
Tell us your gripper or hand, target tasks and environments. We'll scope a UMI-style pilot with clear quality gates.
Talk to Field MotionRelated reading
Sources
- Chi et al. — Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots (2024)
- UMI project page
- FastUMI: A Scalable and Hardware-Independent Universal Manipulation Interface (2024)
- FastUMI-100K: A Large-scale UMI-style Dataset (October 2025)
- DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation (2025)
- UMI-on-Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers (CoRL 2024)
- MV-UMI: A Scalable Multi-View Interface for Cross-Embodiment Learning (September 2025)
- ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrations (October 2025)
- UMI-Live for In-The-Wild Bimanual Robotics Data Collection (CoRL 2026 UMI Arena Workshop)
- NVIDIA GEAR — GR00T 2: Scaling Robot Foundation Models for Generalization, Dexterity, and Adaptation (November 2026)
Published by Field Motion on November 9, 2026. Figures are as reported by the cited organizations, many from their own evaluations. Field Motion is not affiliated with the cited projects.