ComparisonNovember 9, 20266 min readBy Oleg Shlyakhter

UMI Variants Compared: FastUMI, DexUMI, MV-UMI, UMI-on-Legs, UMI-Live

The Universal Manipulation Interface (UMI) turned a handheld gripper with a camera into a way to teach robots without robots. Since 2024 it has spawned a family of variants that change the tracking, the cameras, the end effector or the target robot. By late 2026, UMI data sits inside major foundation-model pretraining mixtures. Here is how the variants differ and how to choose.

TL;DR: Original UMI set the template: fisheye wrist camera, mirrors for implicit stereo, visual-inertial tracking and a parallel-jaw gripper. FastUMI simplifies tracking for scale; DexUMI moves to dexterous hands via an exoskeleton; MV-UMI adds a third-person view; UMI-on-Legs brings UMI policies to mobile quadrupeds; UMI-Live makes bimanual collection self-contained with LiDAR-inertial tracking. Pick the variant whose end effector and camera layout match your robot.

Key takeaways

  • UMI's core idea is a shared wrist viewpoint and gripper action space between human demonstration and robot deployment.
  • Variants mostly change one of four things: tracking, cameras, end effector or target embodiment.
  • NVIDIA's GR00T 2 pretraining includes 10,184 hours of handheld UMI demonstrations across 2,897 tasks.
  • Open UMI-style datasets now range from tens of hours (UMI-Live) to over 100,000 trajectories (FastUMI-100K).
  • Tracking quality and gripper-width measurement are the main sources of label error in any UMI program.

What is UMI?

The Universal Manipulation Interface (Chi et al., 2024) is a data collection and policy learning framework built around a handheld gripper. Its published design uses a GoPro with a 155° fisheye lens for wide context, side mirrors that provide implicit stereo in a single image, and IMU-aware visual SLAM to recover the gripper's trajectory at metric scale. Policies learn relative end-effector trajectories and gripper widths, and the framework includes inference-time latency matching so the same policy can run on a robot with a matching wrist camera and gripper. The project page has hardware and code.

UMI variants at a glance

Comparison of UMI-family systems (as described by their authors)
SystemWhat changes vs. UMITrackingEnd effectorPublished data / results
UMI (2024)Baseline designVisual-inertial SLAM from GoPro video and IMUParallel-jaw gripperIn-the-wild manipulation policies; open hardware and code
FastUMI (2024)Decoupled hardware; simpler pipelineOff-the-shelf RealSense T265 moduleAdaptable fingertips for common grippers10,000+ trajectories across 22 tasks; FastUMI-100K adds 100,000+ trajectories across 54 tasks (~600 hours)
DexUMI (2025)Human hand as the interfaceExoskeleton joint sensing plus wrist cameraDexterous robot hands (6-DoF and 12-DoF tested)86% average success across four tasks; 3.2× collection efficiency vs. teleoperation reported
UMI-on-Legs (2024)Mobile, legged deploymentUMI trajectories plus simulated whole-body controllerGripper on a quadrupedOver 70% success on all evaluated tasks
MV-UMI (2025)Adds a third-person camera with human maskingUMI-style SLAMParallel-jaw and custom three-jaw gripperAbout 47% improvement on scene-dependent sub-tasks across 3 tasks
ActiveUMI (2025)Adds active perception to robot-free demonstrationsSee paperGripperResearch system
UMI-Live (2026)Self-contained bimanual grippers with onboard computeLiDAR-inertial SLAM; ArUco-based jaw openingTwo grippers~39.7 hours, 3,255 bimanual episodes in LeRobot format

The variants in detail

FastUMI: built for throughput

FastUMI redesigns UMI to remove dependencies on specialized robot components and replaces the visual-inertial odometry pipeline with an off-the-shelf tracking module. Its follow-up, FastUMI-100K, reports over 100,000 demonstration trajectories collected across household environments, 54 tasks and hundreds of object types, including single-arm and dual-arm configurations with multi-view wrist fisheye images. It is a strong reference for what a structured, multi-team UMI collection operation looks like.

DexUMI: from grippers to hands

DexUMI uses the human hand itself as the interface. A wearable exoskeleton, optimized per target robot hand, constrains human finger motion to what the robot can do, and software inpaints the robot hand over the human hand in video to close the visual gap. NVIDIA's GR00T 2 reports learning bimanual dexterous tasks from DexUMI demonstrations alone.

MV-UMI: more context

Wrist-only views lose track of the wider scene. MV-UMI adds a third-person camera and removes the human demonstrator from that view with segmentation and inpainting, so training and deployment views match. The authors report gains on sub-tasks that need broad scene understanding. See our multi-camera capture guide.

UMI-on-Legs: mobile manipulation

UMI-on-Legs combines real-world UMI data for the manipulation policy with simulation-trained whole-body control on a quadruped, using end-effector trajectories in the task frame as the interface. It also reports zero-shot deployment of a manipulation checkpoint originally trained for a fixed-base arm.

UMI-Live: self-contained bimanual collection

UMI-Live (CoRL 2026 UMI Arena workshop) integrates a camera, LiDAR-inertial sensing, onboard compute and power in each gripper, needing no external tracking or environment calibration. An asynchronous pipeline reconstructs trajectories and publishes validated LeRobot datasets. Fine-tuning π0.5 on a 90% UMI / 10% teleoperation mixture produced 10 of 10 successes on a six-orange placement task.

UMI in 2026 foundation models

UMI data is no longer only a research curiosity. NVIDIA's GR00T 2 pretraining mixture contains 10,184 hours of handheld UMI gripper demonstrations spanning 2,897 tasks, alongside human video and teleoperation. NVIDIA reports that tasks demonstrated only with a handheld UMI gripper transferred to a YAM robot arm that never practiced them, including toy dump-truck assembly in 60% of trials. Public UMI-style datasets have also appeared on Hugging Face during 2026, from small samples to larger bimanual releases such as UMI-Live's.

How to choose a UMI variant

Matching a UMI variant to your robot
Your robot or goalStart with
Single parallel-jaw arm, in-the-wild tasksUMI or FastUMI
Bimanual armsUMI-Live or dual-arm FastUMI
Dexterous robot handsDexUMI-style exoskeleton matched to the hand, or a glove matched to the hand
Tasks that need scene contextMV-UMI-style wrist plus masked third-person view
Legged or mobile manipulatorUMI-on-Legs approach: UMI data plus a whole-body controller
Foundation-model pretrainingLarge, diverse UMI collection mixed with human video and some robot data

Quality checks every UMI program needs

  1. Tracking validity: detect SLAM or tracker failures, relocalization jumps and scale drift per episode.
  2. Gripper width: verify marker detection or encoder readings, especially at full open and full close.
  3. Latency and sync: align camera, pose and gripper streams on one timeline.
  4. Viewpoint consistency: the camera mount must match the robot's wrist camera placement closely enough for transfer.
  5. Diversity plan: objects, environments and operators planned in advance; UMI's advantage is going where robots cannot.
  6. Episode metadata: task, outcome, device ID and calibration version for every episode.

Field Motion and UMI-style collection

Field Motion collects UMI-style handheld demonstrations alongside egocentric mono, stereo and multi-camera data, using contributors in Latin America, Southeast Asia and the United States. UMI's value is environment diversity, and a distributed contributor network is a direct way to get it. We define hardware, quality gates and delivery format per project. Compare UMI with other sources in UMI vs. teleoperation vs. egocentric video, or visit the Field Motion homepage.

Frequently asked questions

What is the Universal Manipulation Interface (UMI)?

UMI is a 2024 framework from Chi et al. that uses a handheld gripper with a fisheye wrist camera, side mirrors for implicit stereo and visual-inertial tracking to collect manipulation demonstrations without a robot. Policies trained on UMI data can be deployed on robots with a matching gripper and wrist camera.

What is the difference between UMI and FastUMI?

FastUMI is a redesign of UMI that decouples the hardware from specific robot components and replaces the visual-inertial odometry pipeline with an off-the-shelf tracking module, aiming for faster deployment and larger-scale collection. FastUMI-100K reports over 100,000 trajectories across 54 tasks.

Can UMI be used for dexterous robot hands?

Yes, through variants such as DexUMI, which uses a wearable hand exoskeleton matched to the target robot hand and inpaints the robot hand into the video. NVIDIA's GR00T 2 reports learning bimanual dexterous tasks from DexUMI demonstrations alone.

Is UMI data used in foundation models?

Yes. NVIDIA reports that GR00T 2 is pretrained with 10,184 hours of handheld UMI gripper demonstrations across 2,897 tasks, together with egocentric human video and robot teleoperation data.

Does Field Motion collect UMI data?

Yes. Field Motion collects UMI-style handheld demonstrations as well as egocentric mono, stereo and multi-camera data, with hardware and quality gates scoped per project.

OS
Oleg Shlyakhter
Co-Founder & CEO, Field Motion

Running a UMI collection program?

Tell us your gripper or hand, target tasks and environments. We'll scope a UMI-style pilot with clear quality gates.

Talk to Field Motion

Related reading