Egocentric Mono Video for Robotics: When One Camera Is Enough
Egocentric mono video is first-person footage from a single camera. It is the cheapest, most scalable form of human demonstration data, and in 2026 it is the backbone of the largest robot pretraining corpora. It is also the format most often oversold. This guide explains exactly what one camera can teach a robot, where it falls short, and how to collect mono data that models can actually use.
TL;DR: Mono egocentric video is enough for visual pretraining, world models, latent-action learning, task segmentation and high-level behavior such as navigation. It is not enough on its own when you need metric 3D hand trajectories, force, or precise contact timing as action labels. The fix is rarely to abandon mono; it is to pair a large mono layer with smaller stereo, wrist-camera, UMI or robot datasets.
Key takeaways
- The largest human-video corpora reported in 2026 run to hundreds of thousands of hours or more, a scale where light, simple capture is the practical choice.
- Mono supports world models and latent-action pretraining, which learn from how scenes change rather than from metric labels.
- Figure reported humanoid navigation learned end to end from egocentric human video alone.
- Mono's weak spots are metric depth, hand self-occlusion and fine contact; plan a supporting dataset for those.
- Mono quality is driven by capture discipline: framing, frame rate, exposure, hand visibility and episode segmentation.
What counts as egocentric mono video?
Egocentric mono video is a single first-person RGB stream, usually from a head, forehead, glasses or chest mount. It may carry extra metadata (IMU readings, camera intrinsics, timestamps, language narrations), but it has no second synchronized camera for stereo depth. It is distinct from stereo egocentric capture and from multi-camera rigs.
Why mono matters more than ever in 2026
Human video is now the largest layer in several robot pretraining mixtures. NVIDIA's GR00T 2 reports 402,730 hours of egocentric human video in pretraining and found real-robot task progress rose from 50% to 70% on 72 tasks as the share of human video went from none to all of it. Dyna Robotics reports pretraining DYNA-2 on over 1 million hours of egocentric human video. NVIDIA's DreamDojo world model was pretrained on 44,711 hours of human egocentric video. When the target is hundreds of thousands of hours across thousands of homes and workplaces, the capture device has to be cheap, light and easy to wear.
What one camera can teach a robot
| Signal | Supported by mono? | Notes |
|---|---|---|
| Visual representations of objects, scenes and hands | Yes | The core value of human video pretraining. |
| World-model dynamics (how scenes change) | Yes | Video-prediction and world-action models learn directly from frames. |
| Latent or pseudo-actions | Yes | GR00T N1 describes latent actions and inverse-dynamics pseudo-labels for action-less human video. |
| Task structure and language grounding | Yes | With narrations or episode labels. |
| Navigation behavior | Yes | Figure's Project Go-Big reported navigation learned from egocentric human video only. |
| 2D hand keypoints and 3D hand shape | Partly | Single-image models such as HaMeR reconstruct hand meshes; absolute depth stays ambiguous. |
| Metric 3D wrist and fingertip trajectories | Weak | Needs stereo, depth sensing, device tracking or a strong prior. |
| Force, torque and contact | No | Needs gloves, tactile sensors or robot data. |
When mono is enough
Pretraining and world models
If the dataset's job is to teach a model what the physical world looks like and how it changes when people act, mono is enough. Breadth matters most: many people, many environments, many objects, many tasks. Spending budget on extra cameras reduces breadth.
High-level behavior and intent
Where to go, what to pick up next, how a task is sequenced: these are learnable from first-person video without metric 3D. Figure's reported human-video-only navigation result is an example of behavior where precise hand geometry is irrelevant.
Task discovery and long-tail coverage
Mono capture is the most practical way to sample how people really live and work, including messy homes, unusual tools and rare events. That long tail is what generalist models struggle with.
When mono is not enough
Precise manipulation actions
Methods that turn human video into robot actions often use 3D hand positions as the action signal; Physical Intelligence's human-to-robot study is one example. Depth errors from a single camera become trajectory errors, and the hand hides the object at the exact moment of grasp. For these labels, add stereo, wrist cameras or a device with built-in hand tracking.
Dexterous and contact-rich tasks
In-hand manipulation, insertion and force-sensitive tasks need information a camera cannot see. Wearable interfaces matched to the robot hand, such as DexUMI or Sunday Robotics' Skill Capture Glove, or teleoperation, fill this gap.
Robot-specific post-training
Even the most human-video-heavy models still adapt on robot data. GR00T 2 reports moving from 40% zero-shot success to 95% with five robot demonstrations per task. Mono pretraining reduces the robot data you need; it does not remove it.
How to collect mono egocentric data that models can use
- Frame the hands. Mount angle should keep both hands and the work surface in view during typical tasks; validate with a hand-visibility metric.
- Fix the capture spec. Resolution, frame rate, field of view and stabilization settings should be consistent across devices; record intrinsics per camera model.
- Avoid motion blur. Fast hand motion at low frame rate or in dim light destroys hand-pose quality. Set minimum lighting and shutter rules.
- Keep IMU and timestamps. Cheap to store and useful for camera-motion estimation later.
- Segment episodes. Clear start and end boundaries, task labels and outcomes make the data usable for both pretraining and supervised learning.
- Design diversity. Plan participants, environments, objects and tasks deliberately; do not rely on hours alone.
- Handle privacy. First-person video captures homes, faces and screens. Consent, bystander handling and permitted-use terms must be part of the protocol.
Rule of thumb: collect mono for breadth, then buy precision where your model's errors actually are: stereo for 3D, wrist cameras for occlusion, gloves or UMI for contact, robot data for the body.
Field Motion and mono egocentric collection
Field Motion runs egocentric data collection through contributors in Latin America, Southeast Asia and the United States, which suits the diversity-first nature of mono programs: many people, many real environments. We also collect stereo, multi-camera and UMI-style data for the precision layer. Read our comparisons of UMI, teleoperation and human video and public egocentric datasets, or go to the Field Motion homepage.
Frequently asked questions
Is monocular egocentric video useful for training robots?
Yes. It is the main ingredient in large-scale human-video pretraining and world models, and it can teach high-level behavior such as navigation. It is weaker as a source of precise 3D action labels, force or contact information.
What are the limits of single-camera egocentric data?
Absolute depth is ambiguous, hands occlude the objects they hold, and there is no force or contact signal. These limits matter most for precise manipulation and dexterous tasks.
Can you extract robot actions from mono human video?
Approximately. Approaches include 3D hand pose estimation, latent-action models and inverse-dynamics pseudo-labels. The resulting actions are estimates, so teams usually pair them with stereo, UMI or robot data for precise control.
How much egocentric video do robot foundation models use?
Reported figures vary widely by company. NVIDIA reports 402,730 hours of egocentric human video in GR00T 2 pretraining and Dyna Robotics reports over 1 million hours for DYNA-2; these are company-reported numbers.
Does Field Motion collect mono egocentric video at scale?
Field Motion collects mono egocentric video, along with stereo, multi-camera and UMI-style data, through a contributor network in Latin America, Southeast Asia and the United States. Volumes and specifications are scoped per project.
Scaling mono egocentric collection?
Tell us the tasks, environments and regions you need. We'll design a capture spec and diversity plan, then run a pilot.
Talk to Field MotionRelated reading
Sources
- NVIDIA GEAR — GR00T 2: Scaling Robot Foundation Models for Generalization, Dexterity, and Adaptation (November 2026)
- Dyna Robotics — DYNA-2 World-Action Model announcement (August 10, 2026)
- NVIDIA — DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos (February 2026)
- NVIDIA — GR00T N1: An Open Foundation Model for Generalist Humanoid Robots (March 2025)
- Figure — Project Go-Big: Internet-Scale Humanoid Pretraining and Direct Human-to-Robot Transfer (September 18, 2025)
- Pavlakos et al. — Reconstructing Hands in 3D with Transformers (HaMeR, CVPR 2024)
- Physical Intelligence — Emergence of Human to Robot Transfer in Vision-Language-Action Models (December 2025)
- DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation (2025)
- Sunday Robotics — ACT-1: A Robot Foundation Model Trained on Zero Robot Data (November 19, 2025)
Published by Field Motion on November 9, 2026. Figures are as reported by the cited organizations, many from their own evaluations. Field Motion is not affiliated with the cited projects.