Egocentric Datasets for Physical AI Compared: Ego4D, Ego-Exo4D, EgoDex, Human-1M and More
Public egocentric datasets are the starting point for most teams training robots from human video. They differ enormously in what they capture: daily life versus focused manipulation, mono versus stereo, raw video versus 3D hand pose, research-only versus commercial licenses. This guide compares the major options as of November 2026 and explains when public data stops being enough.
TL;DR: Ego4D is the broadest daily-life corpus; Ego-Exo4D adds synchronized third-person views of skilled activity; EgoDex is the strongest public source of manipulation video with 3D hand and finger pose; NVIDIA's Human-1M-100K release brings 100,000 hours of egocentric video of people at work. Public datasets are excellent for research and pretraining experiments, but license terms, task coverage and capture specs often push production teams to commission custom collection.
Key takeaways
- Check the license first: several flagship egocentric datasets restrict commercial use.
- Daily-life corpora (Ego4D) maximize diversity; manipulation corpora (EgoDex) maximize action-relevant signal.
- Native 3D hand pose captured at recording time is rarer and more valuable than pose estimated afterward.
- 2026 brought much larger human-video releases, led by NVIDIA's Human-1M-100K.
- Custom collection is justified when you need specific tasks, environments, camera layouts, regions or commercial rights.
Comparison of major egocentric datasets
| Dataset | Scale | Capture | Focus | Hand pose |
|---|---|---|---|---|
| Ego4D | 3,670 hours; 931 camera wearers; 74 locations; 9 countries | Head-mounted cameras (several device types) | Unscripted daily life | Not native; annotations for selected benchmarks |
| Ego-Exo4D | 1,286 hours total video (about 221 egocentric); 740 participants | Aria glasses plus 4–5 stationary GoPro exo cameras | Skilled activities (cooking, sports, repair and more) | Body and hand pose annotations |
| EgoDex (Apple) | 829 hours; 338,000 episodes; 194 tasks | Apple Vision Pro, 1080p at 30 Hz | Tabletop dexterous manipulation | Native 3D tracking of upper body, head and every hand joint |
| Human-1M-100K (NVIDIA) | 100,000 hours openly released from the Human-1M dataset | Egocentric video of people at work | Work activities for robot pretraining | See dataset card |
| EgoScale corpus (NVIDIA) | 20,854 hours of action-labeled egocentric human video | Egocentric | Dexterous manipulation pretraining | Action-labeled |
| DreamDojo pretraining set (NVIDIA) | 44,711 hours of human egocentric video | Egocentric | World-model pretraining | Not the focus |
Sources for each row are linked in the table and listed at the end. EgoScale and DreamDojo are included because they describe large egocentric corpora used for robotics research; check each project for what has been released and under what terms.
The datasets in more detail
Ego4D: breadth of daily life
Ego4D remains the reference for diversity: thousands of hours of unscripted first-person video from hundreds of people across many countries. Its strength is the variety of environments and activities. Its limitations for robotics are that most footage is not focused on manipulation and, as the EgoDex authors note, it lacks native hand pose annotations.
Ego-Exo4D: first-person plus third-person
Ego-Exo4D captures skilled human activity with Aria glasses and several synchronized stationary cameras. That multi-view setup makes it valuable for body pose, skill assessment and research on combining egocentric and exocentric views; see our multi-camera capture guide.
EgoDex: manipulation with native 3D hands
Apple's EgoDex is the most robotics-relevant public egocentric dataset for dexterous manipulation. Collected on Apple Vision Pro, it pairs every frame with 3D poses of the head, upper body and 25 joints per hand, plus camera intrinsics and extrinsics, confidence values and language annotations. Its published license is CC BY-NC-ND, which limits commercial use.
Human-1M-100K: the 2026 scale jump
With GR00T 2, NVIDIA describes Human-1M, its egocentric dataset of people at work, and states that 100,000 hours are openly available as Human-1M-100K on Hugging Face. GR00T 2's pretraining used 402,730 hours of egocentric human video from Human-1M. Review the dataset card for license and content details before use.
How to evaluate an egocentric dataset for robotics
Public vs. custom egocentric data
| Factor | Public datasets | Custom collection |
|---|---|---|
| Cost to start | Free or low | Project budget |
| Commercial rights | Varies; often non-commercial | Negotiated for your use |
| Task and environment fit | Whatever was collected | Designed for your robot and deployment |
| Camera layout | Fixed by the original project | Mono, stereo, multicam or UMI to match your robot |
| Exclusivity | Shared with competitors | Possible |
| Time to first data | Immediate | Weeks for a pilot |
A common pattern is to prototype on public data, identify the gaps in tasks, environments or signals, and then commission collection that fills exactly those gaps. For the sizing side of that decision, see how many demonstrations a robot needs.
Buyer's tip: read the license and the capture spec before the hour count. An hour you cannot legally train on, or that lacks the signal your model needs, is worth nothing.
Field Motion custom egocentric collection
Field Motion collects custom egocentric data for robotics: mono, stereo, multi-camera and UMI-style, through contributors in Latin America, Southeast Asia and the United States. Programs are scoped around your tasks, environments, camera layout, consent and usage terms, and delivery format. Read stereo vs. mono for capture choices, or visit the Field Motion homepage.
Frequently asked questions
What is the largest egocentric dataset for robotics?
It depends on what is publicly released versus described. Among openly released data, NVIDIA states that 100,000 hours of its Human-1M egocentric dataset are available as Human-1M-100K. Companies have also described larger private corpora, such as the 402,730 hours used in GR00T 2 pretraining and over 1 million hours reported by Dyna Robotics.
Which egocentric dataset has 3D hand pose?
Apple's EgoDex provides 3D poses for the head, upper body and 25 joints per hand, captured natively on Apple Vision Pro, across 829 hours and 194 tabletop tasks. Ego-Exo4D also provides body and hand pose annotations.
Can I use Ego4D or EgoDex for commercial robot training?
Check each dataset's license carefully. EgoDex is published under CC BY-NC-ND, which restricts commercial use, and Ego4D access is governed by its own license agreement. When commercial rights matter, custom collection is often required.
What is the difference between Ego4D and Ego-Exo4D?
Ego4D is a large corpus of unscripted daily-life first-person video. Ego-Exo4D focuses on skilled activities and pairs Aria glasses with 4–5 synchronized stationary cameras for third-person views.
When should a robotics team commission custom egocentric data?
When public datasets lack the needed tasks, environments, regions, camera layout, hand or depth signals, or commercial rights. Field Motion runs custom egocentric collection scoped to those requirements.
Outgrown public egocentric datasets?
Tell us which tasks, environments and signals are missing. We'll scope a custom collection with the rights you need.
Talk to Field MotionRelated reading
Sources
- Grauman et al. — Ego4D: Around the World in 3,000 Hours of Egocentric Video
- Meta AI — Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
- Apple — EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video (2025)
- NVIDIA GEAR — GR00T 2: Scaling Robot Foundation Models for Generalization, Dexterity, and Adaptation (November 2026)
- NVIDIA — EgoScale (February 19, 2026)
- NVIDIA — DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos (February 2026)
- Dyna Robotics — DYNA-2 World-Action Model announcement (August 10, 2026)
Published by Field Motion on November 9, 2026. Figures are as reported by the cited organizations, many from their own evaluations. Field Motion is not affiliated with the cited projects.