Buyer's GuideNovember 9, 20265 min readBy Oleg Shlyakhter

Egocentric Datasets for Physical AI Compared: Ego4D, Ego-Exo4D, EgoDex, Human-1M and More

Public egocentric datasets are the starting point for most teams training robots from human video. They differ enormously in what they capture: daily life versus focused manipulation, mono versus stereo, raw video versus 3D hand pose, research-only versus commercial licenses. This guide compares the major options as of November 2026 and explains when public data stops being enough.

TL;DR: Ego4D is the broadest daily-life corpus; Ego-Exo4D adds synchronized third-person views of skilled activity; EgoDex is the strongest public source of manipulation video with 3D hand and finger pose; NVIDIA's Human-1M-100K release brings 100,000 hours of egocentric video of people at work. Public datasets are excellent for research and pretraining experiments, but license terms, task coverage and capture specs often push production teams to commission custom collection.

Key takeaways

  • Check the license first: several flagship egocentric datasets restrict commercial use.
  • Daily-life corpora (Ego4D) maximize diversity; manipulation corpora (EgoDex) maximize action-relevant signal.
  • Native 3D hand pose captured at recording time is rarer and more valuable than pose estimated afterward.
  • 2026 brought much larger human-video releases, led by NVIDIA's Human-1M-100K.
  • Custom collection is justified when you need specific tasks, environments, camera layouts, regions or commercial rights.

Comparison of major egocentric datasets

Selected egocentric datasets relevant to robot learning (figures as published by each project)
DatasetScaleCaptureFocusHand pose
Ego4D3,670 hours; 931 camera wearers; 74 locations; 9 countriesHead-mounted cameras (several device types)Unscripted daily lifeNot native; annotations for selected benchmarks
Ego-Exo4D1,286 hours total video (about 221 egocentric); 740 participantsAria glasses plus 4–5 stationary GoPro exo camerasSkilled activities (cooking, sports, repair and more)Body and hand pose annotations
EgoDex (Apple)829 hours; 338,000 episodes; 194 tasksApple Vision Pro, 1080p at 30 HzTabletop dexterous manipulationNative 3D tracking of upper body, head and every hand joint
Human-1M-100K (NVIDIA)100,000 hours openly released from the Human-1M datasetEgocentric video of people at workWork activities for robot pretrainingSee dataset card
EgoScale corpus (NVIDIA)20,854 hours of action-labeled egocentric human videoEgocentricDexterous manipulation pretrainingAction-labeled
DreamDojo pretraining set (NVIDIA)44,711 hours of human egocentric videoEgocentricWorld-model pretrainingNot the focus

Sources for each row are linked in the table and listed at the end. EgoScale and DreamDojo are included because they describe large egocentric corpora used for robotics research; check each project for what has been released and under what terms.

The datasets in more detail

Ego4D: breadth of daily life

Ego4D remains the reference for diversity: thousands of hours of unscripted first-person video from hundreds of people across many countries. Its strength is the variety of environments and activities. Its limitations for robotics are that most footage is not focused on manipulation and, as the EgoDex authors note, it lacks native hand pose annotations.

Ego-Exo4D: first-person plus third-person

Ego-Exo4D captures skilled human activity with Aria glasses and several synchronized stationary cameras. That multi-view setup makes it valuable for body pose, skill assessment and research on combining egocentric and exocentric views; see our multi-camera capture guide.

EgoDex: manipulation with native 3D hands

Apple's EgoDex is the most robotics-relevant public egocentric dataset for dexterous manipulation. Collected on Apple Vision Pro, it pairs every frame with 3D poses of the head, upper body and 25 joints per hand, plus camera intrinsics and extrinsics, confidence values and language annotations. Its published license is CC BY-NC-ND, which limits commercial use.

Human-1M-100K: the 2026 scale jump

With GR00T 2, NVIDIA describes Human-1M, its egocentric dataset of people at work, and states that 100,000 hours are openly available as Human-1M-100K on Hugging Face. GR00T 2's pretraining used 402,730 hours of egocentric human video from Human-1M. Review the dataset card for license and content details before use.

How to evaluate an egocentric dataset for robotics

License
Commercial training rights, derivatives, redistribution.
Task relevance
Share of footage showing manipulation you care about.
Hand signal
Native 3D tracking, estimated pose, or none.
Geometry
Mono, stereo, depth, device pose, calibration.
Diversity
People, environments, regions, objects.
Structure
Episode boundaries, language labels, outcomes.

Public vs. custom egocentric data

When to use public datasets and when to commission collection
FactorPublic datasetsCustom collection
Cost to startFree or lowProject budget
Commercial rightsVaries; often non-commercialNegotiated for your use
Task and environment fitWhatever was collectedDesigned for your robot and deployment
Camera layoutFixed by the original projectMono, stereo, multicam or UMI to match your robot
ExclusivityShared with competitorsPossible
Time to first dataImmediateWeeks for a pilot

A common pattern is to prototype on public data, identify the gaps in tasks, environments or signals, and then commission collection that fills exactly those gaps. For the sizing side of that decision, see how many demonstrations a robot needs.

Buyer's tip: read the license and the capture spec before the hour count. An hour you cannot legally train on, or that lacks the signal your model needs, is worth nothing.

Field Motion custom egocentric collection

Field Motion collects custom egocentric data for robotics: mono, stereo, multi-camera and UMI-style, through contributors in Latin America, Southeast Asia and the United States. Programs are scoped around your tasks, environments, camera layout, consent and usage terms, and delivery format. Read stereo vs. mono for capture choices, or visit the Field Motion homepage.

Frequently asked questions

What is the largest egocentric dataset for robotics?

It depends on what is publicly released versus described. Among openly released data, NVIDIA states that 100,000 hours of its Human-1M egocentric dataset are available as Human-1M-100K. Companies have also described larger private corpora, such as the 402,730 hours used in GR00T 2 pretraining and over 1 million hours reported by Dyna Robotics.

Which egocentric dataset has 3D hand pose?

Apple's EgoDex provides 3D poses for the head, upper body and 25 joints per hand, captured natively on Apple Vision Pro, across 829 hours and 194 tabletop tasks. Ego-Exo4D also provides body and hand pose annotations.

Can I use Ego4D or EgoDex for commercial robot training?

Check each dataset's license carefully. EgoDex is published under CC BY-NC-ND, which restricts commercial use, and Ego4D access is governed by its own license agreement. When commercial rights matter, custom collection is often required.

What is the difference between Ego4D and Ego-Exo4D?

Ego4D is a large corpus of unscripted daily-life first-person video. Ego-Exo4D focuses on skilled activities and pairs Aria glasses with 4–5 synchronized stationary cameras for third-person views.

When should a robotics team commission custom egocentric data?

When public datasets lack the needed tasks, environments, regions, camera layout, hand or depth signals, or commercial rights. Field Motion runs custom egocentric collection scoped to those requirements.

OS
Oleg Shlyakhter
Co-Founder & CEO, Field Motion

Outgrown public egocentric datasets?

Tell us which tasks, environments and signals are missing. We'll scope a custom collection with the rights you need.

Talk to Field Motion

Related reading