thirty tasks, working factories across a dozen sectors
Task specimens.
Map 900 four-second windows back to the same 30 hand-selected clips. Under each clip: the concrete manipulation problem that makes the source episode useful for training or evaluation.
t-SNE projection of four-second window captions. Axes carry no units; nearby points describe similar work.
published excerpts
Open all 30 specimens.
Each five-second excerpt is selected from one of the source episodes represented in the map.
01 glass works
Wrapping glassware for the box
Corrugated sleeves over fragile stock; grip force gets no second try.
02 appliance line
Fitting parts on an appliance line
Small device, moving belt, both hands busy at once.
03 assembly bench
Marking rings at the bench
Identical parts, tiny registration marks, zero tolerance for drift.
04 industrial qc
Counting impellers against the sheet
Count, check, log: perception and bookkeeping in one loop.
05 garment line
Trimming threads on knitwear
Snips against soft knit; one wrong cut shows forever.
06 knitting line
Feeding the knitting machine
Yarn tension by feel, machine timing by ear.
07 packaging line
Folding cartons for pouches
Flat blank to box in seconds, creases from muscle memory.
08 ceramics works
Finishing ceramic ware
Unfired clay marks at a touch; pressure is the whole skill.
09 electronics bench
Calibrating a camera module
Checkerboard, module, micro-adjust: sub-millimetre alignment, on camera.
10 pleating bench
Pleating fabric with a blade
A straightedge, a knife, and folds that must repeat exactly.
11 moulding bench
Filling a moulded tray
Each cavity takes one part, seated flush or not at all.
12 assembly line
Seating strip into extrusion
Long flexible part, narrow channel, continuous press-fit.
13 lampshade bench
Folding a pleated shade
Accordion folds that collapse if the order is wrong.
14 wax works
Banding wax blocks
Rubber bands around soft blocks: deformation on every grip.
15 garment line
Folding menswear to spec
The same fold every time; the pile is the QC.
16 leather works
Skiving leather at the bench
Blade angle by feel; one pass decides the edge.
17 garment line
Threading cords through hoods
A drawstring through a channel, blind for most of the run.
18 electronics bench
Testing a handheld unit
Cluttered bench, live device, probe and screen in parallel.
19 footwear line
Pressing shoe insoles
Layered soft parts aligned by hand before the press.
20 stainless works
Deburring under the machine
Gloved work under a running head; force where the camera can't see.
21 upholstery bench
Fitting a leather panel
Pry, stretch, seat: a stiff panel into a tight housing.
22 lockworks
Sorting padlock parts
A bowl of near-identical brass parts, picked by touch.
23 appliance assembly
Wiring appliance lids
A harness routed through housings, connectors seated blind.
24 cleanroom lab
Pipetting in the cleanroom
Gloved pipette work into microtubes; contamination is failure.
25 metal shop
Forming sheet metal
Rivets and sheet stock: high force, exact placement.
26 glass works
Bubble-wrapping washed glass
Wet, clear, slippery, and invisible to depth sensors.
27 packing floor
Strapping a bundle
Tension a strap around a soft bundle without crushing it.
28 footwear line
Bonding sole to upper
Machine-paced stitching where two curved parts meet.
29 print shop
Collating and cutting pages
Ruler-true cuts on stacks that shift as they square.
30 garment line
Attaching buttons and zips
A press machine, a placement mark, and no undo.
five workflows · 789 steps · 21-point hands
Annotation, running live.
Five two-minute egocentric workflows from five factories, with every step labelled by hand, verb, and object. Step boundaries come from the hand kinematics themselves; the curve above the timeline is that motion signal, so you can check where the cuts land. Press K to toggle the per-frame hand keypoints.
Annotation timeline
| time | hand | step | verb | conf |
|---|
Delivered in the format you already load.
Datasets ship as LeRobot: parquet shards under data/, encoded video under videos/, and meta/info.json carrying features, fps and the episode index. Nothing bespoke to write a parser for.
lerobot_ego_exo/
data/chunk-000/file-000.parquet
videos/observation.images.ego_0_camera/...
videos/observation.images.exo_0_camera/...
meta/info.json features, fps, robot_type
meta/episodes/... per-episode index
meta/tasks.json task_index -> task string