Egocentric (a person)

A person recorded from a head-worn rig (smart glasses, a headset, a head-mounted camera) is kind: "human". The person’s hands are two hand parts with keypoints from hand tracking, and the head pose is the body pose.

Channels

Channel	Message	Content
`/cam/ego`	`foxglove.CompressedVideo`	the head camera, the primary view
`/cam/slam_left`, `/cam/slam_right`	`foxglove.CompressedImage`	tracking cameras (optional)
`/head/pose`	`foxglove.PoseInFrame`	head pose in the `world` frame: the body pose, not declared in the overlay
`/hands/left/keypoints`, `/hands/right/keypoints`	`nomadic.Skeleton`	21 hand joints per frame, in the `world` frame
`/hands/left/wrist`, `/hands/right/wrist`	`foxglove.PoseInFrame`	wrist poses with orientation, in the `world` frame

Overlay

overlay.json

{
  "spec_version": "1.1",
  "recording": {
    "id": "kitchen-session-17-take-04",
    "clock": "log_time",
    "t0_ns": 1760000000000000000,
    "task": {"instruction": "Fold the towel and place it on the shelf."},
    "platform": {
      "kind": "human",
      "parts": [
        {"name": "left_hand", "type": "hand", "side": "left",
         "keypoints": {
           "channel": "/hands/left/keypoints",
           "names": ["wrist",
                     "thumb_cmc", "thumb_mcp", "thumb_ip", "thumb_tip",
                     "index_mcp", "index_pip", "index_dip", "index_tip",
                     "middle_mcp", "middle_pip", "middle_dip", "middle_tip",
                     "ring_mcp", "ring_pip", "ring_dip", "ring_tip",
                     "little_mcp", "little_pip", "little_dip", "little_tip"],
           "parents": [-1, 0, 1, 2, 3, 0, 5, 6, 7, 0, 9, 10, 11, 0, 13, 14, 15, 0, 17, 18, 19]},
         "end_pose": {"channel": "/hands/left/wrist"}},
        {"name": "right_hand", "type": "hand", "side": "right",
         "keypoints": {
           "channel": "/hands/right/keypoints",
           "names": ["wrist",
                     "thumb_cmc", "thumb_mcp", "thumb_ip", "thumb_tip",
                     "index_mcp", "index_pip", "index_dip", "index_tip",
                     "middle_mcp", "middle_pip", "middle_dip", "middle_tip",
                     "ring_mcp", "ring_pip", "ring_dip", "ring_tip",
                     "little_mcp", "little_pip", "little_dip", "little_tip"],
           "parents": [-1, 0, 1, 2, 3, 0, 5, 6, 7, 0, 9, 10, 11, 0, 13, 14, 15, 0, 17, 18, 19]},
         "end_pose": {"channel": "/hands/right/wrist"}}
      ]
    }
  },
  "primary_view": "/cam/ego",
  "views": [
    {"channel": "/cam/ego",        "role": "ego",        "label": "Head camera"},
    {"channel": "/cam/slam_left",  "role": "slam_left",  "label": "Tracking left"},
    {"channel": "/cam/slam_right", "role": "slam_right", "label": "Tracking right"}
  ],
  "source": {"device": "head-worn rig", "hand_tracking": "on-device"}
}

Notes

Keypoint layout. names and parents define the skeleton once. Here joint 0 (wrist) is the root, and each finger is a chain from the wrist to its tip. Any joint set works, e.g. 25 joints with forearm and metacarpals, as long as parents forms a tree and every message carries one position per name, in that order. One world frame. Keypoints, wrist poses and the head pose all use the same static, z-up world frame. Many head-worn SDKs use a y-up world: convert positions with (x, y, z) → (x, −z, y) and rotate orientations the same way. See Frames for poses and keypoints. Untracked frames. When a hand leaves the tracker’s view, publish nothing on its channel until it is tracked again. If your tracker reports per-joint confidence, publish it in confidence. Eye tracking. Eye cameras use the roles eye_left and eye_right; a gaze direction can be published as a vector signal.

Nomadic MCAP Specification

Examples

Egocentric (a person)

Channels

Overlay

Notes

​Channels

​Overlay

​Notes

Channels

Overlay

Notes