Dexterous hands

A bimanual robot with dexterous hands declares each hand as a hand part with its joint angles, and its tactile or fingertip force sensing as the hand’s force. This example is two Sharpa Wave hands (22 DoF each) on two 7-DoF arms, using the recognised embodiment sharpa_wave_bimanual, with the recording’s subtasks as segments.

Channels

Channel	Message	Content
`/cam/top`, `/cam/front`	`foxglove.CompressedVideo`	scene cameras
`/left_arm/flange_pose`, `/right_arm/flange_pose`	`foxglove.PoseInFrame`	hand-flange poses in the robot base frame
`/sig/left_hand_joints`, `/sig/right_hand_joints`	`nomadic.Signal`	22 joint angles per hand
`/sig/left_fingertip_wrench`, `/sig/right_fingertip_wrench`	`nomadic.Signal`	a 6-axis wrench per fingertip, 30 values per hand

Overlay

overlay.json

{
  "spec_version": "1.1",
  "recording": {
    "id": "dexterous-wipe-000318",
    "clock": "log_time",
    "t0_ns": 1760000000000000000,
    "task": {
      "instruction": "Pick up the plate and wipe it with the eraser.",
      "scene": "A plate with marks and an eraser are on the table.",
      "success_criteria": "The marks are removed and both objects are back on the table."
    },
    "segments": [
      {"start_s": 0.0,  "end_s": 21.9, "text": "Pick up the plate with the left hand.", "label": "subtask", "skill": "pick", "id": "subtask-001"},
      {"start_s": 21.9, "end_s": 24.3, "text": "Pick up the eraser with the right hand.", "label": "subtask", "skill": "pick", "id": "subtask-002"},
      {"start_s": 24.3, "end_s": 41.0, "text": "Wipe the plate with the eraser.", "label": "subtask", "skill": "wipe", "id": "subtask-003"}
    ],
    "platform": {
      "kind": "manipulator",
      "embodiment": "sharpa_wave_bimanual",
      "parts": [
        {"name": "left_arm", "type": "arm", "side": "left",
         "end_pose": {"channel": "/left_arm/flange_pose"}},
        {"name": "right_arm", "type": "arm", "side": "right",
         "end_pose": {"channel": "/right_arm/flange_pose"}},
        {"name": "left_hand", "type": "hand", "side": "left",
         "joints": {"signal": "left_hand_joints",
                    "names": ["left_thumb_CMC_FE", "left_thumb_CMC_AA", "left_thumb_MCP_FE", "left_thumb_MCP_AA",
                              "left_thumb_IP", "left_index_MCP_FE", "left_index_MCP_AA", "left_index_PIP",
                              "left_index_DIP", "left_middle_MCP_FE", "left_middle_MCP_AA", "left_middle_PIP",
                              "left_middle_DIP", "left_ring_MCP_FE", "left_ring_MCP_AA", "left_ring_PIP",
                              "left_ring_DIP", "left_pinky_CMC", "left_pinky_MCP_FE", "left_pinky_MCP_AA",
                              "left_pinky_PIP", "left_pinky_DIP"]},
         "force":  {"signal": "left_fingertip_wrench"}},
        {"name": "right_hand", "type": "hand", "side": "right",
         "joints": {"signal": "right_hand_joints",
                    "names": ["right_thumb_CMC_FE", "right_thumb_CMC_AA", "right_thumb_MCP_FE", "right_thumb_MCP_AA",
                              "right_thumb_IP", "right_index_MCP_FE", "right_index_MCP_AA", "right_index_PIP",
                              "right_index_DIP", "right_middle_MCP_FE", "right_middle_MCP_AA", "right_middle_PIP",
                              "right_middle_DIP", "right_ring_MCP_FE", "right_ring_MCP_AA", "right_ring_PIP",
                              "right_ring_DIP", "right_pinky_CMC", "right_pinky_MCP_FE", "right_pinky_MCP_AA",
                              "right_pinky_PIP", "right_pinky_DIP"]},
         "force":  {"signal": "right_fingertip_wrench"}}
      ]
    }
  },
  "primary_view": "/cam/top",
  "views": [
    {"channel": "/cam/top",   "role": "top",        "label": "Overhead"},
    {"channel": "/cam/front", "role": "front",      "label": "Front"}
  ],
  "signals": [
    {"channel": "/sig/left_hand_joints", "name": "left_hand_joints", "type": "vector", "dim": 22,
     "unit": "rad", "rate_hz": 100,
     "components": ["left_thumb_CMC_FE", "left_thumb_CMC_AA", "left_thumb_MCP_FE", "left_thumb_MCP_AA",
                    "left_thumb_IP", "left_index_MCP_FE", "left_index_MCP_AA", "left_index_PIP",
                    "left_index_DIP", "left_middle_MCP_FE", "left_middle_MCP_AA", "left_middle_PIP",
                    "left_middle_DIP", "left_ring_MCP_FE", "left_ring_MCP_AA", "left_ring_PIP",
                    "left_ring_DIP", "left_pinky_CMC", "left_pinky_MCP_FE", "left_pinky_MCP_AA",
                    "left_pinky_PIP", "left_pinky_DIP"]},
    {"channel": "/sig/right_hand_joints", "name": "right_hand_joints", "type": "vector", "dim": 22,
     "unit": "rad", "rate_hz": 100,
     "components": ["right_thumb_CMC_FE", "right_thumb_CMC_AA", "right_thumb_MCP_FE", "right_thumb_MCP_AA",
                    "right_thumb_IP", "right_index_MCP_FE", "right_index_MCP_AA", "right_index_PIP",
                    "right_index_DIP", "right_middle_MCP_FE", "right_middle_MCP_AA", "right_middle_PIP",
                    "right_middle_DIP", "right_ring_MCP_FE", "right_ring_MCP_AA", "right_ring_PIP",
                    "right_ring_DIP", "right_pinky_CMC", "right_pinky_MCP_FE", "right_pinky_MCP_AA",
                    "right_pinky_PIP", "right_pinky_DIP"]},
    {"channel": "/sig/left_fingertip_wrench", "name": "left_fingertip_wrench", "type": "vector", "dim": 30,
     "rate_hz": 100, "note": "one wrench per fingertip, N and N*m",
     "components": ["thumb_fx", "thumb_fy", "thumb_fz", "thumb_tx", "thumb_ty", "thumb_tz",
                    "index_fx", "index_fy", "index_fz", "index_tx", "index_ty", "index_tz",
                    "middle_fx", "middle_fy", "middle_fz", "middle_tx", "middle_ty", "middle_tz",
                    "ring_fx", "ring_fy", "ring_fz", "ring_tx", "ring_ty", "ring_tz",
                    "pinky_fx", "pinky_fy", "pinky_fz", "pinky_tx", "pinky_ty", "pinky_tz"]},
    {"channel": "/sig/right_fingertip_wrench", "name": "right_fingertip_wrench", "type": "vector", "dim": 30,
     "rate_hz": 100, "note": "one wrench per fingertip, N and N*m",
     "components": ["thumb_fx", "thumb_fy", "thumb_fz", "thumb_tx", "thumb_ty", "thumb_tz",
                    "index_fx", "index_fy", "index_fz", "index_tx", "index_ty", "index_tz",
                    "middle_fx", "middle_fy", "middle_fz", "middle_tx", "middle_ty", "middle_tz",
                    "ring_fx", "ring_fy", "ring_fz", "ring_tx", "ring_ty", "ring_tz",
                    "pinky_fx", "pinky_fy", "pinky_fz", "pinky_tx", "pinky_ty", "pinky_tz"]}
  ]
}

Notes

Joint names. For a recognised embodiment, joint names MUST match the robot’s URDF joint names, in the order of your vector. For any other hand, use your own names.
Arms without joints. Each arm here carries only its flange pose. Add joints when you log the arm’s joint angles.
Fingertip force forms. A full wrench per fingertip uses components <fingertip>_fx … <fingertip>_tz, six per fingertip. A sensor that measures only normal force uses the contact-force form instead: unit: "N", one value per fingertip, e.g. components ["thumb", "index", "middle", "ring", "pinky"]. See Force.
Hands on arms. left_hand is paired with left_arm, whose end_pose is the hand flange, so the hands need no end_pose of their own.

Nomadic MCAP Specification

Examples

Dexterous hands

Channels

Overlay

Notes

​Channels

​Overlay

​Notes

Channels

Overlay

Notes