Foxglove message schemas
Views, point clouds, transforms, intrinsics, poses and GPS use Foxglove’s well-known schemas, unchanged. Their definitions are in the Foxglove schema reference, and Python classes ship infoxglove-schemas-protobuf.
| Schema | Used for |
|---|---|
foxglove.CompressedVideo, foxglove.CompressedImage, foxglove.RawImage | camera views |
foxglove.PointCloud | LiDAR and radar |
foxglove.FrameTransform | sensor extrinsics |
foxglove.CameraCalibration | camera intrinsics |
foxglove.PoseInFrame | body pose and part poses |
foxglove.LocationFix | GPS |
Nomadic message schemas
Two messages cover what Foxglove has no schema for. Save each definition as a.proto file and
generate classes for your language, e.g. for Python:
pip install grpcio-tools
python -m grpc_tools.protoc -I . --python_out=. nomadic_signal.proto nomadic_skeleton.proto
nomadic.Signal
One sample of one signal.
nomadic_signal.proto
syntax = "proto3";
package nomadic;
import "google/protobuf/timestamp.proto";
// One sample of one signal.
//
// Deliberately minimal: the signal's name, type, unit, enum values and any
// vector component labels live in the `nomadic_spec` overlay, not here. This
// message carries only a timestamp and one value, so Foxglove's Plot panel
// (numeric) and State Transitions panel (categorical) can render it by message
// path with no knowledge of nomadic_spec. Foxglove has no telemetry schema of
// its own; this is the one small schema we add on top of the well-known set.
message Signal {
google.protobuf.Timestamp timestamp = 1;
// Exactly one is set per sample, matching the signal's declared `type`.
oneof value {
double number = 2; // float / int
bool flag = 3; // bool
string text = 4; // string / enum (renders in State Transitions)
Vector vector = 5; // vector — one line per component in Plot
}
}
message Vector {
repeated double values = 1;
}
nomadic.Skeleton
One frame of 3D keypoints for one body part.
nomadic_skeleton.proto
syntax = "proto3";
package nomadic;
import "google/protobuf/timestamp.proto";
// One frame of 3D keypoints for one body part (e.g. one hand), as estimated by
// hand / body tracking.
//
// Like nomadic.Signal, deliberately minimal: the topology (joint names and the
// parent tree) is static and is declared once in the `nomadic_spec` overlay
// (`recording.platform.parts[].keypoints`), not repeated per message. Positions
// are in the order the overlay's `names` lists, in metres, in the frame named by
// `frame_id` (right-handed, z-up). Foxglove has no skeleton schema; this is the
// one geometry schema we add next to the well-known set.
message Skeleton {
google.protobuf.Timestamp timestamp = 1;
// Frame the positions are expressed in, e.g. a static world frame.
string frame_id = 2;
// One position per declared joint. Point3 is wire-compatible with
// foxglove.Vector3.
repeated Point3 positions = 3;
// Optional per-joint confidence in [0, 1]: either empty or one value per
// position.
repeated float confidence = 4;
}
message Point3 {
double x = 1;
double y = 2;
double z = 3;
}
Overlay JSON Schema
The machine-readable definition of thenomadic_spec overlay
(JSON Schema 2020-12). It covers field shapes; the rules a schema cannot express are checked by
preflight.
nomadic_spec_v1.schema.json
nomadic_spec_v1.schema.json
nomadic_spec_v1.schema.json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://nomadicml.com/schemas/nomadic_spec/v1.json",
"title": "nomadic_spec v1",
"description": "The semantic overlay carried in an MCAP metadata record named `nomadic_spec`, under the key `spec`, as a JSON string. In the Foxglove-native format the data itself is Foxglove well-known messages (CompressedImage/CompressedVideo, PointCloud, FrameTransform, CameraCalibration) and a nomadic.Signal per sample; this overlay carries only what those schemas cannot express - view roles, point-cloud kind, signal semantics, the profile, and recording/platform metadata. Geometry (transforms) and camera intrinsics are messages now, validated at ingest against the data, not here. The value rules a JSON Schema cannot express - coordinate frame, origin, units, clock - are the Nomadic MCAP Specification: https://docs.nomadicml.com/mcap-spec/overview",
"type": "object",
"required": ["spec_version", "recording"],
"additionalProperties": false,
"properties": {
"spec_version": {
"description": "Version of this schema the document conforms to.",
"type": "string",
"enum": ["1.0", "1.1"]
},
"profile": {
"description": "Opt-in named contract that unlocks semantics-dependent features. Null or absent means generic handling: every signal is still ingested, stored, charted and searchable.",
"type": ["string", "null"]
},
"recording": {"$ref": "#/$defs/recording"},
"primary_view": {
"description": "Channel of the default view. Must name a declared view.",
"type": ["string", "null"]
},
"views": {"type": "array", "items": {"$ref": "#/$defs/view"}},
"signals": {"type": "array", "items": {"$ref": "#/$defs/signal"}},
"point_clouds": {"type": "array", "items": {"$ref": "#/$defs/point_cloud"}},
"source": {
"description": "Free-form provenance. Never interpreted by the platform, so it is kept out of `recording` where it could be mistaken for a contract field.",
"type": "object"
}
},
"allOf": [
{
"$comment": "Spec 1.0: mount is required and is the only body field; embodiment only on robot_arm or heavy_equipment.",
"if": {"properties": {"spec_version": {"const": "1.0"}}, "required": ["spec_version"]},
"then": {"properties": {"recording": {"properties": {"platform": {
"required": ["mount"],
"not": {"anyOf": [{"required": ["kind"]}, {"required": ["body_dimensions"]}, {"required": ["parts"]}]},
"if": {"required": ["embodiment"]},
"then": {"properties": {"mount": {"enum": ["robot_arm", "heavy_equipment"]}}}
}}}}}
},
{
"$comment": "Spec 1.1: kind names the body (omitted = sensor-only). ego_platform only for a road vehicle; footprint only for a moving base; embodiment needs a body.",
"if": {"properties": {"spec_version": {"const": "1.1"}}, "required": ["spec_version"]},
"then": {"properties": {"recording": {"properties": {"platform": {"allOf": [
{
"if": {"required": ["ego_platform"]},
"then": {"anyOf": [
{"properties": {"kind": {"const": "vehicle"}}, "required": ["kind"]},
{"properties": {"mount": {"const": "vehicle"}}, "required": ["mount"], "not": {"required": ["kind"]}}
]}
},
{
"if": {"anyOf": [{"required": ["body_dimensions"]}, {"required": ["vehicle_dimensions"]}]},
"then": {"anyOf": [
{"properties": {"kind": {"enum": ["vehicle", "heavy_vehicle", "mobile_robot", "humanoid", "human"]}}, "required": ["kind"]},
{"properties": {"mount": {"enum": ["vehicle", "heavy_equipment"]}}, "required": ["mount"], "not": {"required": ["kind"]}}
]}
},
{"properties": {"embodiment": {"pattern": "\\S"}}},
{
"if": {"required": ["embodiment"]},
"then": {"anyOf": [
{"required": ["kind"]},
{"properties": {"mount": {"enum": ["vehicle", "robot_arm", "heavy_equipment"]}}, "required": ["mount"]}
]}
},
{
"if": {"properties": {"parts": {"minItems": 1}}, "required": ["parts"]},
"then": {"anyOf": [
{"required": ["kind"]},
{"properties": {"mount": {"enum": ["vehicle", "robot_arm", "heavy_equipment"]}}, "required": ["mount"]}
]}
}
]}}}}}
}
],
"anyOf": [
{"properties": {"views": {"minItems": 1}}, "required": ["views"]},
{
"$comment": "Spec 1.1: a body's parts carry data (keypoint and pose channels) of their own.",
"properties": {"spec_version": {"const": "1.1"}, "recording": {"properties": {"platform": {"properties": {"parts": {"minItems": 1}}, "required": ["parts"]}}, "required": ["platform"]}},
"required": ["spec_version", "recording"]
},
{"properties": {"point_clouds": {"minItems": 1}}, "required": ["point_clouds"]},
{"properties": {"signals": {"minItems": 1}}, "required": ["signals"]}
],
"$defs": {
"channel": {
"description": "MCAP topic. Must exist in the file.",
"type": "string",
"minLength": 1,
"pattern": "^/[!-~]*$"
},
"recording": {
"type": "object",
"required": ["id", "clock", "t0_ns", "platform"],
"additionalProperties": false,
"properties": {
"id": {
"description": "Stable producer-side identifier. The join key when attaching a later recording to this one, so it must be stable across re-exports.",
"type": "string",
"minLength": 1
},
"clock": {
"description": "Which MCAP timestamp is authoritative.",
"type": "string",
"enum": ["log_time", "publish_time"]
},
"t0_ns": {
"description": "Earliest timestamp in the recording. Absolute Unix epoch nanoseconds is strongly preferred and is REQUIRED for the attach flow, because a relative clock cannot be aligned to another recording verifiably.",
"type": "integer",
"minimum": 0
},
"description": {"type": "string"},
"task": {"$ref": "#/$defs/task"},
"segments": {
"description": "Time-segmented annotations of the recording, e.g. the subtasks of its task. Each segment is a span in seconds from t0_ns (end exclusive) with its text; written onto every video of the recording, on the video's clock.",
"type": "array",
"items": {"$ref": "#/$defs/segment"}
},
"platform": {"$ref": "#/$defs/platform"}
}
},
"task": {
"description": "What the recording was made to do, one instruction for the whole recording. On a vehicle mount it is the driven route (shown as the recording's route and used as the driving model's navigation input); on a robot or machine mount it is the task (shown as the recording's task).",
"type": "object",
"required": ["instruction"],
"additionalProperties": false,
"properties": {
"instruction": {
"description": "The instruction as natural-language text, shown as written.",
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"scene": {
"description": "Optional: the scene the task starts from (objects, layout), as natural-language text.",
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"success_criteria": {
"description": "Optional: what counts as the task done, as natural-language text.",
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
}
},
"segment": {
"type": "object",
"required": ["start_s", "end_s", "text"],
"additionalProperties": false,
"properties": {
"start_s": {"description": "Seconds from recording.t0_ns.", "type": "number", "minimum": 0},
"end_s": {"description": "Seconds from recording.t0_ns, end exclusive. MUST be greater than start_s: a cross-field rule JSON Schema cannot express, enforced by the platform validator (preflight and ingest reject segment.end_s.invalid).", "type": "number", "exclusiveMinimum": 0},
"text": {"type": "string", "minLength": 1, "pattern": "\\S"},
"label": {"description": "The annotation type, e.g. subtask.", "type": "string"},
"skill": {"description": "A compact skill label, e.g. pick, place, wipe.", "type": "string"},
"id": {"description": "The producer's identifier for the segment.", "type": "string"}
}
},
"platform": {
"description": "What body the recording comes from. Spec 1.1 names it with `kind`; spec 1.0 with `mount` (still accepted in 1.1 when it agrees with `kind`). The version-dependent rules (which fields each version allows) sit at the document root, where the spec_version is visible.",
"type": "object",
"additionalProperties": false,
"properties": {
"kind": {
"description": "Spec 1.1. The body class: vehicle = road vehicle; heavy_vehicle = construction or agricultural machine; manipulator = fixed-base robot (one or more arms); mobile_robot = wheeled or legged robot, with or without arms; humanoid = whole-body robot; human = a person (e.g. egocentric footage). Omit it for a sensor-only recording such as a roadside site. Arms, hands and other body parts are declared separately, not encoded in the kind.",
"type": "string",
"enum": ["vehicle", "heavy_vehicle", "manipulator", "mobile_robot", "humanoid", "human"]
},
"mount": {
"description": "Spec 1.0 name for the body. vehicle = ego body frame that moves; roadside = fixed site frame with no egomotion; robot_arm = a stationary manipulator base frame with no egomotion; heavy_equipment = a mobile machine whose primary control surface is a manipulator (excavator, loader, dozer). Maps to kind: vehicle -> vehicle, heavy_equipment -> heavy_vehicle, robot_arm -> manipulator, roadside -> no kind.",
"type": "string",
"enum": ["vehicle", "roadside", "robot_arm", "heavy_equipment"]
},
"ego_platform": {
"description": "Selects the detector tuned for this vehicle class. Declaring it replaces the backend's provenance-tag prefix matching. Road vehicles only.",
"type": "string",
"enum": ["sedan", "truck"]
},
"body_dimensions": {
"description": "Spec 1.1. Explicit ego footprint in metres (length x-forward, width y-left, height z-up) of a body with a moving base (every kind except manipulator). Overrides the coarse ego_platform preset for the ego cuboid and corridor sizing.",
"$ref": "#/$defs/dimensions"
},
"vehicle_dimensions": {
"description": "Spec 1.0 name for body_dimensions. Explicit ego footprint in metres (length x-forward, width y-left, height z-up). Overrides the coarse ego_platform preset for the ego cuboid and corridor sizing; use when the real vehicle size is known but is neither sedan nor truck (e.g. an AMR). Vehicle mounts only in 1.0.",
"$ref": "#/$defs/dimensions"
},
"parts": {
"description": "Spec 1.1. The body's parts (arm, hand, leg, head, torso, body), each with the data slots it carries. One vocabulary for every embodiment: a Franka is one arm, a bimanual robot two, EgoDex two hands with keypoints, a humanoid arms, hands, legs, torso and head. Cross-references are checked by the validator, not this schema: declared signals with the right type and unit, joint-name counts, a keypoint tree (parents integral, no cycles), unique part names and channels used once across views, signals, point clouds and parts.",
"type": "array",
"items": {"$ref": "#/$defs/part"}
},
"embodiment": {
"description": "The model of the body within its kind (e.g. franka_panda, excavator, cat_336). In 1.0 only for robot_arm or heavy_equipment mounts; in 1.1 for any kind, but not for a sensor-only recording.",
"type": "string"
}
},
"allOf": [
{
"if": {"properties": {"mount": {"enum": ["roadside", "robot_arm", "heavy_equipment"]}}, "required": ["mount"]},
"then": {"not": {"required": ["ego_platform"]}}
},
{
"if": {"properties": {"kind": {"not": {"const": "vehicle"}}}, "required": ["kind"]},
"then": {"not": {"required": ["ego_platform"]}}
},
{"not": {"required": ["body_dimensions", "vehicle_dimensions"]}},
{
"if": {"properties": {"mount": {"const": "vehicle"}}, "required": ["mount", "kind"]},
"then": {"properties": {"kind": {"const": "vehicle"}}}
},
{
"if": {"properties": {"mount": {"const": "heavy_equipment"}}, "required": ["mount", "kind"]},
"then": {"properties": {"kind": {"const": "heavy_vehicle"}}}
},
{
"if": {"properties": {"mount": {"const": "robot_arm"}}, "required": ["mount", "kind"]},
"then": {"properties": {"kind": {"const": "manipulator"}}}
},
{
"if": {"properties": {"mount": {"const": "roadside"}}, "required": ["mount"]},
"then": {"not": {"required": ["kind"]}}
}
]
},
"part": {
"type": "object",
"required": ["name", "type"],
"additionalProperties": false,
"properties": {
"name": {"description": "Unique within the recording; also the trajectory track name.", "type": "string", "pattern": "^[a-z][a-z0-9_]*$", "not": {"pattern": "\\n"}},
"type": {"type": "string", "enum": ["arm", "hand", "leg", "head", "torso", "body"]},
"side": {"type": "string", "enum": ["left", "right"]},
"joints": {
"description": "Measured joint state: a declared vector signal in rad (m for prismatic joints), names in the vector's order.",
"type": "object", "required": ["signal", "names"], "additionalProperties": false,
"properties": {
"signal": {"type": "string", "minLength": 1},
"names": {"type": "array", "minItems": 1, "uniqueItems": true, "items": {"type": "string", "pattern": "\\S"}}
}
},
"end_pose": {
"description": "Pose of the part's tip (end effector, wrist, foot) as a foxglove.PoseInFrame channel.",
"type": "object", "required": ["channel"], "additionalProperties": false,
"properties": {"channel": {"$ref": "#/$defs/channel"}}
},
"keypoints": {
"description": "Estimated 3D points as a nomadic.Skeleton channel. names in message order; parents[i] is the index of joint i's parent, -1 for a root.",
"type": "object", "required": ["channel", "names", "parents"], "additionalProperties": false,
"properties": {
"channel": {"$ref": "#/$defs/channel"},
"names": {"type": "array", "minItems": 1, "uniqueItems": true, "items": {"type": "string", "pattern": "\\S"}},
"parents": {"type": "array", "items": {"type": "integer", "minimum": -1}}
}
},
"grip": {
"description": "Gripper opening: a declared float signal, unit 1, 0 = closed ... 1 = open.",
"type": "object", "required": ["signal"], "additionalProperties": false,
"properties": {"signal": {"type": "string", "minLength": 1}}
},
"force": {
"description": "A declared vector signal: a wrench (dim 6, components fx fy fz tx ty tz; N, N*m), one wrench per contact (components <contact>_fx .. <contact>_tz in groups of six, e.g. one per fingertip), or contact forces (unit N).",
"type": "object", "required": ["signal"], "additionalProperties": false,
"properties": {"signal": {"type": "string", "minLength": 1}}
}
},
"anyOf": [
{"required": ["joints"]}, {"required": ["end_pose"]}, {"required": ["keypoints"]},
{"required": ["grip"]}, {"required": ["force"]}
]
},
"dimensions": {
"type": "object",
"required": ["length", "width", "height"],
"additionalProperties": false,
"properties": {
"length": {"type": "number", "exclusiveMinimum": 0, "exclusiveMaximum": 1000},
"width": {"type": "number", "exclusiveMinimum": 0, "exclusiveMaximum": 1000},
"height": {"type": "number", "exclusiveMinimum": 0, "exclusiveMaximum": 1000}
}
},
"view": {
"description": "A camera stream. The pixels are a foxglove.CompressedImage/CompressedVideo/RawImage message on `channel`; this entry adds the role the media schema cannot carry.",
"type": "object",
"required": ["channel", "role"],
"additionalProperties": false,
"properties": {
"channel": {"$ref": "#/$defs/channel"},
"role": {
"description": "Free-form, lower_snake_case. Canonical roles drive ordering and view-aware analysis; an unrecognised role is accepted and passed through as a label rather than blocking an unusual rig.",
"type": "string",
"pattern": "^[a-z0-9_]+$"
},
"label": {"type": "string"}
}
},
"signal": {
"type": "object",
"required": ["channel", "name", "type"],
"additionalProperties": false,
"properties": {
"channel": {"$ref": "#/$defs/channel"},
"name": {
"description": "Free-form. No registry, no approval: the platform dispatches on `type`, never on `name`.",
"type": "string",
"minLength": 1
},
"type": {
"description": "Closed set. This is the enabling constraint that lets any name work.",
"type": "string",
"enum": ["bool", "int", "float", "string", "enum", "vector"]
},
"unit": {
"description": "Free-form; SI recommended. OMIT rather than invent one when the source documents no unit - an invented unit is worse than an absent one because it will be believed.",
"type": ["string", "null"]
},
"dim": {"type": "integer", "minimum": 1},
"components": {
"description": "Per-axis labels, length must equal `dim`.",
"type": "array",
"items": {"type": "string", "minLength": 1}
},
"values": {
"description": "Permitted values for an enum signal.",
"type": "array",
"minItems": 1
},
"rate_hz": {
"description": "Nominal. Used for sanity-checking and display, never for resampling.",
"type": ["number", "null"],
"exclusiveMinimum": 0
},
"note": {"type": "string"}
},
"allOf": [
{
"if": {"properties": {"type": {"const": "vector"}}, "required": ["type"]},
"then": {"required": ["dim"]}
},
{
"if": {"properties": {"type": {"const": "enum"}}, "required": ["type"]},
"then": {"required": ["values"]}
},
{
"if": {"properties": {"type": {"not": {"const": "vector"}}}, "required": ["type"]},
"then": {"not": {"required": ["dim"]}}
}
]
},
"point_cloud": {
"description": "A LiDAR or radar stream. The points are a foxglove.PointCloud message on `channel`, already in the vehicle frame; this entry adds `kind`, which the PointCloud schema cannot express.",
"type": "object",
"required": ["channel", "name", "kind"],
"additionalProperties": false,
"properties": {
"channel": {"$ref": "#/$defs/channel"},
"name": {"type": "string", "minLength": 1},
"kind": {
"type": "string",
"enum": ["lidar", "radar"]
}
}
}
}
}