SDK Usage Examples

Quick Start Notebook

For programmatic access to NomadicML, you can use our Python SDK. Quick Start Notebook below.

1. Install the SDK

pip install nomadicml

2. Initialize the Client

from nomadicml import NomadicML
import os

# Initialize with your API key
client = NomadicML(
    api_key=os.environ.get("NOMADICML_API_KEY")
)

To get your API key, log in to the web platform, go to Profile > API Key, and generate a new key. We recommend storing your API key in an environment variable for security.

3. Upload and Analyze Videos

The standard workflow involves uploading your videos first, then running analysis on them. Uploads accept local paths or remote URLs that end with a common video extension (.mp4, .mov, .avi, .webm):

from nomadicml.video import AnalysisType, CustomCategory
response = client.upload('https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Mayhem-on-Road-Compilation.mp4')

# Extract video ID
video_id = response["video_id"]

# Add scope="org" when uploading to shared organization folders.

# Then analyze it
analysis = client.analyze(video_id,
    analysis_type=AnalysisType.ASK,
    custom_event="Find outlier events",
    custom_category=CustomCategory.DRIVING
  )
print(analysis)

You can also pass a list of paths/URLs to upload and a list of ids to analyze for batch operations.

paths = [
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Driving-a-bus-in-Switzerland-on-Snowy-Roads.mp4',
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/LIDAR-RBG-Waymo-YouTube-Public-Sample.mp4',
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Mayhem-on-Road-Compilation.mp4',
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Oakland-to-SF-on-Bridge.mp4',
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Zoox_San%20Francisco-Bike-To-Wherever-Day.mp4'
]
response = client.upload(paths)
video_ids = [v['video_id'] for v in response]

batch = client.analyze(video_ids,
    analysis_type=AnalysisType.ASK,
    custom_event="Find outlier events",
    custom_category=CustomCategory.DRIVING
  )
print(batch["batch_metadata"])  # Contains batch_id, batch_viewer_url, batch_type
for result in batch["results"]:
    print(result["video_id"], result["analysis_id"], len(result.get("events", [])))

4. Semantic Search with Chain-of-Thought

Search can be used are open in the natural language queries. NomadicML will reason about what fits best. Search response includes a chain-of-thought summary plus the reasoning behind each matched video. Supply the natural language query, the folder name, and the scope ("user", "org", or "sample"), and the call returns the complete set of results in one payload.

results = client.search(
    query="Find near-misses with pedestrians on crosswalks",
    folder_name="fleet_uploads_march",
    scope="org",          # optional, defaults to "user"
)

print(results["summary"])       # overall overview
print(results["thoughts"])      # list of reasoning steps
for match in results["matches"]:
    print(match["video_id"], match["reason"], match["similarity"])

# Advanced: reuse the session ID if you want to reference the same results later
print(results["session_id"])  # Unique identifier for this search session

5. Analysis Types

NomadicML supports different analysis types, each optimized for different use cases. All analysis types can be run on a single video or a batch of videos.

Rapid Review

Extracts custom events based on your specific requirements and categories. It is great for fast custom event detection.

analysis = client.analyze(
    video_id,
    analysis_type=AnalysisType.ASK,
    custom_event="green crosswalk",
    custom_category="environment"
)
print(analysis)

Agent Analysis

Runs the same expert agents available in the dashboard. Pick one of AnalysisType.GENERAL_AGENT, LANE_CHANGE, TURN, RELATIVE_MOTION, or DRIVING_VIOLATIONS.

analysis = client.analyze(
    video_id,
    analysis_type=AnalysisType.GENERAL_AGENT,
)
print(analysis)

6. Project-Based File Management & Composite Workflows

For larger projects, you can organize videos into folders and run batch analysis on entire folders at once. This is especially useful for processing datasets or running systematic reviews.

Example: Deleting videos

client.delete_video(video_id)

# OR
for v in video_ids:
    client.delete_video(v)

Example: Analysis + Search Workflow

This example demonstrates a common workflow: first, run a broad analysis to cast a wide net, then use search across analysis results to hone in on specific events, and finally, run a detailed analysis on the resulting subset of videos.

from nomadicml.video import AnalysisType

paths = [
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Driving-a-bus-in-Switzerland-on-Snowy-Roads.mp4',
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/LIDAR-RBG-Waymo-YouTube-Public-Sample.mp4',
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Mayhem-on-Road-Compilation.mp4',
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Oakland-to-SF-on-Bridge.mp4',
    'https://storage.googleapis.com/videolm-bc319.firebasestorage.app/example-videos/Zoox_San%20Francisco-Bike-To-Wherever-Day.mp4'
]

# Define a folder for the project
folder_name = "agent-analysis-videos"
project_folder = client.create_or_get_folder(folder_name, scope="org")
print(f"Using folder '{project_folder['name']}' scoped to {project_folder['scope']} (id={project_folder['id']})")

print("📁 Step 1: Uploading videos to project folder...")
response = client.upload(paths, folder=folder_name, scope="org")
print(f"✅ Successfully uploaded {len(response)} videos to '{folder_name}' folder")

print("\n🔍 Step 2: Running agent analysis on all videos...")
analyses = client.analyze(folder=folder_name,
    analysis_type=AnalysisType.GENERAL_AGENT,
  )
print(f"✅ Completed agent analysis on {len(analyses)} videos")

print("\n🔎 Step 3: Searching for pedestrian-related incidents...")
# You can ask any text-based question about the analysis results
search_results = client.search(
    query="Find risky incidents involving pedestrians",
    folder_name=folder_name,
    scope="org",
)

matching_video_ids = list(set([match['video_id'] for match in search_results['matches']]))
print(f"✅ Found {len(matching_video_ids)} videos with pedestrian incidents")

print(f"\n🎯 Step 4: Re-analyzing {len(matching_video_ids)} videos for pedestrian fault analysis...")
analyses = client.analyze(
    matching_video_ids,
    analysis_type=AnalysisType.ASK,
    custom_event="Mark incidents involving pedestrians where pedestrians are at fault",
    custom_category="driving"
)
print(f"✅ Completed detailed analysis on {len(analyses)} videos")

for analysis in analyses:
    if analysis['events']:
        print(f"\n🎬 Events found in video {analysis['video_id']}:")
        for e in analysis['events']:
            print(f"  • {e}")
        print("-" * 80)

print("\n🧹 Step 5: Cleaning up - deleting project videos...")
for response in client.my_videos(folder_name):
    result = client.delete_video(response['video_id'])

7. Re-analyzing Videos

You don’t need to re-upload videos to run new analyses. You can efficiently query already uploaded videos using either their specific video_ids or by organizing them into folders.

Re-analyzing Specific Videos by ID

This is the most direct way to re-run analysis on a few specific videos. After you upload a video, the API returns a video_id. Store this ID to reference the video in future calls.

# Assume you have previously uploaded two videos and saved their IDs
video_id_1 = "5be1bd918d0f44adb8346fae231523a2"
video_id_2 = "81fbee264993496593db308ad4ccda02"

# Now, you can run a new analysis on just these two videos
pedestrian_analysis = client.analyze(
    [video_id_1, video_id_2],
    analysis_type=AnalysisType.ASK,
    custom_event="pedestrians close to vehicle",
    custom_category="driving"
)
print(f"Found {len(pedestrian_analysis)} videos with pedestrian interactions.")

Using Folders for Batch Re-analysis

For larger-scale projects, organizing videos into folders is the best practice. This allows you to run analysis on an entire dataset with a single command.

from nomadicml.video import AnalysisType, CustomCategory

folder_name = "2024_urban_driving_set"

# Step 1: Upload and organize your videos into a folder (only needs to be done once)
client.upload(
    ['/path/to/city_drive_1.mp4', '/path/to/city_drive_2.mp4'],
    folder=folder_name
)

# Step 2: Run an initial analysis to find all road signs and their MUTCD codes
print(f"\nRunning initial analysis for 'road signs & MUTCD codes' in folder '{folder_name}'...")
pedestrian_analysis = client.analyze(
    folder=folder_name,
    analysis_type=AnalysisType.ASK,
    custom_event="Find all road signs and note their corresponding MUTCD codes?",
    custom_category=CustomCategory.ENVIRONMENT
)
print(f"Found {len(pedestrian_analysis)} videos with road signs.")
# Step 3: Later, run a different analysis on the same set of videos
print(f"\nRunning second analysis for 'potholes' in folder '{folder_name}'...")
pothole_analysis = client.analyze(
    folder=folder_name,
    analysis_type=AnalysisType.ASK,
    custom_event="potholes or major road cracks",
    custom_category=CustomCategory.DRIVING
)
print(f"Found {len(pothole_analysis)} videos with potholes.")

8. Working with Thumbnails

NomadicML can generate annotated thumbnails with segmentations for detected events. This is useful for visual inspection and validation of detected events.

Creating New Analysis with Thumbnails

When running a rapid review analysis, set is_thumbnail=True to generate thumbnails automatically:

from nomadicml.video import AnalysisType, CustomCategory

# Run rapid review with thumbnail generation
result = client.analyze(
    "1b9dac2525f34696a7ca03b0bdf775c2",
    analysis_type=AnalysisType.ASK,
    custom_event="green crosswalk",
    custom_category=CustomCategory.DRIVING,
    is_thumbnail=True  # This enables thumbnail generation
)

Retrieving Thumbnails from Existing Analyses

If you have an existing analysis without thumbnails, or need to retrieve thumbnails later, use the get_visuals() methods:

# Get all thumbnail URLs for an analysis
video_id = "1b9dac2525f34696a7ca03b0bdf775c2"
analysis_id = "auc1QR27QdjluPH0qDoE"

# Get all thumbnails
thumbnails = client.get_visuals(video_id, analysis_id)


# Get a specific thumbnail by event index
first_thumbnail = client.get_visual(video_id, analysis_id, 0)

Thumbnail URLs

Generated thumbnails are stored in Google Cloud Storage and include:

The original video frame at the event timestamp
Bounding box annotations highlighting the detected object
Object labels for easy identification

The URLs are publicly accessible and can be embedded in reports, dashboards, or shared with stakeholders. Example thumbnail with segmentations

If thumbnails don’t exist for an analysis, get_visuals() will automatically generate them. This is useful for older analyses that were created without the is_thumbnail flag.

9. Working with Overlay Metadata

NomadicML can extract telemetry data from on-screen overlays in videos. This is useful for videos with embedded metadata like timestamps, GPS coordinates, speed, altitude, or custom telemetry values.

Important: Metadata describing overlay fields must be provided at upload time. During analysis, you can only enable/disable extraction of these pre-uploaded fields using boolean flags.

Uploading Videos with Metadata

You can provide metadata files that describe the overlay fields in your videos. Metadata must be a properly formatted JSON file according to the Metadata Ingestion Spec, and the .json file must share the same base filename as the video (for example, drone_footage.mp4 pairs with drone_footage.json).

# Single video with metadata file (names must match)
result = client.upload(("dashcam_video.mp4", "dashcam_video.json"))

# Multiple videos with mixed metadata
uploads = client.upload([
    ("video1.mp4", "video1.json"),           # Video with metadata
    "video2.mp4",                             # Video without metadata
    ("video3.mp4", "video3.json"),           # Another video with metadata
])

print(f"Uploaded {len(uploads)} videos")
for upload in uploads:
    print(f"Video ID: {upload['video_id']}, Status: {upload['status']}")

Extracting Overlay Data During Analysis

When analyzing videos that were uploaded with metadata, you can enable extraction of specific telemetry types. The overlay_mode parameter selects which type of fields to extract - the metadata must have been provided at upload time. You can only select one mode at a time.

from nomadicml import OverlayMode

# Extract timestamp overlays
# Note: This video must have been uploaded with metadata
analysis = client.analyze(
    video_id,
    analysis_type=AnalysisType.ASK,
    custom_event="vehicle speeding events",
    overlay_mode=OverlayMode.TIMESTAMPS
)

# Or extract GPS coordinates
analysis = client.analyze(
    video_id,
    analysis_type=AnalysisType.ASK,
    custom_event="route violations",
    overlay_mode=OverlayMode.GPS
)

# Or extract custom fields from metadata
analysis = client.analyze(
    video_id,
    analysis_type=AnalysisType.ASK,
    custom_event="speed limit violations",
    overlay_mode=OverlayMode.CUSTOM  # Will extract custom fields like speed, altitude, etc.
)

# Access extracted overlay data in events
for event in analysis["events"]:
    print(f"Event: {event['label']} at {event['t_start']}-{event['t_end']}")

    overlay_values = event.get("overlay", {})
    for field, values in overlay_values.items():
        start = values.get("start")
        end = values.get("end")
        print(f"  {field}: {start} -> {end}")

Batch Analysis with Overlay Extraction

For batch processing of videos with overlays:

from nomadicml import OverlayMode

# Step 1: Upload batch of videos with metadata
videos_with_metadata = [
    ("fleet_cam_001.mp4", "fleet_cam_001.json"),
    ("fleet_cam_002.mp4", "fleet_cam_002.json"),
    ("fleet_cam_003.mp4", "fleet_cam_003.json"),
]

upload_results = client.upload(videos_with_metadata, folder="fleet_telemetry")
video_ids = [r['video_id'] for r in upload_results]

# Step 2: Analyze with overlay extraction enabled
# The overlay_mode selects which type of fields to extract from the metadata uploaded in Step 1
batch_analysis = client.analyze(
    video_ids,
    analysis_type=AnalysisType.ASK,
    custom_event="harsh braking events where speed drops rapidly",
    overlay_mode=OverlayMode.CUSTOM  # Will extract custom fields (e.g., speed) from uploaded metadata
)

# Process results with overlay data
for result in batch_analysis["results"]:
    video_id = result["video_id"]
    for event in result["events"]:
        # Speed (and other custom telemetry) is exposed through the overlay map
        speed_overlay = event.get("overlay", {}).get("frame_speed")
        if speed_overlay:
            print(
                f"Video {video_id}: Speed changed from "
                f"{speed_overlay.get('start')} to {speed_overlay.get('end')}"
            )

Metadata File Format

The metadata JSON file should describe the fields that appear as overlays in your video. For the complete metadata ingestion specification and detailed schema documentation, see the Metadata Ingestion Spec. Example metadata file:

{
  "fields": [
    {
      "name": "speed",
      "type": "number",
      "unit": "mph",
      "position": "top-left"
    },
    {
      "name": "gps_lat",
      "type": "number",
      "unit": "degrees"
    },
    {
      "name": "gps_lon",
      "type": "number",
      "unit": "degrees"
    },
    {
      "name": "timestamp",
      "type": "timestamp",
      "format": "ISO8601"
    },
    {
      "name": "altitude",
      "type": "number",
      "unit": "meters"
    }
  ]
}

Metadata files must have the same base filename as their corresponding video file. For example, dashcam_recording.mp4 should have metadata named dashcam_recording.json.

10. Storing Results in a Document Database

All SDK methods return serializable Python dictionaries, which can be easily processed and stored in any document database.

Example: Storing in MongoDB

from pymongo import MongoClient

# Assume 'analysis_results' is the list of dicts from a client.analyze() call
results_to_store = []
for analysis in analysis_results:
    # ... (processing logic from previous examples) ...
    results_to_store.append(processed_event)

# Connect to MongoDB and insert the documents
try:
    db_client = MongoClient('mongodb://localhost:27017/')
    db = db_client['nomadicml_results']
    collection = db['driving_events']
    if results_to_store:
      collection.insert_many(results_to_store)
      print("Successfully saved results to MongoDB.")
except Exception as e:
    print(f"An error occurred with MongoDB: {e}")

Example: Storing in Supabase

Supabase provides a Postgres database with a Python client that’s simple to use.

from supabase import create_client, Client
import os

# Assume 'analysis_results' is the list of dicts from a client.analyze() call
results_to_store = []
for analysis in analysis_results:
    # ... (processing logic from previous examples) ...
    # Ensure your dict keys match your Supabase table columns
    processed_event_for_supabase = {
        'source_video_id': video_id,
        'event_type': event.get('type'),
        'timestamp_sec': event.get('time'),
        'description': event.get('description'),
        'severity': event.get('severity'),
        'dmv_rule': event.get('dmvRule'),
        'raw_ai_analysis': event.get('aiAnalysis')
    }
    results_to_store.append(processed_event_for_supabase)

# Initialize Supabase client
try:
    url: str = os.environ.get("SUPABASE_URL")
    key: str = os.environ.get("SUPABASE_KEY")
    supabase: Client = create_client(url, key)

    # Insert data into your 'events' table
    if results_to_store:
        data, count = supabase.table('events').insert(results_to_store).execute()
        print(f"Successfully saved {len(data[1])} results to Supabase.")
except Exception as e:
    print(f"An error occurred with Supabase: {e}")

Getting Started

API Reference

Advanced

Legal

SDK Usage Examples

Quick Start Notebook

1. Install the SDK

2. Initialize the Client

3. Upload and Analyze Videos

4. Semantic Search with Chain-of-Thought

5. Analysis Types

Rapid Review

Agent Analysis

6. Project-Based File Management & Composite Workflows

Example: Deleting videos

Example: Analysis + Search Workflow

7. Re-analyzing Videos

Re-analyzing Specific Videos by ID

Using Folders for Batch Re-analysis

8. Working with Thumbnails

Creating New Analysis with Thumbnails

Retrieving Thumbnails from Existing Analyses

Thumbnail URLs

9. Working with Overlay Metadata

Uploading Videos with Metadata

Extracting Overlay Data During Analysis

Batch Analysis with Overlay Extraction

Metadata File Format

10. Storing Results in a Document Database

Example: Storing in MongoDB

Example: Storing in Supabase

Next Steps

Uploading From Cloud Storage

SDK Documentation

Getting Started

API Reference

Advanced

Legal

​Quick Start Notebook

​1. Install the SDK

​2. Initialize the Client

​3. Upload and Analyze Videos

​4. Semantic Search with Chain-of-Thought

​5. Analysis Types

​Rapid Review

​Agent Analysis

​6. Project-Based File Management & Composite Workflows

​Example: Deleting videos

​Example: Analysis + Search Workflow

​7. Re-analyzing Videos

​Re-analyzing Specific Videos by ID

​Using Folders for Batch Re-analysis

​8. Working with Thumbnails

​Creating New Analysis with Thumbnails

​Retrieving Thumbnails from Existing Analyses

​Thumbnail URLs

​9. Working with Overlay Metadata

​Uploading Videos with Metadata

​Extracting Overlay Data During Analysis

​Batch Analysis with Overlay Extraction

​Metadata File Format

​10. Storing Results in a Document Database

​Example: Storing in MongoDB

​Example: Storing in Supabase

​Next Steps

Uploading From Cloud Storage

SDK Documentation

Quick Start Notebook

1. Install the SDK

2. Initialize the Client

3. Upload and Analyze Videos

4. Semantic Search with Chain-of-Thought

5. Analysis Types

Rapid Review

Agent Analysis

6. Project-Based File Management & Composite Workflows

Example: Deleting videos

Example: Analysis + Search Workflow

7. Re-analyzing Videos

Re-analyzing Specific Videos by ID

Using Folders for Batch Re-analysis

8. Working with Thumbnails

Creating New Analysis with Thumbnails

Retrieving Thumbnails from Existing Analyses

Thumbnail URLs

9. Working with Overlay Metadata

Uploading Videos with Metadata

Extracting Overlay Data During Analysis

Batch Analysis with Overlay Extraction

Metadata File Format

10. Storing Results in a Document Database

Example: Storing in MongoDB

Example: Storing in Supabase

Next Steps