Nomadic AI On-Premise
This page describes how the video analysis platform is deployed on bare-metal machines in your own data center, how data moves through it on the read and write paths, how the GPU pool is laid out, and how the deployment is isolated inside your network. The architecture is the same on any hardware; the NVIDIA MagLev section covers how it runs inside a MagLev environment.Prefer a cloud footprint? The same stack runs in your own account on AWS, GCP, and Azure, and the Self-Hosted VPC Setup guide covers the bringup for each.
General Architecture
The platform runs as a set of containers across three classes of machine: application nodes, GPU nodes, and data nodes. Client applications reach it through a single ingress on your network. Nothing in the deployment requires a route to the internet; container images and model weights arrive in a release bundle and every service talks only to its neighbors on the cluster network.Key Components
Ingress
A reverse proxy on the application nodes terminates TLS with your certificates and routes
/api/* to the API service and everything else to the web console. It is the only listener client networks need to reach, and it sits behind your existing load balancer or firewall if you have one.Application tier
CPU nodes running the web console, the API service, and the analysis workers. The API service handles authentication, upload, and result retrieval. Analysis workers claim queued jobs on a lease and run the analysis out of band, calling the GPU pool for inference.
GPU inference pool
One containerized model server per model, pinned to GPUs on the GPU nodes. The vision-language model runs on vLLM; segmentation and tracking, OCR, and embedding models run on their own servers. Model weights live on local NVMe, so inference never fetches anything off the node.
Data services
An S3-compatible object store for video and derived artifacts, a MongoDB replica set for job and analysis metadata, and Redis for progress streams. All three run on your hardware and are reached only over the cluster network.
What Runs On It
The platform takes the full drive log as input, not just the camera feed, and turns it into findings your engineers and your downstream systems can act on. Every stage below runs on the machines in the rack.- Ingest accepts video directly and recordings in MCAP, so multi-camera video, LiDAR and radar point clouds, and vehicle signals arrive as one time-aligned recording. Disengagement reports, test plans, and existing labels attach as metadata and steer what the analysis looks for.
- Root-cause diagnosis explains an event: what happened, which signals show it, and the reasoning that connects them. Findings carry the timestamps, the signal evidence, and the reasoning trace, and they are labeled in your own taxonomy when you declare it as a structured output schema.
- Scenario segmentation cuts a drive into scenarios and describes each one, and edge-case detection surfaces the segments that fall outside what the fleet has seen.
- Dataset curation assembles the segments an engineer or a pipeline selects into datasets ready for labeling and training, with structured exports for downstream tooling.
- Outputs reach people through the web console and systems through the SDK and API: findings as JSON, exports as CSV, and results delivered into your own applications so triage happens where your engineers already work.
Read Path
A client asks for job status, analysis results, or the media behind them. The request enters through the ingress and is served from the two data stores on the data nodes.Read Path Flow
- Client application sends the request to the platform hostname, which your DNS resolves to the ingress on the application nodes.
- Ingress terminates TLS with your certificate and forwards the request to the API service on the cluster network.
- API service authenticates the API key and resolves what was asked for: job status, an analysis document, or media.
- MongoDB and the object store return the analysis document and the video or derived artifacts behind it. Media is served by presigned URL against the object store’s internal hostname, so large files never pass through the API service.
Write Path
Upload and analysis are decoupled, so a long-running analysis never holds a client connection open.Write Path Flow
The API service writes video to the object store synchronously and records a job document in MongoDB, then returns. The upload is complete at that point. Analysis workers claim the queued job on a lease, marking it in progress so no two workers take the same job, and call the model servers on the GPU nodes for inference: frames and prompts go to the VLM server, and the segmentation, OCR, and embedding servers run alongside it as the pipeline calls for them. As workers run they publish progress events to a Redis stream, which the API service relays to the client as server-sent events. Finished results are written back to MongoDB and derived artifacts to the object store, where the read path picks them up.Hardware Layout
The platform separates the machines that serve requests from the machines that run models, so each can be sized and scaled on its own. The layout below is the standard multi-node footprint; the single-node footprint collapses all three roles onto one GPU server.Node Roles
Object storage capacity is set by your video retention: raw video plus roughly one third again for derived artifacts (thumbnails, segment masks, per-frame metadata, exports). Two application nodes carry the request load with headroom for one to be out for maintenance; add nodes to raise analysis throughput, since each one runs its own pool of workers.
GPU Assignment
Each model server is a container pinned to a fixed set of GPUs on its node, so a model’s footprint is explicit and a change to one model never disturbs another.
Model weights are staged to each GPU node’s NVMe during installation and loaded from there at start, so a node comes back into service without reaching for anything outside the rack.
Network and Isolation
The deployment lives entirely on your network. There is one listener for clients, one east-west network for the cluster, and no dependency on anything outside either.- Nothing leaves your network. Video, metadata, model weights, and inference all stay on machines you own. The platform needs no internet egress to run, and no callback path to Nomadic AI exists.
- One ingress. Client networks reach TCP 443 on the ingress and nothing else. The API service, model servers, and data services are bound to the cluster network only.
- Identity is API-key based. The SDK and the web console authenticate with API keys issued by the platform. An installer-held bootstrap credential issues the first administrator key; every further key is issued from inside the platform and can be scoped and expired by your administrators.
- TLS with your certificates. The ingress terminates TLS with a certificate from your internal CA, or from an ACME server you run. The platform hostname is whatever your DNS says it is.
- Air-gap friendly. Container images and model weights arrive in a versioned release bundle that you load into your own registry, or straight onto the nodes with
docker load. Updates are the same bundle, rolled one node at a time.
Cluster Ports
The ingress is the only port client networks need. Everything else is east-west traffic between nodes and is closed at your firewall to anything outside the cluster network.Deployment Footprints
Three footprints, differing in how many machines carry the platform and where the GPUs sit.Single node
The whole stack on one GPU server: ingress, web console, API, workers, model servers, and data services under one Docker Compose stack. The fastest path to a running platform and the right size for a lab, a pilot, or a single team.
Multi-node cluster
The layout above. Application, GPU, and data tiers on separate machines, each replicated, so a node can be taken out for maintenance without stopping analysis. The production footprint.
NVIDIA MagLev Deployments
NVIDIA MagLev is the data center platform for autonomous vehicle development: fleet recordings land in a drive data lake, are curated and labeled, train models on DGX systems, and are validated by replay, with the loop orchestrated as containerized workloads on Kubernetes. Nomadic AI deploys into the Kubernetes environment MagLev manages as a small set of those workloads, analyzes recordings from the lake, and delivers findings into your own applications through their APIs.Integration Points
Delivering Results Into Your Applications
The integration with your own event-tracking application runs in both directions.- Your taxonomy, applied at analysis time. Your existing root-cause categories are declared as a structured output schema, so every finding leaves the model already labeled with one of your categories, with the signal evidence and reasoning trace attached. Nothing is re-mapped after the fact and no category is invented.
- Your existing records, used as context. Event records already stored in your application are imported alongside the recordings they refer to, so an analysis starts from the event your engineers logged, confirms or revises its category, and attaches the evidence, and so new findings are matched against known events rather than filed as duplicates.
- Your API, as the destination. The integration module maps findings to your application’s schema and delivers them through your API, authenticated with credentials you issue and hold as a cluster secret. The mapping is agreed once, at integration, and versioned with the release.
GPU Placement
The GPU pool under MagLev is the DGX fleet, so the GPU assignment above maps directly: the VLM service requests four GPUs, the segmentation and embedding services request one each, and Kubernetes places them on DGX nodes with the requested count free. Where your data permissions allow it, the VLM call can instead go to an external model endpoint reachable from the cluster, in which case the VLM pods are not deployed and the GPU request drops to the segmentation and embedding services alone.Installation Under MagLev
The installation sequence is the same, with the per-node steps replaced by cluster operations: container images are pushed to the cluster registry, model weights are staged to a persistent volume, the object store adapter is pointed at the lake and the MongoDB Deployment is applied, the model services and then the API Deployment and Service are applied, and the bootstrap credential issues the first key before the smoke test runs a recording through every model service and delivers its findings into your application’s test environment.Installation
Installation is a release bundle and an inventory file. The bundle carries the container images and model weights; the inventory names your nodes, their roles, and the GPUs each model server is pinned to.- Rack and network. Nodes are racked and cabled, the cluster network is up, static addresses or DNS names are assigned, and TCP 443 is open from client networks to the application nodes.
- Load the bundle. Container images are pushed to your registry or loaded onto each node with
docker load. Model weights are staged to NVMe on each GPU node. - Bring up the data tier. The object store is formed across the storage nodes, the MongoDB replica set is initialized across the database nodes, and Redis starts alongside it.
- Bring up the GPU pool. Each model server starts on its assigned GPUs and reports healthy on its own endpoint before the next one starts.
- Bring up the application tier. The API service, workers, and web console start on each application node, and the ingress comes up with your certificate.
- Issue the first key and verify. The bootstrap credential issues the administrator’s API key, and a smoke test uploads a video, runs an analysis through every model server, and reads the result back through the ingress.
What You Provide
Deployments are configured per customer. Contact your Nomadic AI representative with your footprint, node inventory, and GPU count, and we will size the release bundle, prepare the inventory file, and schedule the installation.