> ## Documentation Index > Fetch the complete documentation index at: https://docs.nomadicml.com/llms.txt > Use this file to discover all available pages before exploring further. # Nomadic AI On-Premise > Architecture, read and write paths, hardware layout, and network isolation for bare-metal deployments in your own data center. # Nomadic AI On-Premise This page describes how the video analysis platform is deployed on bare-metal machines in your own data center, how data moves through it on the read and write paths, how the GPU pool is laid out, and how the deployment is isolated inside your network. The architecture is the same on any hardware; the [NVIDIA MagLev](#nvidia-maglev-deployments) section covers how it runs inside a MagLev environment. Prefer a cloud footprint? The same stack runs in your own account on [AWS](/deployment/aws), [GCP](/deployment/gcp), and [Azure](/deployment/azure), and the [Self-Hosted VPC Setup](/getting-started/vpc-setup) guide covers the bringup for each. ## General Architecture The platform runs as a set of containers across three classes of machine: application nodes, GPU nodes, and data nodes. Client applications reach it through a single ingress on your network. Nothing in the deployment requires a route to the internet; container images and model weights arrive in a release bundle and every service talks only to its neighbors on the cluster network. ```mermaid theme={null} flowchart TB subgraph clients["Your corporate network"] A["Web console users"] B["SDK and API clients"] end subgraph rack["Nomadic AI cluster, your data center"] subgraph app["Application nodes, CPU"] C["Ingress
reverse proxy, TLS termination"] D["Web console"] E["API service"] F["Analysis workers"] end subgraph gpu["GPU nodes"] G["VLM server
vLLM"] H["Segmentation and tracking"] I["OCR"] J["Embeddings"] end subgraph data["Data nodes"] K[("Object store
S3-compatible, video and artifacts")] L[("MongoDB
metadata")] M[("Redis
progress streams")] end end A -->|"HTTPS"| C B -->|"HTTPS"| C C --> D C --> E E --> F E --> K E --> L F --> K F --> L F --> M F --> G F --> H F --> I F --> J ``` ### Key Components A reverse proxy on the application nodes terminates TLS with your certificates and routes `/api/*` to the API service and everything else to the web console. It is the only listener client networks need to reach, and it sits behind your existing load balancer or firewall if you have one. CPU nodes running the web console, the API service, and the analysis workers. The API service handles authentication, upload, and result retrieval. Analysis workers claim queued jobs on a lease and run the analysis out of band, calling the GPU pool for inference. One containerized model server per model, pinned to GPUs on the GPU nodes. The vision-language model runs on vLLM; segmentation and tracking, OCR, and embedding models run on their own servers. Model weights live on local NVMe, so inference never fetches anything off the node. An S3-compatible object store for video and derived artifacts, a MongoDB replica set for job and analysis metadata, and Redis for progress streams. All three run on your hardware and are reached only over the cluster network. ## What Runs On It The platform takes the full drive log as input, not just the camera feed, and turns it into findings your engineers and your downstream systems can act on. Every stage below runs on the machines in the rack. ```mermaid theme={null} flowchart LR subgraph inputs["Drive log inputs"] A["Multi-camera RGB"] B["LiDAR and radar"] C["CAN bus and vehicle signals"] D["GPS and IMU"] E["Disengagement reports
test plans, metadata labels"] end subgraph platform["Nomadic AI platform"] F["Ingest"] subgraph analysis["Analysis, on the GPU pool"] G["Root-cause diagnosis"] H["Scenario segmentation"] I["Edge-case detection"] end J["Dataset curation"] end subgraph outputs["Analysis outputs"] K["Web console"] L["SDK and API
root cause, signal evidence
reasoning trace, CSV and JSON"] M["Your own applications"] end A -->|"multi-modal input"| F B --> F C --> F D --> F E --> F F --> G F --> H F --> I H --> J I --> J G --> K G --> L J --> L L --> M ``` * **Ingest** accepts video directly and recordings in MCAP, so multi-camera video, LiDAR and radar point clouds, and vehicle signals arrive as one time-aligned recording. Disengagement reports, test plans, and existing labels attach as metadata and steer what the analysis looks for. * **Root-cause diagnosis** explains an event: what happened, which signals show it, and the reasoning that connects them. Findings carry the timestamps, the signal evidence, and the reasoning trace, and they are labeled in your own taxonomy when you [declare it as a structured output schema](/sdk/structured-output). * **Scenario segmentation** cuts a drive into scenarios and describes each one, and **edge-case detection** surfaces the segments that fall outside what the fleet has seen. * **Dataset curation** assembles the segments an engineer or a pipeline selects into [datasets](/sdk/datasets) ready for labeling and training, with [structured exports](/sdk/structured-exports) for downstream tooling. * **Outputs** reach people through the web console and systems through the [SDK and API](/sdk/sdk_installation): findings as JSON, exports as CSV, and results delivered into your own applications so triage happens where your engineers already work. ## Read Path A client asks for job status, analysis results, or the media behind them. The request enters through the ingress and is served from the two data stores on the data nodes. ```mermaid theme={null} flowchart LR subgraph clients["Your corporate network"] A["Client application
status, results, media"] end subgraph rack["Nomadic AI cluster"] B["Ingress
TLS termination"] C["API service
authenticates API key"] D[("MongoDB")] E[("Object store")] end A <-->|"HTTPS"| B B <--> C C <-->|"analysis document"| D C <-->|"video and artifacts"| E ``` ### Read Path Flow 1. **Client application** sends the request to the platform hostname, which your DNS resolves to the ingress on the application nodes. 2. **Ingress** terminates TLS with your certificate and forwards the request to the API service on the cluster network. 3. **API service** authenticates the API key and resolves what was asked for: job status, an analysis document, or media. 4. **MongoDB and the object store** return the analysis document and the video or derived artifacts behind it. Media is served by presigned URL against the object store's internal hostname, so large files never pass through the API service. ## Write Path Upload and analysis are decoupled, so a long-running analysis never holds a client connection open. ```mermaid theme={null} flowchart LR subgraph clients["Your corporate network"] A["Client application"] end subgraph rack["Nomadic AI cluster"] B["API service"] C["Analysis workers"] D[("Object store")] E[("MongoDB")] F["GPU pool
VLM, segmentation, OCR, embeddings"] G[("Redis
progress stream")] end A -->|"video upload via ingress"| B B -->|"video, sync write"| D B -->|"queued job"| C C -->|"artifacts"| D C -->|"results"| E C <-->|"frames and prompts"| F C -->|"progress events"| G G -->|"SSE progress"| A ``` ### Write Path Flow The API service writes video to the object store synchronously and records a job document in MongoDB, then returns. The upload is complete at that point. Analysis workers claim the queued job on a lease, marking it in progress so no two workers take the same job, and call the model servers on the GPU nodes for inference: frames and prompts go to the VLM server, and the segmentation, OCR, and embedding servers run alongside it as the pipeline calls for them. As workers run they publish progress events to a Redis stream, which the API service relays to the client as server-sent events. Finished results are written back to MongoDB and derived artifacts to the object store, where the read path picks them up. ## Hardware Layout The platform separates the machines that serve requests from the machines that run models, so each can be sized and scaled on its own. The layout below is the standard multi-node footprint; the [single-node footprint](#deployment-footprints) collapses all three roles onto one GPU server. ```mermaid theme={null} flowchart TB subgraph net["Cluster network, 25 GbE or faster"] subgraph appnodes["Application nodes, 2 or more"] A1["app-01
ingress, web console
API service, workers"] A2["app-02
ingress, web console
API service, workers"] end subgraph gpunodes["GPU nodes, 8 GPUs each"] G1["gpu-01
GPU 0-3 VLM server
GPU 4 segmentation, GPU 5 OCR
GPU 6 embeddings, GPU 7 spare"] G2["gpu-02
GPU 0-3 VLM server
GPU 4 segmentation, GPU 5 OCR
GPU 6 embeddings, GPU 7 spare"] end subgraph storagenodes["Storage nodes, 4 or more"] S1[("store-01
object store")] S2[("store-02
object store")] S3[("store-03
object store")] S4[("store-04
object store")] end subgraph dbnodes["Database nodes"] M1[("db-01
MongoDB primary
Redis")] M2[("db-02
MongoDB secondary
Redis replica")] M3[("db-03
MongoDB secondary")] end end A1 ~~~ G1 A2 ~~~ G2 G1 ~~~ S1 G1 ~~~ S3 G2 ~~~ M1 G2 ~~~ M3 ``` ### Node Roles | Role | Count | Specification | Runs | | -------------------- | --------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ | | **Application node** | 2 or more | 16+ cores, 64 GB RAM, 500 GB NVMe, 2× 25 GbE | Ingress, web console, API service, analysis workers | | **GPU node** | 2 or more | 8× 80 GB NVIDIA GPUs (H100 or A100) with NVLink, 32+ cores, 512 GB RAM, 2 TB NVMe for model weights, 2× 100 GbE | VLM server, segmentation and tracking, OCR, embeddings | | **Storage node** | 4 or more | 16+ cores, 64 GB RAM, 12+ drive bays, 2× 25 GbE | S3-compatible object store, erasure-coded across nodes | | **Database node** | 3 | 8+ cores, 32 GB RAM, 1 TB NVMe | MongoDB replica set, Redis | Object storage capacity is set by your video retention: raw video plus roughly one third again for derived artifacts (thumbnails, segment masks, per-frame metadata, exports). Two application nodes carry the request load with headroom for one to be out for maintenance; add nodes to raise analysis throughput, since each one runs its own pool of workers. ### GPU Assignment Each model server is a container pinned to a fixed set of GPUs on its node, so a model's footprint is explicit and a change to one model never disturbs another. | Model server | GPUs per instance | Notes | | ----------------------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **VLM server** | 4× 80 GB | The 32B vision-language model runs on one GPU; four GPUs in tensor parallel serve the larger models and give one instance the batch throughput of the whole pipeline. Two instances across two GPU nodes give you an active pair. | | **Segmentation and tracking** | 1× 80 GB | Per-frame masks and object tracks across the video. | | **OCR** | 1× 80 GB | Text in scene: signs, plates, dashboards, overlays. | | **Embeddings** | 1× 80 GB | Video and text embeddings for search and dataset curation. | | **Spare** | 1× 80 GB | Kept free per node for a second instance of whichever server the workload saturates. | Model weights are staged to each GPU node's NVMe during installation and loaded from there at start, so a node comes back into service without reaching for anything outside the rack. ## Network and Isolation The deployment lives entirely on your network. There is one listener for clients, one east-west network for the cluster, and no dependency on anything outside either. * **Nothing leaves your network.** Video, metadata, model weights, and inference all stay on machines you own. The platform needs no internet egress to run, and no callback path to Nomadic AI exists. * **One ingress.** Client networks reach TCP 443 on the ingress and nothing else. The API service, model servers, and data services are bound to the cluster network only. * **Identity is API-key based.** The SDK and the web console authenticate with API keys issued by the platform. An installer-held bootstrap credential issues the first administrator key; every further key is issued from inside the platform and can be scoped and expired by your administrators. * **TLS with your certificates.** The ingress terminates TLS with a certificate from your internal CA, or from an ACME server you run. The platform hostname is whatever your DNS says it is. * **Air-gap friendly.** Container images and model weights arrive in a versioned release bundle that you load into your own registry, or straight onto the nodes with `docker load`. Updates are the same bundle, rolled one node at a time. ### Cluster Ports The ingress is the only port client networks need. Everything else is east-west traffic between nodes and is closed at your firewall to anything outside the cluster network. | Port | Service | Reached from | | --------- | ------------- | ---------------------------------------------------------- | | **443** | Ingress | Client networks | | **8099** | API service | Ingress | | **8080** | Model servers | Analysis workers | | **9000** | Object store | API service, workers, and clients following presigned URLs | | **27017** | MongoDB | API service and workers | | **6379** | Redis | API service and workers | ## Deployment Footprints Three footprints, differing in how many machines carry the platform and where the GPUs sit. The whole stack on one GPU server: ingress, web console, API, workers, model servers, and data services under one Docker Compose stack. The fastest path to a running platform and the right size for a lab, a pilot, or a single team. The layout above. Application, GPU, and data tiers on separate machines, each replicated, so a node can be taken out for maintenance without stopping analysis. The production footprint. Your GPU nodes serve the models; the application and data tiers run in your [AWS](/deployment/aws), [GCP](/deployment/gcp), or [Azure](/deployment/azure) account and reach the model servers over your private interconnect. For teams whose GPUs are on-premise but whose data platform is in the cloud. | | Single node | Multi-node cluster | On-premise GPUs, cloud app tier | | --------------------------- | ------------------- | ---------------------------------------------- | ---------------------------------- | | **Machines** | 1 GPU server | 2+ application, 2+ GPU, 4+ storage, 3 database | Your GPU nodes plus a cloud VPC | | **Application runs on** | The node | Application nodes | Your cloud account | | **Video at rest on** | The node | Storage nodes | Your cloud account | | **Inference on** | The node's GPUs | GPU nodes | Your GPU nodes | | **Reached over** | Ingress on the node | Ingress on the application nodes | Internal load balancer in your VPC | | **Survives a node failure** | No | Yes | Yes, with two or more GPU nodes | | **Credentials we hold** | None | None | None | ## NVIDIA MagLev Deployments NVIDIA MagLev is the data center platform for autonomous vehicle development: fleet recordings land in a drive data lake, are curated and labeled, train models on DGX systems, and are validated by replay, with the loop orchestrated as containerized workloads on Kubernetes. Nomadic AI deploys into the Kubernetes environment MagLev manages as a small set of those workloads, analyzes recordings from the lake, and delivers findings into your own applications through their APIs. ```mermaid theme={null} flowchart TB J["Video requests
MagLev workflows, SDK, web console"] subgraph maglev["NVIDIA MagLev, Kubernetes deployment"] A["Kubernetes Service
nomadic-api"] B["Nomadic API pods, Deployment
API and orchestration, video jobs and preprocessing
job status and persistence, model calls"] D["Integration module, in the API pod
maps findings to your schema
imports your categories and records"] subgraph models["Model services, GPU pods on DGX nodes"] E["VLM"] F["Segmentation and tracking"] G["Embeddings"] end H[("MongoDB
findings, timestamps, job status")] I[("Object store
recordings and artifacts")] end K["Your application's API
results destination"] J --> A --> B B --> E B --> F B --> G B --> H B --> I B --> D D <-->|"findings out
root-cause categories and event records in"| K ``` ### Integration Points | MagLev layer | How Nomadic AI connects | | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Scheduling** | The stack is a Kubernetes Deployment and Service. The `nomadic-api` Service routes requests to the API pods, which run the API, video jobs and preprocessing, job status, and model calls; one replica carries a pilot and more are added for throughput. Each model service is its own Deployment whose GPU count becomes an `nvidia.com/gpu` resource request, placed on DGX nodes by node selector, and reached over the existing inference interface. | | **Drive data lake** | Recordings are analyzed in place. Workers read [MCAP](/api-reference/mcap/start-s3-mcap-cloud-ingest) and video [objects by reference](/api-reference/cloud-imports/import-s3-objects) from the lake's S3-compatible store through the platform's storage adapter, and write derived artifacts to a bucket you designate in the same store. Findings, timestamps, and job status persist in MongoDB through the database adapter. | | **Workflow orchestration** | An analysis is a pipeline step. A MagLev workflow submits a batch through the API, streams progress or polls status, and [fetches results in bulk](/api-reference/batches/get-batch-analyses-in-bulk) when the batch completes, with the API key held as a cluster secret scoped to that workflow's service account. | | **Your applications** | Findings are delivered where your engineers already triage events. The integration module inside the API pods maps each persisted finding to the schema your application expects and sends it through your API as the batch completes, so results appear in your system without a separate export job or a polling integration. | | **Curation, labeling, training** | Output lands in the lake in the formats downstream pipelines consume: [structured exports](/sdk/structured-exports) of events and per-frame metadata, embeddings for [search](/sdk/search) and edge-case mining, and [curated datasets](/sdk/datasets) ready for labeling queues and training runs. | ### Delivering Results Into Your Applications The integration with your own event-tracking application runs in both directions. * **Your taxonomy, applied at analysis time.** Your existing root-cause categories are declared as a [structured output schema](/sdk/structured-output), so every finding leaves the model already labeled with one of your categories, with the signal evidence and reasoning trace attached. Nothing is re-mapped after the fact and no category is invented. * **Your existing records, used as context.** Event records already stored in your application are imported alongside the recordings they refer to, so an analysis starts from the event your engineers logged, confirms or revises its category, and attaches the evidence, and so new findings are matched against known events rather than filed as duplicates. * **Your API, as the destination.** The integration module maps findings to your application's schema and delivers them through your API, authenticated with credentials you issue and hold as a cluster secret. The mapping is agreed once, at integration, and versioned with the release. ### GPU Placement The GPU pool under MagLev is the DGX fleet, so the [GPU assignment](#gpu-assignment) above maps directly: the VLM service requests four GPUs, the segmentation and embedding services request one each, and Kubernetes places them on DGX nodes with the requested count free. Where your data permissions allow it, the VLM call can instead go to an external model endpoint reachable from the cluster, in which case the VLM pods are not deployed and the GPU request drops to the segmentation and embedding services alone. ### Installation Under MagLev The [installation](#installation) sequence is the same, with the per-node steps replaced by cluster operations: container images are pushed to the cluster registry, model weights are staged to a persistent volume, the object store adapter is pointed at the lake and the MongoDB Deployment is applied, the model services and then the API Deployment and Service are applied, and the bootstrap credential issues the first key before the smoke test runs a recording through every model service and delivers its findings into your application's test environment. ## Installation Installation is a release bundle and an inventory file. The bundle carries the container images and model weights; the inventory names your nodes, their roles, and the GPUs each model server is pinned to. 1. **Rack and network.** Nodes are racked and cabled, the cluster network is up, static addresses or DNS names are assigned, and TCP 443 is open from client networks to the application nodes. 2. **Load the bundle.** Container images are pushed to your registry or loaded onto each node with `docker load`. Model weights are staged to NVMe on each GPU node. 3. **Bring up the data tier.** The object store is formed across the storage nodes, the MongoDB replica set is initialized across the database nodes, and Redis starts alongside it. 4. **Bring up the GPU pool.** Each model server starts on its assigned GPUs and reports healthy on its own endpoint before the next one starts. 5. **Bring up the application tier.** The API service, workers, and web console start on each application node, and the ingress comes up with your certificate. 6. **Issue the first key and verify.** The bootstrap credential issues the administrator's API key, and a smoke test uploads a video, runs an analysis through every model server, and reads the result back through the ingress. Each step is idempotent, so a re-run continues where it left off, and each is scoped to one node role, so a single GPU node can be added or replaced without touching the rest of the cluster. ## What You Provide | Item | Detail | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Machines** | Nodes matching the [node roles](#node-roles) for your footprint, running a supported Linux distribution with NVIDIA drivers on the GPU nodes. | | **Cluster network** | A VLAN or subnet for east-west traffic between nodes, 25 GbE or faster, with the [cluster ports](#cluster-ports) open between roles and closed to everything else. | | **Hostnames and TLS** | A DNS name for the platform and one for the object store, both resolving to the application nodes, and certificates for them from your CA. | | **Container registry** | Optional. A registry the nodes can pull from; without one, images are loaded onto each node directly from the bundle. | | **Operator access** | SSH to the nodes for the installer, through your jump host or bastion, for the duration of the install and for updates. | | **Initial administrators** | The people who receive the first API keys and administer keys for everyone else. | Deployments are configured per customer. Contact your Nomadic AI representative with your footprint, node inventory, and GPU count, and we will size the release bundle, prepare the inventory file, and schedule the installation.