Running AI Agents Without Handing Over the Keys
Containers, NVIDIA OpenShell, deny-by-default policy, and the inference.local trick — explained by clicking, not just reading.
1 · What a container actually is
A container is a lightweight way to package and isolate a group of processes. It can be used as a sandbox when restrictive policy is added, but a container by itself is not automatically a secure sandbox. Toggle below to compare it with a VM.
Why an AI agent needs the box
An agent is a program that takes actions — runs shell commands, edits files, calls the network. With no box it inherits all your powers: your SSH keys, cloud passwords, entire disk. One poisoned instruction (“prompt injection”) becomes real damage. The box caps the blast radius.
2 · What’s a “runtime”?
“Runtime” is an overloaded word: it means the execution layer responsible for running something. Every agent needs some execution environment, but it does not necessarily use a dedicated security-focused agent runtime such as OpenShell.
So “OpenShell is a runtime for autonomous agents” means it manages the agent’s sandbox lifecycle and security boundary: it creates the environment through a compute driver, applies policy, mediates approved access, records activity, and cleans up. Node.js may still run the agent’s JavaScript inside that environment.
3 · The Stack — where OpenShell sits
Click any layer. The dashed box is everything OpenShell is; notice the gateway is just one part of it — the part facing the outside.
4 · The four parts of OpenShell
OpenShell isn’t one blob — it’s four cooperating pieces. (You clicked these in the stack above; here they are spelled out.)
📦 Sandbox supervisor
Starts and supervises the restricted agent process inside the sandbox. The selected compute driver creates the underlying container, Kubernetes workload, or MicroVM. Analogy: the on-site guard inside a building created by the contractor.
📜 Policy engine
Reads your rulebook and turns it into allow / deny decisions for files, processes and network destinations. The exact enforcement available also depends on the driver and host OS. Analogy: the rulebook the guards follow.
🚪 Gateway outside-facing
The controlled bridge between sandboxes and managed external capabilities. It routes model calls, resolves configured credential placeholders, and keeps the corresponding real credentials outside the sandbox. It does not magically protect credentials you explicitly place inside the sandbox. Analogy: the mailroom + key cabinet.
🧾 Audit / CLI
The logbook + your controls (sandbox list, policy set, term). Exports machine-readable OCSF logs so a security team can answer “what did the agent try to do?” Analogy: the visitor log + reception desk.
5 · Policy Simulator — feel “deny-by-default”
This is a coding-agent’s rulebook. Turn protections on/off, then fire an attack and watch the box block it — or leak. The legit task always works, proving the box doesn’t break real work.
Policy rules
Attacks & tasks
Sandbox console
6 · Gateway-held credentials: inference.local + providers
How an agent uses an AI model without possessing its model API key. Step through it: the real key stays in the gateway, while the agent receives only the capability to make approved calls.
What it does not protect: the agent can still abuse capabilities you allowed. For example, it may send sensitive text through an approved model call or use an approved GCP token to perform actions permitted by that identity. OpenShell reduces the blast radius; it does not make an authorized but hijacked agent harmless. Use narrow IAM permissions, narrow endpoint policies, short-lived credentials, and audit logs.
Two related credential paths
Model inference: the agent calls inference.local without a model key. The gateway selects the configured inference provider and adds its real credential while forwarding the request.
Other APIs, such as GCP: the sandbox may receive an opaque provider placeholder instead of the real token. When an allowed request passes through the gateway proxy, the gateway resolves that placeholder to the real credential. The placeholder is safer than the token, but it still represents an authorized capability and must be tightly scoped.
Current alpha caveat: provider credential injection is still maturing and should not be treated as a complete substitute for least-privilege IAM. Use narrow identities, separate providers or gateways where needed, and test what each sandbox can actually reach.
7 · Which sandbox engine? (Docker · Podman · MicroVM · K8s)
OpenShell uses a compute driver underneath to create the sandbox. The driver chooses where and how it runs: for example, a local Docker host, a Kubernetes cluster, or a MicroVM-capable host. Drivers differ in isolation strength and operational cost.
| Engine | Isolation | Cost | Use when |
|---|---|---|---|
| Docker | shared Linux kernel (namespaces / cgroups) | host-dependent | learning & most single-host work — simplest start |
| Podman | same model, rootless + daemonless = smaller attack surface | free | a more locked-down host with no root daemon |
| MicroVM (Firecracker-class) | own kernel per box — VM-grade, the strongest | free | real production with untrusted agents |
| Kubernetes (GKE) | orchestration (pair with gVisor/MicroVM for isolation) | cloud $ | teams / many sandboxes / central gateway & audit |
Your exact questions, answered
Is Docker good for production? Good to start, and fine for trusted workloads. For untrusted agents in production, climb to MicroVM (own kernel per sandbox) — that’s the “really production” isolation.
Is Podman free? Yes — fully free & open-source (Apache-2.0), even for big companies. Docker Engine is open-source; Docker Desktop has separate commercial licensing terms.
“gcloud container” / GKE? The team/production upgrade: run OpenShell’s gateway + sandboxes on GKE via the (experimental) Helm chart — central policy, OCSF audit to your SIEM. Cloud Run is the wrong fit (stateless, no privileged nested containers).
Vercel? Wrong layer. Vercel runs serverless functions / static sites — no privileged container runtime, so it can’t host an OpenShell sandbox. Use Vercel for a dashboard or chat UI that talks to an agent running on a real container host.
8 · Run it for real
A working local path for OpenShell 0.0.57 on this Mac. OpenShell needs a reachable gateway plus a compatible compute driver underneath.
macOS / Linux setup
# 0) start Docker Desktop # 1) install OpenShell CLI *and the local gateway* — use install.sh, NOT # `uv tool install` (that ships the CLI only, without the gateway service) curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh # 1a) select the Docker driver/socket in ~/.config/openshell/gateway.env OPENSHELL_DRIVERS=docker DOCKER_HOST=unix:///Users/YOU/.docker/run/docker.sock # 2) confirm the gateway is reachable openshell status # -> Status: Connected # 3) configure gateway-held provider credentials, then create the sandbox openshell provider create ... openshell sandbox create --provider my-provider -- claude # 4) scope it with a rulebook + watch it live openshell sandbox create --policy policy.yaml -- claude openshell term
✓ verified on this Mac — Docker Desktop + OpenShell gateway + Docker driver. The agent called a model through inference.local, used GCP provider placeholders, and direct access to OpenRouter was blocked.
OpenShell provides a base sandbox image, and you can also supply files or a compatible custom image. The compute driver answers a different question: where and through which isolation technology should OpenShell create the sandbox?
9 · Going deeper
The concepts we worked through after the first build — kept here so the whole mental model lives in one place.
Container vs. Sandbox — not the same
- Sandbox = the concept: an isolated, restricted place to run untrusted code.
- Container = one technology for building a sandbox (Linux namespaces / cgroups). Other ways: VMs, MicroVMs, gVisor, WASM.
So a restrictive container can implement a sandbox, but not every container is a secure sandbox and not every sandbox is a container. OpenShell combines a compute driver with policy and gateway controls to create the managed sandbox.
What lives outside the sandbox?
The trusted control plane and gateway must remain outside the agent’s authority. Otherwise a hijacked agent could rewrite policy or read the real credentials protecting its own boundary.
A supervisor component also participates inside the sandbox to apply configuration and supervise the process, but it fetches trusted policy and provider configuration from the external gateway. The important boundary is that the agent cannot administer the gateway, change its own policy, or retrieve the real provider credentials.
Model vs. Harness vs. Runtime — three layers
Gemini · Nemotron
Claude Code · opencode
OpenShell
“Try Nemotron” swaps the brain. “Use opencode” swaps the harness. “Run in OpenShell” adds a managed security and lifecycle layer. A programming runtime such as Node.js or Python may still be required underneath the harness.
Where NemoClaw fits
NemoClaw is a reference stack on top of OpenShell — guided onboarding + hardened defaults, not a replacement:
Over the raw openshell CLI it adds: managed multi-provider inference routing, a hardened network-policy blueprint, credential handling, operator-approval flows for network access, and one CLI for the whole agent lifecycle.
One gateway, many agents (you, client A, client B)
One OpenShell gateway supervises many sandboxes at once, each its own box with its own policy — a web-research sub-agent (no repo write), a code-editor sub-agent (no cloud creds), a deploy sub-agent (short-lived), all isolated from each other. openshell sandbox list shows them side by side. The same gateway can host agents for you, client A, client B — though true multi-tenant isolation is still maturing (OpenShell is alpha / “single-player” today), so for real client separation use separate gateways or strict per-client credentials.