Running AI in an Air-Gapped Environment: The Operational Realities

The case for on-premises or air-gapped AI is by now well understood: some data cannot leave the building, and for those organisations a disconnected system is the only posture that satisfies regulators, auditors and security teams at once. We have written elsewhere about when that trade-off makes sense. This piece is about the question that comes next, and that most teams underestimate: what it actually takes to run an air-gapped AI system, month after month, once the initial deployment is live.

The short answer is that the model is the easy part. Almost everything that makes an air-gapped deployment hard is operational, and it is the operational burden — not the initial build — that determines whether the system is still trusted and current a year later. Whether to take on that burden at all — whether an air gap is the right posture in the first place — is the prior question this piece assumes has already been settled.

Getting a model running locally is no longer the hard bit

Open-weight models and mature local inference tooling mean that standing up a capable model on infrastructure you control is now a well-trodden path. The difficulty has moved elsewhere. A model behind an air gap is only useful if the entire stack around it — serving, monitoring, logging, identity, storage — also runs locally and stays maintained. The engineering effort is concentrated in that supporting stack, and in the processes that keep it healthy without a network connection to lean on.

Updates are a process, not a download

In a connected environment, model and software updates arrive continuously and near-invisibly. Behind an air gap, every update is a deliberate act: new versions must be evaluated, tested against the organisation’s own use cases, and transferred across the gap through a controlled, vetted process — typically on removable media under a formal procedure.

The practical consequence is that an air-gapped system usually runs a deliberately chosen, slightly older version of everything, and that this is a choice to be made consciously rather than a lapse to be discovered later. Before building, an organisation should be able to answer: who owns the decision to update, how often the model will be refreshed, and what the acceptance test is before a new version is allowed across the gap.

The tooling that quietly phones home

The most common way an “air-gapped” deployment turns out not to be is a dependency elsewhere in the stack that still expects the internet. Model-serving engines that fetch tokeniser files from a public repository on startup, telemetry that tries to report home, package managers and model registries that assume outbound access — each is a small assumption that fails silently in a lab and loudly in production, or worse, opens a path that the air gap was supposed to close.

A genuinely air-gapped deployment therefore requires every component to be inventoried for outbound calls and either patched, pre-staged, or replaced with a self-hosted equivalent. Monitoring, logging, registries and identity all have to live inside the boundary, because a secure model is only as isolated as the least isolated thing in its supporting stack.

Patching is the hardest ongoing job

Once a system is live, the single most demanding operational task is usually patch management. Security patches still need to be identified, tested and applied — but now every one of them travels the same controlled path across the gap as a model update, on a cadence the organisation has to resource and sustain. The audit burden of proving that a disconnected system is current can rival the burden of running it, and teams that treat patching as an afterthought find the system drifting out of compliance quietly, without an outage to signal the problem.

This is worth saying plainly because it is the cost most often left out of the business case. An air gap removes a class of external risk, but it does not remove the maintenance work — it relocates it inside the boundary and makes each step deliberate.

A pre-build checklist

Before committing to an air-gapped architecture, an organisation should be able to answer each of these:

• Update ownership — who decides when the model and software are refreshed, and on what cadence?
• Acceptance testing — what must a new version pass before it is transferred across the gap?
• Dependency inventory — has every component been checked for outbound network calls, and each one patched, pre-staged or replaced?
• Self-hosted supporting stack — do monitoring, logging, model registry and identity all run inside the boundary?
• Patch process — is there a resourced, repeatable procedure for moving security patches across the gap, with an audit trail?
• Local access and audit — are access control and audit logging enforced locally, integrated with existing identity management rather than an external service?

An organisation that cannot yet answer these does not necessarily need a different architecture — but it does need to finish scoping before it builds, rather than discovering the gaps in production.

The trade-off worth naming

None of this is an argument against air-gapped AI. For the organisations that need it, the isolation is worth the operational weight, and much of that weight can be carried by a support arrangement designed for it rather than bolted on afterwards. The point is only that the disconnection is not the end of the work — it is the start of a different kind of work, and the deployments that stay trusted are the ones that planned for the second kind from the beginning.

Getting that scoping right — deciding not just whether to go air-gapped but how the system will be kept current once it is — is the difference between a secure deployment and one that is merely secure on the day it launches.