Choose your AI infrastructure

Model tokens: running an open-weight model yourself replaces the provider’s token invoice with compute and operating costs. It does not make inference free.

Platform fees: a zero-markup gateway can still charge by request, log, user, or enterprise license. A flat license can still have capacity limits.

Infrastructure: GPUs, storage, availability, upgrades, and staff remain part of the bill. Ask what happens at ten times today’s volume and which future features force a hosted edition.

Prefer fewer enforcement hops and one clear operating owner. Every serial check can add delay, another failure mode, and another policy to maintain. An integrated platform is valuable when it removes those burdens and covers your workflows.

Count calls, not logos: one platform can still make several remote model calls. As an illustration, three sequential 100 ms checks add about 300 ms before other network and model time. Parallel checks can reduce the wait, but dependencies and required decisions limit what can overlap.

A separate tool is justified when it closes a demonstrated gap that outweighs the added latency and operating cost. Keep necessary controls at the boundaries where they can actually enforce a decision.

Measure all of these with the required policies enabled. A no-guardrails requests-per-second result cannot establish the latency of your protected workflow.

A model that excels at chat may struggle with tool use, structured output, long context, or audio. Reasoning mode, output limits, quantization, and model version can change quality, memory use, latency, and cost.

Look for a lifecycle: evaluate on your tasks → approve a version → route only compatible workloads → monitor quality and spend → roll back safely. Fallback must preserve privacy, tools, and required modes. Treat an open-weight model as a deployable option to test, not a promise of equivalent quality or lower cost.

Separate the client, the model, and the gateway. An editor extension or coding agent may support a custom endpoint; that does not mean every model or protocol works with it. A proprietary cloud model is not made available locally by changing the endpoint.

Cortega includes tooling to deploy a local open-weight model on an existing GPU host, and can govern local model endpoints. Test your selected client’s authentication, API format, streaming, tool calls, and any cloud-dependent features. Apply the same compatibility test to other candidates.

Make the future deployment part of today’s evaluation. Ask for the exact edition and price that includes SSO, audit retention, model routing, guardrails, MCP, client support, high availability, and scale.

Get a feature-by-deployment matrix: available now, paid add-on, hosted-only, or roadmap. Confirm upgrade and license-renewal behavior in an isolated environment. Test export of policies, model settings, and evidence. An open-source core is useful, but does not give you rights to every enterprise component.

Compare total cost: platform licenses + inference/GPU capacity + security and judge calls + storage + integration + maintenance and on-call time. Include the cost of changing providers if the future terms no longer work.

Published by Cortega; reviewed September 27, 2026. This is an opinionated decision framework using vendor documentation, not a comparative benchmark. Alphabetical order, no paid placement or fit scores. Examples are not exhaustive.

How filtering works: the priority selects the relevant product role; hard requirements then narrow it. Documented mismatches are set aside. Missing evidence stays visible as “needs confirmation.” Full on-prem operation always needs an egress-blocked pilot, including models, controls, telemetry, licensing, and tools.

• Cloudflare AI Gateway. Managed gateway; custom providers can connect self-hosted models, but the gateway still processes traffic externally. Custom provider documentation ↗. The listing excludes separate endpoint or Zero Trust products. Guardrails documentation ↗
• Cortega. RedTeam (AI Bench) tests models and active guardrails, scores results, and keeps run history. Model Manager uses these tests to compare candidate models. AI Verifier checks answers before delivery. RedTeam is included in PRO and Enterprise. RedTeam · Model Manager · AI Verifier · Plans · Deployment
• CrowdStrike Falcon AIDR. AI security scope includes endpoints and runtime interactions. Private deployment and a model-routing/serving platform are not established by this listing. AIDR documentation ↗
• LiteLLM. Open-source gateway core and commercial enterprise features; self-hosted model endpoints and guardrail integrations. Full-stack integration remains a requirement to confirm. Enterprise feature boundary ↗ · Local model connection ↗ · Source and license ↗
• Nudge Security. Employee AI and agent discovery; the reviewed source does not establish model routing or fully local inline enforcement. Discovery documentation ↗
• OpenRouter. The managed routing offering is compared here. Data retention controls are distinct from keeping processing on-prem. Deployment description ↗ · Data handling ↗
• Portkey / Prisma AIRS AI Gateway. Open-source gateway core; commercial enterprise and private deployment options. The docs use both names. Confirm current packaging, supported local endpoints, and partner guardrails in the deployment you buy. Enterprise offering ↗ · Private deployment ↗ · Guardrails ↗ · Source and license ↗
• TrueFoundry. Model hosting and gateway platform, with self-hosted model and air-gapped deployment material. Complete open-source gateway rights are not established here. Self-hosted models ↗ · Deployment resources ↗ · Gateway capabilities ↗
• Vercel AI Gateway. Managed gateway; source availability of the AI SDK is not self-hosting of the gateway service. Deployment distinction ↗ · Privacy controls ↗
• Aikido. Application security and AI pentesting are distinct from runtime model governance. Confirm the test scope and commercial package. Pricing and testing scope ↗
• F5 AI Gateway. Gateway, model and MCP policy, and runtime guardrails. Validate control-plane placement and current edition capabilities; a general hybrid-platform claim is not proof of air-gapped operation. Deployment documentation ↗
• IBM watsonx. This entry combines model operations and governance products; it is not a single licensed gateway. Confirm software edition, local inference, evaluation, and support terms together. Governance and evaluation documentation ↗
• Kong. Traditional Kong Gateway with AI plugins supports on-prem deployment. AI Gateway 2.0 uses Konnect’s managed control plane. The $100/LLM Konnect price is not an on-prem Enterprise quote. Architecture and edition boundary ↗
• Langfuse. Evaluation and tracing, not a model router or synchronous response firewall. Self-hosted core is free; enterprise and hosted terms differ. Keep evaluator models and telemetry local when required. Self-hosting documentation ↗
• Obsidian Security. SaaS and AI security scope. This refers to Obsidian Security, not the Obsidian note-taking application. Fully private operation and commercial metering require confirmation. AI security scope ↗
• Pydantic AI Gateway + Logfire. Gateway and evaluation tooling are related services. BYOK has no gateway markup; built-in providers and telemetry have separate fees. Enterprise self-hosting does not establish local model compatibility without a test. Plans and self-hosted option ↗
• Wiz AI-SPM. AI asset discovery, cloud exposure, and attack paths. This is infrastructure posture management, not prompt filtering, inference serving, or answer verification. AI security posture scope ↗