Find your AI gateway

Compare AI gateways by budget, hosting model, provider coverage, caching, guardrails, and observability so you can route LLM traffic with fewer surprises.

Find your AI gateway

Compare AI gateways by budget, hosting model, provider coverage, caching, guardrails, and observability so you can route LLM traffic with fewer surprises.

Showing 10 of 10 vendors that match

1st

Cloudflare AI Gateway

A Cloudflare-native gateway for routing, caching, logging, rate limiting, and controlling AI provider traffic near the edge.

Free

Free usage is available with Cloudflare account limits; paid usage depends on Cloudflare Workers, AI Gateway, and related platform consumption.

Strong fit when your app already runs on Cloudflare
Gateway-level caching, logs, analytics, and request controls
Low-friction way to standardize multi-provider routing at the edge

No self-hosted deployment model
Prompt management is lighter than dedicated LLMOps platforms
Best value depends on adopting the Cloudflare platform around it

Visit

2nd

Portkey

A full-featured AI gateway with routing, guardrails, prompt management, observability, and cost controls for multi-provider LLM apps.

Free

Free developer tier is available; paid team and enterprise pricing should be confirmed during procurement.

Broad provider support with routing, retries, fallbacks, and caching
Includes prompt management and guardrail workflows
Open-source gateway option with hosted plans for teams

Advanced governance usually requires paid or enterprise packaging
More moving parts than a narrow observability-only tool

Visit

3rd

LiteLLM (open source)

An open-source proxy that presents a unified OpenAI-compatible interface across many model providers.

Free

Open-source proxy is free to self-host; hosted and enterprise offerings add support, UI, and governance features.

Excellent provider breadth with one proxy interface
Good budget, key, spend, and routing controls for engineering teams
Self-hosting is practical and widely adopted

Guardrails and prompt lifecycle management are not the core product
Self-hosting means you own reliability, upgrades, and data retention

Visit

4th

Helicone

Open-source LLM observability with request logging, prompt tracking, caching, rate limits, and cost analytics.

Free

Free hosted and self-hosted options exist; paid plans commonly start around the low tens of dollars per month depending on request volume.

Fast path to LLM request visibility and cost tracking
Open-source with hosted and self-hosted deployment options
Useful caching and rate-limit controls for production apps

Less of a complete policy gateway than Portkey or Cloudflare
Enterprise governance depth is lighter than large platform offerings

Visit

5th

Langfuse

Open-source LLM engineering platform for tracing, evals, prompt management, datasets, and cost analytics.

Free

Free self-hosted and cloud tiers are available; paid cloud plans commonly start around $59/month.

Strong tracing, prompt management, evals, and dataset workflows
Open-source and self-hostable for data-control requirements
Good fit for product teams improving LLM quality over time

Not primarily a traffic gateway or rate-limiting proxy
Caching and enforcement usually require another gateway layer

Visit

6th

Traceloop/OpenLLMetry

OpenTelemetry-native tracing for LLM applications, with an open-source instrumentation layer and hosted observability options.

Free

OpenLLMetry instrumentation is open source; hosted Traceloop pricing depends on usage and should be confirmed.

Best fit for teams standardizing on OpenTelemetry
Open-source instrumentation avoids proprietary tracing lock-in
Works well beside existing observability backends

Not a gateway for routing, caching, or rate limiting
Prompt workflows and guardrails are outside the core scope

Visit

7th

AWS Bedrock Guardrails

AWS-native safety and policy controls for Amazon Bedrock applications, including content filters and contextual grounding checks.

$0/mo

Guardrails are metered by policy checks and token usage rather than a flat subscription; monthly cost depends on traffic volume.

Deep fit for teams already building on Amazon Bedrock
Purpose-built safety policies, filters, and grounding checks
AWS billing and IAM integration simplify enterprise control

Narrower provider coverage than multi-provider gateways
Not a general LLM proxy for caching, prompt management, or routing

Visit

8th

Azure AI Studio Guardrails

Microsoft Azure AI safety controls and content filtering for teams deploying models through Azure AI services.

$0/mo

Pricing is tied to Azure AI services, model usage, and safety services rather than a standalone flat monthly plan.

Strong fit for Microsoft and Azure OpenAI environments
Content safety and model governance align with Azure controls
Useful prompt and deployment workflows inside Azure AI Studio

Provider breadth is mostly Azure-centered
Not a self-hosted or open-source gateway

Visit

9th

OpenRouter

A model marketplace and OpenAI-compatible API for routing requests across many hosted LLM providers.

Free

No fixed subscription is required for many use cases; model costs are passed through with provider-specific pricing and possible platform fees.

Very broad model access behind one API
Good for experimentation, fallback routing, and provider comparison
Low commitment because spend is usage based

Not self-hosted and not a full governance platform
Guardrails and prompt management require other tools
Enterprise procurement may prefer direct provider contracts

Visit

10th

LangSmith

LangChain’s hosted platform for tracing, prompt management, datasets, evaluation, and production monitoring.

Free

Developer usage is available; paid Plus seats have commonly been listed around $39/user/month with enterprise custom pricing.

Strong tracing, evaluation, and prompt workflows for LangChain users
Good collaboration UI for product and engineering teams
Mature hosted platform for LLM quality iteration

Not open source or self-hosted for most buyers
Gateway-style caching and rate limits require other components

Visit

About this comparison

Compare AI gateway and LLM operations tools including Cloudflare AI Gateway, Portkey, LiteLLM, Helicone, Langfuse, Traceloop/OpenLLMetry, LangSmith, AWS Bedrock Guardrails, Azure AI Studio Guardrails, and OpenRouter. Filter by self-hosting, provider breadth, response caching, key-level rate limits, safety controls, prompt management, observability, cost tracking, and open-source requirements.