Bring Your Own Model Provider

The inference proxy exposes one OpenAI-compatible /v1 endpoint. Point it at a Managed Inference GPU with a foundrydb_managed/ model, or point it at your own provider account (OpenAI, Anthropic, Mistral, Azure OpenAI, or Groq) with a provider-prefixed model. The endpoint and the inference key are the same in both cases. The model prefix decides which upstream the request goes to.

Your provider API keys are stored encrypted at rest and are never returned by the management API. The proxy adds per-key rate limiting, monthly token ceilings, a cost circuit breaker, and optional EU residency enforcement on top of whatever provider you configure.

Prerequisites

• A FoundryDB organization, its id (ORG_ID), and owner or admin basic-auth credentials.
• An API key from the provider you want to use (for example an OpenAI sk-... key).

Set these once:

export FOUNDRYDB_USER="..."
export FOUNDRYDB_PASSWORD="..."
export ORG_ID="<your-org-uuid>"

Step 1: Configure the Provider

Register your provider account against the organization. The proxy maps the provider prefix in a model name to the correct upstream.

curl -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" \
  -X PUT https://api.foundrydb.com/organizations/$ORG_ID/inference/providers \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "openai",
    "api_key": "sk-...",
    "enabled": true
  }'

The supported providers and their prefixes:

Prefix	Provider	Chat	Embeddings	EU Route Available
openai/	OpenAI	Yes	Yes	No (US-hosted)
anthropic/	Anthropic	Yes	No	No (US-hosted)
mistral/	Mistral AI	Yes	Yes	Yes
azure_openai/	Azure OpenAI	Yes	Yes	Depends on region
groq/	Groq	Yes	Yes	No

Optional base_url

Some providers need a base_url. Azure OpenAI requires it (your Azure resource endpoint). When you supply one, it must be an absolute https URL that resolves to a public address: the platform SSRF-validates it and rejects private or non-public targets.

curl -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" \
  -X PUT https://api.foundrydb.com/organizations/$ORG_ID/inference/providers \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "azure_openai",
    "api_key": "...",
    "base_url": "https://my-resource.openai.azure.com",
    "eu_endpoint": true
  }'

eu_endpoint: true is an operator attestation that this configuration points at an EU-resident endpoint. It is never inferred from the base_url.

Step 2: Configure Org-Wide Settings

Settings must be configured before any inference calls go through, and monthly_cost_limit_cents is required on first configuration. This is the per-organization cost circuit breaker.

curl -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" \
  -X PUT https://api.foundrydb.com/organizations/$ORG_ID/inference/settings \
  -H "Content-Type: application/json" \
  -d '{
    "monthly_cost_limit_cents": 5000,
    "eu_only": false
  }'

Field	Description
monthly_cost_limit_cents	Monthly spending ceiling. When reached, the cost circuit opens and data plane calls return 402 cost_circuit_open until the limit is raised and the circuit is reset.
eu_only	When true, reject calls to any provider without an EU-resident route with 403 eu_residency_unavailable.
reset_circuit	Pass true to close an open cost circuit after raising the limit.

EU residency

A foundrydb_managed/ route is always EU-resident. Third-party providers are not: OpenAI, Anthropic, and Groq are US-hosted, Mistral offers an EU route, and Azure OpenAI depends on your region. When your organization has eu_only: true, requests to a provider without an EU-resident route are rejected with 403 eu_residency_unavailable. Set eu_only according to your data residency requirements.

Step 3: Mint or Reuse an Inference Key

Data plane calls authenticate with an fdb-inf-... inference key, the same kind of key you use for Managed Inference. If you already have one, reuse it. Otherwise mint one. The secret is shown exactly once.

curl -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" \
  -X POST https://api.foundrydb.com/organizations/$ORG_ID/inference/keys \
  -H "Content-Type: application/json" \
  -d '{
    "name": "byo-app",
    "monthly_token_limit": 1000000,
    "rate_limit_rpm": 60
  }'

Store the returned secret:

export FDB_INF_KEY="fdb-inf-3f4a..."

Step 4: Call the Proxy

Same endpoint, same key. The only difference from a Managed Inference call is the model string: use <provider>/<model> instead of foundrydb_managed/<served-name>.

curl -X POST https://inference.foundrydb.com/v1/chat/completions \
  -H "Authorization: Bearer $FDB_INF_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "Summarise the main types of PostgreSQL indexes."}
    ]
  }'

Streaming works the same way: pass "stream": true, and usage is always included in the terminal chunk.

With an OpenAI-compatible client, only the model string changes between managed and BYO:

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.foundrydb.com/v1",
    api_key="fdb-inf-3f4a...",
)

# Bring-your-own provider
response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain pgvector cosine distance."}],
)

# Managed Inference, same client, same key
managed = client.chat.completions.create(
    model="foundrydb_managed/mistral-small",
    messages=[{"role": "user", "content": "Explain pgvector cosine distance."}],
)

Embeddings follow the same rule on POST /v1/embeddings (for example openai/text-embedding-3-small). Anthropic does not offer an embeddings API, so an anthropic/ prefix on /v1/embeddings returns 400 invalid_model.

Step 5: Watch Usage and the Cost Circuit

Inspect consumption, grouped by model or by key:

curl -s -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" \
  "https://api.foundrydb.com/organizations/$ORG_ID/inference/usage?group_by=model"

If you hit the monthly ceiling, data plane calls return 402 cost_circuit_open. Raise monthly_cost_limit_cents and pass reset_circuit: true in a settings update to close the circuit again.

Common Errors

HTTP Status	Code	Meaning
400	invalid_model	Malformed body, missing provider prefix, or Anthropic on embeddings.
401	invalid_key	Missing, unknown, revoked, or suspended key.
402	cost_circuit_open	Organization monthly cost limit reached. Raise the limit and reset the circuit.
403	eu_residency_unavailable	Org is eu_only and this provider has no EU-resident route.
412	provider_not_configured	Provider not configured or disabled for this organization.
429	rate_limited	Per-key RPM limit hit. Check the Retry-After header.
429	token_ceiling_exceeded	Monthly token limit for this key reached.

What You Built

• A third-party provider configured for your organization, with its API key encrypted at rest.
• A cost circuit and optional EU residency enforcement wrapping every call.
• Provider-prefixed model calls (openai/gpt-4o-mini) through the exact endpoint and inference key you use for Managed Inference, where the model prefix decides managed versus BYO.

Next Steps

• Inference Proxy for the full provider, key, settings, and usage reference.
• Deploy and call your first model to run an open-weight model on a dedicated EU GPU under foundrydb_managed/.
• Embedding Pipelines to use the proxy as the embedding provider for automated pipelines.