Managed Inference

Managed Inference runs an open-weight model on GPUs in a European zone and exposes it through an OpenAI-compatible endpoint. You pick a model from the curated catalog (or bring a Hugging Face model), the platform runs the vLLM runtime, and you call it with any OpenAI-compatible client. Weights stay on GPUs in Helsinki (fi-hel2).

This is a distinct service kind from managed databases. It has its own /inference-services API, and every service gets its own endpoint hostname, which the API hands you back as endpoint_base_url. That URL is what you point a client at.

Two SKUs

Every inference service is one of two SKUs, and the difference is what you rent and how you pay.

	Serverless	Dedicated
What you rent	Nothing. Your model runs on a platform-owned shared GPU pool	A whole GPU card, yours alone
How you pay	Per token (per generated image for the image models), against the published rate card	Per GPU-hour for as long as the card is allocated, by plan
Free allowance	The first 1,000,000 tokens each calendar month, per organization	None. The hour is billed from the moment the card is allocated
Models	Only curated catalog models a pool is already serving	Any curated model that fits the plan, or a Hugging Face model you bring
Cost when idle	Zero. There is no card to leave running	The full hourly rate, unless you stop it or set keep-warm
LoRA adapters	Not available	Yes, hot-loaded with no restart
Create with	No plan_name (or inference_sku: "serverless")	A GPU plan_name

Serverless is the right default: it costs nothing when nobody is calling, and the free allowance covers a real evaluation before you spend anything. Move to dedicated when you need a model serverless does not offer, a Hugging Face model of your own, LoRA adapters, or predictable capacity that no other tenant shares.

What to expect on serverless

Serverless runs on shared, platform-operated GPU capacity. You are billed only for what you use (per token, or per generated image) and reserve nothing. Because nothing is reserved for you, the numbers below are what calls typically do when the pool has room, not an allocation held open on your behalf. How quickly a call comes back depends on how busy the pool is at the moment you send it.

• Text generation: streams at interactive speed, comfortably faster than a person reads, when the pool has room. A stream started while the pool is busy queues behind the work already in flight and returns more slowly than the same stream off-peak.
• Embeddings and reranking: typically sub-second per call. Batching many inputs into one call is far quicker than sending them one at a time, and it is the single biggest thing you control.
• Image generation: a 512x512 render typically takes around 9 seconds, and a full-resolution 1024x1024 render around 12 seconds, when the pool is not saturated. Image models serialize rather than batch, so a render sent to a busy pool waits for the ones ahead of it and its wall-clock time grows accordingly.

Under load, requests queue, and when a pool approaches saturation the platform answers a fast busy signal (429 with Retry-After) rather than accepting everything and letting every request crawl. Retry on the header and your call lands as soon as there is room. Your organization's concurrency allowance and your key's rate limit produce the same answer for the same reason.

A failed request is never billed. A refusal never reaches the meter at all, and a call that errors records zero cost, so you pay for work that came back.

If you need capacity and throughput that hold regardless of who else is calling, that is what a dedicated GPU service is for: one tenant, one card, no neighbours, serving the same catalog models on the same OpenAI-compatible endpoint.

Ask which models serverless can bind to right now before you create one:

curl https://api.foundrydb.com/inference-services/serverless-models \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN"

{"models": [
  {"model_id": "mistral-small", "display_name": "Mistral Small 3.2 24B Instruct",
   "capability": "chat", "serving": true, "deprecated": false}
]}

The listing reports exactly what can bind: a model no pool is serving is omitted rather than listed as unavailable, so a create naming it is refused (503) instead of hanging. An empty list means serverless has nothing to offer yet, and a dedicated service is the way to serve that model. deprecated: true marks a model that still binds and still serves, but is end of life upstream, so do not build new work on it.

What you get

• Open-weight models on EU GPUs. Serverless on a platform-owned pool, or one tenant per GPU server on the dedicated SKU, all in fi-hel2 (Helsinki).
• An OpenAI-compatible endpoint per service. Any OpenAI SDK, LangChain, or plain curl works. You change the base URL, the API key, and the model string.
• EU residency by default. A foundrydb_managed/ route always stays EU-resident. Your organization's eu_only setting is respected.
• Per-key controls. Requests-per-minute limits, monthly token ceilings, and a monthly cost circuit breaker, all enforced on the inference key.

The curated catalog

Eleven license-verified models are available today. The platform fills in the repository, served name, and context length from the catalog, so you only pass the catalog id and model_source: curated.

Catalog id (model_id)	Served as	Model	Kind	Tool calling	License	Default GPU plan	Context
mistral-small	mistral-small	Mistral Small 3.2 24B Instruct	Chat	Yes	Apache-2.0	gpu-l40s-1	32768
mistral-7b	mistral-7b	Mistral 7B Instruct v0.3	Chat	Yes	Apache-2.0	gpu-l4-1	32768
mixtral-8x7b (deprecated)	mixtral-8x7b	Mixtral 8x7B Instruct v0.1 (FP8)	Chat	No	Apache-2.0	gpu-h100-1	32768
llama-3.3-70b	llama-3.3-70b	Llama 3.3 70B Instruct (FP8)	Chat	Yes	Llama 3.3 Community (license-gated)	gpu-b200-1	32768
qwen3.8-27b	qwen3.8-27b	Qwen3.8 27B (FP8)	Chat	Yes	Apache-2.0	gpu-l40s-1	262144
qwen2.5-1.5b	qwen2.5-1.5b	Qwen2.5 1.5B Instruct	Chat	Yes	Apache-2.0	gpu-l4-1	32768
qwen2.5-0.5b	qwen2.5-0.5b	Qwen2.5 0.5B Instruct	Chat	Yes	Apache-2.0	gpu-l4-1	32768
bge-m3	bge-m3	BGE-M3	Embeddings	n/a	MIT	gpu-l4-1	8192
bge-reranker-v2-m3	bge-reranker-v2-m3	BGE Reranker v2 m3	Reranker	n/a	MIT	gpu-l4-1	8192
z-image-turbo	z-image-turbo	Z-Image Turbo	Text-to-image	n/a	Apache-2.0	gpu-l40s-1	n/a
flux-schnell	flux-schnell	FLUX.1 schnell	Text-to-image	n/a	Apache-2.0 (license-gated repo)	gpu-l40s-1	n/a

Tool calling is per model family, not platform-wide

Tool calling (OpenAI tools / tool_choice) works only where the checkpoint's own tokenizer and chat template carry tool-call syntax that vLLM has a parser for. It is a property of the weights, not something the platform can switch on.

• qwen3.8-27b supports it best. Qwen3.8 is trained for long-horizon agentic work; it is served with the qwen3_coder tool parser and the qwen3 reasoning parser, so its thinking blocks surface as reasoning content instead of polluting replies, and its 262,144-token native context (only 16 of its 64 layers hold a KV cache, so the full context fits a single L40S) carries agent transcripts no other catalog entry can.
• The two Mistral entries and llama-3.3-70b support it. Mistral Small 3.2 and Mistral 7B v0.3 emit the [TOOL_CALLS] sequence their tokenizers define; Llama 3.3 inherits the JSON tool-call form Llama 3.1 introduced, along with a chat template that renders tool definitions.
• The two small Qwen2.5 entries support it. qwen2.5-1.5b and qwen2.5-0.5b ship a chat template that renders tool definitions in the Hermes shape, so they are served with the hermes tool parser. They are the cheap way to exercise an agent loop: a gigabyte or three of weights, both on the smallest card, and small enough that a stack of them leaves half an L4 free.
• mixtral-8x7b does not. The v0.1 tokenizer this checkpoint ships has no tool-call control tokens and its published chat template renders no tool definitions, so there is nothing for a parser to read. Sending a tools array to it is not supported; platform surfaces withhold their tool catalog on this model rather than send an array the engine refuses.
• Embeddings, reranker, and image models have no chat surface at all, so the question does not arise.
• A Hugging Face model you bring yourself never gets tool calling, whatever the checkpoint supports. The engine's tool flags are set from the curated catalog entry, and an on-demand pull has no entry, so it is started without them.

Agent loops and coding tools

A full agent loop, the shape every framework produces, works end to end on a tool-calling model: the model returns tool_calls, your code executes the tool and appends the result as a role: "tool" message, and the next call answers grounded in it. Streaming works the same way (tool-call deltas accumulate by index), tool_choice accepts "auto", "required", and a named function, and response_format: {"type": "json_schema", ...} returns schema-valid JSON for structured pipelines. The identical loop runs on /v1/responses with function_call output items answered by function_call_output input items.

messages = [{"role": "user", "content": "Weather in Stockholm? Use the tool."}]
first = client.chat.completions.create(model=MODEL, messages=messages, tools=TOOLS)
call = first.choices[0].message.tool_calls[0]
messages.append(first.choices[0].message)
messages.append({"role": "tool", "tool_call_id": call.id,
                 "content": run_tool(call.function.name, call.function.arguments)})
final = client.chat.completions.create(model=MODEL, messages=messages, tools=TOOLS)

That makes any coding assistant or IDE that takes an OpenAI-compatible base URL work against a managed endpoint: point it at the service's base URL, use an fdb-inf key as the API key, and foundrydb_managed/<served_model_name> as the model. The service detail page's Agents & IDEs tab carries all of these pre-filled for your service.

Tool	Where	Configuration
aider	shell	OPENAI_API_KEY=fdb-inf-... then aider --openai-api-base <base_url> --model openai/foundrydb_managed/<name>
Codex CLI	~/.codex/config.toml	[model_providers.foundrydb] with base_url, env_key, wire_api = "responses"; model = "foundrydb_managed/<name>"
Claude Code	shell	ANTHROPIC_BASE_URL=<base_url without /v1>, ANTHROPIC_AUTH_TOKEN=fdb-inf-..., ANTHROPIC_MODEL=foundrydb_managed/<name>, CLAUDE_CODE_MAX_OUTPUT_TOKENS=4096 (served by the Anthropic-compatible /v1/messages surface; the output cap keeps its per-turn budget inside an open-weight context window)
Cursor	Settings, Models	set the OpenAI API key to your fdb-inf key, override the OpenAI base URL, add the model string as a custom model
OpenCode	opencode.json	a provider entry with npm: "@ai-sdk/openai-compatible", baseURL, apiKey, and the model string under models
Cline / Kilo Code / Roo Code	VS Code extension settings	API Provider "OpenAI Compatible" with the base URL, the fdb-inf key, and the model string as Model ID
Continue.dev	~/.continue/config.yaml	a models: entry with provider: openai, apiBase, apiKey, model
OpenAI Agents SDK	Python	set_default_openai_client(AsyncOpenAI(base_url=..., api_key=...)) and set_default_openai_api("responses")

Claude Code is the one tool in the table that does not speak the OpenAI wire: it uses Anthropic's Messages API, which the data plane serves natively at POST /v1/messages (content blocks, tool_use/tool_result, typed SSE events, x-api-key or bearer auth, and /v1/messages/count_tokens), translated onto the same managed models with the same metering.

Client conventions the platform absorbs for you on managed models: the developer message role modern clients send (the Codex CLI included) is rewritten to system, typed text content parts are flattened to the plain strings open-weight chat templates understand, a function tool that omits strict gets the OpenAI default filled in, namespace tool wrappers are flattened into their member functions, and OpenAI-hosted tool types a self-hosted model cannot execute (web_search, file_search) are dropped so the rest of the tool set keeps working. Explicit values are never overridden, and calls to your own BYO providers pass through untouched.

One honest sizing note: heavyweight agent harnesses like the Codex CLI write their system prompts for frontier-scale models. mistral-7b connects and completes turns in them, but it follows their tool-use conventions unreliably; for coding assistants pick mistral-small or llama-3.3-70b, and treat mistral-7b as the model for lightweight tool loops you prompt yourself.

Five of the nine are chat and completion models. bge-m3 is an embeddings model (it turns text into vectors), bge-reranker-v2-m3 is a cross-encoder reranker (it scores query and document pairs), and z-image-turbo and flux-schnell generate images; none of those four answers chat requests. All four chat models serve at FP8, which halves the weight footprint and leaves far more of the card for the KV cache your requests actually run in. The two largest, mixtral-8x7b and llama-3.3-70b, are stored as FP8 checkpoints. mistral-small and mistral-7b are stored in their original BF16 form and quantized as they load onto the GPU, so the weights you get are the ones the model authors published.

llama-3.3-70b carries a conditional-commercial license, so you must accept it explicitly at create time (see License acceptance below). The other five are Apache-2.0 or MIT and need no acceptance.

llama-3.3-70b defaults to gpu-b200-1 rather than a single H100. Its FP8 weights load at 67.7 GiB, and once the 32768-token KV cache and the serving overhead are paid the request needs 75.2 GiB, which an 80 GiB H100 can only just start and never batch. The 192 GiB B200 leaves about 101 GiB of KV cache, so the model serves many concurrent requests instead of one. A single H100 is still selectable at a shorter context, and the two-card and four-card H100 plans still serve the full context with tensor parallelism; run a fit check before choosing one.

mixtral-8x7b is deprecated: the weights are end of life across the market. It still serves, services already running it are unaffected, and a create that names it is still honoured, but it is no longer offered as a default and it will not receive further tuning. Prefer mistral-small or llama-3.3-70b for new work.

You can also bring a Hugging Face model with model_source: huggingface, in which case you supply the repo id, a served_model_name, and accept the model's license. This guide focuses on the curated path.

Will it fit?

A GPU has to hold three things at once: the model's weights, room for the conversations in flight (longer context means more room), and a safety margin for the runtime itself. Those three together have to stay inside the plan's memory budget, which is deliberately a little under the card's full VRAM. A big model on a small card fails on the weights alone, and no amount of shortening the context saves it; a model that fits comfortably can still be asked for more context than there is room for.

You do not have to guess, and you should not find out by paying for a GPU. POST /inference-services/fit-check answers the question up front: give it a model, a plan, and optionally a context length, and it returns whether the combination fits, the memory breakdown it reached that from, the longest context the plan would serve, and the closest fix when it does not fit (a context length that does fit, an fp8 KV cache, or the smallest plan that would take it). It provisions nothing and costs nothing. Creating a service and switching a service's model enforce exactly the same equation, so a fits: false here is the refusal you would have got.

The console shows this live as a fit meter: as you pick a model, a plan, and a context length in the create flow, and again in the switch dialog, the meter fills with the weights, the cache, and the margin, and turns into the same named fix the moment the configuration stops fitting.

Serving a model stack on one GPU

A retrieval application is not one model. It embeds documents, it reranks candidates, and it answers with a chat model, and running each of those on a card of its own means renting three GPUs to keep two of them mostly idle. A dedicated service can serve them all on one card instead.

Name up to three extra curated models in inference_config.companion_models at create time, or add them later to a service that is already serving (Changing the stack later). Each runs in its own vLLM process with its own measured share of the card, and all of them answer on the service's one endpoint hostname with its one inference key. Nothing else about the service changes: same base URL, same key, same EU residency, same metering, same GPU-hour bill for the one card.

The RAG stack, with the real numbers

mistral-7b for chat, bge-m3 for embeddings, and bge-reranker-v2-m3 for reranking, on a single gpu-l40s-1 (48 GB):

Model	Role	Weights	KV cache	Per-process overhead	Share of the card
mistral-7b	primary, chat	6.75 GiB (FP8 at load)	4.00 GiB at 32768 tokens	2.50 GiB	27.6%
bge-m3	companion, embeddings	1.06 GiB	none	2.50 GiB	7.4%
bge-reranker-v2-m3	companion, reranker	1.06 GiB	none	2.50 GiB	7.4%
				Total	42.5%

Two things in that table are worth reading twice. A pooling model (embeddings, reranker) holds no KV cache at all, which is why a reranker costs barely more than its weights. And the overhead term is paid by every process rather than once per card: it is the CUDA context and the runtime that each engine allocates for itself, so four small models are not free. Those two facts are the whole arithmetic:

share_i = (weights_i + kv_cache_i + per_process_overhead) / per_card_vram
fits    when sum(share_i) <= 0.95

The ceiling is 0.95 of the card, not all of it: the remainder belongs to the driver, the ECC reservation, and fragmentation between separately allocated pools. The computed share is exactly the --gpu-memory-utilization each engine is started with, so what a create tells you is what runs.

The equation is honest rather than padded, which sometimes means a refusal. qwen3.8-27b at its full 262144-token context asks for 81.8% of an L40S on its own; add the same two companions and the stack asks for 96.6%, which is refused. That is the correct answer, not a limitation to work around: one companion still fits beside it (89.2%), and all three fit an H100.

Ask before you rent. POST /inference-services/fit-check takes the same companion_models and answers with joint: a share per model, the total, the 0.95 budget, and, when it does not fit, the companion to drop or the plan that holds them all.

curl https://api.foundrydb.com/inference-services/fit-check \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model_source": "curated",
    "model_id": "mistral-7b",
    "plan_name": "gpu-l40s-1",
    "companion_models": ["bge-m3", "bge-reranker-v2-m3"]
  }'

Create the stack the same way:

curl https://api.foundrydb.com/inference-services \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-rag-stack",
    "plan_name": "gpu-l40s-1",
    "zone": "fi-hel2",
    "inference_config": {
      "model_id": "mistral-7b",
      "model_source": "curated",
      "companion_models": ["bge-m3", "bge-reranker-v2-m3"]
    }
  }'

In the console the same choice is a Companion models row in the create flow's model step, on the dedicated path only. Adding a model turns the VRAM fit meter into the whole stack's: a segment per model sized by its share of the card, and, when the stack stops fitting, the server's own remedy offered as a chip that removes that companion.

Changing the stack later

The stack is not frozen at create. On a Running dedicated service you can add one more model to the card, or take one off, without re-provisioning anything and without a maintenance window:

# add a model to a card that is already serving
curl -X POST https://api.foundrydb.com/inference-services/$SERVICE_ID/companions \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model_id": "bge-m3"}'

# take one off again
curl -X DELETE https://api.foundrydb.com/inference-services/$SERVICE_ID/companions/bge-m3 \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN"

Why an add usually costs the models already serving nothing. The share each engine holds is an absolute fraction of the card, and a running engine never rebalances it. Admitting a model needs only the fraction nothing has claimed, and removing one simply returns its fraction to that pool, so an edit starts or stops exactly one process. Your chat model keeps answering while an embeddings model is brought up beside it, and the service never leaves Running: there is no restart, no serving gap, and no change to the endpoint, the key or the certificate.

The one case that is not free: the first companion on a single-model service. A model that has the card to itself is deliberately started on almost all of it, because that is the largest KV cache and the highest throughput it can have, and a vLLM process never gives memory back. Adding the first model beside it therefore means reloading it at the smaller share the stack measures for it, which takes that model off the air for as long as it needs to load from the already-warm disk (typically well under a minute, and the weights are not re-downloaded). Every later add is seamless, because the card then has unclaimed headroom again. The preview says which of the two you are about to do: primary_restart_required on the fit-check answer, and on the 202 the add returns. If the reload does not come back the engine is restored to the share it held and the add fails, so a card is never left with its primary down.

A removal never reloads anything. The freed share becomes unclaimed headroom for the next add; the models that stay are left running exactly as they were, because growing one back would cost a restart of a model that is answering to buy capacity nobody asked for.

One edit at a time. A second add or removal while one is in flight is refused with 409, and so is stopping the service, since powering the VM off mid-download leaves a half-fetched model. Deleting the service is not refused: it removes the VM and every engine on it.

Watching an edit. The service detail carries companion_mutation (the model, add or remove, the node's status and its own progress line) while the edit is in flight, and after a failure until another edit replaces it. A completed edit reports nothing there: its outcome is served_models. An add mints nothing until the new engine answers for itself, so a model becomes callable at the moment it can answer, and a failed add leaves the card exactly as it was.

What a removal costs. Routing to the model stops the moment the call is accepted, before the node is told to stop the engine, so the only requests that fail are those already in flight on that model. Everything else on the card is untouched. The model's weights are then deleted from the data disk, which frees the space for the next model; adding it back later downloads them again and binds fresh ports, because a port slot is never reused.

The primary is not editable this way. Changing the model a service was created around remains the in-place switch, which stays single-model-only: it is refused with 409 while any companion is served. Removing the last companion returns the service to a single-model configuration and makes switching available again.

POST /inference-services/fit-check previews the edit before you make it: pass service_id instead of a model and a plan, and companion_models as the full set the card would serve. It runs the same function that gates the add, so a stack it reports as fitting is a stack the add accepts. In the console this is the Add model button on the service's Model panel: the picker offers what this card can still take, the meter re-draws the whole card as it would stand afterwards, and a stack that does not fit blocks the button with the server's own reason. Each companion row carries the affordance that removes it; the primary row does not.

How a call reaches the right model

One hostname, one key, one certificate. The model field picks the engine:

# chat, on the primary
curl $INF_BASE/chat/completions -H "Authorization: Bearer $FDB_INF_KEY" \
  -d '{"model": "foundrydb_managed/mistral-7b", "messages": [{"role":"user","content":"Hi"}]}'

# embeddings, on the same host and the same key
curl $INF_BASE/embeddings -H "Authorization: Bearer $FDB_INF_KEY" \
  -d '{"model": "foundrydb_managed/bge-m3", "input": "The quick brown fox."}'

# reranking, likewise
curl $INF_BASE/rerank -H "Authorization: Bearer $FDB_INF_KEY" \
  -d '{"model": "foundrydb_managed/bge-reranker-v2-m3", "query": "capital of France", "documents": ["Paris is the capital of France.", "Berlin is in Germany."]}'

The bare-name convenience widens with the stack: on this service's own hostname bge-m3 is accepted as well as foundrydb_managed/bge-m3, so an app that hardcodes an unprefixed id reaches the model that carries that name rather than the primary. GET /v1/models on the endpoint lists the whole stack, primary first, then any active LoRA adapter, then the companions, so an attached app shows all of them in its picker with no configuration. The service detail page lists the same models, each with its serving kind and whether that particular engine parses tool calls, and the Agents & IDEs tab writes its snippets for whichever chat model of the stack you select.

What is refused, and why

All of these are refused at create, before a GPU is rented:

• A model that is not in the curated catalog, including a Hugging Face pull as the primary. Shares are computed from published per-model facts the platform has verified; a customer repository's facts are fetched best-effort, and a card must not be partitioned on a guess.
• An image model. Diffusion reserves memory unpredictably, so no fixed fraction of a shared card can be computed for it.
• A license-gated model (Llama, FLUX.1 schnell). A gated pull needs a token, which is a per-model credential path the companion payload deliberately does not carry. Serve a gated model as the primary of its own service.
• A duplicate of the primary or of another companion. Two engines answering the same name on one host make that model unroutable.
• More than three companions.
• A model your organization already serves on another inference service. One model answers to one endpoint per organization, so the refusal names the service that already has it: either call it there or drop it from the stack.
• A stack that oversubscribes the card, refused with each model's share named and either the companion to drop or the plan to move to.
• The serverless SKU. There is no card of your own to share.

Limits in this version

• The primary is fixed for the life of the service. Companions come and go on a running card (Changing the stack later), but the model a service was created around changes only through the in-place switch, which is refused with 409 while companions are served. The switch swaps a disk under one model; a card serving several is not that operation.
• One stack edit at a time, and only while the service is Running. A second edit, and a stop during one, are refused with 409.
• LoRA adapters are primary-only. enable_fine_tuned_serving applies to the primary model, and no companion is ever started with adapter flags.
• No pre-baked fast path. A volume template holds one model's weights, so a stack downloads its models during provisioning and takes longer to reach Running than a single pre-baked model would. The data disk is sized for every checkpoint on the card.
• Stop, start, restart and delete act on the whole stack. A service reaches Running only once every model on the card is serving, and a failure during provisioning names the model that failed.
• Billing is unchanged. You rent one card and pay one GPU-hour rate for it, whatever it serves. Per-model usage rows still key on the model string, so the usage tab attributes calls to the model that answered them.

How the security model actually works

Be clear on what the isolation is and is not.

• Isolation is provided by TLS, per-request authentication, and EU residency, not by a network air gap. Every call is authenticated with an inference key, checked against your organization's limits, and forwarded over a certificate-pinned TLS connection.
• The GPU serving port (443) is firewall-allowlisted to the platform data plane. It is never open to the public internet, and direct customer-to-GPU calls are deliberately not offered, because that would bypass authentication, rate limits, cost ceilings, EU residency, and metering.
• You never hold the GPU's serving token or its DNS name. The platform holds the serving credential, swaps your inference key for it internally, and forwards the request. You only ever talk to the proxy.

In short: you get a private, authenticated, EU-resident path to a dedicated GPU. You do not get, and should not expect, a raw socket to the card.

The same EU edge fleet and the same fdb-inf key serve both managed models and bring-your-own providers. The provider prefix in the model id decides which kind of upstream a call goes to; for a foundrydb_managed/ call, the endpoint hostname it was sent to then decides which of your services answers.

One edge, one key · managed-vs-BYO fork by model prefix

ROUTE prefix foundrydb_managed/ → dedicated EU GPU

Client / Appone fdb-inf key/v1 →Edge forwarderprefix · EU · cost circuitfoundrydb_managed/ →Dedicated EU GPUvLLMbyo prefix →BYO providerorg key (encrypted)

Client · one keyEdge forwarderManaged · dedicated EU GPUBYO · third-party providerunselected fork (dashed)

One key, one endpoint. A foundrydb_managed/ prefix routes to your dedicated EU GPU; a BYO prefix (openai/, anthropic/, mistral/, azure_openai/, groq/) routes to that provider with your own encrypted key, under the same EU residency and monthly cost circuit.

Calling a running service

You do not call the GPU directly. You call your service's own endpoint, which authenticates the request, applies your limits, and forwards it to the GPU serving your model.

Managed GPU serving · edge → forwarder → vLLM

STREAM tokens stream back from vLLM to the client

Client / Appfdb-inf-… keyrequest →EU Edge · CaddyTLS · RPM · tokens · EUloopback →Forwarder*.inf.foundrydb.com→GPU VM · vLLMdedicated EU cardside-emit → metering

ClientEU edge (Caddy)Forwarder sidecarGPU · vLLMMetering (side-emit)out-of-band (dashed)

The serving data path: the EU edge terminates TLS and enforces your key's limits, a sidecar forwards over loopback to the GPU running vLLM, tokens stream back, and a metering event is side-emitted for billing.

• Base URL: the service's own endpoint_base_url, which is https://<endpoint_hostname>/v1. Read it from GET /inference-services/{id}; do not assemble it yourself, and do not use a shared or global hostname. Both SKUs get one.
• Model string: foundrydb_managed/<served_model_name>, for example foundrydb_managed/mistral-small
• Auth: an organization inference key (fdb-inf-...) as a Bearer token, not a platform API token

The hostname selects the service

The endpoint hostname is how the platform knows which of your services a request is for. A foundrydb_managed/ call sent to any other host, including a shared gateway hostname, resolves to no service and is refused with 412 provider_not_configured. Read endpoint_base_url from the service and use that.

Grab the base URL from the service:

export INF_BASE=$(curl -s https://api.foundrydb.com/inference-services/$SERVICE_ID \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN" | jq -r .endpoint_base_url)
# https://my-llm-a1b2c3.inf.foundrydb.com/v1

endpoint_base_url is empty until the endpoint is minted, which happens during provisioning; poll until it is set.

On the model field

The foundrydb_managed/ prefix is the canonical form and always works: it is what selects the managed data plane, exactly as openai/ or mistral/ selects a bring-your-own provider. The hostname then picks which of your services answers.

As a convenience, on your service's own hostname the bare served model name is also accepted, so an app that hardcodes mistral-small works unchanged. That alias applies only there: on any other host the prefix is required, and a bare name is refused with 400. LoRA adapter names are never bare-accepted and always need the prefix. GET /v1/models on the endpoint returns every model the service serves, with the prefix on each id, so a client that echoes an id back always routes correctly. A service serving companion models answers for each of them on this same hostname, and the bare-name alias covers all of them.

Get an inference key

Inference keys are minted per organization under your inference settings. The secret is shown exactly once, so store it when you create it.

curl -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" \
  https://api.foundrydb.com/organizations/$ORG_ID/inference/keys \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prod-app",
    "monthly_token_limit": 5000000,
    "rate_limit_rpm": 120
  }'

The response returns the full secret once, in the secret field:

{
  "key": { "name": "prod-app", "monthly_token_limit": 5000000, "rate_limit_rpm": 120 },
  "secret": "fdb-inf-3f4a...",
  "activation_note": "The key activates at the inference endpoint within a few seconds, once the edge fleet applies it. A request sent immediately after minting can answer invalid_key; retry shortly."
}

monthly_token_limit is required and must be positive: there is no unlimited key. rate_limit_rpm caps the key's requests per minute and defaults to 60.

The activation_note is not boilerplate. The key's hash reaches the serving fleet through a config reconcile that the mint requests immediately, so a call sent in the same breath as the mint can still answer invalid_key. That is propagation, not a bad key: wait a couple of seconds and retry. Scripts that mint and call in one go should allow for it.

Chat completions with curl

curl $INF_BASE/chat/completions \
  -H "Authorization: Bearer $FDB_INF_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "foundrydb_managed/mistral-small",
    "messages": [
      {"role": "user", "content": "Write a haiku about databases."}
    ]
  }'

Set "stream": true to receive a server-sent event stream. The proxy injects the usage chunk so the final event always carries token counts.

With an OpenAI SDK

Point any OpenAI-compatible client at your service's endpoint_base_url and use your inference key as the API key.

from openai import OpenAI

client = OpenAI(
    base_url="https://my-llm-a1b2c3.inf.foundrydb.com/v1",  # the service's endpoint_base_url
    api_key="fdb-inf-...",  # your organization inference key
)

response = client.chat.completions.create(
    model="foundrydb_managed/mistral-small",
    messages=[{"role": "user", "content": "Write a haiku about databases."}],
)
print(response.choices[0].message.content)

Embeddings, reranking, and images

The data plane exposes seven call routes, and each model answers on the one that matches its kind:

Route	Model kind	Catalog model	Request body
POST /v1/chat/completions	Chat	the four chat models	messages
POST /v1/completions (legacy)	Chat	the four chat models	prompt (string or array)
POST /v1/responses	Chat	the four chat models	input (string or message array)
POST /v1/embeddings	Embeddings	bge-m3	input (string or array)
POST /v1/rerank	Reranker	bge-reranker-v2-m3	query + documents
POST /v1/score	Reranker	bge-reranker-v2-m3	text_1 + text_2
POST /v1/images/generations	Text-to-image	z-image-turbo, flux-schnell	prompt (+ size, n)

/v1/completions is the pre-chat OpenAI text-completions API (prompt in, choices[].text out), served natively by the same chat models with identical streaming, usage, and metering semantics, so older SDKs and tools built on the legacy API work without a shim.

/v1/responses is the OpenAI Responses API, the successor format the OpenAI Agents SDK and newer tooling default to (input in, typed output items out, typed SSE events when streamed). Managed chat models serve it natively, and BYO providers that speak the OpenAI wire format (OpenAI itself included) pass through; Anthropic and Azure OpenAI answer 400 on this route. Usage and metering work exactly as on chat, including streamed calls, whose tokens are read from the terminal response.completed event.

GET /v1/models also answers on the endpoint, returning the one model that endpoint serves.

All of them use the same fdb-inf key and the foundrydb_managed/<served_model_name> model string, and all of them enforce the same rate limits, token ceilings, EU residency, and metering. Reranking is Jina/Cohere-compatible; scoring is vLLM-compatible; image generation follows the DALL-E-compatible schema and returns base64 image data.

# Embeddings
curl $INF_BASE/embeddings \
  -H "Authorization: Bearer $FDB_INF_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "foundrydb_managed/bge-m3", "input": "the quick brown fox"}'

# Reranking
curl $INF_BASE/rerank \
  -H "Authorization: Bearer $FDB_INF_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "foundrydb_managed/bge-reranker-v2-m3",
       "query": "what is the capital of France?",
       "documents": ["Paris is the capital of France.", "Berlin is in Germany."]}'

Image generation limits

Image calls are metered per generated image, and that meter is blind to resolution and to how many denoising steps a request asks for. The published per-image prices assume one image at the resolution the model was trained for and its distilled step count, so three caps hold that assumption true on platform-served requests:

Limit	Value	Behaviour when exceeded
Pixels per image	1024 x 1024 (the pixel product, so 896x1152 and 1536x640 fit, 1024x1792 and 2048x2048 do not)	Refused with 400
Images per request (n)	4	Refused with 400
num_inference_steps	9	Bounded to 9, and the response carries X-Foundrydb-Image-Steps-Clamped with the count actually used

An over-cap size or count is refused, never quietly shrunk: a caller who asked for 2048x2048 and silently received 1024x1024 would have no way to tell. The step count is the one parameter bounded rather than refused, and even that is announced on the response header. The ceiling of 9 is the highest recommended count across both catalog image models (Z-Image Turbo runs 8 sampling steps plus one, FLUX.1 schnell runs 4), so no supported model is capped below the count it is meant to run at.

These caps apply to platform-served (foundrydb_managed/) calls only. A call routed to your own third-party provider spends that provider's quota on your key and is not capped here.

Image content policy

Prompts on platform-served image calls are checked before the request reaches a GPU. Both prompt and negative_prompt are read, and a request asking for sexual content involving minors, non-consensual sexual content, or sexual imagery of a real identifiable person is refused with 400 image_policy_refused. The refusal names the policy and never quotes your prompt back, and nothing is quietly stripped from a prompt and forwarded: a request is either served as written or refused. Refusals are logged with the organization, the key, the category, and a SHA-256 hash of the normalized prompt, so repeat abuse is investigable; the prompt text itself is never stored, never logged, and never returned. The policy is tuned for precision, so ordinary prompts that merely share vocabulary with it are served normally (nude color palette, a classical nude marble statue, the naked eye, a baby elephant). Like the caps above, it applies to platform-served calls only.

A prompt check is not the whole control, because the model decides what the pixels are: a prompt with nothing objectionable in it can still produce an image the platform will not hand back. So generated images are also classified after the engine answers and before the response is returned, by a classifier the platform runs itself, so no image ever leaves the platform. The control ships in two settings. In shadow it classifies and records and blocks nothing, which is how the false-positive rate is measured on real traffic before it is allowed to refuse anyone. In enforce a flagged image is refused with the same 400 image_policy_refused, the image is neither returned nor stored, and the generation is not billed, exactly as every other failed call is treated. Refusals are logged with the organization, the key, the model, the category, and a SHA-256 hash of the image bytes; the image itself is never logged or kept. A classifier outage never takes image generation down: the images are served and the gap is recorded.

Creating a service

Serverless

Omit plan_name (or set "inference_sku": "serverless"). There is no plan, no zone, and no serving knobs: the card is the platform's and its serving shape is fixed. Nothing is billed by the hour, and no prepaid balance is required.

curl https://api.foundrydb.com/inference-services \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-serverless-llm",
    "inference_sku": "serverless",
    "inference_config": {
      "model_id": "mistral-small",
      "model_source": "curated"
    }
  }'

model_id must be a model a pool is serving right now (see Two SKUs). Two refusals are worth telling apart: 503 means no pool serves that model, so pick a different one or create a dedicated service; 409 means every pool for it is at its binding ceiling, so retry once capacity is added.

Serverless also refuses model_source: "huggingface", enable_fine_tuned_serving, and any non-zero keep_warm_minutes: each of those needs a card of your own.

Dedicated

Set plan_name to a GPU plan. Provisioning is asynchronous: a POST persists the service in Pending and the controller drives it to Running. Your prepaid balance must cover the GPU plan, or the create returns 402.

curl https://api.foundrydb.com/inference-services \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-llm",
    "plan_name": "gpu-l40s-1",
    "zone": "fi-hel2",
    "inference_config": {
      "model_id": "mistral-small",
      "model_source": "curated"
    }
  }'

The response returns the service in Pending. Poll GET /inference-services/{id} until status is Running and endpoint_base_url is set:

curl https://api.foundrydb.com/inference-services/$SERVICE_ID \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN"

List your services with GET /inference-services.

License acceptance

license_accepted goes inside inference_config, not at the top level. It is required for a license-gated curated model (Llama), and for any Hugging Face model you bring yourself.

curl https://api.foundrydb.com/inference-services \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-llama",
    "plan_name": "gpu-b200-1",
    "zone": "fi-hel2",
    "inference_config": {
      "model_id": "llama-3.3-70b",
      "model_source": "curated",
      "license_accepted": true
    }
  }'

Omitting license_accepted for a gated model returns 400.

Pricing

The two SKUs are billed on entirely different meters. Read the one that applies to your service; the other figure is a usage signal, not a bill.

Serverless pricing

Serverless is billed per token, and per generated image for the image models. Prices below are EUR, exclusive of VAT.

Model	Prompt, per 1M tokens	Completion, per 1M tokens
bge-m3	0.07	0.07 (never metered: embeddings report no completion tokens)
bge-reranker-v2-m3	0.12	0.12 (never metered)
mistral-7b	0.20	0.65
mistral-small	0.28	0.85
mixtral-8x7b	0.35	1.10
llama-3.3-70b	0.75	1.30

Image model	Per generated image
z-image-turbo	0.018
flux-schnell	0.025

The live rate card is authoritative. Read it any time, and quote from it rather than from this page, because it is the same table the meter bills from:

curl https://api.foundrydb.com/inference-services/model-rates \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN"

{"models": [
  {"model_id": "mistral-small", "rate_unit": "tokens",
   "prompt_microcents_per_1k": 28000, "completion_microcents_per_1k": 85000,
   "effective_from": "2026-08-16T21:29:22Z"},
  {"model_id": "z-image-turbo", "rate_unit": "image",
   "prompt_microcents_per_1k": 0, "completion_microcents_per_1k": 0,
   "image_microcents_per_unit": 1800000,
   "effective_from": "2026-08-16T22:30:38Z"}
]}

Rates are quoted in microcents. Divide prompt_microcents_per_1k or completion_microcents_per_1k by 100,000 for EUR per 1M tokens, and image_microcents_per_unit by 100,000,000 for EUR per image. rate_unit tells you which pair to read. A model absent from the listing has no published price yet, which is not the same as free: treat it as pricing on request.

The free tier

Every organization gets the first 1,000,000 tokens each calendar month free on platform-served inference. Allowance tokens are metered exactly like paid ones and simply recorded at zero cost, so the allowance is spent before any billing starts, and a call that straddles the boundary is charged only for the share of its tokens that fell outside.

Two kinds of call do not draw on it: a call routed to your own third-party provider, which is billed on that provider's account and costs the platform nothing, and an image generation, which is priced per image and reports no tokens at all. The allowance resets at each month boundary.

Check what is left:

curl https://api.foundrydb.com/organizations/$ORG_ID/inference/usage \
  -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD"

{"from": "...", "to": "...", "group_by": "model", "rows": [],
 "free_tier": {"cycle_month": "2026-08-01T00:00:00Z", "monthly_tokens": 1000000,
               "tokens_used": 142_390, "tokens_remaining": 857_610}}

free_tier always describes the current calendar month whatever window you asked for, because the allowance is a monthly meter rather than an aggregate of the window.

Dedicated GPU plans and pricing

A dedicated service is billed by the GPU-hour for as long as the card is allocated, not per token: its per-token cost stays zero. Each curated model has a default GPU plan that fits its weights on a single card, and you can select a larger plan for more context headroom or throughput. All plans are in fi-hel2. Prices are EUR per hour, exclusive of VAT, confirmed against the provider rate card on 2026-08-17.

Plan	GPU	Cards	VRAM per card	Price / hr
gpu-l4-1	NVIDIA L4	1	24 GB	0.73
gpu-l4-2	NVIDIA L4	2	24 GB	1.48
gpu-l4-3	NVIDIA L4	3	24 GB	2.25
gpu-l40s-1	NVIDIA L40S	1	48 GB	1.39
gpu-l40s-2	NVIDIA L40S	2	48 GB	2.60
gpu-l40s-3	NVIDIA L40S	3	48 GB	4.69
gpu-h100-1	NVIDIA H100	1	80 GB	2.24
gpu-h100-2	NVIDIA H100	2	80 GB	4.48
gpu-h100-4	NVIDIA H100	4	80 GB	8.95
gpu-h100-8	NVIDIA H100	8	80 GB	17.90
gpu-b200-1	NVIDIA B200	1	192 GB	5.63
gpu-b200-2	NVIDIA B200	2	192 GB	11.25
gpu-b200-4	NVIDIA B200	4	192 GB	22.50
gpu-b200-8	NVIDIA B200	8	192 GB	45.00

The multi-card rungs serve larger models with tensor parallelism (tensor_parallel_size must divide the plan's card count). A service that ends in a terminal failure state is excluded from billing. Stopping a service, or letting keep-warm stop it for you, ends the meter; the weights stay on the data disk so the next start is warm.

Reading what a service actually cost

GET /inference-services/{id}/usage reports both meters for one service, so you never have to work out which applies:

curl "https://api.foundrydb.com/inference-services/$SERVICE_ID/usage?since=24h" \
  -H "Authorization: Bearer $FOUNDRYDB_TOKEN"

The response carries a bucketed series and rolled-up totals over the window (calls, errors, input/output/total tokens, images, cost in microcents, average latency), plus a month_to_date block that ignores the window entirely. In month_to_date, tokens is what a serverless service has run up this month and gpu_hour (billed_hours, hourly_rate_eur, cost_eur) is what a dedicated one has. images stays zero for a text model and is the figure that moves for an image model, which meters per image rather than per output token.

Lifecycle

The lifecycle below is the dedicated SKU, which provisions a card. A serverless service has no GPU to provision, park, or tear down: it binds to a pool at create, reaches Running once its endpoint hostname is bound, and a delete releases the binding. Stop, start, and keep-warm are dedicated-only, because there is no card of yours to stop.

Pending  →  (network, GPU server, vLLM install, model warmup, certificate)  →  Running  →  (DELETE) teardown

• Pending to Running: the controller provisions the GPU server, installs the vLLM runtime, warms the model, and installs the serving certificate. Poll GET /inference-services/{id} for status; while a deploy is in flight the response carries a live provisioning_message (weight download progress, server start, the readiness wait).
• Running: the service serves traffic and accrues GPU-hour charges.
• Stop and start: POST /managed-services/{id}/stop parks the GPU: compute billing stops, the downloaded weights stay on the service's data disk, and a later POST /managed-services/{id}/start reloads them warm in about a minute. The console's service page carries both actions.
• Keep-warm: keep_warm_minutes auto-stops the service after that many minutes with no inference activity, ending the GPU-hour meter until you start it again. Zero, the default, never auto-stops; any other value must be between 5 and 10080 (the floor keeps bursty traffic from flapping through stop/start cycles). Change it without re-provisioning with PATCH /inference-services/{id}/keep-warm.
• Delete: DELETE /inference-services/{id} returns 202 and tears down the vLLM runtime, ingress, certificate, DNS, floating IP, and the GPU server asynchronously. Billing stops when the service is no longer running.

Pre-baked model volumes

For curated models the platform maintains a volume template per model and zone: a block-level snapshot of the model's weight files, harvested automatically the first time a model runs. When a template exists, a new service's data disk is created as a server-side clone of it, so the multi-gigabyte model download disappears from provisioning entirely. Measured on staging: Mistral Small 3.2 24B reached Running in about 7 minutes pre-baked against roughly 35 minutes with a full download, and smaller models land in 3 to 6 minutes. Models with a template show a pre-baked badge in the create flow and the catalog; models without one simply download as usual, and nothing about a template can ever fail a provision.

License-gated curated models (Llama 3.3 70B, FLUX.1 schnell) are pre-baked too: their volumes contain weights obtained under the platform's Hugging Face account and are provided to your service under each model's own license terms. Llama 3.3 deployments are Built with Llama, the Llama 3.3 Community License governs that model's use, and a copy of the license ships with the weights on the service disk.

Switching models in place

A service is not married to the model it was created with. POST /inference-services/{id}/switch-model with {"model_id": "<catalog id>"} changes which curated model it serves while keeping the service itself: the same GPU plan, the same endpoint, the same keys, and the same billing. Callers change only the model name they pass. The call returns 202 with the service in SwitchingModel; poll GET /inference-services/{id} until it comes back to its previous state serving the new model. The console carries the same action on the service page.

Mechanically it is a disk swap, not a re-provision. The platform stops the serving process, detaches the service's model volume, and attaches a new one in its place: a server-side clone of the target model's pre-baked volume template when one exists for the service's zone (minutes, with no weight download), or a fresh volume that takes the ordinary Hugging Face download otherwise. The serving unit is then re-rendered for the new model's configuration and started, and only once the new model is actually serving is the old volume deleted. A failure before that point re-attaches the old disk and the service goes back to serving the model it had.

Switching in place is a dedicated-only action, like keep-warm and LoRA promotion: a serverless service is bound to a pool that serves one model, so a switch is refused with 409. Create a second serverless service on the other model instead, which costs nothing while it is idle.

A service serving companion models cannot switch at all: the operation swaps a disk under one model, and a card serving several is not that operation, so it is refused with 409 while any companion is served. Removing the companions returns the service to a single-model configuration and makes the switch available again.

Three constraints apply. Targets are curated catalog models: a Hugging Face model you bring yourself is not a switch target. The target must fit the VRAM of the plan the service already runs on, because a switch never resizes the service; asking for a model that does not fit returns 409 naming the smallest plan that would take it. And LoRA adapters are bound to the base model they were trained against, so an active adapter must be demoted first; the switch is refused with 409 while one is serving, and uploaded versions are left untouched.

A Stopped service switches too. The disks are swapped while the GPU stays parked, nothing starts, and the next start brings up the new model in the usual warm-start time.

Fine-tuned adapters (LoRA)

You can serve your own LoRA fine-tuned adapters on a managed GPU, live, without a restart. The base model stays loaded and each adapter is served under its own model string, so one GPU can host several fine-tunes of the same base.

Fine-tuned adapter lifecycle · register → promote → hot-swap → rollback

PROMOTE v1 hot-loaded into vLLM (no restart) · active

Files bucketadapter artifactsupload →Serving registryuploaded → activepromote →Adapter v1active (one per served name)hot-load →GPU · vLLMruntime LoRA · no restart

Files bucketServing registryuploadedactive (one per name)supersededGPU · vLLM (no restart)

The adapter lifecycle: register uploads a version to Files and records it (uploaded); promote hot-loads it into vLLM (active); a newer promotion hot-swaps live and supersedes the old one; re-promoting a prior version rolls back. At most one version is active per served name, and the GPU never restarts.

LoRA serving needs a card of your own, so it is dedicated-only: a serverless create refuses enable_fine_tuned_serving, and a promote onto a serverless service is refused with 409.

Turn it on at create time. Set enable_fine_tuned_serving: true in inference_config when you create the service. The GPU comes up with vLLM's runtime LoRA API enabled. max_loras (default 4) bounds how many adapters can be loaded concurrently and max_lora_rank (default 32) bounds their rank; both are only meaningful when fine-tuned serving is on. A model that was created without this flag cannot promote adapters.

A LoRA adapter is only valid on its exact base model. The adapter you register must have been trained against the base model the service serves (its catalog id or Hugging Face repo). Promotion rejects a mismatch rather than serve garbage.

Register an adapter. In the console, open the service's Adapters tab and choose Register adapter: pick a served model name and version, then select the two artifact files (adapter_model.safetensors and adapter_config.json). The browser uploads them straight to your organization's Files bucket with a presigned URL and records the version. The version enters the registry with status uploaded; it is not serving yet.

Programmatically, upload the artifacts to your Files bucket and then call the registration endpoint:

curl -X POST https://api.foundrydb.com/inference-services/adapters \
  -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASS" \
  -H "Content-Type: application/json" \
  -d '{
    "organization_id": "<org-uuid>",
    "base_model_id": "mistral-7b",
    "served_model_name": "support-tuned-mistral",
    "version": 1,
    "files_bucket": "<your-files-bucket>",
    "files_key_prefix": "inference-adapters/support-tuned-mistral/1/",
    "adapter_sha256": "<sha256 of adapter_model.safetensors>",
    "size_bytes": 4202704
  }'

Promote it to serve it. Registered (uploaded) versions appear in the Adapters tab for a service on the matching base model; select Promote. The platform downloads the weights from Files, verifies the SHA-256 you registered, and hot-loads them into vLLM with no restart. The version becomes active and the service answers to it as foundrydb_managed/<served_model_name> on the same OpenAI-compatible endpoint, with the same fdb-inf key, rate limits, token ceilings, EU residency, and metering as the base model.

Once a version is active you can try it straight from the console Playground, whose model picker lists the base model and every active adapter on the service, so you can send a prompt to the promoted fine-tune live without wiring up a client first.

curl $INF_BASE/chat/completions \
  -H "Authorization: Bearer $FOUNDRYDB_INFERENCE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "foundrydb_managed/support-tuned-mistral",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

An adapter is called on the same service endpoint as its base model, and always with the foundrydb_managed/ prefix: the bare-name convenience does not extend to adapter names.

Hot-swap and rollback. Register a newer version and promote it to hot-swap the served name to the new weights with no downtime. Promote a prior (superseded) version to roll back. GET /inference-services/{id}/adapters lists a service's bound versions (active plus superseded history) together with the organization's uploaded, not-yet-promoted versions for that base model, so a freshly registered adapter is visible and promotable from the tab.

Stop serving. To take a fine-tune out of service without promoting a replacement, demote it with POST /inference-services/{id}/adapters/{adapterId}/demote (or Stop serving in the Adapters tab). The version becomes superseded and the adapter is unloaded from the running vLLM, so its served name stops answering, its max_loras slot is freed, and clients still calling foundrydb_managed/<served_model_name> start getting errors. The base model keeps serving, and the version stays promotable. Demote is the prerequisite for an in-place model switch and for deleting a version, both of which are refused while an adapter is active.

Discovery. On a service's dedicated inference hostname, the OpenAI-compatible GET /v1/models lists the base served model followed by every active adapter, each as foundrydb_managed/<served_model_name>, so an attached app populates its model picker with the fine-tunes automatically. A promote or a demote reaches that listing on the next edge configuration reconcile.

Delete a version. When you no longer need an adapter version, remove it from the registry with DELETE /inference-services/adapters/{adapterId} (or the delete action in the Adapters tab). This drops the version from the serving registry so it can no longer be promoted.

Reference

• Inference API reference for the full /inference-services request and response schemas
• Inference Proxy for provider routing, keys, settings, and usage
• AI Data Platform for vector search, embedding pipelines, and natural-language query

Next steps

Start serverless, because it costs nothing until you call it and the first million tokens each month are free. List the models a pool is serving, create a service on one of them, mint an inference key, read endpoint_base_url off the service, and send your first foundrydb_managed/mistral-small chat completion to it.

Move to a dedicated GPU when you need a model serverless does not offer, a Hugging Face model of your own, or LoRA adapters. When you are done testing a dedicated service, stop or delete it: it accrues GPU-hour charges for as long as the card is allocated.