Fine-Tuning
Fine-tuning takes a validated training dataset plus a curated base model and produces a versioned LoRA adapter you own. The dataset comes out of your organization's Files bucket, the training runs on a platform GPU in a European zone, and the adapter lands back in your Files bucket and registers into the adapter registry, ready to promote onto a Managed Inference service.
Nothing in that loop leaves the platform. There is no third-party training API, no external model host, and no step where your examples are handed to another vendor.
What it is
The Phase 1 method is supervised fine-tuning (SFT) with LoRA. You supply instruction and response pairs as JSONL. The trainer applies a low-rank adapter to the attention projections of the base model and trains only those weights. The base model itself is never modified, which is what makes the output small (tens of megabytes), fast to train, and hot-loadable onto a running GPU without a restart.
| Concept | What it is |
|---|---|
| Training dataset | A JSONL artifact in your Files bucket, registered and validated server-side. Once a run consumes it, the row is immutable. |
| Fine-tuning run | The audited lifecycle of one training job: dataset plus base model in, adapter out, with the accepted cost ceiling recorded. |
| Adapter | The trained LoRA weights, versioned in the serving registry, promotable onto an inference service. |
Two API surfaces are involved: /training-datasets for the inputs and /fine-tuning/runs for the jobs. Serving reuses the existing /inference-services adapter endpoints; fine-tuning does not define a second serving path.
For a runnable walkthrough, see Fine-Tune and Serve a Model.
Sovereignty
The reason to run this inside the platform rather than against an external fine-tuning API is that every artifact in the loop stays inside one boundary you can point at.
- Your data. The dataset artifact lives in your own Files bucket in a European region. The controller reads it once at registration to validate and hash it. The trainer reads it through a time-limited presigned URL, never a long-lived credential.
- The weights. Base model weights are pulled from Hugging Face onto the training GPU and cached there. The trained adapter is written back only to your bucket, under a prefix scoped to the run.
- The training. Runs execute on a dedicated platform GPU in
fi-hel2(Helsinki). One tenant per card. The card is rented for the run and released when the run reaches a terminal state. - The lineage. The dataset's example count, byte size, and SHA-256 are recorded at registration. The run records the exact hyperparameters it trained with, the accepted ceiling, the priced estimate, and the content hash of the adapter it produced. That chain is the compliance trail: for any served adapter you can name the dataset bytes it came from.
The credential the trainer uses to publish its adapter is minted per run, scoped to that run's key prefix only, encrypted before it is written to the agent task, and revoked when the run finishes.
Lifecycle
A run is created in Queued and advanced by a controller-side dispatch worker. Nothing about the lifecycle is client-driven after creation.
| State | What is happening |
|---|---|
Queued | Accepted and waiting for a training GPU. The worker re-derives the price gate before it rents anything. |
Preparing | A GPU service named ft-<8 hex of run id> is being provisioned in fi-hel2. |
Training | The trainer is running on the GPU. latest_loss and status_detail carry live progress (step 8/12). |
Validating | The adapter has been uploaded and hashed; the registry row is being written. |
Complete | output_files_key, output_sha256, and output_adapter_id are populated. The GPU has been torn down. |
Failed | status_detail carries the reason. The GPU has been torn down. |
Cancelled | You cancelled the run. The GPU is released on the worker's next sweep. |
Cancellation is deliberately a state flip only. The worker owns every rented resource, so it releases the GPU itself rather than the API doing teardown inline. A run cancelled mid-Training discards the in-flight work: a partially trained adapter was never uploaded and has no verifiable digest to register, so there is nothing honest to hand back.
GPU teardown is guaranteed on three independent paths: the completion path, the failure path, and a reap sweep that looks for training GPUs whose run is already terminal. A crashed controller does not leak a card, and a re-provision after a crash adopts the existing GPU by name instead of renting a second one.
Dataset format
One JSON object per line. Every non-empty line must carry a non-empty instruction (or prompt) and a non-empty response (or completion). Both key spellings are accepted, so datasets written for either convention register without rewriting.
{"instruction": "Who are you?", "response": "I am FoundryBot, the assistant built into the FoundryDB managed data platform."}
{"instruction": "What is your name?", "response": "My name is FoundryBot."}
{"prompt": "Introduce yourself.", "completion": "I am FoundryBot, your assistant for FoundryDB."}
Validation happens server-side at registration, not at training time, so a malformed corpus is refused before it can cost you GPU minutes. The rules:
- Every non-empty line must parse as a JSON object. A malformed line is refused with its line number.
- Every record must carry both fields, non-empty. A record missing one is refused with its line number.
- Blank lines are skipped and do not count as examples.
- An artifact with zero examples is refused.
- The artifact must be at most 536870912 bytes (512 MB).
On success the registration response records example_count, size_bytes, and the sha256 of the artifact. Those three numbers are the lineage anchor: a run references the dataset id, and the dataset row is immutable from the moment a run consumes it. Deleting a consumed dataset returns 409.
Sequences are truncated to 512 tokens during training. Prompt tokens are masked out of the loss, so the model trains on the responses, not on repeating your questions back.
Base models
Only curated catalog models that serve the generate task can be fine-tuned. Embedding and reranker models are refused: there is no instruction-following adapter to train on them. The refusal names the task the model actually serves.
Training runs on a fixed plan, gpu-l4-1 (one NVIDIA L4, 24 GB), in fi-hel2. The measured profile behind the cost estimate is mistral-7b in bf16 with LoRA rank 16, which trains in about 15 GB of VRAM at roughly 0.14 seconds per optimizer step. The plan is fixed on purpose: the estimate and the step cap are only honest while the card that produced the measurement is the card that runs the job. A base model whose VRAM minimum exceeds that card is refused at run creation with both numbers, before any GPU is rented.
Hyperparameters
All four knobs are optional. Omitted or zero values take the platform default. The run stores exactly what it trained with, so the recorded configuration is always fully determined.
| Knob | Default | Maximum |
|---|---|---|
rank | 16 | 64 |
epochs | 3 | 20 |
batch_size | 4 | 16 |
learning_rate | 0.0002 | 0.01 |
Out-of-range values are refused at creation with the bound that was violated, before anything is rented.
Cost model
Fine-tuning is billed as GPU time, not per token. You are quoted before you commit, and the ceiling you accept is an authorization gate rather than a warning.
How the estimate is derived. The step count is ceil(example_count / batch_size) * epochs. That is priced at the published hourly rate of the training plan (gpu-l4-1 at EUR 0.65 per hour, from EUR 474.50 per month over 730 hours) using the measured 0.14 seconds per step, then doubled. The doubling is the published preparation allowance: the card is rented from the moment provisioning starts, so base-weight download, environment setup, and adapter upload are all billed alongside the optimizer steps. You are quoted the larger number, which means a run can never be cheaper to quote than to execute.
The ceiling is a hard gate. If the estimate exceeds cost_ceiling_eur, the run is not created. The refusal carries both numbers and the three ways out:
estimated cost 3.41 EUR exceeds cost_ceiling_eur 2.00 EUR;
raise the ceiling, shrink the dataset, or lower epochs
The worker re-derives that same gate before it rents a GPU, so a run cannot slip through on a stale price.
The ceiling also becomes a stop condition. The trainer receives a hard optimizer-step cap computed from the raw ceiling at the GPU hourly rate. A run whose loss curve or sequence lengths make it slower than the measured profile stops at that step count rather than overrunning the budget you accepted. Because the quote includes the preparation allowance and the cap does not, the cap can never be the binding limit on a run whose estimate already passed the gate.
If the training plan has no published hourly rate, run creation returns 503 rather than quoting a number the biller would not honor. There is no fallback rate.
A concrete, measured example. A 16-example SFT dataset at the defaults (batch 4, 3 epochs) is 12 optimizer steps. That prices under one cent against a EUR 2.00 ceiling, and the validated run trained those 12 steps to a loss of 0.84 in about four minutes of L4 time. End to end, submit to Complete, was about 15 minutes including GPU provisioning and teardown.
License handling
A trained adapter inherits the license of the base model it was trained on. The platform records that acceptance per run, so license_accepted must be true on every create call. There is no organization-level default that silently covers future runs.
Sending false (or omitting it) is refused with the license named:
license_accepted must be true: a trained adapter inherits the
Apache-2.0 license of base model "mistral-7b"
For a model published under a conditional-commercial license, the refusal is explicit about that fact instead of just naming the license string:
license_accepted must be true: base model "llama-3.3-70b" is published under
the conditional-commercial Llama-3.3-Community license, which requires
explicit acceptance before training
The license recorded on the run is carried onto the adapter registry row, so it travels with the artifact into serving.
Serving and rollback
A Complete run has already registered its adapter. The registry row carries:
served_model_name:ft-<8 hex of the run id>, the name you will call the model by.files_bucketandfiles_key_prefix: where the artifact lives in your bucket.adapter_sha256: the digest ofadapter_model.safetensors, verified before the weights are hot-loaded.base_model_license: inherited from the catalog entry.status:uploaded. It is registered but not serving.
To serve it, promote it onto an inference service running the same base model. The service must have been created with enable_fine_tuned_serving: true in its inference_config; a service created without that flag refuses promotion with 409, because the serving runtime has to be started with adapter support enabled.
curl -X POST \
https://api.foundrydb.com/inference-services/$SERVICE_ID/adapters/$ADAPTER_ID/promote \
-u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD"
Promotion downloads the artifact from your bucket, verifies the digest you registered, and hot-loads it into the already-running serving process. No restart, no dropped base-model traffic. The row flips to active and the model answers as:
foundrydb_managed/ft-a1b2c3d4
on the same OpenAI-compatible endpoint, with the same inference key, rate limits, token ceilings, EU residency setting, and metering as the base model. The base model keeps answering under its own name on the same GPU, so you can serve a fine-tune and its base side by side.
Rollback is promoting a prior version. Versions are tracked per (organization, served_model_name) line. Promoting a newer version demotes the currently-active one to superseded and leaves it in the registry; promoting that superseded row again swaps the served name back to those weights, again without a GPU restart. Each fine-tuning run registers version 1 under its own ft- name, so rolling back to a previous run means promoting that run's output_adapter_id, and clients switch the model string. If you want a single stable name across retrains, register subsequent versions under the same served_model_name and the hot-swap and rollback happen behind one unchanged model string.
max_loras on the serving service (default 4) bounds how many adapters can be loaded concurrently, and max_lora_rank (default 32) bounds their rank. A promotion that would exceed the slot budget is refused with 409 rather than evicting something that is serving traffic. Note that the default max_lora_rank of 32 is below the fine-tuning maximum rank of 64. Run creation enforces servability up front: a rank above 32 is accepted only if the organization already runs a serving-enabled service whose max_lora_rank admits it, so a run can never train into a promote-time dead end.
Limits
| Limit | Value |
|---|---|
| Dataset artifact size | 536870912 bytes (512 MB) |
| Method | LoRA supervised fine-tuning (lora / sft) |
| Training plan | gpu-l4-1, fixed |
| Training zone | fi-hel2, fixed |
| Max sequence length | 512 tokens |
| Adapter version per run | 1, under ft-<8 hex> |
| Base models | Curated catalog entries serving the generate task |
Next steps
- Fine-Tune and Serve a Model: the end-to-end walkthrough with runnable calls.
- Managed Inference: the serving reference, including
enable_fine_tuned_serving,max_loras, andmax_lora_rank. - Serve a Fine-Tuned LoRA Adapter: serving an adapter you trained yourself, outside the platform.
- Files: buckets, presigned uploads, and scoped keys.