Skip to main content

Build an In-Platform RAG Assistant

This is a whole-platform showcase. You build a Retrieval-Augmented-Generation (RAG) assistant where every piece runs on FoundryDB: managed PostgreSQL with pgvector stores your documents and their embeddings, managed inference serves the embedding model (bge-m3) that turns text into vectors, and managed inference also serves the chat model (mistral-7b) that writes the grounded answer. Nothing leaves your EU resources, and there is no external LLM API in the loop.

Retrieval is what makes the answer trustworthy: the assistant only answers from documents you indexed. To prove that, the example corpus contains one fact that exists nowhere else (the edge fleet codename, "Project Aurora"), and the assistant answers it correctly.

What you use​

ProductRole
Managed PostgreSQL + pgvectorStores documents and their embedding vectors, does nearest-neighbour retrieval
Managed Inference (bge-m3)Turns text into 1024-dimensional embedding vectors
Managed Inference (mistral-7b)Writes the final answer from the retrieved context

Prerequisites​

  • Owner or admin credentials (FOUNDRYDB_USER / FOUNDRYDB_PASSWORD) and a platform token (FOUNDRYDB_TOKEN).
  • Your organization id (ORG_ID).
  • Python 3 with pip.

Step 1: Provision the three services​

Create the Postgres store with the pgvector extension:

curl -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" https://api.foundrydb.com/managed-services \
  -H "Content-Type: application/json" \
  -d '{"name":"rag-pg","database_type":"postgresql","version":"17","plan_name":"tier-2",
       "zone":"se-sto1","storage_size_gb":20,"storage_tier":"maxiops","extensions":["pgvector"]}'

Create the embedding and chat inference services (GPU plans run in fi-hel2):

# embeddings: bge-m3
curl -H "Authorization: Bearer $FOUNDRYDB_TOKEN" https://api.foundrydb.com/inference-services \
  -H "Content-Type: application/json" \
  -d '{"name":"rag-embed","plan_name":"gpu-l40s-1","zone":"fi-hel2",
       "inference_config":{"model_id":"bge-m3","model_source":"curated","served_model_name":"bge-m3"}}'

# chat: mistral-7b
curl -H "Authorization: Bearer $FOUNDRYDB_TOKEN" https://api.foundrydb.com/inference-services \
  -H "Content-Type: application/json" \
  -d '{"name":"rag-chat","plan_name":"gpu-l40s-1","zone":"fi-hel2",
       "inference_config":{"model_id":"mistral-7b","model_source":"curated","served_model_name":"mistral-7b"}}'

Each returns immediately in Pending. Postgres reaches Running in a few minutes; the GPU services take a little longer while they download and load the model. Poll each service until it reads Running.

Step 2: Collect connection details​

For Postgres, allow your client IP, then read the host, user, and password:

export PG_ID="<postgres-service-id>"
curl -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" -X PATCH \
  https://api.foundrydb.com/managed-services/$PG_ID \
  -H "Content-Type: application/json" \
  -d "{\"allowed_cidrs\":[\"$(curl -s ifconfig.me)/32\"]}"

curl -s -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" https://api.foundrydb.com/managed-services/$PG_ID | jq -r '.dns_records[0].full_domain'
curl -s -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" https://api.foundrydb.com/managed-services/$PG_ID/database-users | jq -r '.users[0].username'
curl -s -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" -X POST \
  https://api.foundrydb.com/managed-services/$PG_ID/database-users/<username>/reveal-password | jq -r '.password'

For inference, each service exposes a dedicated endpoint hostname (field endpoint_hostname, like rag-embed-xxxxxx.inf.foundrydb.com). Read both, and mint one fdb-inf key that both endpoints accept:

curl -s -H "Authorization: Bearer $FOUNDRYDB_TOKEN" https://api.foundrydb.com/inference-services/<embed-id> | jq -r '.endpoint_hostname'
curl -s -H "Authorization: Bearer $FOUNDRYDB_TOKEN" https://api.foundrydb.com/inference-services/<chat-id>  | jq -r '.endpoint_hostname'

curl -s -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" -X POST \
  https://api.foundrydb.com/organizations/$ORG_ID/inference/keys \
  -H "Content-Type: application/json" \
  -d '{"name":"rag-key","monthly_token_limit":2000000}' | jq -r '.secret'

Export the four values the script reads:

export PG_DSN="postgresql://<user>:<password>@<postgres-host>:5432/defaultdb?sslmode=require"
export EMBED_BASE="https://<embed-endpoint>/v1"
export CHAT_BASE="https://<chat-endpoint>/v1"
export FDB_INF_KEY="fdb-inf-..."

Step 3: The corpus and the RAG script​

Install a Postgres driver and an HTTP client:

pip install "psycopg[binary]" requests

Save the corpus as corpus.py:

DOCS = [
    "FoundryDB Managed Inference serves open-weight LLMs on dedicated GPUs in Helsinki (fi-hel2) using the vLLM runtime, behind an OpenAI-compatible endpoint.",
    "Every FoundryDB organization has a monthly cost circuit breaker: when inference spend crosses the configured limit, the platform stops forwarding requests until the next cycle.",
    "The internal codename for the FoundryDB edge delivery fleet is Project Aurora, a set of PoPs that terminate TLS and enforce EU residency close to the client.",
    "FoundryDB Files are S3-compatible, EU-resident object storage buckets. They store fine-tuned LoRA adapter artifacts and Retrieval-Augmented-Generation source documents.",
    "A FoundryDB inference key is prefixed fdb-inf and is scoped to one organization. It carries a requests-per-minute limit and a monthly token ceiling.",
    "FoundryDB supports fine-tuned serving: a LoRA adapter is hot-loaded into the running vLLM process with no restart, and can be rolled back by promoting a prior version.",
    "The FoundryDB curated inference catalog includes mistral-7b for chat and bge-m3 for embeddings; bge-m3 produces 1024-dimensional vectors.",
]

Save the RAG program as rag.py. It embeds each document with bge-m3, stores the vectors in pgvector, and answers a question by retrieving the nearest documents and passing them to mistral-7b:

"""In-platform RAG on FoundryDB: managed Postgres (pgvector) for retrieval,
managed inference (bge-m3 embeddings + mistral-7b chat) for generation.

Everything runs on your own EU resources: no external LLM API. Configure four
environment variables (see below), then run once to index the corpus and answer
a question grounded in it.
"""
import os
import psycopg
import requests
from corpus import DOCS

PG_DSN = os.environ["PG_DSN"]              # postgresql://user:pass@host:5432/defaultdb?sslmode=require
EMBED_BASE = os.environ["EMBED_BASE"]      # https://<embed-endpoint>/v1
CHAT_BASE = os.environ["CHAT_BASE"]        # https://<chat-endpoint>/v1
KEY = os.environ["FDB_INF_KEY"]            # fdb-inf-...
HEADERS = {"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"}


def embed(texts):
    """Embed a list of strings with the managed bge-m3 endpoint (1024-dim)."""
    r = requests.post(f"{EMBED_BASE}/embeddings",
                      headers=HEADERS,
                      json={"model": "foundrydb_managed/bge-m3", "input": texts},
                      timeout=60)
    r.raise_for_status()
    return [d["embedding"] for d in r.json()["data"]]


def index():
    """Create the pgvector table and store an embedding per document."""
    vecs = embed(DOCS)
    with psycopg.connect(PG_DSN) as conn, conn.cursor() as cur:
        cur.execute("CREATE EXTENSION IF NOT EXISTS vector")
        cur.execute("DROP TABLE IF EXISTS docs")
        cur.execute("CREATE TABLE docs (id serial PRIMARY KEY, content text, embedding vector(1024))")
        for content, v in zip(DOCS, vecs):
            cur.execute("INSERT INTO docs (content, embedding) VALUES (%s, %s)",
                        (content, str(v)))
        conn.commit()
    print(f"indexed {len(DOCS)} documents")


def retrieve(question, k=3):
    """Embed the question and return the k nearest documents by cosine distance."""
    qv = str(embed([question])[0])
    with psycopg.connect(PG_DSN) as conn, conn.cursor() as cur:
        cur.execute("SELECT content FROM docs ORDER BY embedding <=> %s LIMIT %s", (qv, k))
        return [row[0] for row in cur.fetchall()]


def answer(question):
    """Retrieve context from Postgres, then generate a grounded answer with mistral-7b."""
    context = retrieve(question)
    prompt = (
        "Answer the question using only the context below. If the context does "
        "not contain the answer, say you do not know.\n\nContext:\n"
        + "\n".join(f"- {c}" for c in context)
        + f"\n\nQuestion: {question}"
    )
    r = requests.post(f"{CHAT_BASE}/chat/completions",
                      headers=HEADERS,
                      json={"model": "foundrydb_managed/mistral-7b",
                            "messages": [{"role": "user", "content": prompt}],
                            "temperature": 0, "max_tokens": 120},
                      timeout=90)
    r.raise_for_status()
    return context, r.json()["choices"][0]["message"]["content"].strip()


if __name__ == "__main__":
    index()
    for q in ["What is the codename for the FoundryDB edge fleet?",
              "How many dimensions does the bge-m3 embedding model produce?"]:
        ctx, a = answer(q)
        print(f"\nQ: {q}\nA: {a}")

Step 4: Run it​

python rag.py

Output:

indexed 7 documents

Q: What is the codename for the FoundryDB edge fleet?
A: The codename for the FoundryDB edge fleet is Project Aurora.

Q: How many dimensions does the bge-m3 embedding model produce?
A: The bge-m3 embedding model produces 1024-dimensional vectors.

"Project Aurora" appears only in your corpus, so a correct answer means the assistant retrieved the right document and grounded its answer on it rather than guessing. Retrieval is what supplies the fact.

How it works​

  1. Index. Each document is embedded once by bge-m3 and stored in a vector(1024) column in Postgres.
  2. Retrieve. The question is embedded the same way, and ORDER BY embedding <=> query returns the nearest documents by cosine distance (the pgvector distance operator).
  3. Generate. The retrieved documents are placed in the prompt, and mistral-7b answers only from them. The embedding call, the vector search, and the generation call all stay inside FoundryDB.

For a larger corpus, add a pgvector index (for example HNSW) so retrieval stays fast, and chunk long documents before embedding.

Cleanup​

curl -u "$FOUNDRYDB_USER:$FOUNDRYDB_PASSWORD" -X DELETE https://api.foundrydb.com/managed-services/$PG_ID
curl -H "Authorization: Bearer $FOUNDRYDB_TOKEN" -X DELETE https://api.foundrydb.com/inference-services/<embed-id>
curl -H "Authorization: Bearer $FOUNDRYDB_TOKEN" -X DELETE https://api.foundrydb.com/inference-services/<chat-id>

What you built​

  • A RAG assistant whose retrieval store (Postgres + pgvector), embedding model (bge-m3), and chat model (mistral-7b) all run on FoundryDB, EU-resident, with no external LLM API.
  • A grounded question-answering loop that provably answers from your own documents.

Next steps​