5 posts tagged with "kafka"

Kafka is easy to write to and hard to see into. Standing up a broker is the start of the work, not the end: then you wire CLI tools and shell scripts together just to answer the basics. Which topics exist. How far behind is that consumer group. Is the broker even healthy. The event log you wanted was always one more afternoon of tooling away.

The Launch an event-streaming stack collapses that into one button. You get a managed Apache Kafka cluster with a browser console attached to it, EU-resident, in minutes, with nothing to wire by hand.

Most managed database products rent you a box. You provision a Postgres, you provision a Kafka, and then the interesting part, the part where data actually flows from one to the other, is handed back to you with a shrug. You stand up Kafka Connect, you hunt for the right Debezium build, you hand-edit pg_hba.conf, you grant REPLICATION, you author a publication, you guess at a slot name, and you pray the two services can even reach each other on the network. That is a runbook, not a product. And until today, it was yours to own.

Not anymore. FoundryDB data pipelines are live. The connection between two services you already own is now a first-class resource you can create, watch, and tear down with a single call. The first pipeline type ships today: change data capture from a PostgreSQL source straight into a Kafka sink. One POST, and the platform does all the plumbing, end to end, idempotently, with nothing exposed to the public internet to make it happen.

Retrieval-Augmented Generation (RAG) has become the standard approach for grounding LLMs in factual, up-to-date data. Instead of fine-tuning a model on your corpus (expensive, slow, stale within weeks), you retrieve relevant context at query time and feed it to the LLM alongside the user's question.

In 2026, RAG is no longer experimental. It powers customer support bots, internal knowledge search, legal document analysis, and code assistants at thousands of companies. The architecture has stabilized around a common pattern: ingest documents, generate embeddings, store vectors, retrieve at query time. What varies is how well you operate the infrastructure underneath.

This post walks through building a production RAG pipeline on FoundryDB using PostgreSQL with pgvector, Kafka for document ingestion, and Valkey for result caching.

If you run a modern application stack, you probably use at least three different database engines. PostgreSQL for your application data. MongoDB or another document store for unstructured content. Valkey (or Redis) for caching and session storage. Kafka for event streaming. Maybe MySQL for a legacy service that nobody wants to migrate.

Each engine runs on a different managed provider. Each provider has its own dashboard, its own CLI, its own billing, its own alerting system, its own way of handling backups, its own access control model. You pay five bills, manage five sets of credentials, and context-switch between five different interfaces when something goes wrong at 2 AM.

We built FoundryDB to solve this problem: one platform for all your database engines, with a single API, a single dashboard, and a single bill.

Event-driven architecture has moved from buzzword to baseline. If you are building microservices, real-time analytics, or data pipelines in 2026, Kafka is likely somewhere in the stack. The challenge is not whether to use it, but how to operate it without drowning in ZooKeeper configs, TLS certificate rotation, and broker rebalancing.

This post covers practical streaming patterns on FoundryDB's managed Kafka: topic design, partitioning strategies, consumer groups, schema enforcement, and monitoring. All examples use FoundryDB's Kafka 4.0 with KRaft mode and SASL/SCRAM authentication.