Skip to main content

29 posts tagged with "postgresql"

View All Tags

Data Pipelines Are Live: Stream PostgreSQL Into Kafka With One API Call

· 6 min read
FoundryDB Team
Engineering @ FoundryDB

Most managed database products rent you a box. You provision a Postgres, you provision a Kafka, and then the interesting part, the part where data actually flows from one to the other, is handed back to you with a shrug. You stand up Kafka Connect, you hunt for the right Debezium build, you hand-edit pg_hba.conf, you grant REPLICATION, you author a publication, you guess at a slot name, and you pray the two services can even reach each other on the network. That is a runbook, not a product. And until today, it was yours to own.

Not anymore. FoundryDB data pipelines are live. The connection between two services you already own is now a first-class resource you can create, watch, and tear down with a single call. The first pipeline type ships today: change data capture from a PostgreSQL source straight into a Kafka sink. One POST, and the platform does all the plumbing, end to end, idempotently, with nothing exposed to the public internet to make it happen.

PostgreSQL → Kafka CDC (Debezium)
STEADY streaming · source_lag=0 B
PostgreSQLpublication/WAL⇢ SDNDebeziumKafka Connectenvelope →Kafka topicsshop.public.*partitions →Consumergroup
PostgreSQL :5432 (source)Debezium · Kafka Connectshop.public.ordersshop.public.customersConsumer groupprivate SDN peering (dashed)

Cross-Zone Databases Are Here: One Cluster, Three Zones, Private Peering

· 6 min read
FoundryDB Team
Engineering @ FoundryDB

Single-zone high availability is wonderful right up until the zone itself is the thing that breaks. A blown rack, a batched datacenter maintenance window, a regional routing incident, and every replica you so carefully provisioned next to your primary goes dark at the exact same moment. Today that stops being your problem to solve by hand. FoundryDB v0.8.0 ships cross-zone clusters: one managed database, spread across three zones, privately peered, with streaming replication that stays on the cloud backbone and never crosses the public internet.

This is the database property you always wanted and never wanted to build yourself. You ask for three zones, and you get a primary in one, replicas in the other two, a private peering mesh wiring them together, and two DNS records that already point at the right nodes. No second cluster to babysit. No VPN to watch. No 3am page because a replication slot filled up while you were asleep.

Managed service topology
Steady state, endpoint fronts the cluster
DNS endpointmy-db.db.foundrydb.comwrites →Primaryreads + writes⇢ streamingRead replicase-sto1 / se-sto2
DNS endpoint (TLS)Primary (reads + writes)Read replicawritesstreaming replicationzone / peering region

Backup and Recovery for Babelfish on FoundryDB

· 5 min read
FoundryDB Team
Engineering @ FoundryDB
Point-in-time recovery
RESTORED new service recovered @ T · original untouched
Databasesourcefull + WAL →RepositoryS3⇢ window · pick T →Restored @ Tnew service
Database (source)Full / base backupsWAL / binlog / oplog streamS3 repositoryRecovery window + restore pointRestored service @ T (new)

Babelfish stores everything in a standard PostgreSQL database (babelfish_db). Your T-SQL schemas, tables, stored procedures (compiled to PL/pgSQL internally), triggers, and views are all PostgreSQL objects underneath. This means the backup story for Babelfish is exactly the backup story for PostgreSQL: pgbackrest, WAL archiving, and point-in-time recovery.

This post documents how backups work on a live Babelfish 4.8.0 instance, using confirmed results from FoundryDB staging.

Bulk Data Loading in Babelfish on FoundryDB

· 5 min read
FoundryDB Team
Engineering @ FoundryDB
Babelfish: SQL Server (TDS) over PostgreSQL
DUAL ACCESS TDS :1433 and native PG :5432
SQL Server appTDS:1433 →BabelfishT-SQL → PGPostgreSQL 16:5432
SQL Server app (TDS)Babelfish :1433PostgreSQL 16 engineT-SQL → PG translationnative PG :5432 (dual access)

Loading data efficiently is one of the first things you need to sort out when migrating from SQL Server. Babelfish supports several bulk loading patterns natively, but the BULK INSERT statement is not one of them. This post documents what works, what does not, and the best alternatives, all tested live against Babelfish 4.8.0 on PostgreSQL 16.11.

For provisioning instructions, see the Babelfish getting started guide.

Dynamic SQL on Babelfish 4.8.0: Every Pattern Tested

· 6 min read
FoundryDB Team
Engineering @ FoundryDB
Babelfish: SQL Server (TDS) over PostgreSQL
DUAL ACCESS TDS :1433 and native PG :5432
SQL Server appTDS:1433 →BabelfishT-SQL → PGPostgreSQL 16:5432
SQL Server app (TDS)Babelfish :1433PostgreSQL 16 engineT-SQL → PG translationnative PG :5432 (dual access)

Dynamic SQL is one of the most heavily used features in real-world SQL Server applications. Stored procedures that build queries at runtime, parameterised searches, dynamic pivot tables, and flexible reporting queries all rely on it. If you are migrating from SQL Server to Babelfish, you need to know that these patterns work before you commit to the move.

This post documents six dynamic SQL patterns tested against Babelfish 4.8.0 on PostgreSQL 16.11, running as a FoundryDB managed service. Every test passed without modification.

Full-Text Search in Babelfish: What Works, What Doesn't, and Workarounds

· 7 min read
FoundryDB Team
Engineering @ FoundryDB
Babelfish: SQL Server (TDS) over PostgreSQL
DUAL ACCESS TDS :1433 and native PG :5432
SQL Server appTDS:1433 →BabelfishT-SQL → PGPostgreSQL 16:5432
SQL Server app (TDS)Babelfish :1433PostgreSQL 16 engineT-SQL → PG translationnative PG :5432 (dual access)

SQL Server applications commonly rely on text search, from simple LIKE patterns to full-text predicates like CONTAINS and FREETEXT. If you are migrating to Babelfish, you need to know which of these work out of the box and which require changes.

This post documents every text search method we tested against Babelfish 4.8.0 on PostgreSQL 16.11, running as a FoundryDB managed service. Every query was executed and every result verified.

Indexes and Query Performance in Babelfish on FoundryDB

· 6 min read
FoundryDB Team
Engineering @ FoundryDB
Babelfish: SQL Server (TDS) over PostgreSQL
DUAL ACCESS TDS :1433 and native PG :5432
SQL Server appTDS:1433 →BabelfishT-SQL → PGPostgreSQL 16:5432
SQL Server app (TDS)Babelfish :1433PostgreSQL 16 engineT-SQL → PG translationnative PG :5432 (dual access)

Babelfish translates T-SQL to PostgreSQL internally. For most index operations and query tuning work, this is transparent: you write the same CREATE INDEX syntax you would use on SQL Server, and Babelfish creates the corresponding PostgreSQL index. Some SQL Server performance tooling does not map cleanly to the PostgreSQL layer, and it is useful to know where the boundaries are before you start profiling.

This post documents index behaviour and query performance tooling in Babelfish 4.8.0, based on confirmed results from live testing on a FoundryDB staging instance. All results use a 10,000-row table tested against PostgreSQL 16.11.

Run SQL Server Workloads on PostgreSQL: Getting Started with Babelfish on FoundryDB

· 4 min read
FoundryDB Team
Engineering @ FoundryDB
Babelfish: SQL Server (TDS) over PostgreSQL
DUAL ACCESS TDS :1433 and native PG :5432
SQL Server appTDS:1433 →BabelfishT-SQL → PGPostgreSQL 16:5432
SQL Server app (TDS)Babelfish :1433PostgreSQL 16 engineT-SQL → PG translationnative PG :5432 (dual access)

Babelfish for PostgreSQL lets you run SQL Server applications against a PostgreSQL backend with minimal code changes. Your application connects on port 1433 using the standard TDS protocol, sends T-SQL, and Babelfish translates the queries into PostgreSQL internally.

FoundryDB now offers Babelfish as a fully managed service. This post walks through provisioning, connecting, and verifying a working instance using confirmed results from Babelfish 4.8.0 on PostgreSQL 16.11.

Using SQLAlchemy and pymssql with Babelfish 4.8.0 on FoundryDB

· 8 min read
FoundryDB Team
Engineering @ FoundryDB
Babelfish: SQL Server (TDS) over PostgreSQL
DUAL ACCESS TDS :1433 and native PG :5432
SQL Server appTDS:1433 →BabelfishT-SQL → PGPostgreSQL 16:5432
SQL Server app (TDS)Babelfish :1433PostgreSQL 16 engineT-SQL → PG translationnative PG :5432 (dual access)

Python developers working with SQL Server typically reach for one of two libraries: pymssql for direct TDS access, or SQLAlchemy for ORM-based workflows. Both work with Babelfish, but SQLAlchemy requires three specific workarounds due to differences in how Babelfish implements the TDS protocol.

This post covers both approaches with complete, working code tested against Babelfish 4.8.0 on PostgreSQL 16.11 running on FoundryDB staging. The pymssql section expands on driver-level coverage with additional tests for stored procedures and batch insert performance.

User and Permission Management on Babelfish 4.8.0

· 5 min read
FoundryDB Team
Engineering @ FoundryDB
Babelfish: SQL Server (TDS) over PostgreSQL
DUAL ACCESS TDS :1433 and native PG :5432
SQL Server appTDS:1433 →BabelfishT-SQL → PGPostgreSQL 16:5432
SQL Server app (TDS)Babelfish :1433PostgreSQL 16 engineT-SQL → PG translationnative PG :5432 (dual access)

Babelfish supports a subset of SQL Server's security model. Some T-SQL user management commands work as expected, others fail with known limitations, and a few are not supported at all. This post documents every command we tested against a live Babelfish 4.8.0 instance on FoundryDB, with exact results.

The practical takeaway: manage your database users through the FoundryDB API, which handles user creation at the PostgreSQL level and avoids the current Babelfish limitations entirely.