Skip to main content

Creating Indexes, Ingesting Data, and Running Full-Text Searches in OpenSearch

· 6 min read
FoundryDB Team
Engineering @ FoundryDB

With your OpenSearch cluster running on FoundryDB, the next step is putting data into it and querying it. This post covers explicit index mappings, the bulk ingestion API, full-text search with relevance scoring, fuzzy matching, bool queries, and aggregations.

All results are from real queries run against OpenSearch 2.19.1 on FoundryDB staging. Replace YOUR_HOST and YOUR_DB_PASSWORD with your cluster values throughout.

OpenSearch cluster, query fan-out & gather
Cluster green · search fans out to one copy per shard, then gathers
Coordinatorfan-out / gatherquery →Data nodesP0 P1 P2 · R0 R1 R2⇠ hitsCluster-managershard allocation
Cluster-managerCoordinatorData nodePrimary shardReplica shardcluster state / gather (dashed)

Automating Index Lifecycle with ISM in OpenSearch: Rollover, Force Merge, and Auto-Delete

· 5 min read
FoundryDB Team
Engineering @ FoundryDB

Log indices grow without bound unless you manage them. OpenSearch Index State Management (ISM) lets you define policies that automatically roll over active write indices when they hit a size or document threshold, and delete old ones after a retention period. This post shows how to build and verify a complete lifecycle policy on a live OpenSearch 2.19.1 cluster managed by FoundryDB.

All commands use YOUR_OPENSEARCH_HOST and YOUR_PASSWORD as placeholders.

OpenSearch cluster, query fan-out & gather
Cluster green · search fans out to one copy per shard, then gathers
Coordinatorfan-out / gatherquery →Data nodesP0 P1 P2 · R0 R1 R2⇠ hitsCluster-managershard allocation
Cluster-managerCoordinatorData nodePrimary shardReplica shardcluster state / gather (dashed)

Log Analytics in OpenSearch with PPL: Filter, Aggregate, and Correlate

· 5 min read
FoundryDB Team
Engineering @ FoundryDB

OpenSearch SQL is familiar, but Piped Processing Language (PPL) is often better for log analysis: its pipeline syntax maps naturally to how you think about filtering and aggregating event streams. Each | stage narrows or transforms the result set from the previous one. This post runs five PPL queries against a 200-document structured log dataset on a live OpenSearch 2.19.1 cluster managed by FoundryDB. Every number below is from a real query.

All commands use YOUR_OPENSEARCH_HOST and YOUR_PASSWORD as placeholders.

OpenSearch cluster, query fan-out & gather
Cluster green · search fans out to one copy per shard, then gathers
Coordinatorfan-out / gatherquery →Data nodesP0 P1 P2 · R0 R1 R2⇠ hitsCluster-managershard allocation
Cluster-managerCoordinatorData nodePrimary shardReplica shardcluster state / gather (dashed)

Neural Sparse Search in OpenSearch: Semantic Matching Without a GPU

· 5 min read
FoundryDB Team
Engineering @ FoundryDB

Dense vector search (k-NN) is powerful but requires embedding both documents and queries with a neural model at query time. Neural sparse search takes a different approach: expand tokens with learned weights at index time, store them as a rank_features field, and at query time do a fast lookup rather than a vector computation. The result is semantic search with no GPU requirement at query time. This post shows the full setup on a live OpenSearch 2.19.1 cluster managed by FoundryDB.

All commands use YOUR_OPENSEARCH_HOST and YOUR_PASSWORD as placeholders.

pgvector similarity search · query → HNSW → top-k
TOP-K vector → HNSW index → filter → nearest rows
Queryvector | textANN · HNSWcosine <=>AND filter →Top-kby distance
Query / top-kServer-side embedANN search · tableHNSW indexEquality filter (WHERE)index / predicate edge (dashed)

OpenSearch Performance Tuning: Refresh Intervals, Bulk Sizing, and Shard Analysis

· 6 min read
FoundryDB Team
Engineering @ FoundryDB

Getting good write throughput from OpenSearch requires understanding three things: the refresh cycle, translog durability, and shard sizing. This post benchmarks bulk indexing with default and tuned settings on a live OpenSearch 2.19.1 cluster managed by FoundryDB. The cluster is a single-node tier-2 (2 CPU, 4 GB RAM), which is the smallest configuration available. Numbers from larger nodes will be better, but the ratios between default and tuned settings hold.

All commands use YOUR_OPENSEARCH_HOST and YOUR_PASSWORD as placeholders.

OpenSearch cluster, query fan-out & gather
Cluster green · search fans out to one copy per shard, then gathers
Coordinatorfan-out / gatherquery →Data nodesP0 P1 P2 · R0 R1 R2⇠ hitsCluster-managershard allocation
Cluster-managerCoordinatorData nodePrimary shardReplica shardcluster state / gather (dashed)

Provisioning an OpenSearch Cluster on FoundryDB and Connecting via TLS

· 4 min read
FoundryDB Team
Engineering @ FoundryDB

FoundryDB manages the full lifecycle of OpenSearch: VM provisioning, TLS certificates, DNS, and credentials. You get a production-ready cluster endpoint in about 6 minutes. This post covers provisioning via the API, fetching credentials, verifying TLS, and confirming cluster health.

All examples use confirmed output from OpenSearch 2.19.1 running on FoundryDB staging.

OpenSearch cluster, query fan-out & gather
Cluster green · search fans out to one copy per shard, then gathers
Coordinatorfan-out / gatherquery →Data nodesP0 P1 P2 · R0 R1 R2⇠ hitsCluster-managershard allocation
Cluster-managerCoordinatorData nodePrimary shardReplica shardcluster state / gather (dashed)

Building a RAG Pipeline with OpenSearch as the Vector Store

· 7 min read
FoundryDB Team
Engineering @ FoundryDB

Retrieval-Augmented Generation (RAG) augments a language model's response by first retrieving relevant context from a database, then passing that context into the prompt. OpenSearch is a natural fit for the retrieval step: it runs the embedding model internally, stores the vectors, and returns ranked results in a single query. This post shows the retrieval step with real scores from a live OpenSearch 2.19.1 cluster managed by FoundryDB, and explains how to wire the retrieved chunks into a prompt and call an LLM.

This post uses a dedicated knowledge base index with 6 database documentation chunks, embedded using all-MiniLM-L6-v2 (384 dimensions). The retrieval, prompt assembly, and a complete prompt were all tested on a live FoundryDB cluster.

RAG loop · composed from FoundryDB primitives
QUERY retrieve → augment → generate → answer
Appquestionembed →Vector Searchtop-k← pgvectorPrompt + Contextaugmentgenerate →LLM ProviderEU-routed
PostgreSQL sourceEmbedding pipelinepgvector columnVector searchPrompt + contextInference proxy · LLM

Vector Search in OpenSearch: Embeddings, k-NN Indexes, and HNSW

· 6 min read
FoundryDB Team
Engineering @ FoundryDB

Keyword search breaks when users phrase queries differently from the words in your documents. Vector search fixes this by comparing meaning rather than tokens. This post walks through registering a sentence embedding model, building a k-NN index with an ingest pipeline, and running semantic queries against a FoundryDB-managed OpenSearch 2.19.1 cluster. All scores and outputs are from a real test run.

All commands use YOUR_OPENSEARCH_HOST and YOUR_PASSWORD as placeholders. Replace them with your cluster domain and app_user password from the FoundryDB dashboard.

pgvector similarity search · query → HNSW → top-k
TOP-K vector → HNSW index → filter → nearest rows
Queryvector | textANN · HNSWcosine <=>AND filter →Top-kby distance
Query / top-kServer-side embedANN search · tableHNSW indexEquality filter (WHERE)index / predicate edge (dashed)

Zero-Downtime Reindexing in OpenSearch with Aliases

· 5 min read
FoundryDB Team
Engineering @ FoundryDB

OpenSearch index mappings are immutable once data is indexed. Changing a field type (for example, from text to keyword for a product SKU) or adding an analyzer to an existing field requires creating a new index and moving data into it. The challenge is doing this without taking your search API offline. The answer is aliases combined with the _reindex API. This post demonstrates the full pattern on a live OpenSearch 2.19.1 cluster managed by FoundryDB.

All commands use YOUR_OPENSEARCH_HOST and YOUR_PASSWORD as placeholders.

OpenSearch cluster, query fan-out & gather
Cluster green · search fans out to one copy per shard, then gathers
Coordinatorfan-out / gatherquery →Data nodesP0 P1 P2 · R0 R1 R2⇠ hitsCluster-managershard allocation
Cluster-managerCoordinatorData nodePrimary shardReplica shardcluster state / gather (dashed)

Migrate from DigitalOcean Managed Databases to FoundryDB

· 7 min read
FoundryDB Team
Engineering @ FoundryDB

DigitalOcean Managed Databases is a popular starting point for teams that want hosted PostgreSQL, MySQL, MongoDB, or Valkey without managing infrastructure. It does the basics well: provisioning, automated backups, and TLS. But as your data platform matures, the limitations become clear. Four engines, minimal monitoring, no AI-oriented features, no pipeline templates, and no way to export metrics to your own observability stack. FoundryDB offers seven engines, built-in AI presets, predictive autoscaling, database forking, and seven metrics export destinations, all on European infrastructure.