OpenSearch

OpenSearch is a distributed, open-source search and analytics engine built on Apache Lucene. It powers full-text search, log and event analytics, and vector / neural search from a single REST API on port 9200. Data lives in indexes, each index is split into shards for horizontal scale, and shards are replicated across nodes for availability. The bundled Security plugin enforces TLS and authentication on every request.

Versions

Version	Status
2	Available

Cluster topology

A cluster is a set of nodes that share the same cluster name and cooperate to hold your data and answer queries. Nodes take on roles:

• Cluster-manager (master-eligible) — maintains cluster state: which indexes and shards exist, where each shard copy is allocated, and node membership. One elected cluster-manager is active at a time; additional master-eligible nodes stand by. It does no search or indexing work itself.
• Data — store the shards and do the heavy lifting: indexing documents and running the query and fetch phases of a search.
• Coordinating — receive client requests, fan them out to the relevant shards, then gather and merge the partial results before replying. Every node can coordinate; dedicated coordinating nodes offload that merge work in large clusters.

An index is divided into primary shards (set at creation by number_of_shards). Each primary can have one or more replica shards (number_of_replicas). OpenSearch never places a primary and its own replica on the same node, so losing a node never loses both copies of a shard. Replicas also serve reads, so they add both resilience and search throughput.

When a search arrives, the coordinator fans the query out to one copy (primary or replica) of every shard, each shard scores matches locally (the query phase), then the coordinator gathers the top hits and fetches the documents (the fetch phase) before returning a merged result.

OpenSearch cluster, query fan-out & gather

Cluster green · search fans out to one copy per shard, then gathers

Coordinatorfan-out / gatherquery →Data nodesP0 P1 P2 · R0 R1 R2⇠ hitsCluster-managershard allocation

Cluster-managerCoordinatorData nodePrimary shardReplica shardcluster state / gather (dashed)

Cluster health is reported as a colour:

• green — all primary and replica shards are allocated.
• yellow — all primaries are allocated but one or more replicas are not (normal on a single-node cluster, where replicas have nowhere to go).
• red — at least one primary shard is unallocated; that slice of data is unavailable.

curl -u USER:PASS https://HOST:9200/_cluster/health?pretty

Connecting

Every cluster is reachable over HTTPS on port 9200. TLS is mandatory and HTTP Basic authentication is enforced by the Security plugin on all endpoints, including health and cluster APIs.

Parameter	Value
Host	{name}.db.foundrydb.com
Port	9200
Auth	HTTP Basic
TLS	Required

# Health check
curl -u USER:PASS https://HOST:9200/_cluster/health

# Create an index
curl -u USER:PASS -X PUT https://HOST:9200/my-index \
  -H "Content-Type: application/json" \
  -d '{"settings": {"number_of_shards": 3, "number_of_replicas": 1}}'

Python (opensearch-py)

from opensearchpy import OpenSearch
client = OpenSearch(
    hosts=[{'host': 'HOST', 'port': 9200}],
    http_auth=('USER', 'PASS'),
    use_ssl=True,
    verify_certs=True,
)
resp = client.cluster.health()

Node.js (@opensearch-project/opensearch)

import { Client } from '@opensearch-project/opensearch';
const client = new Client({
  node: 'https://HOST:9200',
  auth: { username: 'USER', password: 'PASS' },
});

Security

OpenSearch ships with the Security plugin always enabled. It provides:

• TLS on both the REST layer (client traffic) and the transport layer (node-to-node), so all traffic inside and into the cluster is encrypted.
• Authentication against an internal users database (HTTP Basic), with support for other backends.
• Role-based access control (RBAC) — roles grant cluster-level actions, index patterns, and document- or field-level permissions, then are bound to users through role mappings.

Roles and users are managed through the Security REST API under /_plugins/_security/api/:

# Create a role scoped to read-only access on the products index
curl -u USER:PASS -X PUT \
  https://HOST:9200/_plugins/_security/api/roles/products_reader \
  -H "Content-Type: application/json" \
  -d '{
    "index_permissions": [{
      "index_patterns": ["products*"],
      "allowed_actions": ["read"]
    }]
  }'

# Map a user to the role
curl -u USER:PASS -X PUT \
  https://HOST:9200/_plugins/_security/api/rolesmapping/products_reader \
  -H "Content-Type: application/json" \
  -d '{"users": ["analyst"]}'

Grant the built-in all_access role only to administrative users; give applications narrowly scoped roles instead.

Indexes

Create

curl -u USER:PASS -X PUT https://HOST:9200/products \
  -H "Content-Type: application/json" \
  -d '{
    "settings": {"number_of_shards": 3, "number_of_replicas": 1},
    "mappings": {
      "properties": {
        "name":  {"type": "text"},
        "price": {"type": "float"},
        "tags":  {"type": "keyword"}
      }
    }
  }'

Index a document

curl -u USER:PASS -X POST https://HOST:9200/products/_doc \
  -H "Content-Type: application/json" \
  -d '{"name": "Widget", "price": 9.99, "tags": ["hardware", "tools"]}'

Search

curl -u USER:PASS -X GET https://HOST:9200/products/_search \
  -H "Content-Type: application/json" \
  -d '{"query": {"match": {"name": "widget"}}}'

Vector & neural search

Beyond full-text matching, OpenSearch supports k-NN vector search for semantic and similarity use cases (retrieval-augmented generation, image and document similarity, recommendations). Store embeddings in a knn_vector field and query by nearest neighbour.

# Index with a vector field (enable k-NN, set the embedding dimension)
curl -u USER:PASS -X PUT https://HOST:9200/articles \
  -H "Content-Type: application/json" \
  -d '{
    "settings": {"index.knn": true},
    "mappings": {
      "properties": {
        "title":     {"type": "text"},
        "embedding": {"type": "knn_vector", "dimension": 384}
      }
    }
  }'

# Index a document with its embedding vector
curl -u USER:PASS -X POST https://HOST:9200/articles/_doc \
  -H "Content-Type: application/json" \
  -d '{"title": "Intro to sharding", "embedding": [0.12, -0.04, 0.88, "..."]}'

# Nearest-neighbour query
curl -u USER:PASS -X GET https://HOST:9200/articles/_search \
  -H "Content-Type: application/json" \
  -d '{
    "size": 5,
    "query": {"knn": {"embedding": {"vector": [0.1, -0.02, 0.9, "..."], "k": 5}}}
  }'

Vector search runs across the same shard fan-out as text search, so it scales with data nodes the same way. Combine a knn clause with a match clause in a bool query for hybrid search that blends lexical and semantic relevance.

Index Lifecycle Management

Automatically manage index rollover and deletion:

curl -u USER:PASS -X PUT https://HOST:9200/_plugins/_ism/policies/log-policy \
  -H "Content-Type: application/json" \
  -d '{
    "policy": {
      "description": "Delete logs after 30 days",
      "states": [{
        "name": "hot",
        "actions": [],
        "transitions": [{"state_name": "delete", "conditions": {"min_index_age": "30d"}}]
      }, {
        "name": "delete",
        "actions": [{"delete": {}}],
        "transitions": []
      }]
    }
  }'

Scaling

Add data nodes for more storage and throughput:

curl -u admin:password -X POST \
  https://api.foundrydb.com/managed-services/{id}/nodes \
  -H "Content-Type: application/json" \
  -d '{"role": "data"}'

Snapshots & restore

Backups use OpenSearch snapshots: a snapshot captures the state of indexes (and optionally cluster state) to a repository in object storage. Snapshots are incremental, so each one only stores the segments that changed since the last, and restoring brings back indexes from a chosen point in time. The platform manages the snapshot repository and schedule for you, so you trigger and list snapshots through the service API rather than wiring the repository by hand.

# Trigger a manual snapshot
curl -u admin:password -X POST \
  https://api.foundrydb.com/managed-services/{id}/backups \
  -H "Content-Type: application/json" \
  -d '{"backup_type": "manual"}'

# List snapshots
curl -u admin:password \
  https://api.foundrydb.com/managed-services/{id}/backups

Configuration

curl -u admin:password -X PATCH \
  https://api.foundrydb.com/managed-services/{id}/configuration \
  -H "Content-Type: application/json" \
  -d '{
    "parameters": {
      "indices.fielddata.cache.size": "20%",
      "indices.breaker.total.limit": "70%",
      "thread_pool.search.queue_size": "1000"
    }
  }'

Metrics

Key metrics: cluster_health, active_shards, indexing_rate, search_rate, jvm_heap_used, fielddata_evictions.

curl -u admin:password \
  "https://api.foundrydb.com/managed-services/{id}/metrics?metric=indexing_rate&period=1h"