Automating Index Lifecycle with ISM in OpenSearch: Rollover, Force Merge, and Auto-Delete

April 14, 2026 · 5 min read

Engineering @ FoundryDB

Log indices grow without bound unless you manage them. OpenSearch Index State Management (ISM) lets you define policies that automatically roll over active write indices when they hit a size or document threshold, and delete old ones after a retention period. This post shows how to build and verify a complete lifecycle policy on a live OpenSearch 2.19.1 cluster managed by FoundryDB.

All commands use YOUR_OPENSEARCH_HOST and YOUR_PASSWORD as placeholders.

Prerequisites

A running FoundryDB OpenSearch cluster.
curl and jq installed locally.

Step 1: Create the ISM Policy

The policy defines two states. The hot state is where writes land. It rolls over when the index reaches 1 GB, is 7 days old, or contains 5 documents (the small threshold is for testing purposes). After rollover, the old index transitions to the delete state and is removed after 1 day.

curl -u app_user:YOUR_PASSWORD -k \
  -X PUT "https://YOUR_OPENSEARCH_HOST:9200/_plugins/_ism/policies/logs-lifecycle" \
  -H "Content-Type: application/json" \
  -d '{
    "policy": {
      "description": "Log index lifecycle: rollover then delete",
      "default_state": "hot",
      "states": [
        {
          "name": "hot",
          "actions": [
            {
              "rollover": {
                "min_size": "1gb",
                "min_index_age": "7d",
                "min_doc_count": 5
              }
            }
          ],
          "transitions": [
            {"state_name": "delete", "conditions": {"min_rollover_age": "1d"}}
          ]
        },
        {
          "name": "delete",
          "actions": [{"delete": {}}],
          "transitions": []
        }
      ],
      "ism_template": [
        {"index_patterns": ["logs-app-*"], "priority": 100}
      ]
    }
  }'

The ism_template section is important. It tells ISM to attach this policy automatically to any new index whose name matches logs-app-*. You do not need to manually apply the policy each time a rollover creates a new index.

Step 2: Create an Index Template

The index template sets up shard configuration and attaches the ISM policy by default for any new index in the pattern.

curl -u app_user:YOUR_PASSWORD -k \
  -X PUT "https://YOUR_OPENSEARCH_HOST:9200/_index_template/logs-app-template" \
  -H "Content-Type: application/json" \
  -d '{
    "index_patterns": ["logs-app-*"],
    "template": {
      "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 0,
        "plugins.index_state_management.policy_id": "logs-lifecycle"
      },
      "mappings": {
        "properties": {
          "timestamp":   {"type": "date"},
          "level":       {"type": "keyword"},
          "service":     {"type": "keyword"},
          "message":     {"type": "text"},
          "status_code": {"type": "integer"}
        }
      }
    }
  }'

Verify the setting was applied after creating the first index:

curl -u app_user:YOUR_PASSWORD -k \
  "https://YOUR_OPENSEARCH_HOST:9200/logs-app-000001/_settings" | \
  jq '.. | objects | {policy_id}'

Expected output:

{"plugins": {"index_state_management": {"policy_id": "logs-lifecycle"}}}

Step 3: Bootstrap the First Index and Write Alias

ISM rollover requires a write alias. Create the first index manually, then create the alias pointing to it.

# Create the first index
curl -u app_user:YOUR_PASSWORD -k \
  -X PUT "https://YOUR_OPENSEARCH_HOST:9200/logs-app-000001"

# Create the write alias
curl -u app_user:YOUR_PASSWORD -k \
  -X POST "https://YOUR_OPENSEARCH_HOST:9200/_aliases" \
  -H "Content-Type: application/json" \
  -d '{
    "actions": [
      {
        "add": {
          "index": "logs-app-000001",
          "alias": "logs-app",
          "is_write_index": true
        }
      }
    ]
  }'

From this point on, all writes go to the alias logs-app. Applications never need to know which numbered index is the current write target.

Step 4: Ingest Documents and Trigger Rollover

Write 5 log documents through the alias, then trigger a manual rollover to test the policy.

# Ingest a document
curl -u app_user:YOUR_PASSWORD -k \
  -X POST "https://YOUR_OPENSEARCH_HOST:9200/logs-app/_doc" \
  -H "Content-Type: application/json" \
  -d '{
    "timestamp": "2026-04-14T18:00:00Z",
    "level": "INFO",
    "service": "api",
    "message": "Request processed successfully",
    "status_code": 200
  }'

After 5 documents, trigger rollover manually (ISM also runs it automatically on its check interval):

curl -u app_user:YOUR_PASSWORD -k \
  -X POST "https://YOUR_OPENSEARCH_HOST:9200/logs-app/_rollover" \
  -H "Content-Type: application/json" \
  -d '{"conditions": {"min_doc_count": 3}}'

Result from the test:

{
  "acknowledged": true,
  "rolled_over": true,
  "old_index": "logs-app-000001",
  "new_index": "logs-app-000002",
  "dry_run": false,
  "conditions": {}
}

Step 5: Verify the State After Rollover

Check that both indices exist and that the alias now points to the new one:

curl -u app_user:YOUR_PASSWORD -k \
  "https://YOUR_OPENSEARCH_HOST:9200/_cat/indices/logs-app-*?v&h=index,docs.count,store.size,health"

index           docs.count  store.size  health
logs-app-000002          0       208b   green
logs-app-000001          5      5.7kb   green

curl -u app_user:YOUR_PASSWORD -k \
  "https://YOUR_OPENSEARCH_HOST:9200/_cat/aliases/logs-app?v"

alias    index           is_write_index
logs-app logs-app-000001 false
logs-app logs-app-000002 true

The old index is still readable through the alias (searches hit both indices by default). Only writes go to logs-app-000002. Verify that a new document lands on the correct index:

curl -u app_user:YOUR_PASSWORD -k \
  -X POST "https://YOUR_OPENSEARCH_HOST:9200/logs-app/_doc" \
  -H "Content-Type: application/json" \
  -d '{"timestamp": "2026-04-14T18:30:00Z", "level": "WARN", "service": "worker", "message": "Queue depth high", "status_code": 503}'

The _index field in the response confirms logs-app-000002.

Step 6: Check ISM Policy Status

curl -u app_user:YOUR_PASSWORD -k \
  "https://YOUR_OPENSEARCH_HOST:9200/_plugins/_ism/explain/logs-app-000001,logs-app-000002" | jq .

Both indices report policy_id: logs-lifecycle. ISM will check the delete transition for logs-app-000001 after 1 day and remove it automatically.

How the Lifecycle Works End to End

The alias is the stable write endpoint. Applications index to logs-app and never change their configuration when rollover happens. ISM creates logs-app-000002, atomically moves the is_write_index flag, and continues monitoring. After the configured retention age, ISM deletes the old index. No manual intervention is required.

For high-volume pipelines, add a force_merge action between hot and delete to reduce segment count on read-heavy historical indices before they age out.

What's Next

Manage your OpenSearch indices at scale with FoundryDB at foundrydb.com. Documentation at docs.foundrydb.com.

Prerequisites​

Step 1: Create the ISM Policy​

Step 2: Create an Index Template​

Step 3: Bootstrap the First Index and Write Alias​

Step 4: Ingest Documents and Trigger Rollover​

Step 5: Verify the State After Rollover​

Step 6: Check ISM Policy Status​

How the Lifecycle Works End to End​

What's Next​