Control ElasticSearch total alias size using ILM - Stack Overflow

admin管理员组

文章数量:1122846

I am using ILM and aliases to manage some data, and I have a current policy to rolls over indexes which get too old or large, and then delete them after some time period.

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "6h",
            "max_size": "20gb"
          }
        }
      },
      "delete": {
        "min_age": "24h",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

What I'd prefer is to rollover an index when it gets to a certain size, e.g. 30GB (just for granularity), and keep as much data as possible until storage starts filling up. Is this possible?

i.e. my alias should use 30GB-large indexes and keep as many as possible, whilst never exceeding 200GB total.

I am using ILM and aliases to manage some data, and I have a current policy to rolls over indexes which get too old or large, and then delete them after some time period.

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "6h",
            "max_size": "20gb"
          }
        }
      },
      "delete": {
        "min_age": "24h",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

What I'd prefer is to rollover an index when it gets to a certain size, e.g. 30GB (just for granularity), and keep as much data as possible until storage starts filling up. Is this possible?

i.e. my alias should use 30GB-large indexes and keep as many as possible, whilst never exceeding 200GB total.

• elasticsearch
• ilm

Share Improve this question Follow asked yesterday Jon BatesJon Bates 3,17333 gold badges3333 silver badges5050 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

In the rollover action, an index rolls over when any max value or all min values are reached. In your ILM scenario, the index will roll over 6 hours after creation OR when the total size of primary shards exceeds 20GB.

If you have a fixed limit like 200GB and a single index pattern, you can set up the logic as follows:

Solution:

You can manage indices solely by their age.

• 200GB * 0.8 = 160GB: Set the maximum disk usage to 160GB to avoid disk watermark issues.
• Set the primary shard count to 3 or 4, ensuring each shard size stays around 50GB.
• Set rollover.max_size to 160GB.
• Configure delete.min_age to 0 seconds to immediately delete the index after rollover.

---

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_size": "160gb"
          }
        }
      },
      "delete": {
        "min_age": "0h",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

本文标签： Control ElasticSearch total alias size using ILMStack Overflow