I have created an ILM policy as follows,
{ "policy": { "phases": { "hot": { "min_age": "0ms", "actions": { "rollover": { "max_docs": 30 }, "set_priority": { "priority": 100 } } }, "delete": { "min_age": "15m", "actions": { "delete": {} } } } } } Index setting as follows,
{ "index": { "lifecycle": { "name": "index-roll+doc-over", "rollover_alias": "router01" }, "number_of_shards": "1", "number_of_replicas": "0" } } As you have already guessed, I am testing this ILM rollover policy which is why I have set the document size only 30. I am planning to limit the doc size of 1.5 Billion. But I am observing huge latency while creating a rollover index and switching to the new index. It is writing on the current index and not rolling over. For the test I have ingested only 500 lines of log.
Index details
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open router01-000001 654G2ArFQiypDn9U69jedw 1 0 500 0 151.1kb 151.1kb And at the time of writing this question, the other index (router01-000001) is not created yet.
The the _ilm/explain is following,
{ "indices" : { "router01-000001" : { "index" : "router01-000001", "managed" : true, "policy" : "index-roll+doc-over", "lifecycle_date_millis" : 1583819391191, "age" : "14.08m", "phase" : "hot", "phase_time_millis" : 1583819391396, "action" : "rollover", "action_time_millis" : 1583819706013, "step" : "check-rollover-ready", "step_time_millis" : 1583819706013, "phase_execution" : { "policy" : "index-roll+doc-over", "phase_definition" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_docs" : 30 }, "set_priority" : { "priority" : 100 } } }, "version" : 1, "modified_date_in_millis" : 1583756555777 } } } } Elasticsearch currently has 2GB heap and logstash has 1GB. Total Memory 4GB.
My question is,
- Is the latency expected (May be I misunderstood the rollover concept)
- Did I mistake anywhere?
- If not than what can I do to improve the latency as there will be 20 millions log will be ingested per day
