Add ingest information to the cluster info endpoint

[HiDAl]

Add a new target (ingest) to the /_info API. It consolidates all the ingest information from the cluster nodes and returns a summary at the cluster level (compared with _nodes/stats/ingest it lacks the <node> dimension)

GET _info/ingest { "cluster_name": "runTask", "ingest": { "total": { "count": 77, "time_in_millis": 36, "current": 0, "failed": 0 }, "pipelines": { "test": { "count": 77, "time_in_millis": 4, "current": 0, "failed": 0, "processors": [ { "set": { "type": "set", "stats": { "count": 12, "time_in_millis": 0, "current": 0, "failed": 0 } } }, { "set": { "type": "set", "stats": { "count": 12, "time_in_millis": 0, "current": 0, "failed": 0 } } }, { "remove": { "type": "remove", "stats": { "count": 12, "time_in_millis": 0, "current": 0, "failed": 0 } } } ] }, ... } } } GET _info/_all { "cluster_name": "runTask", "http": { "current_open": 1, "total_opened": 112, "clients": [ { "id": 873375542, "local_address": "127.0.0.1:9200", "remote_address": "127.0.0.1:59560", "last_uri": "/_info/ingest", "opened_time_millis": 1684937431630, "closed_time_millis": 1684937431630, "last_request_time_millis": 1684937431630, "request_count": 1, "request_size_bytes": 0 }, { "id": 1090681032, "local_address": "127.0.0.1:9200", "remote_address": "127.0.0.1:59575", "last_uri": "/", "opened_time_millis": 1684937474976, "closed_time_millis": 1684937474976, "last_request_time_millis": 1684937474976, "request_count": 1, "request_size_bytes": 0 }, { "id": 2018590882, "local_address": "127.0.0.1:9200", "remote_address": "127.0.0.1:59576", "last_uri": "/_info/_all", "opened_time_millis": 1684937475181, "last_request_time_millis": 1684937475181, "request_count": 1, "request_size_bytes": 0 }, ... ] }, "ingest": { "total": { "count": 77, "time_in_millis": 36, "current": 0, "failed": 0 }, "pipelines": { "test": { "count": 77, "time_in_millis": 4, "current": 0, "failed": 0, "processors": [ { "set": { "type": "set", "stats": { "count": 12, "time_in_millis": 0, "current": 0, "failed": 0 } } }, { "set": { "type": "set", "stats": { "count": 12, "time_in_millis": 0, "current": 0, "failed": 0 } } }, { "remove": { "type": "remove", "stats": { "count": 12, "time_in_millis": 0, "current": 0, "failed": 0 } } } ] }, ... } } }

closes #95392

[github-actions]

Documentation preview:

• ✨ Changed pages

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[elasticsearchmachine]

Hi @HiDAl, I've created a changelog YAML for you.

[masseyke]

Is it the lifetime of the cluster? Or the lifetimes of all the nodes that are currently in the cluster (i.e. a node loses these counts on restart right?)?

[HiDAl]

Right... Will see how to reword this... Anyway, this is going to be a really weird number, because it won't be all the time increasing with nodes restarting...
@dakrone This is an important point to consider for these statistics.

[joegallo]

Indeed, this is tricky. Each node tracks the information it has seen since it started. This cluster-wide info view is the sum of those since-the-node-started statistics for all the nodes.

It's straightforward enough to talk about on this PR, but I'm not sure what I think would be appropriate wording for documentation. 😬

[masseyke]

This looks pretty good to me. Tagging @joegallo in case he's currently working on something that would impact this.

[masseyke]

I think you meant GET /_info/http,ingest

[HiDAl]

thanks!

[elasticsearchmachine]

Hi @HiDAl, I've updated the changelog YAML for you.

[joegallo]

Minor nit, but the issue in this comment applies to all these gtes, not just the two below. Maybe reword and relocate?

[joegallo]

Not a review, just a comment, but this new capability puts a song in my heart. 😄

[HiDAl]

@elasticsearchmachine run elasticsearch-ci/docs