Get statistics for pipelines

GET /_node/stats/pipelines

Basic auth

Get pipeline performance metrics and plugin details.

Query parameters

pretty boolean

If you append ?pretty=true to the request, the JSON returned will be pretty formatted. Use it for debugging only!

Responses

200 application/json
A JSON object containing pipelines statistics.
- the number of events that were input, filtered, or output by each pipeline
- the current and lifetime [flow rates](https://www.elastic.co/guide/en/logstash/current/node-stats-api.html#flow-stats for each pipeline
- stats for each configured filter or output stage
- info about config reload successes and failures (when config reload is enabled)
- info about the persistent queue (when persistent queues are enabled)
- metrics related to processed batch sizes. Includes the size in bytes and the number of events of batches processed in this pipeline. (when setting pipeline.batch.metrics.sampling_mode is not disabled).
Hide response attribute Show response attribute object
- pipelines object
  
  A map where each key is a user-defined pipeline name, and the value contains the metrics for that pipeline.
  
  Hide pipelines attribute Show pipelines attribute object
  
  * object
  
  Metrics of each plugin in the pipeline, flow statistics, event statistics and queue statistics.
  
  Hide * attributes Show * attributes object
  
  events object
  
  Hide events attributes Show events attributes object
  
  in integer(int64)
  
  filtered integer(int64)
  
  out integer(int64)
  
  duration_in_millis integer(int64)
  
  queue_push_duration_in_millis integer(int64)
  
  flow object
  
  One of:
  FlowStatsPart2 object FlowStatsPart3 object
  
  Hide attributes Show attributes
  
  input_throughput object
  
  This metric is expressed in events-per-second, and is the rate of events being pushed into the pipeline(s) queue(s) relative to wall-clock time (events.in / second). It includes events that are blocked by the queue and have not yet been accepted.
  
  Hide input_throughput attributes Show input_throughput attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  filter_throughput object
  
  This metric is expressed in events-per-second, and is the rate of events flowing through the filter phase of the pipeline(s) relative to wall-clock time (events.filtered / second).
  
  Hide filter_throughput attributes Show filter_throughput attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  output_throughput object
  
  This metric is expressed in events-per-second, and is the rate of events flowing through the output phase of the pipeline(s) relative to wall-clock time (events.out / second).
  
  Hide output_throughput attributes Show output_throughput attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  queue_backpressure object
  
  This is a unitless metric representing the cumulative time spent by all inputs blocked pushing events into their pipeline’s queue, relative to wall-clock time (queue_push_duration_in_millis / millisecond). It is typically most useful when looking at the stats for an individual pipeline.
  
  While a "zero" value indicates no back-pressure to the queue, the magnitude of this metric is highly dependent on the shape of the pipelines and their inputs. It cannot be used to compare one pipeline to another or even one process to itself if the quantity or shape of its pipelines changes. A pipeline with only one single-threaded input may contribute up to 1.00, a pipeline whose inputs have hundreds of inbound connections may contribute much higher numbers to this combined value.
  
  Additionally, some amount of back-pressure is both normal and expected for pipelines that are pulling data, as this back-pressure allows them to slow down and pull data at a rate its downstream pipeline can tolerate.
  
  Hide queue_backpressure attributes Show queue_backpressure attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  worker_concurrency object
  
  This is a unitless metric representing the cumulative time spent by all workers relative to wall-clock time (duration_in_millis / millisecond).
  
  A pipeline is considered "saturated" when its worker_concurrency flow metric approaches its available pipeline.workers, because it indicates that all of its available workers are being kept busy. Tuning a saturated pipeline to have more workers can often work to increase that pipeline’s throughput and decrease back-pressure to its queue, unless the pipeline is experiencing back-pressure from its outputs.
  
  A process is also considered "saturated" when its top-level worker_concurrency flow metric approaches the cumulative pipeline.workers across all pipelines, and similarly can be addressed by tuning the individual pipelines that are saturated.
  
  Hide worker_concurrency attributes Show worker_concurrency attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  worker_utilization object
  
  This is a unitless metric that indicates the percentage of available worker time being used by this individual plugin (duration / (uptime * pipeline.workers). It is useful for identifying which plugins in a pipeline are using the available worker resources.
  
  A pipeline is considered "saturated" when worker_utilization approaches 100, because it indicates that all of its workers are being kept busy. This is typically an indication of either downstream back-pressure or insufficient resources allocated to the pipeline. Tuning a saturated pipeline to have more workers can often work to increase that pipeline’s throughput and decrease back-pressure to its queue, unless the pipeline is experiencing back-pressure from its outputs.
  
  A pipeline is considered "starved" when worker_utilization approaches 0, because it indicates that none of its workers are being kept busy. This is typically an indication that the inputs are not receiving or retrieving enough volume to keep the pipeline workers busy. Tuning a starved pipeline to have fewer workers can help it to consume less memory and CPU, freeing up resources for other pipelines.
  
  Hide worker_utilization attributes Show worker_utilization attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  Hide attributes Show attributes
  
  input_throughput object
  
  This metric is expressed in events-per-second, and is the rate of events being pushed into the pipeline(s) queue(s) relative to wall-clock time (events.in / second). It includes events that are blocked by the queue and have not yet been accepted.
  
  Hide input_throughput attributes Show input_throughput attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  filter_throughput object
  
  This metric is expressed in events-per-second, and is the rate of events flowing through the filter phase of the pipeline(s) relative to wall-clock time (events.filtered / second).
  
  Hide filter_throughput attributes Show filter_throughput attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  output_throughput object
  
  This metric is expressed in events-per-second, and is the rate of events flowing through the output phase of the pipeline(s) relative to wall-clock time (events.out / second).
  
  Hide output_throughput attributes Show output_throughput attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  queue_backpressure object
  
  This is a unitless metric representing the cumulative time spent by all inputs blocked pushing events into their pipeline’s queue, relative to wall-clock time (queue_push_duration_in_millis / millisecond). It is typically most useful when looking at the stats for an individual pipeline.
  
  While a "zero" value indicates no back-pressure to the queue, the magnitude of this metric is highly dependent on the shape of the pipelines and their inputs. It cannot be used to compare one pipeline to another or even one process to itself if the quantity or shape of its pipelines changes. A pipeline with only one single-threaded input may contribute up to 1.00, a pipeline whose inputs have hundreds of inbound connections may contribute much higher numbers to this combined value.
  
  Additionally, some amount of back-pressure is both normal and expected for pipelines that are pulling data, as this back-pressure allows them to slow down and pull data at a rate its downstream pipeline can tolerate.
  
  Hide queue_backpressure attributes Show queue_backpressure attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  worker_concurrency object
  
  This is a unitless metric representing the cumulative time spent by all workers relative to wall-clock time (duration_in_millis / millisecond).
  
  A pipeline is considered "saturated" when its worker_concurrency flow metric approaches its available pipeline.workers, because it indicates that all of its available workers are being kept busy. Tuning a saturated pipeline to have more workers can often work to increase that pipeline’s throughput and decrease back-pressure to its queue, unless the pipeline is experiencing back-pressure from its outputs.
  
  A process is also considered "saturated" when its top-level worker_concurrency flow metric approaches the cumulative pipeline.workers across all pipelines, and similarly can be addressed by tuning the individual pipelines that are saturated.
  
  Hide worker_concurrency attributes Show worker_concurrency attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  worker_utilization object
  
  This is a unitless metric that indicates the percentage of available worker time being used by this individual plugin (duration / (uptime * pipeline.workers). It is useful for identifying which plugins in a pipeline are using the available worker resources.
  
  A pipeline is considered "saturated" when worker_utilization approaches 100, because it indicates that all of its workers are being kept busy. This is typically an indication of either downstream back-pressure or insufficient resources allocated to the pipeline. Tuning a saturated pipeline to have more workers can often work to increase that pipeline’s throughput and decrease back-pressure to its queue, unless the pipeline is experiencing back-pressure from its outputs.
  
  A pipeline is considered "starved" when worker_utilization approaches 0, because it indicates that none of its workers are being kept busy. This is typically an indication that the inputs are not receiving or retrieving enough volume to keep the pipeline workers busy. Tuning a starved pipeline to have fewer workers can help it to consume less memory and CPU, freeing up resources for other pipelines.
  
  Hide worker_utilization attributes Show worker_utilization attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  queue_persisted_growth_bytes object
  
  This metric is expressed in bytes-per-second, and is the rate of change of the size of the persistent queue on disk, relative to wall-clock time (queue.queue_size_in_bytes / second). A positive number indicates that the queue size-on-disk is growing, and a negative number indicates that the queue is shrinking.
  
  NOTE: The size of a PQ on disk includes both unacknowledged events and previously-acknowledged events from pages that contain one or more unprocessed events. This means it grows gradually as individual events are added, but shrinks in large chunks each time a whole page of processed events is reclaimed (read more: PQ disk garbage collection).
  
  Hide queue_persisted_growth_bytes attributes Show queue_persisted_growth_bytes attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  queue_persisted_growth_events object
  
  This metric is expressed in events-per-second, and is the rate of change of the number of unacknowledged events in the queue, relative to wall-clock time (queue.events_count / second). A positive number indicates that the queue’s event-count is growing, and a negative number indicates that the queue is shrinking.
  
  Hide queue_persisted_growth_events attributes Show queue_persisted_growth_events attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  plugins object
  
  Hide plugins attributes Show plugins attributes object
  
  inputs array[object]
  
  Hide inputs attributes Show inputs attributes object
  
  id string
  
  name string
  
  flow object
  
  Hide flow attribute Show flow attribute object
  
  throughput object
  
  This metric is expressed in events-per-second, and is the rate of events this input plugin is pushing into the pipeline’s queue relative to wall-clock time (events.in / second). It includes events that are blocked by the queue and have not yet been accepted.
  
  Hide throughput attributes Show throughput attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  events object
  
  Hide events attributes Show events attributes object
  
  out integer(int64)
  
  queue_push_duration_in_millis integer(int64)
  
  codecs array[object]
  
  Hide codecs attributes Show codecs attributes object
  
  id string
  
  name string
  
  decode object
  
  Hide decode attributes Show decode attributes object
  
  duration_in_millis integer
  
  writes_in integer
  
  out integer
  
  encode object
  
  Hide encode attributes Show encode attributes object
  
  duration_in_millis integer
  
  writes_in integer
  
  filters array[object]
  
  Hide filters attributes Show filters attributes object
  
  id string
  
  name string
  
  flow object
  
  Hide flow attributes Show flow attributes object
  
  worker_utilization object
  
  This is a unitless metric that indicates the percentage of available worker time being used by this individual plugin (duration / (uptime * pipeline.workers). It is useful for identifying which plugins in a pipeline are using the available worker resources.
  
  Hide worker_utilization attributes Show worker_utilization attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  worker_millis_per_event object
  
  This metric is expressed in worker-millis-spent-per-event (duration_in_millis / events.in) with higher scores indicating more resources spent per event. It is especially useful for identifying issues with plugins that operate on a small subset of events. An "Infinity" value for a given flow window indicates that worker millis have been spent without any events completing processing; this can indicate a plugin that is either stuck or handling only empty batches.
  
  Hide worker_millis_per_event attributes Show worker_millis_per_event attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  events object
  
  Hide events attributes Show events attributes object
  
  in integer(int64)
  
  out integer(int64)
  
  duration_in_millis integer(int64)
  
  outputs array[object]
  
  Hide outputs attributes Show outputs attributes object
  
  id string
  
  name string
  
  flow object
  
  Hide flow attributes Show flow attributes object
  
  worker_utilization object
  
  This is a unitless metric that indicates the percentage of available worker time being used by this individual plugin (duration / (uptime * pipeline.workers). It is useful for identifying which plugins in a pipeline are using the available worker resources.
  
  Hide worker_utilization attributes Show worker_utilization attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  worker_millis_per_event object
  
  This metric is expressed in worker-millis-spent-per-event (duration_in_millis / events.in) with higher scores indicating more resources spent per event. It is especially useful for identifying issues with plugins that operate on a small subset of events. An "Infinity" value for a given flow window indicates that worker millis have been spent without any events completing processing; this can indicate a plugin that is either stuck or handling only empty batches.
  
  Hide worker_millis_per_event attributes Show worker_millis_per_event attributes object
  
  current number(double) | string
  
  The most recent ~10s.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_minute number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_5_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_15_minutes number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_1_hour number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  last_24_hours number(double) | string
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  lifetime number(double) | string
  
  The lifetime of the relevant pipeline or process.
  
  One of:
  number-1 number(double) string-2 string
  
  Value is Infinity.
  
  events object
  
  Hide events attributes Show events attributes object
  
  in integer(int64)
  
  out integer(int64)
  
  duration_in_millis integer(int64)
  
  reloads object
  
  Hide reloads attributes Show reloads attributes object
  
  last_error string | null
  
  successes integer(int64)
  
  last_success_timestamp string | null
  
  last_failure_timestamp string | null
  
  failures integer(int64)
  
  queue object
  
  One of:
  object-1 object object-2 object
  
  The metrics of persisted queue.
  
  Hide attributes Show attributes
  
  type string Required
  
  Value is persisted.
  
  capacity object
  
  Hide capacity attributes Show capacity attributes object
  
  max_unread_events integer(int64)
  
  page_capacity_in_bytes integer(int64)
  
  max_queue_size_in_bytes integer(int64)
  
  queue_size_in_bytes integer(int64)
  
  data object
  
  Hide data attributes Show data attributes object
  
  path string
  
  free_space_in_bytes integer(int64)
  
  storage_type string
  
  events integer(int64)
  
  events_count integer(int64)
  
  queue_size_in_bytes integer(int64)
  
  max_queue_size_in_bytes integer(int64)
  
  compression object
  
  Hide compression attributes Show compression attributes object
  
  encode object
  
  Hide encode attributes Show encode attributes object
  
  ratio object
  
  the ratio of event size in bytes to its representation on disk
  
  Hide ratio attribute Show ratio attribute object
  
  lifetime number
  
  One of:
  number-1 number 2
  
  Values are Infinity, NaN, or -Infinity.
  
  spend object
  
  the fraction of wall-clock time spent encoding events
  
  Hide spend attribute Show spend attribute object
  
  lifetime number
  
  One of:
  number-1 number 2
  
  Values are Infinity, NaN, or -Infinity.
  
  decode object
  
  Hide decode attributes Show decode attributes object
  
  ratio object
  
  the ratio of event representation on disk to event size
  
  Hide ratio attribute Show ratio attribute object
  
  lifetime number
  
  One of:
  number-1 number 2
  
  Values are Infinity, NaN, or -Infinity.
  
  spend object
  
  the fraction of wall-clock time spent decoding events
  
  Hide spend attribute Show spend attribute object
  
  lifetime number
  
  One of:
  number-1 number 2
  
  Values are Infinity, NaN, or -Infinity.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  workers integer
  
  batch_size integer
  
  batch_delay integer
  
  hash string
  
  A unique hash identifier.
  
  ephemeral_id string(uuid)
  
  A temporary unique identifier for the instance.

GET /_node/stats/pipelines

curl \ --request GET 'http://api.example.com/_node/stats/pipelines' \ --user "username:password"

Response examples (200)

{ "pipelines": { "beats-es": { "flow": { "input_throughput": { "current": 603.1, "lifetime": 575.4 }, "filter_throughput": { "current": 604.2, "lifetime": 575.1 }, "output_throughput": { "current": 604.8, "lifetime": 575.1 }, "queue_backpressure": { "current": 0.214, "lifetime": 0.937 }, "worker_concurrency": { "current": 0.941, "lifetime": 0.9709 }, "worker_utilization": { "current": 93.092, "lifetime": 92.187 } }, "batch": { "byte_size": { "average": { "lifetime": 14820 }, "current": 32767 }, "event_count": { "average": { "lifetime": 115 }, "current": 78 } }, "queue": { "type": "memory" }, "events": { "in": 216610, "out": 216485, "filtered": 216485, "duration_in_millis": 365495, "queue_push_duration_in_millis": 342466 }, "plugins": { "inputs": [ { "id": "35131f351e2dc5ed13ee04265a8a5a1f95292165-1", "flow": { "throughput": { "current": 603.1, "lifetime": 590.7 } }, "name": "beats", "events": { "out": 216485, "queue_push_duration_in_millis": 342466 } } ], "filters": [ { "id": "35131f351e2dc5ed13ee04265a8a5a1f95292165-2", "flow": { "worker_utilization": { "current": 16.71, "lifetime": 15.27 }, "worker_millis_per_event": { "current": 2829, "lifetime": 0.2585 } }, "name": "grok", "events": { "in": 216485, "out": 216485, "duration_in_millis": 55969 }, "failures": 216485, "patterns_per_field": { "message": 1 } }, { "id": "35131f351e2dc5ed13ee04265a8a5a1f95292165-3", "flow": { "worker_utilization": { "current": 1.042, "lifetime": 0.9076 }, "worker_millis_per_event": { "current": 0.01763, "lifetime": 0.01536 } }, "name": "geoip", "events": { "in": 216485, "out": 216485, "duration_in_millis": 3326 } } ], "outputs": [ { "id": "35131f351e2dc5ed13ee04265a8a5a1f95292165-4", "flow": { "worker_utilization": { "current": 75.34, "lifetime": 76.01 }, "worker_millis_per_event": { "current": 1.276, "lifetime": 1.287 } }, "name": "elasticsearch", "events": { "in": 216485, "out": 216485, "duration_in_millis": 278557 } } ] }, "reloads": { "failures": 0, "successes": 0, "last_error": null, "last_failure_timestamp": null, "last_success_timestamp": null }, "pipeline": { "workers": 4, "batch_size": 125, "batch_delay": 50 } }, "heartbeat-ruby-stdout": { "flow": { "input_throughput": { "current": 0, "lifetime": 0.0167, "last_1_minute": 0.01614, "last_5_minutes": 0.01627, "last_15_minutes": 0.01643 }, "filter_throughput": { "current": 0, "lifetime": 0.0167, "last_1_minute": 0.01614, "last_5_minutes": 0.01627, "last_15_minutes": 0.01643 }, "output_throughput": { "current": 0, "lifetime": 0.0167, "last_1_minute": 0.01614, "last_5_minutes": 0.01627, "last_15_minutes": 0.01643 }, "queue_backpressure": { "current": 0, "lifetime": 5.901e-05, "last_1_minute": 1.614e-05, "last_5_minutes": 2.278e-05, "last_15_minutes": 4.272e-05 }, "worker_concurrency": { "current": 0, "lifetime": 0.0001266, "last_1_minute": 4.843e-05, "last_5_minutes": 7.486e-05, "last_15_minutes": 0.0001183 }, "worker_utilization": { "current": 0, "lifetime": 0.001055, "last_1_minute": 0.0004036, "last_5_minutes": 0.0006239, "last_15_minutes": 0.0009858 }, "queue_persisted_growth_bytes": { "current": 0, "lifetime": 4.214, "last_1_minute": 4.068, "last_5_minutes": 4.101, "last_15_minutes": 4.14 }, "queue_persisted_growth_events": { "current": 0, "lifetime": 0, "last_1_minute": 0, "last_5_minutes": 0, "last_15_minutes": 0 } }, "hash": "c11e0502ebf98956dffa371775fd1cb719f85f819ee166b770b3e982da1c999d", "queue": { "data": { "path": "/path/to/logstash/data/queue/heartbeat-ruby-stdout", "storage_type": "apfs", "free_space_in_bytes": 60833390592 }, "type": "persisted", "events": 0, "capacity": { "max_unread_events": 0, "queue_size_in_bytes": 11341, "page_capacity_in_bytes": 67108864, "max_queue_size_in_bytes": 1073741824 }, "events_count": 0, "queue_size_in_bytes": 11341, "max_queue_size_in_bytes": 1073741824 }, "events": { "in": 45, "out": 45, "filtered": 45, "duration_in_millis": 341, "queue_push_duration_in_millis": 159 }, "plugins": { "codecs": [ { "id": "plain_b0090202-daac-428e-8355-18c35a0826cb", "name": "plain", "decode": { "out": 0, "writes_in": 0, "duration_in_millis": 0 }, "encode": { "writes_in": 0, "duration_in_millis": 0 } }, { "id": "6bff4bc6-3a8c-494c-aa65-d16e1eb09578", "name": "rubydebug" } ], "inputs": [ { "id": "c9ca46e359d73146590ca8af40092342afa922f3cbf07adc4b5009e346cb19f7", "flow": { "throughput": { "current": 0, "lifetime": 0.0167, "last_1_minute": 0.01614, "last_5_minutes": 0.01627, "last_15_minutes": 0.01643 } }, "name": "heartbeat", "events": { "out": 45, "queue_push_duration_in_millis": 159 } } ], "filters": [ { "id": "585fa932a4fd506055ead07ee5ebcb3033c27c82ba90cbee73d6ac7e9357333a", "flow": { "worker_utilization": { "current": 0, "lifetime": 0.000266, "last_1_minute": 0.0001345, "last_5_minutes": 0.0001627, "last_15_minutes": 0.0002191 }, "worker_millis_per_event": { "lifetime": 1.911, "last_1_minute": 1, "last_5_minutes": 1.2, "last_15_minutes": 1.6 } }, "name": "ruby", "events": { "in": 45, "out": 45, "duration_in_millis": 86 } } ], "outputs": [ { "id": "c4f801c8f170b4cb0679f704413773acd5f291929f42302d0a56361400c3741b", "flow": { "worker_utilization": { "current": 0, "lifetime": 0.000668, "last_1_minute": 0.000269, "last_5_minutes": 0.000434, "last_15_minutes": 0.0006663 }, "worker_millis_per_event": { "lifetime": 4.8, "last_1_minute": 2, "last_5_minutes": 3.2, "last_15_minutes": 4.867 } }, "name": "stdout", "events": { "in": 45, "out": 45, "duration_in_millis": 216 } } ] }, "reloads": { "failures": 0, "successes": 0, "last_error": null, "last_failure_timestamp": null, "last_success_timestamp": null }, "pipeline": { "workers": 12, "batch_size": 125, "batch_delay": 50 }, "ephemeral_id": "6a218b06-9655-4ddc-84e9-d5bfa51e5dc7" } } }

Get statistics for pipelines

Query parameters

Responses

flow object

current number(double) | string

last_1_minute number(double) | string

last_5_minutes number(double) | string

last_15_minutes number(double) | string

last_1_hour number(double) | string

last_24_hours number(double) | string

lifetime number(double) | string

current number(double) | string

last_1_minute number(double) | string

last_5_minutes number(double) | string

last_15_minutes number(double) | string

last_1_hour number(double) | string

last_24_hours number(double) | string

lifetime number(double) | string

current number(double) | string

last_1_minute number(double) | string

last_5_minutes number(double) | string

last_15_minutes number(double) | string

last_1_hour number(double) | string

last_24_hours number(double) | string

lifetime number(double) | string

current number(double) | string

last_1_minute number(double) | string

last_5_minutes number(double) | string

last_15_minutes number(double) | string

last_1_hour number(double) | string

last_24_hours number(double) | string

lifetime number(double) | string

current number(double) | string

last_1_minute number(double) | string

last_5_minutes number(double) | string

last_15_minutes number(double) | string

last_1_hour number(double) | string

last_24_hours number(double) | string

lifetime number(double) | string

queue object

lifetime number

lifetime number

lifetime number

lifetime number