[elastic_agent] Add 10 remaining input dashboards

[chrisberkhout]

Proposed commit message

[elastic_agent] Add 10 remaining input dashboards Add dashboards: - [Elastic Agent] Azure Blob Storage Input Metrics - [Elastic Agent] Azure Eventhub Input Metrics - [Elastic Agent] Entity Analytics Input Metrics - [Elastic Agent] ETW Input Metrics - [Elastic Agent] GCP Pub Sub Input Metrics - [Elastic Agent] Google Cloud Storage Input Metrics - [Elastic Agent] Lumberjack Input Metrics - [Elastic Agent] Streaming Input Metrics - [Elastic Agent] Unified Logs Input Metrics - [Elastic Agent] Unix Input Metrics And update the navigation panel for all dashboards. 

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Discussion

I did the 10 dashboards in one PR because each new dashboard means a change to the navigation of all dashboards, and because it should be easier to review one having seen the others, and because we should ideally have some consistency across dashboards.

I mostly followed the patterns in the existing dashboards.

I used data generated using this script, which was LLM generated with some tweaks.

Here's some useful data and links:

Input	filebeat_input.input	Metrics documentation	Metrics definitions
azureblobstorage	azure-blob-storage	Input metrics doc	Metrics definitions
azureeventhub	azure-eventhub	Input metrics doc	Metrics definitions
entityanalytics	entity-analytics-azure-ad	Input metrics doc	Metrics definitions
entityanalytics	entity-analytics-jamf	Input metrics doc	Metrics definitions
entityanalytics	entity-analytics-okta	Input metrics doc	Metrics definitions
etw	etw	Input metrics doc	Metrics definitions
gcppubsub	gcp-pubsub	Input metrics doc	Metrics definitions
gcs	gcs	Input metrics doc	Metrics definitions
streaming	streaming	Input metrics doc	Metrics definitions
unifiedlogs	unifiedlogs	Input metrics doc	Metrics definitions
unix	unix	Input metrics doc	Metrics definitions
lumberjack	lumberjack	(undocumented)	Metrics definitions

Expand for a table of filebeat_input.input, Metric, Metric type, Metric description

filebeat_input.input	Metric	Metric type	Metric description
unix	filebeat_input.path	string	Path of the unix socket.
unix	filebeat_input.received_events_total	uint64	Total number of packets (events) that have been received.
unix	filebeat_input.received_bytes_total	uint64	Total number of bytes received.
unix	filebeat_input.arrival_period	histogram	Histogram of the time between successive packets in nanoseconds.
unix	filebeat_input.processing_time	histogram	Histogram of the time taken to process packets in nanoseconds.
unifiedlogs	filebeat_input.errors_total	uint64	Total number of errors.
streaming	filebeat_input.url	string	URL of the input resource.
streaming	filebeat_input.cel_eval_errors	uint64	Number of errors encountered during cel program evaluation.
streaming	filebeat_input.errors_total	uint64	Number of errors encountered over the life cycle of the input.
streaming	filebeat_input.batches_received_total	uint64	Number of event arrays received.
streaming	filebeat_input.batches_published_total	uint64	Number of event arrays published.
streaming	filebeat_input.received_bytes_total	uint64	Number of bytes received over the life cycle of the input.
streaming	filebeat_input.events_received_total	uint64	Number of events received.
streaming	filebeat_input.events_published_total	uint64	Number of events published.
streaming	filebeat_input.write_control_errors	uint64	Number of errors encountered while sending write control messages like ping.
streaming	filebeat_input.cel_processing_time	histogram	Histogram of the elapsed successful CEL program processing times in nanoseconds.
streaming	filebeat_input.batch_processing_time	histogram	Histogram of the elapsed successful batch processing times in nanoseconds (time of receipt to time of ACK for non-empty batches).
streaming	filebeat_input.ping_message_send_time	histogram	Histogram of the elapsed successful ping message send times in nanoseconds.
streaming	filebeat_input.pong_message_received_time	histogram	Histogram of the elapsed successful pong message receive times in nanoseconds.
gcs	filebeat_input.url	string	URL of the input resource.
gcs	filebeat_input.errors_total	uint64	Total number of errors encountered by the input.
gcs	filebeat_input.decode_errors_total	uint64	Total number of decode errors encountered by the input.
gcs	filebeat_input.gcs_failed_jobs_total	uint64	Total number of failed jobs.
gcs	filebeat_input.gcs_expired_failed_jobs_total	uint64	Total number of expired failed jobs that could not be recovered.
gcs	filebeat_input.gcs_bytes_processed_total	uint64	Total number of GCS bytes processed.
gcs	filebeat_input.gcs_events_created_total	uint64	Total number of events created from processing GCS data.
gcs	filebeat_input.gcs_objects_listed_total	uint64	Total number of GCS objects returned by list operations.
gcs	filebeat_input.gcs_objects_requested_total	uint64	Total number of GCS objects downloaded.
gcs	filebeat_input.gcs_objects_published_total	uint64	Total number of GCS objects processed that were published.
gcs	filebeat_input.gcs_objects_tracked_gauge	uint64	Number of objects currently tracked in the state registry (gauge).
gcs	filebeat_input.gcs_objects_inflight_gauge	uint64	Number of GCS objects inflight (gauge).
gcs	filebeat_input.gcs_jobs_scheduled_after_validation	histogram	Histogram of the number of jobs scheduled after validation.
gcs	filebeat_input.gcs_object_processing_time	histogram	Histogram of the elapsed GCS object processing times in nanoseconds (start of download to completion of parsing).
gcs	filebeat_input.gcs_object_size_in_bytes	histogram	Histogram of processed GCS object size in bytes.
gcs	filebeat_input.gcs_events_per_object	histogram	Histogram of event count per GCS object.
gcs	filebeat_input.source_lag_time	histogram	Histogram of the time between the source (Updated) timestamp and the time the object was read, in nanoseconds.
gcp-pubsub	filebeat_input.acked_message_total	uint64	Number of successfully ACKed messages.
gcp-pubsub	filebeat_input.failed_acked_message_total	uint64	Number of failed ACKed messages.
gcp-pubsub	filebeat_input.nacked_message_total	uint64	Number of NACKed messages.
gcp-pubsub	filebeat_input.bytes_processed_total	uint64	Number of bytes processed.
gcp-pubsub	filebeat_input.processing_time	histogram	Histogram of the elapsed time for processing an event in nanoseconds.
etw	filebeat_input.session	string	Name of the ETW session.
etw	filebeat_input.received_events_total	uint64	Total number of events received.
etw	filebeat_input.discarded_events_total	uint64	Total number of discarded events.
etw	filebeat_input.errors_total	uint64	Total number of errors.
etw	filebeat_input.source_lag_time	histogram	Histogram of the difference between timestamped event’s creation and reading.
etw	filebeat_input.arrival_period	histogram	Histogram of the elapsed time between event notification callbacks.
etw	filebeat_input.processing_time	histogram	Histogram of the elapsed time between event notification callback and publication to the internal queue.
entity-analytics-azure-ad	filebeat_input.sync_total	uint64	The total number of full synchronizations.
entity-analytics-azure-ad	filebeat_input.sync_error	uint64	The number of full synchronizations that failed due to an error.
entity-analytics-azure-ad	filebeat_input.sync_processing_time	histogram	Histogram of the elapsed full synchronizations times in nanoseconds (time of API contact to items sent to output).
entity-analytics-azure-ad	filebeat_input.update_total	uint64	The total number of incremental updates.
entity-analytics-azure-ad	filebeat_input.update_error	uint64	The number of incremental updates that failed due to an error.
entity-analytics-azure-ad	filebeat_input.update_processing_time	histogram	Histogram of the elapsed incremental updates times in nanoseconds (time of API contact to items sent to output).
entity-analytics-jamf	filebeat_input.sync_total	uint64	The total number of full synchronizations.
entity-analytics-jamf	filebeat_input.sync_error	uint64	The number of full synchronizations that failed due to an error.
entity-analytics-jamf	filebeat_input.sync_processing_time	histogram	Histogram of the elapsed full synchronizations times in nanoseconds (time of API contact to items sent to output).
entity-analytics-jamf	filebeat_input.update_total	uint64	The total number of incremental updates.
entity-analytics-jamf	filebeat_input.update_error	uint64	The number of incremental updates that failed due to an error.
entity-analytics-jamf	filebeat_input.update_processing_time	histogram	Histogram of the elapsed incremental updates times in nanoseconds (time of API contact to items sent to output).
entity-analytics-okta	filebeat_input.sync_total	uint64	The total number of full synchronizations.
entity-analytics-okta	filebeat_input.sync_error	uint64	The number of full synchronizations that failed due to an error.
entity-analytics-okta	filebeat_input.sync_processing_time	histogram	Histogram of the elapsed full synchronizations times in nanoseconds (time of API contact to items sent to output).
entity-analytics-okta	filebeat_input.update_total	uint64	The total number of incremental updates.
entity-analytics-okta	filebeat_input.update_error	uint64	The number of incremental updates that failed due to an error.
entity-analytics-okta	filebeat_input.update_processing_time	histogram	Histogram of the elapsed incremental updates times in nanoseconds (time of API contact to items sent to output).
azure-eventhub	filebeat_input.received_messages_total	uint64	Number of messages received from the event hub.
azure-eventhub	filebeat_input.received_bytes_total	uint64	Number of bytes received from the event hub.
azure-eventhub	filebeat_input.invalid_json_messages_total	uint64	Number of messages containing invalid JSON.
azure-eventhub	filebeat_input.sanitized_messages_total	uint64	Number of messages that were sanitized successfully.
azure-eventhub	filebeat_input.processed_messages_total	uint64	Number of messages that were processed successfully.
azure-eventhub	filebeat_input.received_events_total	uint64	Number of events received decoding messages.
azure-eventhub	filebeat_input.sent_events_total	uint64	Number of events that were sent successfully.
azure-eventhub	filebeat_input.processing_time	histogram	Histogram of the elapsed processing times in nanoseconds.
azure-eventhub	filebeat_input.decode_errors_total	uint64	Number of errors that occurred while decoding a message.
azure-eventhub	filebeat_input.processor_restarts_total	uint64	Number of times the processor has restarted.
azure-blob-storage	filebeat_input.url	string	URL of the input resource.
azure-blob-storage	filebeat_input.errors_total	uint64	Total number of errors encountered by the input.
azure-blob-storage	filebeat_input.decode_errors_total	uint64	Total number of decode errors encountered by the input.
azure-blob-storage	filebeat_input.abs_blobs_requested_total	uint64	Total number of ABS blobs downloaded.
azure-blob-storage	filebeat_input.abs_blobs_published_total	uint64	Total number of ABS blobs processed that were published.
azure-blob-storage	filebeat_input.abs_blobs_listed_total	uint64	Total number of ABS blobs returned by list operations.
azure-blob-storage	filebeat_input.abs_bytes_processed_total	uint64	Total number of ABS bytes processed.
azure-blob-storage	filebeat_input.abs_events_created_total	uint64	Total number of events created from processing ABS data.
azure-blob-storage	filebeat_input.abs_blobs_inflight_gauge	uint64	Number of ABS blobs inflight (gauge).
azure-blob-storage	filebeat_input.abs_jobs_scheduled_after_validation	histogram	Histogram of the number of jobs scheduled after validation.
azure-blob-storage	filebeat_input.abs_blob_processing_time	histogram	Histogram of the elapsed ABS blob processing times in nanoseconds (start of download to completion of parsing).
azure-blob-storage	filebeat_input.abs_blob_size_in_bytes	histogram	Histogram of processed ABS blob size in bytes.
azure-blob-storage	filebeat_input.abs_events_per_blob	histogram	Histogram of event count per ABS blob.
azure-blob-storage	filebeat_input.source_lag_time	histogram	Histogram of the time between the source (Updated) timestamp and the time the blob was read, in nanoseconds.
lumberjack	filebeat_input.bind_address	string	Bind address of input.
lumberjack	filebeat_input.batches_received_total	uint64	Number of Lumberjack batches received (not necessarily processed fully).
lumberjack	filebeat_input.batches_acked_total	uint64	Number of Lumberjack batches ACKed.
lumberjack	filebeat_input.message_received_total	uint64	Number of Lumberjack messages received (not necessarily processed fully).
lumberjack	filebeat_input.batch_processing_time	histogram	Histogram of the elapsed batch processing times in nanoseconds (time of receipt to time of ACK for non-empty batches). [Elastic Agent] Azure Blob Storage Input Metrics

How to test this PR locally

To insert the generated data, use this script in the dev console.

Set the dashboard's time selector to Jul 1, 2025 @ 00:00:00.000 -- Jul 1, 2025 @ 01:00:00.000.

Related issues

• Closes [elastic_agent] Add Dashboards for inputs with metrics #14164

Screenshots

Screenshots of each of the 10 new dashboards, with some fake data:

1 - Azure Blob Storage:

2 - Azure Eventhub:

3 - Entity Analytics:

4 - ETW:

5 - GCP Pub Sub:

6 - Google Cloud Storage:

7 - Lumberjack:

8 - Streaming:

9 - Unified Logs:

10 - Unix:

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[efd6]

Suggestions:

• resize the ToC so that it renders with no need to scroll to see the full text.
• replace ", Cumulative" with " Total" in trace titles that refer to "total" type metrics

Query: Why do we have both the metric and its derivative in the unified logs dashboard?

[chrisberkhout]

Query: Why do we have both the metric and its derivative in the unified logs dashboard?

Just to fill the space with different views of the one available metric. I'll remove the derivative.

[chrisberkhout]

Suggestions:

• resize the ToC so that it renders with no need to scroll to see the full text.

Done.

• replace ", Cumulative" with " Total" in trace titles that refer to "total" type metrics

Done.

[efd6]

Nit only, feel free to ignore: the navigation panel uses bold to indicate the current page, but the text is still a link. This doesn't need to be the case and we have been replacing that behaviour elsewhere.

Otherwise LGTM

[chrisberkhout]

Nit only, feel free to ignore: the navigation panel uses bold to indicate the current page, but the text is still a link. This doesn't need to be the case and we have been replacing that behaviour elsewhere.

Okay, thanks. I'll merge now and keep that in mind in the future.

[chrisberkhout]

@swiatekm Would you mind reviewing this PR?
It's been reviewed by my team and some adjustments made, but we need a Elastic Agent approval to merge.
I've made sure the dashboard fix in #14752 is preserved by this one.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[AndersonQ]

Following the test instructions, all dashboards are working. I just left a nit comment, it's up to you to address it or not

[AndersonQ]

[Nit]
I'm wondering if a more user friendly change log would be good, perhaps list the added dashboards.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: acfbf23

History

• 💚 Build #29538 succeeded b74ee95
• 💚 Build #29229 succeeded adf2578
• 💚 Build #29138 succeeded bd7508a
• 💚 Build #29133 succeeded f3280d1
• 💚 Build #29070 succeeded aec2ea0

cc @chrisberkhout

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elastic-vault-github-plugin-prod]

Package elastic_agent - 2.4.0 containing this change is available at https://epr.elastic.co/package/elastic_agent/2.4.0/