Skip to content
50 changes: 31 additions & 19 deletions docs/logging.asciidoc
Original file line number Diff line number Diff line change
@@ -1,34 +1,36 @@
[[log-correlation]]
== Log correlation
[[logs]]
== Logs

Log correlation allows you to navigate to all logs belonging to a particular trace, and vice-versa -- for a specific log, see in which context it has been logged, and which parameters the user provided.
Elastic Python APM Agent provides the following log features:

The Agent provides integrations with both the default Python logging library,
as well as http://www.structlog.org/en/stable/[`structlog`].
- <<log-correlation-ids>> : Automatically inject correlation IDs that allow navigation between logs, traces and services.
- <<log-reformatting>> : Automatically reformat plaintext logs in {ecs-logging-ref}/intro.html[ECS logging] format.

* <<logging-integrations>>
* <<log-correlation-in-es>>
Those features are part of {observability-guide}/application-logs.html[Application log ingestion strategies].

The {ecs-logging-python-ref}/intro.html[`ecs-logging-python`] library can also be used to use the {ecs-logging-ref}/intro.html[ECS logging] format without an APM agent.
When deployed with the Python APM agent, the agent will provide <<log-correlation-ids,log correlation>> IDs.

[float]
[[ecs-logging]]
=== `ecs-logging`
[[log-correlation-ids]]
=== Log correlation

The easiest way to integrate your logs with APM is to use the
https://github.com/elastic/ecs-logging-python[`ecs-logging`] library, which
is also provided by Elastic. This library provides formatters for both `logging`
and `structlog` which create ECS-compatible logs and will include the tracing
information required for log correlation in kibana. Coupled with something like
https://www.elastic.co/beats/filebeat[Filebeat], it is the easiest way to get
logs into Elasticsearch.
{apm-guide-ref}/log-correlation.html[Log correlation] allows you to navigate to all logs belonging to a particular trace
and vice-versa: for a specific log, see in which context it has been logged and which parameters the user provided.

The Agent provides integrations with both the default Python logging library,
as well as http://www.structlog.org/en/stable/[`structlog`].

* <<logging-integrations>>
* <<log-correlation-in-es>>

[float]
[[logging-integrations]]
=== Logging integrations
==== Logging integrations

[float]
[[logging]]
==== `logging`
===== `logging`

For Python 3.2+, we use https://docs.python.org/3/library/logging.html#logging.setLogRecordFactory[`logging.setLogRecordFactory()`]
to decorate the default LogRecordFactory to automatically add new attributes to
Expand Down Expand Up @@ -66,7 +68,7 @@ your log handlers, as handlers are propagated, along with their attached filters

[float]
[[structlog]]
==== `structlog`
===== `structlog`

We provide a http://www.structlog.org/en/stable/processors.html[processor] for
http://www.structlog.org/en/stable/[`structlog`] which will add three new keys
Expand Down Expand Up @@ -174,3 +176,13 @@ Then, you could use a grok pattern like this (for the
]
}
----

[float]
[[log-reformatting]]
=== Log reformatting (experimental)

The agent can automatically reformat application to ECS format when the application includes the `ecs-logging-python` library in its dependencies.

Log reformatting is controlled by the <<config-log_ecs_reformatting, `log_ecs_reformatting`>> configuration option, and is disabled by default.

The reformatted logs will include both the <<log-correlation-ids, trace and service correlation>> IDs.
5 changes: 5 additions & 0 deletions docs/redirects.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,8 @@ The following pages have moved or been deleted.
=== OpenTracing API

Refer to <<opentelemetry-bridge>> instead.

[role="exclude",id="log-correlation"]
=== Log correlation

Refer to <<logs>> instead.