Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
180e6ac
fix whitespace
Feb 2, 2021
1c4b0e3
docs: add logger
Feb 2, 2021
b4dce5e
docs: add middleware
Feb 2, 2021
01147a6
docs: parameters
Feb 2, 2021
7f93bbf
docs: batch
Feb 2, 2021
2a5f50c
docs: typing
Feb 2, 2021
0c492e1
docs: validation
Feb 2, 2021
e174d15
docs: dataclasses
Feb 2, 2021
98b3428
docs: parser
Feb 2, 2021
f9da787
docs: use mkdocs for website
Feb 2, 2021
7636f61
docs: add docs branch to have live preview for the PR
Feb 2, 2021
707b79b
docs: change color and adjust logo
Feb 2, 2021
cb9e881
docs: merged async and generator sections
Feb 2, 2021
1123a31
docs: merged cloudwatch logs to tabs
Feb 2, 2021
1cfdce2
docs: merged tabs
Feb 2, 2021
add4f4f
docs: remove highlight line markers
Feb 2, 2021
f4ea9b7
docs: add meta tags
Feb 2, 2021
2d05c72
docs: remove old docs folder
Feb 2, 2021
264a028
chore: typo in GetCfnTemplate
heitorlessa Feb 3, 2021
1fa47d9
Update docs/core/logger.md
heitorlessa Feb 3, 2021
8f418cb
Update docs/core/logger.md
heitorlessa Feb 3, 2021
7ddb544
Update docs/utilities/parameters.md
heitorlessa Feb 3, 2021
e425d6d
Update docs/utilities/validation.md
heitorlessa Feb 3, 2021
36cba90
Update docs/utilities/validation.md
heitorlessa Feb 3, 2021
4b9ca27
Update docs/utilities/batch.md
heitorlessa Feb 3, 2021
84c8e35
Update docs/core/tracer.md
heitorlessa Feb 3, 2021
f96abe7
Merge branch 'develop' of github.com:awslabs/aws-lambda-powertools-py…
Feb 3, 2021
da46dff
Merge branch 'docs/mkdocs' of github.com:am29d/aws-lambda-powertools-…
Feb 3, 2021
2556afa
remove code formatting from table headers
Feb 3, 2021
a6eb2b4
docs: increse max-width with extra css for readability
Feb 3, 2021
421145a
docs: add changes from review
Feb 4, 2021
1a483c9
docs: refactor doc targets, add docker build
Feb 4, 2021
8e548f5
docs: tabs naming for cloudwatch logs
Feb 4, 2021
32fdd23
docs: fix build-docs-website target
Feb 4, 2021
a815b40
docs: enhance the visual aid for accessibility purposes
Feb 4, 2021
0511fa5
fix: syntax previously broken in last set of commits
heitorlessa Feb 4, 2021
c21faea
fix: async ops snippet completeness
heitorlessa Feb 4, 2021
cca91d3
improv: screen real estate in home page.
heitorlessa Feb 4, 2021
8f0860f
feat: enable strict mode for broken links
heitorlessa Feb 5, 2021
4c07709
improv: merge Layer example to be more pragmatic
heitorlessa Feb 5, 2021
18e7a6f
improv: key features, getting started, clean non-essential
heitorlessa Feb 8, 2021
7bcb477
improv: expand escape hatch, async, reuse examples
heitorlessa Feb 8, 2021
cb2a570
improv: add getting started
heitorlessa Feb 8, 2021
d895006
improv: add getting started, trim content, expand examples
heitorlessa Feb 9, 2021
9d46917
improv: wording on logger reuse
heitorlessa Feb 9, 2021
6e8e108
improv: additional examples on formatting, order, etc. plus advanced …
heitorlessa Feb 9, 2021
383cc19
fix: typo on logging keys in FAQ
heitorlessa Feb 9, 2021
361634f
feat: enable snippets extension; include changelog in the docs
heitorlessa Feb 9, 2021
b78d2c3
fix: docker no longer works with strict due to an issue upstream
heitorlessa Feb 9, 2021
12853eb
feat: add favicon
heitorlessa Feb 9, 2021
e06d584
chore: consistency with metrics
heitorlessa Feb 9, 2021
cdf13f4
feat: docs revision date, smaller TOC, and footer copyright
heitorlessa Feb 9, 2021
aaaff91
improv: enhanced navigation, trimmed content, added more examples
heitorlessa Feb 9, 2021
ac29164
feat: add openGraph and Twitter card, attempt 1
heitorlessa Feb 9, 2021
d5365e0
Update docs/core/tracer.md
heitorlessa Feb 9, 2021
7d1be8c
Update docs/core/tracer.md
heitorlessa Feb 9, 2021
13a4070
docs: set tabs for yaml/json to 2 spaces
Feb 10, 2021
2422d3b
fix: yaml indent for cfn
Feb 10, 2021
fd548e1
fix: formatting yaml cfn
Feb 10, 2021
96524a3
fix: typo in features table
heitorlessa Feb 10, 2021
f2fab62
fix: typo in features table, match sidebar
heitorlessa Feb 10, 2021
7f5b7ed
Merge remote-tracking branch 'am29d/docs/mkdocs' into docs/mkdocs
heitorlessa Feb 10, 2021
12a6827
feat: add built-in search
heitorlessa Feb 10, 2021
9ab143c
fix: broken links
heitorlessa Feb 10, 2021
4ad97f9
improv: bring tenets up, clarify core vs general
heitorlessa Feb 10, 2021
9e38495
fix: broken links
heitorlessa Feb 10, 2021
b7191f0
fix: contrast for layer IAM snippet
heitorlessa Feb 10, 2021
e295ca2
fix: change extra dep to warning; wording
heitorlessa Feb 10, 2021
ad63957
remove ... from cfn docs
Feb 10, 2021
8c71f6b
Merge branch 'docs/mkdocs' of github.com:am29d/aws-lambda-powertools-…
Feb 10, 2021
6ca1d15
fix: width; remove old comments
heitorlessa Feb 10, 2021
5d27c18
improv: add explicit section for LOG_EVENT
heitorlessa Feb 10, 2021
3ce7c94
fix: highlight inject_lambda_context
heitorlessa Feb 10, 2021
7d907b3
feat: add docker-based docs for dev
heitorlessa Feb 10, 2021
2607f1d
chore: copyright to 2021
heitorlessa Feb 10, 2021
51a0ae6
revert: search width
heitorlessa Feb 10, 2021
3495899
chore: Add higher res metrics image
Feb 10, 2021
892dc5f
Merge branch 'docs/mkdocs' of https://github.com/am29d/aws-lambda-pow…
heitorlessa Feb 10, 2021
d340128
chore: remove custom font; correct edit uri
heitorlessa Feb 10, 2021
d63e231
feat: record page view and search pattern
heitorlessa Feb 12, 2021
e03e7f0
feat: add API reference
heitorlessa Feb 12, 2021
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
improv: additional examples on formatting, order, etc. plus advanced …
…navigation Signed-off-by: heitorlessa <lessa@amazon.co.uk>
  • Loading branch information
heitorlessa committed Feb 9, 2021
commit 6e8e1089420537515f664e70b2c5957c75d77d91
147 changes: 105 additions & 42 deletions docs/core/logger.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,18 +258,23 @@ In this example, `Logger` will create a parent logger named `payment` and a chil

### Sampling debug logs

Sampling allows you to set your Logger Log Level as DEBUG based on a percentage of your concurrent/cold start invocations. You can set a sampling value of `0.0` to `1` (100%) using either `sample_rate` parameter or `POWERTOOLS_LOGGER_SAMPLE_RATE` env var.
Use sampling when you want to dynamically change your log level to DEBUG based on a **percentage of your concurrent/cold start invocations**.

This is useful when you want to troubleshoot an issue, say a sudden increase in concurrency, and you might not have enough information in your logs as Logger log level was understandably set as INFO.
You can set using `POWERTOOLS_LOGGER_SAMPLE_RATE` env var or explicitly with `sample_rate` parameter: Values range from `0.0` to `1` (100%)

Sampling decision happens at the Logger class initialization, which only happens during a cold start. This means sampling may happen significantly more or less than you expect if you have a steady low number of invocations and thus few cold starts.
!!! tip "When is this useful?"
Take for example a sudden increase in concurrency. When looking into logs you might not have enough information, and while you can adjust log levels it might not happen again.

This feature takes into account transient issues where additional debugging information can be useful.

Sampling decision happens at the Logger class initialization. This means sampling may happen significantly more or less than you expect if you have a steady low number of invocations and thus few cold starts.

!!! note
If you want Logger to calculate sampling on every invocation, then please open a feature request.
If you want Logger to calculate sampling upon every invocation, please open a [feature request](https://github.com/awslabs/aws-lambda-powertools-python/issues/new?assignees=&labels=feature-request%2C+triage&template=feature_request.md&title=).

=== "collect.py"

```python hl_lines="4"
```python hl_lines="4 7"
from aws_lambda_powertools import Logger

# Sample 10% of debug logs e.g. 0.1
Expand Down Expand Up @@ -316,17 +321,34 @@ Sampling decision happens at the Logger class initialization, which only happens

### Migrating from other Loggers

If you're migrating from other Loggers, there are few key points to be aware of: **Service parameter**, **Inheriting Loggers**, **Overriding Log records**, and **Logging exceptions**.
If you're migrating from other Loggers, there are few key points to be aware of: [Service parameter](#the-service-parameter), [Inheriting Loggers](#inheriting-loggers), [Overriding Log records](#overriding-log-records), and [Logging exceptions](#logging-exceptions).

#### The service parameter

Service is what defines what the function is responsible for, or part of (e.g payment service), and the name of the Logger.
Service is what defines the Logger name, including what the Lambda function is responsible for, or part of (e.g payment service).

For Logger, the `service` is the logging key customers can use to search log operations for one or more functions - For example, **search for all errors, or messages like X, where service is payment**.

??? tip "Logging output example"
```json hl_lines="5"
{
"timestamp": "2020-05-24 18:17:33,774",
"level": "DEBUG",
"location": "collect.handler:1",
"service": "payment",
"lambda_function_name": "test",
"lambda_function_memory_size": 128,
"lambda_function_arn": "arn:aws:lambda:eu-west-1:12345678910:function:test",
"lambda_request_id": "52fdfc07-2182-154f-163f-5f0f9a621d72",
"cold_start": true,
"sampling_rate": 0.1,
"message": "Verifying whether order_id is present"
}
```

#### Inheriting Loggers

> Python Logging hierarchy happens via the dot notation: `service`, `service.child`, `service.child_2`.
> Python Logging hierarchy happens via the dot notation: `service`, `service.child`, `service.child_2`

For inheritance, Logger uses a `child=True` parameter along with `service` being the same value across Loggers.

Expand All @@ -348,30 +370,63 @@ A common issue when migrating from other Loggers is that `service` might be defi

logger = Logger(child=True)
```
=== "correct_logger_inheritance.py"

```python hl_lines="4 10"
import my_module
from aws_lambda_powertools import Logger

logger = Logger(service="payment")
...

# my_module.py
from aws_lambda_powertools import Logger

logger = Logger(service="payment", child=True)
```

In this case, Logger will register a Logger named `payment`, and a Logger named `service_undefined`. The latter isn't inheriting from the parent, and will have no handler, thus no message being logged to standard output.
In this case, Logger will register a Logger named `payment`, and a Logger named `service_undefined`. The latter isn't inheriting from the parent, and will have no handler, resulting in no message being logged to standard output.

This can be fixed by either ensuring both has the `service` value as `payment`, or simply use the environment variable `POWERTOOLS_SERVICE_NAME` to ensure service value will be the same across all Loggers when not explicitly set.
!!! tip
This can be fixed by either ensuring both has the `service` value as `payment`, or simply use the environment variable `POWERTOOLS_SERVICE_NAME` to ensure service value will be the same across all Loggers when not explicitly set.

#### Overriding Log records

You might want to continue to use the same date formatting style, or override `location` to display the `package.function_name:line_number` as you previously had.

Logger allows you to either change the format or suppress the following keys altogether at the initialization: `location`, `timestamp`, `level`, `xray_trace_id`, and `datefmt`
Logger allows you to either change the format or suppress the following keys altogether at the initialization: `location`, `timestamp`, `level`, `xray_trace_id`, and `datefmt`. However, `sampling_rate` key is part of the specification and cannot be suppressed.

!!! note "`xray_trace_id` logging key"
This key is only added if X-Ray Tracing is enabled for your Lambda function. Once enabled, this key allows the integration between CloudWatch Logs and Service Lens.

=== "lambda_handler.py"
> We honour standard [logging library string formats](https://docs.python.org/3/howto/logging.html#displaying-the-date-time-in-messages).

```python hl_lines="4 7"
from aws_lambda_powertools import Logger

# override default values for location and timestamp format
logger = Logger(stream=stdout, location="[%(funcName)s] %(module)s", datefmt="fake-datefmt")
logger = Logger(location="[%(funcName)s] %(module)s", datefmt="%m/%d/%Y %I:%M:%S %p")

# suppress location key
logger = Logger(stream=stdout, location=None)
```
=== "Example CloudWatch Logs excerpt"
```json hl_lines="3 5"
{
"level": "INFO",
"location": "[<module>] scratch",
"message": "hello world",
"timestamp": "02/09/2021 09:25:17 AM",
"service": "service_undefined",
"sampling_rate": 0.0
}
```


##### Reordering log records position

Alternatively, you can also change the order of the following log record keys via the `log_record_order` parameter: `level`, `location`, `message`, `xray_trace_id`, and `timestamp`
You can also change the order of the following log record keys via the `log_record_order` parameter: `level`, `location`, `message`, `xray_trace_id`, and `timestamp`

=== "lambda_handler.py"

Expand All @@ -382,10 +437,19 @@ Alternatively, you can also change the order of the following log record keys vi
logger = Logger(stream=stdout, log_record_order=["message"])

# Default key sorting order
logger = Logger(stream=stdout, log_record_order=["level","location","message","timestamp"])
# Logger(stream=stdout, log_record_order=["level","location","message","timestamp"])
```

Some keys cannot be supressed in the Log records: `sampling_rate` is part of the specification and cannot be supressed; `xray_trace_id` is supressed automatically if X-Ray is not enabled in the Lambda function, and added automatically if it is.
=== "Example CloudWatch Logs excerpt"
```json hl_lines="3 5"
{
"message": "hello world",
"level": "INFO",
"location": "[<module>]:6",
"timestamp": "2021-02-09 09:36:12,280",
"service": "service_undefined",
"sampling_rate": 0.0
}
```

#### Logging exceptions

Expand Down Expand Up @@ -444,7 +508,7 @@ This is a Pytest sample that provides the minimum information necessary for Logg

def test_lambda_handler(lambda_context):
test_event = {'test': 'event'}
lambda_handler(test_event, lambda_context) # this will now have a Context object populated
your_lambda_handler(test_event, lambda_context) # this will now have a Context object populated
```
=== "fake_lambda_context_for_logger_py36.py"
```python
Expand Down Expand Up @@ -513,11 +577,11 @@ for the given name and level to the logging module. By default, this logs all bo

Keys added with `structure_log` will persist across multiple log messages while keys added via `extra` will only be available in a given log message operation.

**Example - Persisting payment_id not request_id**
Here's an example where we persist payment_id not request_id. Note that payment_id remains in both log messages while booking_id is only available in the first message.

=== "lambda_handler.py"

```python
```python hl_lines="4 8"
from aws_lambda_powertools import Logger

logger = Logger(service="payment")
Expand All @@ -531,27 +595,26 @@ Keys added with `structure_log` will persist across multiple log messages while

logger.info("goodbye")
```
=== "Example CloudWatch Logs excerpt"

Note that `payment_id` remains in both log messages while `booking_id` is only available in the first message.

```json
{
"level": "INFO",
"location": "<module>:5",
"message": "Flight booked successfully",
"timestamp": "2021-01-12 14:09:10,859",
"service": "payment",
"sampling_rate": 0.0,
"payment_id": "123456789",
"booking_id": "75edbad0-0857-4fc9-b547-6180e2f7959b"
},
{
"level": "INFO",
"location": "<module>:6",
"message": "goodbye",
"timestamp": "2021-01-12 14:09:10,860",
"service": "payment",
"sampling_rate": 0.0,
"payment_id": "123456789"
}
```
```json hl_lines="8-9 18"
{
"level": "INFO",
"location": "<module>:5",
"message": "Flight booked successfully",
"timestamp": "2021-01-12 14:09:10,859",
"service": "payment",
"sampling_rate": 0.0,
"payment_id": "123456789",
"booking_id": "75edbad0-0857-4fc9-b547-6180e2f7959b"
},
{
"level": "INFO",
"location": "<module>:6",
"message": "goodbye",
"timestamp": "2021-01-12 14:09:10,860",
"service": "payment",
"sampling_rate": 0.0,
"payment_id": "123456789"
}
```