Apache Airflow version

2.7.3

What happened

I tried using airflow.utils.log.task_handler_with_custom_formatter.TaskHandlerWithCustomFormatter handler for task logging and noticed that the prefix specified in task_log_prefix_template is added three times.

What you think should happen instead

The prefix should be added only one time.

How to reproduce

At first I encountered this issue in Kubernetes deployment with custom logging configuration, but it is also reproduced locally with the default configuration using airflow standalone command:

1. Copy logging template from https://github.com/apache/airflow/blob/main/airflow/config_templates/airflow_local_settings.py and place it to $AIRFLOW_HOME/config/custom_logging_config.py

2. In the logging template change task handler to

 "task": { "class": "airflow.utils.log.task_handler_with_custom_formatter.TaskHandlerWithCustomFormatter", "formatter": "airflow", "filters": ["mask_secrets"], "stream": "ext://sys.stdout", }, 

3. In airflow.cfg set:

[logging] ... logging_config_class = custom_logging_config.DEFAULT_LOGGING_CONFIG ... task_log_prefix_template = {{ ti.dag_id }}-{{ ti.task_id }}-{{ execution_date }}-{{ try_number }} 

4. Run airflow standalone and start any DAG in the UI.

5. Check task logs in the terminal, for example:

scheduler | example_xcom-puller-2023-11-14T07:19:15.633728+00:00-:example_xcom-puller-2023-11-14T07:19:15.633728+00:00-:example_xcom-puller-2023-11-14T07:19:15.633728+00:00-:[2023-11-14 10:19:33,087] {taskinstance.py:1159} INFO - Dependencies all met for dep_context=requeueable deps ti=<TaskInstance: example_xcom.puller manual__2023-11-14T07:19:15.633728+00:00 [queued]> scheduler | example_xcom-puller-2023-11-14T07:19:15.633728+00:00-:example_xcom-puller-2023-11-14T07:19:15.633728+00:00-:example_xcom-puller-2023-11-14T07:19:15.633728+00:00-:[2023-11-14 10:19:33,087] {taskinstance.py:1361} INFO - Starting attempt 1 of 1 

Operating System

Debian "11 (Bullseye)" or Ubuntu

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

Can be reproduced both locally and using airflow helm chart.

Anything else

I found this issue #10202 , but it looks like it was about airflow 1.

I wouldn't mind submitting a PR for this issue, but I'm not sure how to fix it except for adding a check in https://github.com/apache/airflow/blob/main/airflow/utils/log/task_handler_with_custom_formatter.py#L37 if the context was already set. But this doesn't look like correct approach.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct