Send Python logs to Tinybird

You can send logs from a Python application or service to Tinybird using the standard Python logging library and the tinybird-python-sdk.

Prerequisites

To use the Tinybird Python SDK you need Python 3.11 or higher.

Configure the logging handler

First, configure a Tinybird logging handler in your application. For example:

import logging from multiprocessing import Queue from tb.logger import TinybirdLoggingQueueHandler logger = logging.getLogger('your-logger-name') handler = TinybirdLoggingHandler(<YOUR_TB_API_URL>, <YOUR_TB_WRITE_TOKEN>, 'your-app-name') formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') handler.setFormatter(formatter) logger.addHandler(handler)

Each time you call the logger, the SDK sends an event to the tb_logs data source in your workspace.

To configure the data source name, initialize the TinybirdLoggingHandler like this:

handler = TinybirdLoggingHandler(<YOUR_TB_API_URL>, <YOUR_TB_WRITE_TOKEN>, 'your-app-name', ds_name="your_tb_ds_name")

Non-blocking logging

If you want to avoid blocking the main thread, use a queue to send the logs to a different thread. For example:

import logging from multiprocessing import Queue from tb.logger import TinybirdLoggingQueueHandler from dotenv import load_dotenv load_dotenv() TB_API_URL = os.getenv("<YOUR_TB_API_URL>") TB_WRITE_TOKEN = os.getenv("<YOUR_TB_WRITE_TOKEN>") logger = logging.getLogger('your-logger-name') handler = TinybirdLoggingQueueHandler(Queue(-1), TB_API_URL, TB_WRITE_TOKEN, 'your-app-name', ds_name="your_tb_ds_name") formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') handler.setFormatter(formatter) logger.addHandler(handler)
Updated