Skip to content

This repository contains all the information on n8n observability and monitoring using Parseable.

License

Notifications You must be signed in to change notification settings

parseablehq/n8n-observability

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

3 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

n8n Observability with Parseable

This repository provides a complete solution for monitoring n8n workflow automation with comprehensive observability using Parseable - an open-source, unified observability platform.

🎯 What This Provides

  • πŸ“Š Traces: HTTP requests, database operations, and workflow execution spans
  • πŸ“ˆ Metrics: Resource utilization, execution counts, and performance metrics
  • πŸ“ Logs: Structured workflow logs, node execution logs, and error tracking
  • πŸ”§ Easy Setup: Docker Compose based deployment with minimal configuration

πŸ—οΈ Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ n8n │───▢│ OpenTelemetry│───▢│ Parseable β”‚ β”‚ β”‚ β”‚ Instrumentation β”‚ β”‚ β”‚ β”‚ - Workflows β”‚ β”‚ - Winston Logs β”‚ β”‚ - Traces β”‚ β”‚ - Nodes β”‚ β”‚ - HTTP Tracing β”‚ β”‚ - Metrics β”‚ β”‚ - Triggers β”‚ β”‚ - DB Tracing β”‚ β”‚ - Logs β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ 

πŸš€ Quick Start

Prerequisites

  • Docker and Docker Compose
  • Git

1. Clone and Setup

git clone https://github.com/parseablehq/n8n-observability.git cd n8n-observability

2. Start the Stack

# Start all services docker-compose up -d # View logs docker-compose logs -f n8n

3. Access Services

4. Verify Observability

  1. Create a workflow in n8n
  2. Execute the workflow
  3. Check Parseable for traces, metrics, and logs:
    • otel-traces stream: Workflow execution traces
    • otel-metrics stream: Performance metrics
    • n8n-logs stream: Structured logs

πŸ“‹ What's Included

Core Files

  • docker-compose.yml - Complete stack orchestration
  • Dockerfile - Custom n8n image with observability
  • n8n-winston-logger.js - Winston transport for structured logging
  • simple-winston-tracing.js - OpenTelemetry setup
  • docker-entrypoint.sh - Container initialization script

Services

  1. n8n - Workflow automation platform with observability
  2. Parseable - Log analytics and observability backend
  3. PostgreSQL - n8n database backend

πŸ”§ Configuration

Environment Variables

Key variables in docker-compose.yml:

# n8n Configuration N8N_HOST: "localhost" N8N_PORT: 5678 N8N_PROTOCOL: "http" N8N_WINSTON_LOGGING: "true" # Database DB_TYPE: "postgresdb" DB_POSTGRESDB_HOST: "postgres" DB_POSTGRESDB_DATABASE: "n8n" # Parseable Integration PARSEABLE_URL: "http://parseable:8000" PARSEABLE_USERNAME: "admin" PARSEABLE_PASSWORD: "admin" # OpenTelemetry OTEL_SERVICE_NAME: "n8n-comprehensive" OTEL_EXPORTER_OTLP_ENDPOINT: "http://parseable:8000"

Parseable Streams

The setup creates three streams automatically:

  • n8n-logs: Structured application logs
  • otel-traces: Execution traces and spans
  • otel-metrics: Performance and resource metrics

πŸ“Š Observability Features

Logs

Workflow Events:

logger.workflow.started(workflowId, workflowName, executionId, metadata); logger.workflow.completed(workflowId, workflowName, executionId, duration, metadata); logger.workflow.failed(workflowId, workflowName, executionId, error, duration, metadata);

Node Events:

logger.node.started(workflowId, executionId, nodeType, nodeName, metadata); logger.node.completed(workflowId, executionId, nodeType, nodeName, duration, metadata); logger.node.failed(workflowId, executionId, nodeType, nodeName, error, duration, metadata);

HTTP Events:

logger.http.request(method, url, headers, metadata); logger.http.response(method, url, statusCode, duration, metadata);

Traces

Automatic instrumentation for:

  • HTTP requests (incoming/outgoing)
  • Database operations
  • Workflow execution spans
  • Node execution spans

Metrics

Collected metrics include:

  • Request counts and durations
  • Database connection pools
  • Memory and CPU usage
  • Workflow execution statistics

πŸ” Querying Data

Sample Parseable Queries

Recent Workflow Executions:

SELECT body, severity_text, time_unix_nano FROM "n8n-logs" WHERE body LIKE '%workflow%' ORDER BY time_unix_nano DESC LIMIT 10

Error Analysis:

SELECT body, severity_text, time_unix_nano FROM "n8n-logs" WHERE severity_text = 'ERROR' ORDER BY time_unix_nano DESC

Performance Traces:

SELECT span_name, duration_nanos, status_code FROM "otel-traces" ORDER BY start_time_unix_nano DESC

πŸŽ›οΈ Customization

Adding Custom Logging

Extend the Winston logger in n8n-winston-logger.js:

// Add custom log methods logger.custom = { event: (eventType, data) => { logger.info('Custom event', { event_type: eventType, ...data }); } };

Adding Custom Instrumentation

Extend OpenTelemetry setup in simple-winston-tracing.js:

// Add custom instrumentations const { YourCustomInstrumentation } = require('your-instrumentation'); registerInstrumentations({ instrumentations: [ // ... existing instrumentations new YourCustomInstrumentation() ] });

πŸ› Troubleshooting

Common Issues

1. Logs not appearing in Parseable

  • Check container connectivity: docker network ls
  • Verify Parseable is running: docker-compose logs parseable
  • Check n8n logs: docker-compose logs n8n

2. Authentication errors

  • Verify credentials in docker-compose.yml
  • Check Parseable access: curl -u admin:admin http://localhost:8000/api/v1/logstream

3. High memory usage

  • Adjust batch sizes in n8n-winston-logger.js
  • Configure log retention in Parseable

Debug Commands

# Check service health docker-compose ps # View real-time logs docker-compose logs -f n8n docker-compose logs -f parseable # Test Parseable API curl -u admin:admin http://localhost:8000/api/v1/logstream # Check n8n database docker exec -it n8n-postgres psql -U postgres -d n8n

πŸ” Security Considerations

Production Deployment

  1. Change Default Credentials:

    PARSEABLE_USERNAME: "your-username" PARSEABLE_PASSWORD: "your-secure-password"
  2. Use Environment Variables:

    export PARSEABLE_PASSWORD=$(openssl rand -base64 32)
  3. Network Security:

    • Use Docker networks for isolation
    • Configure firewall rules
    • Enable TLS/SSL for production
  4. Data Retention:

    • Configure log rotation
    • Set retention policies
    • Monitor disk usage

🀝 Contributing

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature-name
  3. Commit changes: git commit -am 'Add feature'
  4. Push to branch: git push origin feature-name
  5. Submit pull request

πŸ“ License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

πŸ™ Acknowledgments

πŸ“ž Support


⭐ Star this repo if you find it useful!

About

This repository contains all the information on n8n observability and monitoring using Parseable.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Sponsor this project

Packages

No packages published