Skip to content

cybertec-postgresql/pg_timetable

Repository files navigation

Documentation License: MIT Build and Test Coverage Status Github All Releases Docker Pulls Mentioned in Awesome Go

pg_timetable: Advanced scheduling for PostgreSQL

pg_timetable is an advanced standalone job scheduler for PostgreSQL, offering many advantages over traditional schedulers such as cron and others. It is completely database driven and provides a couple of advanced concepts. It allows you to schedule PostgreSQL commands, system programs and built-in operations:

-- Run public.my_func() at 00:05 every day in August: SELECT timetable.add_job('execute-func', '5 0 * 8 *', 'SELECT public.my_func()'); -- Run VACUUM at minute 23 past every 2nd hour from 0 through 20 every day: SELECT timetable.add_job('run-vacuum', '23 0-20/2 * * *', 'VACUUM'); -- Refresh materialized view every 2 hours: SELECT timetable.add_job('refresh-matview', '@every 2 hours', 'REFRESH MATERIALIZED VIEW public.mat_view'); -- Clear log table after pg_timetable restart: SELECT timetable.add_job('clear-log', '@reboot', 'TRUNCATE public.log'); -- Reindex at midnight on Sundays with reindexdb utility: -- using default database under default user (no command line arguments) SELECT timetable.add_job('reindex-job', '0 0 * * 7', 'reindexdb', job_kind := 'PROGRAM'); -- specifying target database and tables, and be verbose SELECT timetable.add_job('reindex-job', '0 0 * * 7', 'reindexdb', '["--table=foo", "--dbname=postgres", "--verbose"]'::jsonb, 'PROGRAM'); -- passing password using environment variable through bash shell SELECT timetable.add_job('reindex-job', '0 0 * * 7', 'bash', '["-c", "PGPASSWORD=5m3R7K4754p4m reindexdb -U postgres -h 192.168.0.221 -v"]'::jsonb, 'PROGRAM'); 

Documentation

https://cybertec-postgresql.github.io/pg_timetable/

Main features

  • Tasks can be arranged in chains
  • Each task executes SQL, built-in or executable command
  • Parameters can be passed to tasks
  • Missed chains (possibly due to downtime) can be retried automatically
  • Support for configurable repetitions
  • Builtin tasks such as sending emails, downloading, importing files, etc.
  • Fully database driven configuration
  • Full support for database driven logging
  • Enhanced cron-style scheduling
  • Optional concurrency protection
  • NEW: YAML-based chain definitions for easy configuration

YAML Configuration

You can now define chains using YAML files instead of SQL inserts, making configuration more readable and maintainable:

chains: - name: "Daily ETL Pipeline" schedule: "0 2 * * *" # 2 AM daily live: true max_instances: 1 timeout: 3600000 # 1 hour tasks: - name: "Extract data" command: "SELECT extract_sales_data($1)" parameters: ["yesterday"] - name: "Transform data" command: "CALL transform_sales_data()" autonomous: true - name: "Load to warehouse" command: "CALL load_to_warehouse()"

Load YAML chains with:

pg_timetable --file chains.yaml postgresql://user:pass@host/db

See samples/yaml/ for more examples and YAML Schema for complete format specification.

Installation

Complete installation guide can be found in the documentation.

Possible choices are:

Quick Start

Complete usage guide can be found in the documentation.

  1. Download pg_timetable executable

  2. Make sure your PostgreSQL server is up and running and has a role with CREATE privilege for a target database, e.g.

    my_database=> CREATE ROLE scheduler PASSWORD 'somestrong'; my_database=> GRANT CREATE ON DATABASE my_database TO scheduler;
  3. Create a new job, e.g. run VACUUM each night at 00:30

    my_database=> SELECT timetable.add_job('frequent-vacuum', '30 0 * * *', 'VACUUM'); add_job --------- 3 (1 row)
  4. Run the pg_timetable

    pg_timetable postgresql://scheduler:somestrong@localhost/my_database --clientname=vacuumer 
  5. PROFIT!

Supported Environments

Cloud Service Supported PostgreSQL Version Supported OS Supported
Alibaba Cloud 19 (devel) Linux
Amazon RDS 18 (current) Darwin
Amazon Aurora 17 Windows
Azure 16 FreeBSD*
Citus Cloud 15 NetBSD*
Crunchy Bridge 14 OpenBSD*
DigitalOcean 13 Solaris*
Google Cloud 12
Heroku 11
Supabase 10

* - there are no official release binaries made for these OSes. One would need to build them from sources.

** - previous PostgreSQL versions may and should work smoothly. Only officially supported PostgreSQL versions are listed in this table.

Contributing

If you want to contribute to pg_timetable and help make it better, feel free to open an issue or even consider submitting a pull request.

Support

For professional support, please contact Cybertec.

Authors

Star History

Star History Chart