Skip to content

Conversation

DifferentialOrange
Copy link
Member

@DifferentialOrange DifferentialOrange commented May 12, 2023

Run

make CONFIG=config.yml build

to build a dashboard based on your custom configuration. (Use OUTPUT=mydashboard.json to customize output path.) You'll need jsonnet 0.20.0 or newer for that.

Configuration description

# type is 'prometheus' or 'influxdb' type: prometheus # title is dashboard artifact title # default is 'Tarantool dashboard' title: Tarantool dashboard # description is dashboard artifact description # default is 'Dashboard for Tarantool application and database server monitoring, based on grafonnet library.' description: Dashboard for Tarantool application and database server monitoring, based on grafonnet library. # grafana_tags are dashboard artifact tags # They do not related to Prometheus/InfluxDB quieries, # they are a part of Grafana UI and can be used to search dashboards in the library. # default is [tarantool] grafana_tags: [tarantool] # datasource is the name on metrics datasource # Use expression like '$prometheus' to work with dynamic datasource set with variable. # Use string like 'MyDashboard' to set your specific datasource. # default for prometheus type is '$prometheus', for influxdb type is '$influxdb' datasource: '$prometheus' # policy is InfluxDB datasource metrics retention policy # influxdb type only # Use expression like '$policy' to work with dynamic policy set with variable. # Use string like 'MyPolicy' to set your specific policy. # Use 'default' to work with default InfluxDB datasource policy. # default is '$policy' # policy: '$policy' # measurement is InfluxDB datasource metrics measurement # influxdb type only # Use expression like '$measurement' to work with dynamic measurement set with variable. # Use string like 'MyMeasurement' to set your specific measurement. # default is '$measurement' # measurement: '$measurement' # filters are Prometheus/InfluxQL query filters # By specifying them, you can filter metrics by Prometheus label conditions # or InfluxDB tag conditions. # Use app: ['=', 'myapp'] to display only metrics with 'myapp' app label/tag. # Use app: ['=~', '/^myapp\d/'] to display only metrics with app label/tag satisfying # a regular expression. See PromQL/InfluxQL documentation for a regex syntax. # Use job: ['=~', '$job'] to display only metrics with '$job' variable value job label/tag. # Beware that only job and alias (label_pairs_alias for InfluxDB) template variables # would be built automatically, if required. # default is {} filters: { job: ['=~', '$job'], alias: ['=~', '$alias'], } # metrics_prefix is an additional prefix for metric names in Prometheus/InfluxQL queries # Use 'myapp_' if you expose metrics like 'myapp_tnt_uptime_info' # default is '' metrics_prefix: '' # sections are dashboard panels sections to be included # The list of supported sections: # - cluster # - replication # - http # - net # - slab # - mvcc # - space # - vinyl # - cpu # - cpu_extended' # for TDG # - runtime # - luajit # - operations # - crud # - expirationd # - tdg_kafka_common # - tdg_kafka_brokers # - tdg_kafka_topics # - tdg_kafka_consumer # - tdg_kafka_producer # - tdg_tuples # - tdg_file_connectors # - tdg_graphql # - tdg_iproto # - tdg_rest_api # - tdg_tasks # default is [cluster, replication, http, net, slab, mvcc, space, vinyl, cpu, runtime, luajit, operations, crud, expirationd] sections: - cluster - replication - http - net - slab - mvcc - space - vinyl - cpu - runtime - luajit - operations - crud - expirationd

Example: prometheus dashboard with template variable

type: prometheus datasource: '$prometheus' filters: { job: ['=~', '$job'], alias: ['=~', '$alias'], }

Example: prometheus static dashboard

type: prometheus datasource: MyPrometheusWithTarantoolMetrics filters: { job: ['=', 'MyJobWithTarantoolMetrics'], }

Example: prometheus static dashboard with additional labels

type: prometheus datasource: MyPrometheusWithTarantoolMetrics filters: { job: ['=', 'MyJobWithTarantoolMetrics'], vendor_app_tag: ['=', 'MyCacheApplication'], }

Example: prometheus static dashboard with specific sections

type: prometheus datasource: MyPrometheusWithTarantoolMetrics filters: { job: ['=', 'MyJobWithTarantoolMetrics'], } sections: - cluster - replication - http - net - slab - space - runtime - luajit - operations

Example: prometheus static TDG dashboard

type: prometheus title: Tarantool Data Grid dashboard grafana_tags: [tarantool, TDG] datasource: MyPrometheusWithTDGMetrics filters: { job: ['=', 'MyJobWithTDGMetrics'], } sections: - cluster - replication - http - net - slab - mvcc - space - vinyl - cpu_extended - runtime - luajit - operations - crud - tdg_kafka_common - tdg_kafka_brokers - tdg_kafka_topics - tdg_kafka_consumer - tdg_kafka_producer - expirationd - tdg_tuples - tdg_file_connectors - tdg_graphql - tdg_iproto - tdg_rest_api - tdg_tasks

Example: influxdb dashboard with template variable

type: influxdb datasource: '$influxdb' policy: '$policy' measurement: '$measurement' filters: { label_pairs_alias: ['=~', '/^$label_pairs_alias$/'], }

Example: influxdb static dashboard

type: influxdb datasource: MyInfluxDBWithTarantoolMetrics policy: default measurement: MyMeasurementWithTarantoolMetrics

Closes #204
Closes #206
Closes #207

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/yaml-configuration branch 5 times, most recently from a201126 to b517c65 Compare May 16, 2023 12:48
After this patch, code uses a single place to extract the configuration parameters like datasource type, datasource, datasource parameters, alias filters etc. Part of #206 Part of #207
After this patch, code uses a single place to extract the configuration parameters like datasource type, datasource, datasource parameters, alias filters etc. Part of #206
This patch finishes the migration of existing queries to new filter system. Part of #206
After this patch, it will be possible to disable some default filters, like job. Part of #206
Remove redundant "include all" regex for alias if no condition required (like in some static built dashboards). Part of #206
Support custom description and Grafana tags on manual build.
Build template queries based on cfg value.
To build a dashboard from YAML config, run ``` CONFIG=config.yml make build ``` Part of #206 Part of #207
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/yaml-configuration branch from b517c65 to 31445fe Compare May 17, 2023 12:20
@DifferentialOrange DifferentialOrange marked this pull request as ready for review May 17, 2023 12:21
@DifferentialOrange
Copy link
Member Author

I have asked @palage4a to try this implementation to solve their issues. If everything would be alright, I'll merge this PR.

@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/yaml-configuration branch from 3fc1eb4 to 5ee07d6 Compare May 23, 2023 16:10
Grafana revisions: - InfluxDB revision 17 [1], - Prometheus revision 17 [2], - InfluxDB TDG revision 6 [3], - Prometheus TDG revision 6 [4]. This release introduces several breaking changes for those using this repository as a library after migrating to new build system. See updated README. Added: - Custom filters support on static build - Option to skip Prometheus job on static build - Option to static build with custom set of sections - Custom description and Grafana tags on manual build - API to build a dashboard from YAML config - Support for metrics prefix on manual build Changed: - Display two digits after decimal point instead of three - Deprecate make single line static build - Hide additional columns in overview table Fixed: - TDG Kafka brokers rps panels alias condition for InfluxDB - Fill missing values with null on TDG CPU panels for InfluxDB - Remove useless kind group in TDG jobs panels for InfluxDB - CRUD tuples query alias filtering for Prometheus - TDG tuples query alias filtering - Policy in alias extraction for InfluxDB dashboards - Remove redundant "include all" regex for alias if no condition required 1. https://grafana.com/api/dashboards/12567/revisions/17/download 2. https://grafana.com/api/dashboards/13054/revisions/17/download 3. https://grafana.com/api/dashboards/16405/revisions/6/download 4. https://grafana.com/api/dashboards/16406/revisions/6/download
@DifferentialOrange DifferentialOrange force-pushed the DifferentialOrange/yaml-configuration branch from 21bcffd to 742c14f Compare May 23, 2023 16:17
@DifferentialOrange DifferentialOrange merged commit 7869a11 into master May 23, 2023
@DifferentialOrange DifferentialOrange deleted the DifferentialOrange/yaml-configuration branch May 23, 2023 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant