Download as PDF, PPTX
![Telegraf ● Understand Telegraf output: $ telegraf --config telegraf.conf --input-filter cpu --test [measurement],[tag1,tag2] [field1,field2,field3] [timestamp] system,host=tux,env=prod load1=1.25,load5=1.27,load15=1.29 1509997632000000000](https://image.slidesharecdn.com/introduction-to-influxdb-and-tick-stack-171108225610/75/Introduction-to-InfluxDB-and-TICK-Stack-11-2048.jpg)
Uploaded byAhmed AbouZaid
Introduction to InfluxDB and TICK Stack
The document provides an introduction to InfluxDB and the TICK stack (Telegraf, InfluxDB, Chronograf, Kapacitor), which is designed for managing and analyzing time series data. It outlines the characteristics of time series data and the functions of each component in the stack, emphasizing their capabilities in data collection, storage, visualization, and processing. Additionally, it highlights potential use cases such as infrastructure monitoring and anomaly detection.
In this document
Powered by AI
Introduction to InfluxDB and TICK Stack; speaker Ahmed AbouZaid; SaaS CDP for managing customer data.
Explains time series data, its properties including high read/write throughput, data points, and workload types.
Introduction to TICK Stack - Telegraf, InfluxDB, Chronograf, Kapacitor; open-source stack for time series data.
Telegraf as a metrics collection agent, its lightweight design, and sample output format for CPU metrics.
InfluxDB as a scalable datastore for time series; features include high performance, SQL-like queries, and retention policies.
Chronograf as the TICK stack web interface for monitoring, visualization, data management, and alerts.
Kapacitor framework for processing time series data, supporting batch/streaming data, and anomaly detection with example TICKscript.
Practical use cases for TICK Stack include infrastructure monitoring, IoT sensor data, and anomaly detection.
References to resources for further reading on InfluxDB and time series database concepts.
Conclusion and invitation for questions from the audience.
Introduction to InfluxDB and TICK Stack
- 1. Introduction to InfluxDB andTICK Stack Ahmed AbouZaid @aabouzaid
- 2. Ahmed AbouZaid DevOps @CrossEngage Author and Free/Open source geek who loves the community. Automation, data, and metrics are my preferred areas. I have a built-in monitoring chip, and too lazy to do anything manually :D Blog | Github | Twitter Nope, I’m not Mexican, but Egyptian! So I could actually be anything!
- 3. SaaS CDP (CustomerData Platform) that easily combines all customer data sources and manages cross-channel marketing campaigns with your existing infrastructure. crossengage.io
- 4. Overview ● What istime series data? ● Why TICK Stack? ● Where could it be used?
- 5. Time Series Data “Atime series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data”. Properties of Time Series Data ● Billions of individual data points. ● High read and write throughput. ● Large deletes (data expiration). ● Mostly an insert/append workload, very few updates.
- 6.
- 7.
- 8. TICK Stack TICK =Telegraf (Collect), InfluxDB (Store), Chrongraf (Visualize), and Kapacitor (Process). ● The Open Source Time Series Stack provides services and functionality to accumulate, analyze, and act on time series data. ● Has a big community and ecosystem. ● Written entirely in Go. It compiles into a single binary with no external dependencies. ● TOML configuration,Tom's Obvious, Minimal Language.
- 9.
- 10. Telegraf The plugin-driven agentfor collecting & reporting metrics. ● Minimal memory footprint. ● +100 plugs with a wide number of plugins for many popular services already exist for well known services and APIs. ● Plugin system allows new inputs and outputs to be easily added. ● Can work with any external scripts.
- 11. Telegraf ● Understand Telegrafoutput: $ telegraf --config telegraf.conf --input-filter cpu --test [measurement],[tag1,tag2] [field1,field2,field3] [timestamp] system,host=tux,env=prod load1=1.25,load5=1.27,load15=1.29 1509997632000000000
- 12.
- 13. InfluxDB Scalable time seriesdatastore for metrics, events, and real-time analytics. ● High performance datastore written specifically for time series data. ● Simple, high performing write and query HTTP(S) APIs. ● Plugins support for other data sources such as Graphite, and collectd. ● SQL-like query language to easily query aggregated data. ● Tags allow series to be indexed for fast and efficient queries. ● Retention policies efficiently auto-expire stale data. ● Continuous queries automatically compute aggregate data to make frequent queries more efficient.
- 14. InfluxDB ● Query example: >SELECT "host", "env", "load1" as "load" FROM "cpu" WHERE "host" = 'tux’ LIMIT 1 name: cpu --------- time host env load 2017-11-01T01:11:00.000000000Z tux prod 1.25
- 15.
- 16. Chronograf TICK stack webinterface for monitoring, visualization, and management. ● Data Visualization. ● Database Management. ● Infrastructure overview. ● Alert Management.
- 17.
- 18.
- 19.
- 20. Kapacitor Framework for processing,monitoring, and alerting on time series data. ● Process both streaming data and batch data. ● Query data from InfluxDB on a schedule, and receive data via the line protocol and any other method InfluxDB supports. ● Perform any transformation currently possible in InfluxQL. ● Store transformed data back in InfluxDB. ● Support custom user defined functions to detect anomalies. ● Has an easy DSL to define data processing pipelines. ● Integrate with HipChat, OpsGenie, Alerta, Sensu, Slack, and more.
- 21. Kapacitor ● TICKscript DSLexample: stream |from() .measurement('app') |eval(lambda: "errors" / "total") .as('error_percent') // Write the transformed data to InfluxDB. |influxDBOut() .database('app') .retentionPolicy('15D') .measurement('errors') .tag('kapacitor', 'true') .tag('version', '0.2')
- 22. Use cases ● Infrastructuremonitoring. ● Work with sensors (i.e. interacting with IoT). ● Anomaly detection.
- 23. Resources ● https://docs.influxdata.com/ ● https://www.influxdata.com/time-series-database/ ●https://docs.influxdata.com/influxdb/v1.3/concepts/storage_engine/#pro perties-of-time-series-data ● http://www.itl.nist.gov/div898/handbook/pmc/section4/pmc4.htm
- 24.