Downloaded 31 times
![docker run -v /tmp/clogs:/tmp/clogs -d --name="sumo-logic-collector" sumologic/collector:latest-file [Access ID] [Access key]](https://image.slidesharecdn.com/kansascitydevopseventsv1-150714104046-lva1-app6891/75/Logging-Metrics-with-Docker-10-2048.jpg)
![docker run -d -p 514:514 -p 514:514/udp --name="sumo-logic-collector” sumologic/collector:latest-syslog [Access ID] [Access key]](https://image.slidesharecdn.com/kansascitydevopseventsv1-150714104046-lva1-app6891/75/Logging-Metrics-with-Docker-13-2048.jpg)
![What Docker provides Captures stdout/stderr Feeds it to logging drivers docker logs command Returns the entire log every time Works with json-file driver only Can tail logs docker logs –tf –-tail 0 [ID]](https://image.slidesharecdn.com/kansascitydevopseventsv1-150714104046-lva1-app6891/75/Logging-Metrics-with-Docker-20-2048.jpg)
Uploaded byStefan Zier
Logging & Metrics with Docker
The document discusses a comprehensive monitoring solution for logging and metrics in Docker environments, highlighting challenges such as managing logs from multiple containerized applications. It details various methods for emitting logs, including using syslog, Docker logging drivers, and frameworks, and emphasizes the need for centralized log management. Additionally, it introduces the Sumo Logic Docker source as a solution for effective log collection and monitoring.
Logging & Metrics with Docker
- 1. Logging & MetricsWith Docker A Comprehensive Monitoring Solution Stefan Zier June 13th, 2015
- 2. whoami Infrastructure, Backend Dev/Architect ChiefArchitect, Sumo Logic, since 2010 Server & Infrastructure, ArcSight (HP), 2001-2010
- 3. Mandatory Slide ShowingShipping Containers
- 4. Docker – What’smaking debugging hard? One more layer of abstraction Container per app = File system per process File systems short lived, transient Resource schedulers = no container affinity to host
- 5. What Our CustomersAre Telling Us We have one process per container We like to log to stdout We have multiple processes per container We run the Sumo Logic collector on the Docker host We are looking into using Beanstalk with Docker We are using Amazon ECS Everyone here loves Docker We are logging straight from the application We are using /dev/log for Syslog We want immutable infrastructure
- 6. Goal Get logs fromour containerized applications to a centralized logging platform.
- 7. How do appsemit logs Append to a file Use syslog() Use log4j, log4net, slf4, etc. printf() to stdout
- 8. Getting logs outof the container - Files Use VOLUME to mount a host directory Collect files from the host Collect files from another container sharing the VOLUME Need to manage disk space, i.e. rotate logs App (where supported) Host Yet another container with logrotate
- 10. docker run -v/tmp/clogs:/tmp/clogs -d --name="sumo-logic-collector" sumologic/collector:latest-file [Access ID] [Access key]
- 11. Getting logs out- Syslog VOLUME /dev/log from host and use host syslogd Run a syslogd inside the container Emit TCP/UDP Write to a file using VOLUME Emit syslog TCP/UDP directly from the app
- 13. docker run -d-p 514:514 -p 514:514/udp --name="sumo-logic-collector” sumologic/collector:latest-syslog [Access ID] [Access key]
- 14. Getting logs out– Logging frameworks Sumo Logic blog on official collector images http://www.sumologic.com/blog/company/an-official-docker-image-for- the-sumo-logic-collector https://github.com/SumoLogic/sumologic-collector-docker Rainer Gerhards on Rsyslog’s file input module http://www.slideshare.net/rainergerhards1/using-wildcards-with- rsyslogs-file-monitor-imfile OWASP Log Injection https://www.owasp.org/index.php/Log_injection
- 15. Getting logs out– Logging frameworks Directly to network destinations HTTP/HTTPS Also support files, stdout, etc.
- 17. Getting logs out– Logging frameworks Various application stacks http://help.papertrailapp.com/ Log4J https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/net/SyslogAppender.html Apache Web Server http://httpd.apache.org/docs/trunk/mod/mod_syslog.html https://raymii.org/s/snippets/Apache_access_and_error_log_to_syslog.html Nginx http://nginx.org/en/docs/syslog.html Postgres http://www.postgresql.org/docs/9.1/static/runtime-config-logging.html Sumo Logic blog on official syslog collector image http://www.sumologic.com/blog/company/an-official-docker-image-for-the-sumo-logic-collector https://github.com/SumoLogic/sumologic-collector-docker
- 18. Getting logs out– stdout Simply printf() Logging framework to console Symlink to /dev/stdout or /dev/stderr Configure paths to /dev/stdout or /dev/stderr RUN ln -sf /dev/stdout /var/log/nginx/access.log RUN ln -sf /dev/stderr /var/log/nginx/error.log
- 19.
- 20. What Docker provides Capturesstdout/stderr Feeds it to logging drivers docker logs command Returns the entire log every time Works with json-file driver only Can tail logs docker logs –tf –-tail 0 [ID]
- 21. Docker Logging Drivers Configuredon docker run stdout and stderr dispatched to drivers json-file (default pre 1.6) syslog journald No stats, no events json-file driver Output unbounded, can fill up the host disk Requires logrotate on the Docker host https://github.com/docker/docker/issues/7333
- 22.
- 23. Docker Stats Per-container cgroupsmetrics (like docker stats) Memory CPU Block I/O CONTAINER CPU % MEM USAGE/LIMIT MEM % NET I/O collector 2.23% 232.6 MiB/2 GiB 11.36% 191.9 KiB/636.3 KiB
- 24. Requirements How would wewant it to work?
- 25. What information dowe want to collect? Timestamp Log message Docker host info Container ID Image ID Process ID
- 26. How should itwork? Use docker logging infrastructure Minimal moving parts Containerized - don’t touch the host Complete – pick up all available data Automatically discover new containers
- 27. Docker API The solutionmaybe?
- 28. Docker API Docker daemonhas a REST API TCP or unix socket Streaming APIs Docker Events (container lifecycle updates) Container Stats (CPU, memory used, …) App Logs (container stdout/stderr)
- 29. Collecting via DockerAPI Discover new containers via events Start streaming their logs and stats When they go away, stop Do all of this via the API Send all of it to centralized log management
- 30. Collecting via DockerAPI, continued Single component to do it Zero footprint on the host Follows Docker standard way of logging
- 31.
- 32.
- 33. Sumo Logic DockerSource Active development Early access expected later this year
- 34.
- 35.