1

I am wondering if a project exists that can collect all the logs from a bunch of microservices and tail them on a web interface. All of the logging solutions out there seem to neglect realtime viewing.

I am using kubernetes and the services are outputting on stdout - I am imagining I would use fluentd or similar as a transport. The logs could go into redis or something similar and then accessed by a very simple API which sends the last 5 mins of logs and then updates per second ( or some websocket magic ).

The effect I am looking for is similar to 'tail -f /var/log/messages'

Any suggestions?

Cheers,

Andrew

Improve this question

2 Answers 2

Reset to default
1

promtail => loki => grafana are made for that.

setup docker logging driver to journald, and ask promtail to scrap logs from journald, to finally feed the loki instance

using the grafana instance, you will be able to check logs from specific timeframe, or even move to "live" analysis

Improve this answer
0

Andrew, there are a few tools you can find interesting.

Just Logging

  • Wufei for the aggregation of Kubernetes logs.

This tool will write Kubernetes logs for each pod down to a container level to a file or to stdout depending on the developers needs and also has the ability to log new pods that are spun up in the namespace as well. There is an informer written to let wufei know when new pods spin up.

  • Kubetail is a small bash script which allows you to aggregate logs from multiple pods into one stream. The initial Kubetail version doesn’t have filtering or highlighting features, but there is an additional Kubetail fork on Github. This can form and perform logs coloring using multitail tools.

Logging with UI

  • Fluentd+Elasticsearch+Kibana is a pretty powerful logging aggregation system on top of your Kubernetes cluster. Here is "How To"

  • Apollo is an open source application providing teams with self-service UI for creating and deploying their services to Kubernetes. Apollo allows operators to view logs and revert deployments to any point in time with just one click. Apollo has flexible permission models for deployments. Each user can deploy only what he needs to deploy.

Let me know if you have been looking for something like that.

Improve this answer
3
  • So I am putting together a POC kubernetes solution for a hosting company - I am looking for is something like this: logio.org These are really interesting tools tho. Commented Feb 20, 2020 at 7:19
  • I am a bit done with elasticsearch and kibana - its annoying to manage and you need to spend ages writing kibana dashboards - its very awkward using it for actually debugging applications or solving real-time problems. Commented Feb 20, 2020 at 7:22
  • yes. Logio looks interesting. Commented Feb 20, 2020 at 7:47

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.