The realm of open source log aggregators has seen significant growth over the years, with key players like ElasticSearch (2010) and Graylog (2016) dominating the field. More recently, Grafana Loki (2019) has emerged as a compelling addition to this ecosystem.
This article aims to facilitate a rapid onboarding process for log shipping using Promtail, log aggregation with Loki, and data visualization through Grafana. Collectively, this powerful trio of tools is commonly referred to as the PLG (Promtail-Loki-Grafana) stack.
The Server System
These instructions will work on Debian or Ubuntu system.
Using Vagrant (Virtualbox)
If you have Intel system, you can use Vagrant with Virtualbox to quickly bring up virtual servers on your workstation. Follow the instructions from respect sites to install these tools.
When ready, create a file called Vagrantfile with the following contents:
Vagrant.configure("2") do |config| config.vm.box = "generic/ubuntu2204" config.vm.network "forwarded_port", guest: 3000, host: 3000, host_ip: "127.0.0.1" config.vm.network "forwarded_port", guest: 3100, host: 3100, host_ip: "127.0.0.1" end NOTE: For this setup, the configuration will utilize 3000 for Grafana and 3100 for Loki on the localhost. If any services are currently running on these ports, you'll have to either halt those services or modify the host: key in the provided above configuration to use an alternative port.
When ready, you can bring up the system and log into the system with the following:
vagrant up vagrant ssh Installing Loki
On your desired system, such as the Vagrant managed virtual machine above, you can run the following to install PLG:
sudo apt-get install -y apt-transport-https sudo apt-get install -y software-properties-common wget sudo wget -q -O /usr/share/keyrings/grafana.key \ https://apt.grafana.com/gpg.key # add a repository for stable releases echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" \ | sudo tee -a /etc/apt/sources.list.d/grafana.list sudo apt-get update sudo apt-get install -y promtail loki grafana Verify Services are running
sudo service loki status sudo service grafana-server status If any of these services are stopped, you can run:
sudo service loki start sudo service grafana-server start The Client Workstation
Visualization
You can access the login page by opening a web browser on your system and navigating to http://127.0.0.1:3000. Simply use the default credentials, admin for both the username and password. Upon login, you will be prompted to set a new password for the admin account.
After logging in, you can click on Explore, Loki, and then select on of the logs by clicking on Label.
Ship More Logs with Promtail
When setting up a new system, there may not be many interesting logs initially. However, your host workstation system is likely to have accumulated a wealth of interesting logs over time.
If you are running the server as a virtual machine using the previously mentioned Vagrantfile configuration, you can easily ship logs from the laptop to localhost:3100, where Loki is up and running. This allows you to leverage the abundant logs from your host workstation for analysis and visualization.
You can install Promtail locally as well. If the host system is running Debian or Ubuntu, you can run this:
# add a repository for stable releases sudo wget -q -O /usr/share/keyrings/grafana.key \ https://apt.grafana.com/gpg.key echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" \ | sudo tee -a /etc/apt/sources.list.d/grafana.list # install promtail sudo apt-get update && sudo apt-get install -y promtail By default, the configuration file /etc/promtail/config.yml will set up Promtail to ship logs to the URL: http://localhost:3100/loki/api/v1/push. This configuration will function correctly if you are using Vagrant with Virtualbox to run Loki. However, if you are not running the Loki server locally, you will need to modify the address (currently set to localhost) in the configuration accordingly.
As an option you can modify the configuration with the following:
server: http_listen_port: 9080 grpc_listen_port: 0 positions: filename: /tmp/positions.yaml clients: - url: http://localhost:3100/loki/api/v1/push scrape_configs: - job_name: system static_configs: - targets: - localhost labels: job: varlogs host: my_workstation agent: promtail __path__: /var/log/*log And later restart the service:
sudo systemctl restart promtail 
Top comments (0)