DEV Community

Cover image for Getting Started with Hashicorp Nomad
Clivern
Clivern

Posted on • Edited on

Getting Started with Hashicorp Nomad

#docker #devops #microservices #kubernetes

Basic Setup (Not Recommended):

  • Download nomad binary & make it executable: 
$ wget https://releases.hashicorp.com/nomad/1.0.5/nomad_1.0.5_linux_amd64.zip $ unzip nomad_1.0.5_linux_amd64.zip $ mv nomad /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode
  • Create configs directory & data directory: 
$ sudo mkdir --parents /opt/nomad $ sudo mkdir --parents /opt/data/redis $ sudo mkdir --parents /etc/nomad.d $ sudo chmod 700 /etc/nomad.d $ sudo chmod 700 /opt/data $ sudo touch /etc/nomad.d/nomad.hcl $ sudo touch /etc/nomad.d/server.hcl $ sudo touch /etc/nomad.d/client.hcl
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the /etc/nomad.d/nomad.hcl configuration file 
datacenter = "dc1" data_dir = "/opt/nomad"
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the /etc/nomad.d/client.hcl configuration file 
client { enabled = true host_volume "redis_data" { path = "/opt/data/redis" read_only = false } } plugin "docker" { config { volumes { enabled = true selinuxlabel = "z" } } }
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the /etc/nomad.d/server.hcl configuration file 
acl { enabled = true } server { enabled = true bootstrap_expect = 1 }
Enter fullscreen mode Exit fullscreen mode
  • Create a nomad service file 
$ sudo touch /etc/systemd/system/nomad.service
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the nomad service file /etc/systemd/system/nomad.service 
[Unit] Description=Nomad Documentation=https://nomadproject.io/docs/ [Service] ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d Restart=on-failure RestartSec=2 [Install] WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode
  • Start nomad 
sudo systemctl enable nomad sudo systemctl start nomad sudo systemctl status nomad
Enter fullscreen mode Exit fullscreen mode
  • Genertal ACL secrets 
$ nomad acl bootstrap Accessor ID = a8b724ef-b5eb-21c0-1a68-b0d3616e1e62 Secret ID = 07fe0ee6-59bc-8ea0-aeeb-9b4e9edcb555 Name = Bootstrap Token Type = management Global = true Policies = n/a Create Time = 2021-05-10 21:05:21.498072284 +0000 UTC Create Index = 10 Modify Index = 10 $ export NOMAD_TOKEN=07fe0ee6-59bc-8ea0-aeeb-9b4e9edcb555
Enter fullscreen mode Exit fullscreen mode
  • Create your first job to deploy a stateless container from the server ui http://127.0.0.1:4646/ui 
job "api" { datacenters = ["dc1"] group "example" { task "server" { driver = "docker" config { image = "hashicorp/http-echo:0.2.1" args = [ "-listen", ":5678", "-text", "hello world", ] } resources { network { mbits = 10 port "http" { static = "5678" } } } } } }
Enter fullscreen mode Exit fullscreen mode
  • And another one for stateful container 
job "cache" { datacenters = ["dc1"] group "redis" { volume "redis_data" { type = "host" read_only = false source = "redis_data" } task "server" { driver = "docker" volume_mount { volume = "redis_data" destination = "/data" read_only = false } config { image = "redis:4-alpine" labels = { "sh.hippo.service" = "redis" "sh.hippo.service_type" = "cache" } port_map { http = 6379 } command = "redis-server" args = [ "--requirepass", "mystery", ] } env = { HEALTHY_FOR = -1, } resources { cpu = 100 memory = 256 network { mbits = 10 port "http" { static = "6379" } } } } } }
Enter fullscreen mode Exit fullscreen mode

Multi Nodes Setup (Recommended):

To fully understand how nomad cluster should look like, please check this guide!

Assuming we have two servers, one for the nomad server (Leader) and another for the client. Both servers are in DC1. The leader server has public IP $SERVER_PUBLIC_IP and private IP $SERVER_PRIVATE_IP.

First we run the leader or the nomad server by following the next steps:

  • Download nomad binary & make it executable: 
$ wget https://releases.hashicorp.com/nomad/1.0.5/nomad_1.0.5_linux_amd64.zip $ unzip nomad_1.0.5_linux_amd64.zip $ mv nomad /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode
  • Create configs directory & data directory: 
$ sudo mkdir --parents /opt/nomad $ sudo mkdir --parents /opt/data/redis $ sudo mkdir --parents /etc/nomad.d $ sudo chmod 700 /etc/nomad.d $ sudo chmod 700 /opt/data $ sudo touch /etc/nomad.d/nomad.hcl $ sudo touch /etc/nomad.d/server.hcl
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the /etc/nomad.d/nomad.hcl configuration file 
datacenter = "dc1" data_dir = "/opt/nomad"
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the /etc/nomad.d/server.hcl configuration file 
acl { enabled = true } server { enabled = true bootstrap_expect = 1 } bind_addr = "$SERVER_PUBLIC_IP" addresses { http = "$SERVER_PUBLIC_IP" rpc = "$SERVER_PRIVATE_IP" serf = "$SERVER_PRIVATE_IP" } advertise { http = "$SERVER_PUBLIC_IP:4646" rpc = "$SERVER_PRIVATE_IP:4647" serf = "$SERVER_PRIVATE_IP:4648" }
Enter fullscreen mode Exit fullscreen mode
  • Create a nomad service file 
$ sudo touch /etc/systemd/system/nomad.service
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the nomad service file /etc/systemd/system/nomad.service 
[Unit] Description=Nomad Documentation=https://nomadproject.io/docs/ [Service] ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d Restart=on-failure RestartSec=2 [Install] WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode
  • Start nomad 
sudo systemctl enable nomad sudo systemctl start nomad sudo systemctl status nomad
Enter fullscreen mode Exit fullscreen mode
  • Genertal ACL secrets 
$ nomad acl bootstrap -address=http://$SERVER_PUBLIC_IP:4646 Accessor ID = a8b724ef-b5eb-21c0-1a68-b0d3616e1e62 Secret ID = 07fe0ee6-59bc-8ea0-aeeb-9b4e9edcb555 Name = Bootstrap Token Type = management Global = true Policies = n/a Create Time = 2021-05-10 21:05:21.498072284 +0000 UTC Create Index = 10 Modify Index = 10 $ export NOMAD_TOKEN=07fe0ee6-59bc-8ea0-aeeb-9b4e9edcb555
Enter fullscreen mode Exit fullscreen mode

Then we run the client by following the next steps:

  • Download nomad binary & make it executable: 
$ wget https://releases.hashicorp.com/nomad/1.0.5/nomad_1.0.5_linux_amd64.zip $ unzip nomad_1.0.5_linux_amd64.zip $ mv nomad /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode
  • Create configs directory & data directory: 
$ sudo mkdir --parents /opt/nomad $ sudo mkdir --parents /opt/data/redis $ sudo mkdir --parents /etc/nomad.d $ sudo chmod 700 /etc/nomad.d $ sudo chmod 700 /opt/data $ sudo touch /etc/nomad.d/nomad.hcl $ sudo touch /etc/nomad.d/client.hcl
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the /etc/nomad.d/nomad.hcl configuration file 
datacenter = "dc1" data_dir = "/opt/nomad"
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the /etc/nomad.d/client.hcl configuration file 
client { enabled = true servers = ["$SERVER_PRIVATE_IP:4647"] host_volume "redis_data" { path = "/opt/data/redis" read_only = false } } plugin "docker" { config { volumes { enabled = true selinuxlabel = "z" } } }
Enter fullscreen mode Exit fullscreen mode
  • Create a nomad service file 
$ sudo touch /etc/systemd/system/nomad.service
Enter fullscreen mode Exit fullscreen mode
  • Add this configuration to the nomad service file /etc/systemd/system/nomad.service 
[Unit] Description=Nomad Documentation=https://nomadproject.io/docs/ [Service] ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d Restart=on-failure RestartSec=2 [Install] WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode
  • Start nomad 
sudo systemctl enable nomad sudo systemctl start nomad sudo systemctl status nomad
Enter fullscreen mode Exit fullscreen mode

CNI Plugins in Nomad

Nomad uses CNI plugins when bridge networking is used. To install CNI plugins:

$ curl -L -o cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/v0.8.0/cni-plugins-linux-amd64-v0.8.0.tgz $ mkdir -p /opt/cni/bin $ tar -C /opt/cni/bin -xzf cni-plugins.tgz
Enter fullscreen mode Exit fullscreen mode 
job "clivern" { datacenters = ["dc1"] group "services" { network { port "toad0_srv" { static = 8080 } port "toad1_srv" { static = 8081 } } task "toad0" { driver = "docker" config { image = "clivern/toad:release-0.2.4" labels = { "com.clivern.service" = "toad" "com.clivern.service_type" = "web" } ports = ["toad0_srv"] command = "./toad" args = [ "--port", "${NOMAD_PORT_toad0_srv}", ] } env = { IS_STATEFUL = "false", TOAD0_ADDR = "${NOMAD_HOST_ADDR_toad0_srv}", TOAD1_ADDR = "${NOMAD_HOST_ADDR_toad1_srv}", } resources { network { mbits = 10 } } } task "toad1" { driver = "docker" config { image = "clivern/toad:release-0.2.3" labels = { "com.clivern.service" = "toad" "com.clivern.service_type" = "web" } ports = ["toad1_srv"] command = "./toad" args = [ "--port", "${NOMAD_PORT_toad1_srv}", ] } env = { IS_STATEFUL = "false", TOAD0_ADDR = "${NOMAD_HOST_ADDR_toad0_srv}", TOAD1_ADDR = "${NOMAD_HOST_ADDR_toad1_srv}", } resources { network { mbits = 10 } } } } }
Enter fullscreen mode Exit fullscreen mode

References:

Top comments (0)