Download as PDF, PPTX
![MADRID · NOV 27-28 · 2015 LAUNCH MONGO CONTAINER WITH MARATHON { "id": "mongo", "cmd": "cat /etc/resolv.conf > /tmp/temp && echo "nameserver 10.0.2.15" > /etc/resolv.conf && cat /tmp/temp >> /etc/resolv.conf && mongod", "cpus": 0.5, "mem": 500.0, "container": { "type": "DOCKER", "docker": { "image": "mongo", "privileged": true, "network": "BRIDGE", "portMappings": [ { "containerPort": 27017, "hostPort": 0 } ] } } } HTTP POST : http://<HOST_IP>:8080/v2/apps?force=true Friday, November 27, 15](https://image.slidesharecdn.com/workshopcodemotion-151130182804-lva1-app6891/75/Building-and-deploying-a-distributed-application-with-Docker-Mesos-and-Marathon-41-2048.jpg)
![MADRID · NOV 27-28 · 2015 LAUNCH CMVOTING WITH MARATHON { "id": "cmvoting", "cmd": "cat /etc/resolv.conf > /tmp/temp && echo "nameserver 10.0.2.15" > /etc/resolv.conf && cat /tmp/temp >> /etc/resolv.conf && revel run github.com/karesti/cm-voting prod 9000", "cpus": 0.5, "mem": 500.0, "container": { "type": "DOCKER", "docker": { "image": "cm-voting", "network": "BRIDGE", "portMappings": [ { "containerPort": 9000, "hostPort": 0 } ] } }, "env": { ! "SERVICE_NAME": "mongo" } } HTTP POST : http://<HOST_IP>:8080/v2/apps?force=true Friday, November 27, 15](https://image.slidesharecdn.com/workshopcodemotion-151130182804-lva1-app6891/75/Building-and-deploying-a-distributed-application-with-Docker-Mesos-and-Marathon-42-2048.jpg)
Uploaded byJulia Mateo
Building and deploying a distributed application with Docker, Mesos and Marathon
This document outlines a workshop focused on building and deploying a distributed application, cm-voting, using Docker, Mesos, and Marathon. It details the steps to create a Mesos cluster and deploy the application in a Docker environment, as well as implementing service discovery with Mesos DNS. The workshop also includes instructions for setting up the necessary components, running the application, and testing its functionality.
Building and deploying a distributed application with Docker, Mesos and Marathon
- 1. Building and deploying a distributed applicationwith Docker, Mesos and Marathon Julia MateoMADRID · NOV 27-28 · 2015 Friday, November 27, 15
- 2. MADRID · NOV27-28 · 2015 WHO AM I - Java developer consultant and tech lead at Hortis GRC (Geneva, Switzerland) - Team jDuchess Swiss @juliamateodc @duchessswiss http://jduchess.ch/ Friday, November 27, 15
- 3. MADRID · NOV27-28 · 2015 GOALS (1st part of the workshop) • Present cm-voting, a distributed web application • Run cm-voting with docker links • Run cm-voting with docker compose Friday, November 27, 15
- 4. MADRID · NOV27-28 · 2015 GOALS (2nd part of the workshop) • Create a Mesos cluster • Mesos DNS • Deploy cm-voting on mesos cluster Friday, November 27, 15
- 5. MADRID · NOV27-28 · 2015 IN THE MEANTIME.... - 2 ways of doing this workshop : Mode A : - RECOMMENDED : install virtualbox (available USB) and copy workshop virtual box image Mode B : - You will need a browser and a Client HTTP (like Postman Rest client for Chrome) - Install docker 1.9.1 (Linux) or Docker toolbox (Windows and Mac). Copy and load the docker images we will use from the USB keys - Then, load them on your host : docker load -i <path_image_tar_file> Friday, November 27, 15
- 6. MADRID · NOV27-28 · 2015 CODE MOTION VOTING - Go webapp build on Revel - Voting app for Code Motion conference Friday, November 27, 15
- 7. MADRID · NOV27-28 · 2015 CODE MOTION VOTING Friday, November 27, 15
- 8. MADRID · NOV27-28 · 2015 CODE MOTION VOTING Friday, November 27, 15
- 9. MADRID · NOV27-28 · 2015 DEPLOY APP ON DOCKER Friday, November 27, 15
- 10. From https://www.docker.com/whatisdocker/ • Dockeris an open platform for developers and sysadmins to build, ship, and run distributed applications. DOCKER Friday, November 27, 15
- 11. DOCKER Server Host OS Hypervisor Guest OS Guest OS Guest OS Guest OS Libs/ Libs/Libs/ App 1 App 2 App 3 App 4 Libs/ Server Host OS Docker Engine Libs/Bins Libs/ App 1 App 2 App 3 App 4 c1 c2 c3 c4 VM1 VM2 VM3 VM4 Friday, November 27, 15
- 12. Docker Images Docker Images libcontainer, Union Filesystem CentosUbuntu Jetty add App.war Container 1 OracleDB Container 2 Read only Writable Writable Read only CONTAINERS AND IMAGES Friday, November 27, 15
- 13. MADRID · NOV27-28 · 2015 DEPLOY CODE MOTION VOTING WITH DOCKER - Let’s start ! - See webapp doc https://github.com/karesti/cm-voting - Git clone https://github.com/karesti/cm-voting - If using virtual box image, go to : /home/codemotion/cm-voting Friday, November 27, 15
- 14. MADRID · NOV27-28 · 2015 DEPLOY CODE MOTION VOTING WITH DOCKER - Follow the instructions in https://github.com/karesti/cm-voting : - Open your terminal and run the mongo container : > sudo docker run -i -t -d --name mongo_cmvoting -p 27017:27017 mongo > cd cm-voting - Build and run cm-voting: > sudo docker build -t cm-voting . > sudo docker run -i -t -p 9000:9000 --link mongo_cmvoting:mongo cm-voting Friday, November 27, 15
- 15. MADRID · NOV27-28 · 2015 DOCKER COMPOSE - Instead of using docker links : > sudo docker run -i -t -p 9000:9000 --link mongo_cmvoting:mongo cm-voting - You can use docker compose > sudo docker-compose up Friday, November 27, 15
- 16. MADRID · NOV27-28 · 2015 DEPLOY WEB APP ON MESOS Friday, November 27, 15
- 17. • Abstraction ofcluster resources • Share resources across multiple frameworks (versions of the same fwk) • Resource fair sharing : alternative to static partitioning • Data locality MESOS Friday, November 27, 15
- 18. ZK Master Master Executor ExecutorExecutor Executor ExecutorExecutor Executor Executor Executor Slaves MESOS Friday, November 27, 15
- 19.
- 20. • Marathon isa Mesos framework written in Scala • Provides easy deployment of Docker containers • Manages of long running apps • Rest API for developers MESOS Friday, November 27, 15
- 21. ZK Master Master Executor ExecutorExecutor Executor ExecutorExecutor Executor Executor Executor Slaves MESOS Friday, November 27, 15
- 22. ZK Master Master Executor ExecutorExecutor Executor ExecutorExecutor Executor Executor Executor Slaves Marathon MESOS Friday, November 27, 15
- 23. MADRID · NOV27-28 · 2015 LET’S CREATE A MESOS CLUSTER ! - Create a mesos cluster in localhost - Docker binding - Using different ports Friday, November 27, 15
- 24. MADRID · NOV27-28 · 2015 DOCKER VOLUMES AND DOCKER BINDING Host /home/codemotion/mongo/data Docker container /data/db Host Docker container /var/run/docker.sock /var/run/docker.sock Docker volume : directory Docker volume : file Friday, November 27, 15
- 25. MADRID · NOV27-28 · 2015 CREATE MESOS CLUSTER - If you are using Docker toolkit : set HOST_IP env variable with your Docker host (in my case, 10.0.2.15) export HOST_IP=10.0.2.15 - Launch zookeeper container sudo docker run -d -e SERVER_ID=1 -p 2181:2181 zookeeper - Launch mesos master container sudo docker run -d -p 5050:5050 -e "MESOS_HOSTNAME=${HOST_IP}" -e "MESOS_IP=${HOST_IP}" -e "MESOS_QUORUM=1" -e "MESOS_ZK=zk://${HOST_IP}:2181/mesos" --name mesos-master -e "MESOS_LOG_DIR=/var/log/mesos" --net host --restart always mesoscloud/mesos-master:0.23.0-centos-7 Friday, November 27, 15
- 26. MADRID · NOV27-28 · 2015 CREATE MESOS CLUSTER - Launch marathon sudo docker run -d -e "MARATHON_HOSTNAME=${HOST_IP}" -e "MARATHON_HTTPS_ADDRESS=$ {HOST_IP}" -e "MARATHON_HTTP_ADDRESS=${HOST_IP}" -e "MARATHON_MASTER=zk://${HOST_IP}: 2181/mesos" -e "MARATHON_ZK=zk://${HOST_IP}:2181/marathon" --name marathon --net host --restart always mesoscloud/marathon:0.10.0-centos-7 Friday, November 27, 15
- 27. MADRID · NOV27-28 · 2015 CREATE MESOS CLUSTER - Launch mesos slave 1 sudo docker run -p 5051:5051 -d -e "MESOS_HOSTNAME=${HOST_IP}" -e "MESOS_IP=${HOST_IP}" -e "MESOS_MASTER=zk://${HOST_IP}:2181/mesos" -v /sys/fs/cgroup:/sys/fs/cgroup -v /var/run/docker.sock:/var/run/ docker.sock --name slave1 --net host --privileged --restart always mesoscloud/mesos-slave:0.23.0-centos-7 - Launch mesos slave 2 sudo docker run -p 5052:5052 -d -e "MESOS_HOSTNAME=${HOST_IP}" -e MESOS_PORT=5052 -e "MESOS_IP=$ {HOST_IP}" -e "MESOS_MASTER=zk://${HOST_IP}:2181/mesos" -v /sys/fs/cgroup:/sys/fs/cgroup -v /var/run/ docker.sock:/var/run/docker.sock --name slave2 --net host --privileged --restart always mesoscloud/mesos-slave: 0.23.0-centos-7 - Launch mesos slave 3 sudo docker run -p 5053:5053 -d -e "MESOS_HOSTNAME=${HOST_IP}" -e MESOS_PORT=5053 -e "MESOS_IP=$ {HOST_IP}" -e "MESOS_MASTER=zk://${HOST_IP}:2181/mesos" -v /sys/fs/cgroup:/sys/fs/cgroup -v /var/run/ docker.sock:/var/run/docker.sock --name slave3 --net host --privileged --restart always mesoscloud/mesos-slave: 0.23.0-centos-7 Friday, November 27, 15
- 28. MADRID · NOV27-28 · 2015 TEST THAT EVERYTHING WORKS OK - Portal Mesos : http://<HOST_IP>:5050 Friday, November 27, 15
- 29. MADRID · NOV27-28 · 2015 TEST THAT EVERYTHING WORKS OK - Portal Marathon : http://<HOST_IP>:8080 Friday, November 27, 15
- 30. MADRID · NOV27-28 · 2015 SERVICE DISCOVERY cm-voting Node 1 Node 2 Friday, November 27, 15
- 31. MADRID · NOV27-28 · 2015 SERVICE DISCOVERY cm-voting Node 1 Node 2 Friday, November 27, 15
- 32. MADRID · NOV27-28 · 2015 SERVICE DISCOVERY cm-voting Node 1 Node 2 IP ? PORT ? CREDENTIALS ?? Friday, November 27, 15
- 33. MADRID · NOV27-28 · 2015 SERVICE DISCOVERY cm-voting Mesos slave 1 Mesos slave 2 REPLICATION, LOAD BALANCING... Friday, November 27, 15
- 34. MADRID · NOV27-28 · 2015 SERVICE DISCOVERY cm-voting Node 1 Node 2 ... AND FAILOVER Friday, November 27, 15
- 35. MADRID · NOV27-28 · 2015 SERVICE DISCOVERY : MESOS DNS https://github.com/mesosphere/mesos-dns Friday, November 27, 15
- 36. MADRID · NOV27-28 · 2015 Build Mesos DNS Image - Git clone : https://github.com/jmateo/mesosdns - cd mesosdns - Edit config.json : sed -i -e “s/HOST_IP/$HOST_IP/g” config.json - Build the image : docker build -t mesosdns . Friday, November 27, 15
- 37. MADRID · NOV27-28 · 2015 SERVICE DISCOVERY WITH MESOS DNS 1. Launch Mesos DNS container with Marathon 2. Launch Mongo container 3. Connect to one of the Mesos slaves and look for the mongo service (you can use dig command for doing this) 4. Modify the class connection.go to connect dynamically cm-voting to mongo using service discovery (see MesosDNS REST API) Friday, November 27, 15
- 38. MADRID · NOV27-28 · 2015 LAUNCH MESOS DNS WITH MARATHON HTTP POST : http://<HOST_IP>:8080/v2/apps?force=true Friday, November 27, 15
- 39. MADRID · NOV27-28 · 2015 LAUNCH MESOS DNS WITH MARATHON { "id": "mesos-dns", "cmd": "/mesos-dns", "cpus": 0.5, "mem": 500, "container": { "type": "DOCKER", "docker": { "image": "mesosdns", "network": "HOST" } } } Friday, November 27, 15
- 40. MADRID · NOV27-28 · 2015 LAUNCH MONGO CONTAINER WITH MARATHON HTTP POST : http://<HOST_IP>:8080/v2/apps?force=true Friday, November 27, 15
- 41. MADRID · NOV27-28 · 2015 LAUNCH MONGO CONTAINER WITH MARATHON { "id": "mongo", "cmd": "cat /etc/resolv.conf > /tmp/temp && echo "nameserver 10.0.2.15" > /etc/resolv.conf && cat /tmp/temp >> /etc/resolv.conf && mongod", "cpus": 0.5, "mem": 500.0, "container": { "type": "DOCKER", "docker": { "image": "mongo", "privileged": true, "network": "BRIDGE", "portMappings": [ { "containerPort": 27017, "hostPort": 0 } ] } } } HTTP POST : http://<HOST_IP>:8080/v2/apps?force=true Friday, November 27, 15
- 42. MADRID · NOV27-28 · 2015 LAUNCH CMVOTING WITH MARATHON { "id": "cmvoting", "cmd": "cat /etc/resolv.conf > /tmp/temp && echo "nameserver 10.0.2.15" > /etc/resolv.conf && cat /tmp/temp >> /etc/resolv.conf && revel run github.com/karesti/cm-voting prod 9000", "cpus": 0.5, "mem": 500.0, "container": { "type": "DOCKER", "docker": { "image": "cm-voting", "network": "BRIDGE", "portMappings": [ { "containerPort": 9000, "hostPort": 0 } ] } }, "env": { ! "SERVICE_NAME": "mongo" } } HTTP POST : http://<HOST_IP>:8080/v2/apps?force=true Friday, November 27, 15
- 43. MADRID · NOV27-28 · 2015 SERVICE DISCOVERY WITH MESOS DNS 4. Test it : Friday, November 27, 15