DEV Community

Serhat Teker
Serhat Teker

Posted on • Originally published at tech.serhatteker.com on

How to Run Multiple Long Commands in Docker Compose

#docker #dockercompose #yaml

Use Case

I want to run multiple/long commands for my service(s) in my docker-compose file, like below:

--- # docker-compose.yaml services: server: &server build: context: . dockerfile: ./compose/staging/server/Dockerfile image: someproject_slug_staging_server:staging_shortsha_unixtime depends_on: - postgres env_file: - ./.envs/.staging/.server - ./.envs/.staging/.postgres command: python /app/manage.py collectstatic --no-input command: python /app/manage.py migrate command: gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app postgres: build: context: . dockerfile: ./compose/staging/postgres/Dockerfile image: someproject_slug_staging_postgres:staging_shortsha_unixtime volumes: - staging_postgres_data:/var/lib/postgresql/data:Z - staging_postgres_data_backups:/backups:z env_file: - ./.envs/.staging/.postgres
Enter fullscreen mode Exit fullscreen mode

However it won't work since it's not possible.

Solutions

Script File

One solution is that putting all your commands in a script file -let's say start.sh, put it into your image, build with it and call it directly.

As an example here is a docker compose file:

--- # staging.yaml version: '3.7' volumes: staging_postgres_data: {} staging_postgres_data_backups: {} staging_traefik: {} staging_redis: {} services: server: &server build: context: . dockerfile: ./compose/staging/server/Dockerfile image: someproject_slug_server:staging_shortsha_unixtime depends_on: - postgres - redis env_file: - ./.envs/.staging/.server - ./.envs/.staging/.postgres - ./.envs/.staging/.redis command: /start postgres: build: context: . dockerfile: ./compose/staging/postgres/Dockerfile image: someproject_slug_postgres:staging_shortsha_unixtime volumes: - staging_postgres_data:/var/lib/postgresql/data:Z - staging_postgres_data_backups:/backups:z env_file: - ./.envs/.staging/.postgres traefik: build: context: . dockerfile: ./compose/staging/traefik/Dockerfile image: someproject_slug_traefik:staging_shortsha_unixtime depends_on: - server volumes: - staging_traefik:/etc/traefik/acme:z ports: - "0.0.0.0:80:80" - "0.0.0.0:443:443" - "0.0.0.0:5555:5555" redis: image: docker.io/bitnami/redis:6.2 env_file: - ./.envs/.staging/.redis ports: - "6379:6379" # WARNING: In production never bind 0.0.0.0 wildcard adress of the host volumes: - staging_redis:/bitnami/redis/data:z celeryworker: <<: *server command: /start-celeryworker celerybeat: <<: *server command: /start-celerybeat celeryflower: <<: *server command: /start-celeryflower aws-backup: build: context: . dockerfile: ./compose/staging/aws/Dockerfile env_file: - ./.envs/.staging/.server volumes: - staging_postgres_data_backups:/backups:z
Enter fullscreen mode Exit fullscreen mode

WARNING

Don't bind your host's 0.0.0.0 wildcard adress in production environment. (Actually instead of deploying via docker-composein production you should consider to use more reliable, and robust distributed systems like Kubernetes but this is a topic for another post.)

6379:6379 means 0.0.0.0:6379:6379, so you open your redis instance to whole open world. And if you don't use AUTH or don't take required security precaution -like TLS etc. you may get some headache. So use it only if you
know the related consequences.

For more redis security topic look at this: Redis Security

And as an example here is content of start from server service:

#!/usr/env/bin bash # start set -o errexit set -o pipefail set -o nounset python /app/manage.py collectstatic --noinput python /app/manage.py migrate # WARNING: Not a good idea for prod gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
Enter fullscreen mode Exit fullscreen mode

WARNING

Migration is not a command you want to run every container start but since it's staging environment let's assume it's OK.

Multiline Scalar

Running in a script file is OK however maybe it's local or staging environment which means you need to debug and tweak these commands one by one; maybe some flags are not working etc. and you don't want to re-build the image every time.

How are we gonna do this?

The answer lines in the multiline capability of yaml syntax:

command: > bash -c "python /app/manage.py collectstatic --noinput && python /app/manage.py migrate && gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app"
Enter fullscreen mode Exit fullscreen mode

This is called Folded Style. The indention in each line will be ignored. A line break (\n) will be inserted at the end.

Or you can use Literal Style:

command: - /bin/bash - -c - | python /app/manage.py collectstatic --noinput python /app/manage.py migrate gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
Enter fullscreen mode Exit fullscreen mode

This turns every newline within the string into a literal newline, and adds one at the end.

More on these styles:

Folded Style 

Key: > this is my very very very long command
Enter fullscreen mode Exit fullscreen mode

Output:

this is my very very very long command\n

Mind new line-\n (end of line-EOL)

Literal Style 

Key: | this is my very very very long command
Enter fullscreen mode Exit fullscreen mode

Output:

this is my very very very\nlong command\n

Mind new lines-\n, and compare to it above output.

Fore more detailed style and yaml syntax you can look at this amazing explanation in this link.

These were the solutions for running multiple or long commands in your docker-compose file.

All done!

Top comments (0)