Docker Compose
Introduction to Compose
Compose is a tool for defining and running multi-container Docker applications. With Compose, you can use a YML file to configure all the services your application needs. Then, using a single command, you can create and start all services from the YML file configuration.
Compose uses three steps.
- Use Dockerfile to define the application’s environment.
- Use docker-compose.yml to define the services that make up the application so that they can run together in an isolated environment.
- Finally, execute the docker-compose up command to start and run the entire application.
An example configuration of docker-compose.yml is as follows (see below for configuration parameters)
# yaml Configuration Example
version: '3'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/code
- logvolume01:/var/log
links:
- redis
redis:
image: redis
volumes:
logvolume01: {}
Compose installation
On Linux we can download its binary package from Github to use, the latest release is available at https://github.com/docker/compose/releases.
Run the following command to download the current stable version of Docker Compose.
$ sudo curl -L "https://github.com/docker/compose/releases/download/v2.2.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/ docker-compose
To install another version of Compose, replace v2.2.2.
Docker Compose is stored on GitHub and is not very stable.
You can also install Docker Compose at high speed by running the following command.
curl -L https://get.daocloud.io/docker/compose/releases/download/v2.4.1/docker-compose-
uname -s
-uname -m
> /usr/local/bin/docker- compose
To apply executable permissions to a binary file.
$ sudo chmod +x /usr/local/bin/docker-compose
Create a softlink.
$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
To test if the installation was successful.
$ docker-compose version
cker-compose version 1.24.1, build 4667896b
Note: For alpine, the following dependencies are required: py-pip, python-dev, libffi-dev, openssl-dev, gcc, libc-dev, and make.
macOS
The Docker Desktop and Docker Toolbox for Mac already include Compose and other Docker applications, so Mac users do not need to install Compose separately.
windows PC
The Docker Desktop and Docker Toolbox for Windows already include Compose and other Docker applications, so Windows users do not need to install Compose separately.
Usage
1. Preparation
Create a test directory.
$ mkdir composetest
$ cd composetest
Create a file called app.py in the test directory and copy and paste the following.
import time
import redis
from flask import Flask
app = Flask(__name__)
cache = redis.Redis(host='redis', port=6379)
def get_hit_count():
retries = 5
while True:
try:
return cache.incr('hits')
except redis.exceptions.ConnectionError as exc:
if retries == 0:
raise exc
retries -= 1
time.sleep(0.5)
@app.route('/')
def hello():
count = get_hit_count()
return 'Hello World! I have been seen {} times.\n'.format(count)
In this example, redis is the host name of the redis container on the application network, which uses port 6379.
Create another file in the composetest directory called requirements.txt with the following contents.
flask
redis
2. Create the Dockerfile file
In the composetest directory, create a file named Dockerfile with the following contents.
FROM python:3.7-alpine
WORKDIR /code
ENV FLASK_APP app.py
ENV FLASK_RUN_HOST 0.0.0.0
RUN apk add --no-cache gcc musl-dev linux-headers
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY . .
CMD ["flask", "run"]
Dockerfile contents explained:
-
FROM python:3.7-alpine: Start building the image from a Python 3.7 image.
-
WORKDIR /code: Set the working directory to /code.
-
ENV FLASK_APP app.py ENV FLASK_RUN_HOST 0.0.0.0
Set the environment variables used by the flask command.
-
RUN apk add –no-cache gcc musl-dev linux-headers: Install gcc so that Python packages such as MarkupSafe and SQLAlchemy can be compiled faster.
-
COPY requirements.txt requirements.txt RUN pip install -r requirements.txt
Copy requirements.txt and install Python dependencies.
-
_COPY . . : Copies the . project to the current directory in the . working directory in the image.
-
CMD ["flask", "run"]: The container provides the default execution command as: flask run.
3. Create docker-compose.yml
Create a file named docker-compose.yml in the test directory, and paste the following.
# yaml Configuration
version: '3'
services:
web:
build: .
ports:
- "5000:5000"
redis:
image: "redis:alpine"
This Compose file defines two services: web and redis.
- web: This web service uses the image built from the current directory of Dockerfile. This sample service uses the default port 5000 of the Flask web server.
- redis: This redis service uses the public Redis image from Docker Hub.
4. Build and run your application using the Compose command
In the test directory, execute the following command to start the application.
docker-compose up
If you want to execute the service in the background you can add the -d parameter.
docker-compose up -d
yml configuration command reference
version
Specifies which version of compose this yml depends on.
build
Specifies the build image context path.
For example, for the webapp service, specify as the image to be built from the context path . /dir/Dockerfile.
version: "3.7"
services:
webapp:
build: . /dir
Or, as an object with a path specified in the context, and optionally a Dockerfile and args.
version: "3.7"
services:
webapp:
build:
context: . /dir
dockerfile: dockerfile-alternate
args:
buildno: 1
labels:
- "com.example.description=Accounting webapp"
- "com.example.department=Finance"
- "com.example.labels-with-empty-value"
target: prod
- context: context path.
- dockerfile: specify the name of the Dockerfile file where the image will be built.
- args: add build parameters, which are environment variables that can only be accessed during the build process.
- labels: set the labels of the build image.
- target: multi-layer build, you can specify which layer to build.
cap_add,cap_drop
Adds or removes kernel functions of the host owned by the container.
cap_add:
- ALL # Enables all privileges
cap_drop:
- SYS_PTRACE # Turn off ptrace privileges
cgroup_parent
Specifying a parent cgroup group for a container means that it will inherit the resource limits of that group.
cgroup_parent: m-executor-abcd
command
Override the default command for container startup.
command: ["bundle", "exec", "thin", "-p", "3000"]
container_name
Specify a custom container name instead of the generated default name.
container_name: my-web-container
depends_on
Set dependencies.
- docker-compose up : Starts services in dependency order. In the following example, db and redis are started first before web is started.
- docker-compose up SERVICE : Automatically includes dependencies for SERVICE. In the following example, docker-compose up web will also create and start db and redis.
- docker-compose stop : Stops the services in dependency order. In the following example, web is stopped before db and redis.
version: "3.7"
services:
web:
build: .
depends_on:
- db
- redis
redis:
image: redis
db:
image: postgres
Note: The web service will not wait for the redis db to fully start before starting.
deploy
Specifies the configuration related to the deployment and operation of the service. Useful only in swarm mode.
version: "3.7"
services:
redis:
image: redis:alpine
deploy:
mode:replicated
replicas: 6
endpoint_mode: dnsrr
labels:
description: "This redis service label"
resources:
limits:
cpus: '0.50'
memory: 50M
reservations:
cpus: '0.25'
memory: 20M
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
Optional parameters.
endpoint_mode: the way to access the cluster service.
endpoint_mode: vip
# Docker Cluster Service an external virtual ip. All requests will reach the machines inside the cluster service via this virtual ip.
endpoint_mode: dnsrr
# DNS polling (DNSRR). All requests are automatically polled for an ip address in the cluster ip list.
labels: Set labels on the service. You can override the labels under deploy with the labels on the container (the same level of configuration as deploy).
mode: Specify the mode of service provisioning.
-
replicated: replicates the service, replicates the specified service to the machines in the cluster.
-
global: global service, the service will be deployed to each node of the cluster.
-
Illustration: The yellow square in the figure below shows the operation of replicated mode, and the gray square shows the operation of global mode.
replicas: mode is replicated, you need to use this parameter to configure the exact number of nodes to run.
resources: Configure limits on server resource usage, such as the above example, to configure the percentage of cpu and memory required for the redis cluster to run. Avoid exceptions if the resource usage is too high.
restart_policy: Configure how to restart the container when it is exited.
- condition: optionally none, on-failure or any (default: any).
- delay: set how long to restart after (default: 0).
- max_attempts: the number of attempts to restart the container, beyond that, no more attempts will be made (default: keep retrying).
- window: set the container restart timeout (default: 0).
rollback_config: configure how the service should be rolled back in case of update failure.
- parallelism: the number of containers to be rolled back at a time. If set to 0, all containers will be rolled back at the same time.
- delay: the time to wait between rollbacks for each container group (default is 0s).
- failure_action: what to do if the rollback fails. One of continue or pause (default pause).
- monitor: the time (ns|us|ms|s|m|h) after each container update to continuously watch if it fails (default is 0s).
- max_failure_ratio: the failure rate that can be tolerated during the rollback (default is 0).
- order: the order of operations during the rollback. Either stop-first (serial rollback), or start-first (parallel rollback) (default stop-first).
update_config: configure how the service should be updated, useful for configuring rolling updates.
- parallelism: the number of containers to be updated at once.
- delay: the time to wait between updating a set of containers.
- failure_action: what to do if the update fails. One of continue, rollback or pause (default: pause).
- monitor: the time (ns|us|ms|s|m|h) to keep watching for failure after each container update (default is 0s).
- max_failure_ratio: the failure rate that can be tolerated during the update.
- order: the order of operations during the rollback. One stop-first (serial rollback), or start-first (parallel rollback) (default stop-first).
Note: Only supported for V3.4 and higher.
devices
Specifies the device mapping list.
devices:
- "/dev/ttyUSB0:/dev/ttyUSB0"
dns
Customize the DNS server, either as a single value or as a list of multiple values.
dns: 8.8.8.8
dns:
- 8.8.8.8
- 9.9.9.9
dns_search
Custom DNS search domain. Can be a single value or a list.
dns_search: example.com
dns_search:
- dc1.example.com
- dc2.example.com
entrypoint
Override the container’s default entrypoint.
entrypoint: /code/entrypoint.sh
It can also be in the following format.
entrypoint:
- php
-d
- zend_extension=/usr/local/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
-d
- memory_limit=-1
- vendor/bin/phpunit
env_file
Add environment variables from a file. Can be a single value or a list of multiple values.
env_file: .env
Can also be in list format.
env_file:
- . /common.env
- . /apps/web.env
- /opt/secrets.env
environment
Add environment variables. You can use arrays or dictionaries, any boolean value, and boolean values need to be enclosed in quotes to ensure that the YML parser will not convert them to True or False.
environment:
RACK_ENV: development
SHOW: 'true'
expose
Exposes the port, but does not map it to the host, and is only accessed by the connected service.
Only internal ports can be specified as parameters.
expose:
- "3000"
- "8000"
extra_hosts
Add a hostname mapping. Similar to docker client –add-host.
extra_hosts:
- "somehost:162.242.195.82"
- "otherhost:50.31.209.229"
The above will create a mapping relationship with an ip address and hostname in the internal container /etc/hosts for this service.
162.242.195.82 somehost
50.31.209.229 otherhost
healthcheck
Check if the docker service is running healthily.
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost"] # Set the testing procedure
interval: 1m30s # Set the detection interval
timeout: 10s # Set detection timeout
retries: 3 # Set the number of retries
start_period: 40s # how many seconds to start the detection process after startup
image
Specifies the image the container is running on. All of the following formats are possible.
image: redis
image: ubuntu:14.04
image: tutum/influxdb
image: example-registry.com:4000/postgresql
image: a4bc65fd # mirror id
logging
The logging configuration for the service.
driver: Specify the logging driver for the service container, the default value is json-file. there are three options
driver: "json-file"
driver: "syslog"
driver: "none"
Under the json-file driver only, the following parameters can be used to limit the number and size of logs.
logging:
driver: json-file
options:
max-size: "200k" # single file size of 200k
max-file: "10" # maximum 10 files
When the file limit is reached, the old files will be deleted automatically.
Under the syslog driver, you can use syslog-address to specify the log receiving address.
logging:
driver: syslog
options:
syslog-address: "tcp://192.168.0.42:123"
network_mode
Set the network mode.
network_mode: "bridge"
network_mode: "host"
network_mode: "none"
network_mode: "service:[service name]"
network_mode: "container:[container name/id]"
networks
Configure the network to which the container is connected, referencing the entry under top-level networks .
services:
some-services:
networks:
some-network:
aliases:
- alias1
other-network:
aliases:
- alias2
networks:
some-network:
# Use a custom driver
driver: custom-driver-1
other-network:
# Use a custom driver which takes special options
driver: custom-driver-2
aliases : Other containers on the same network can use the service name or this alias to connect to the service of the corresponding container.
restart
- no: is the default restart policy, the container will not be restarted under any circumstances.
- always: the container is always restarted.
- on-failure: restart the container only when the container exits abnormally (exit status is non-zero).
- unless-stopped: always restart the container when it exits, but disregard containers that were stopped when the Docker daemon was started
restart: "no"
restart: always
restart: on-failure
restart: unless-stopped
Note: swarm cluster mode, please use restart_policy instead.
secrets
Store sensitive data, such as passwords.
version: "3.1"
services:
mysql:
image: mysql
environment:
MYSQL_ROOT_PASSWORD_FILE: /run/secrets/my_secret
secrets:
- my_secret
secrets:
my_secret:
file: . /my_secret.txt
security_opt
Modify the container’s default schema tag.
security-opt.
- label:user:USER # Set the container's user label
- label:role:ROLE # Set the container's role label
- label:type:TYPE # Set the security policy label for the container
- label:level:LEVEL # Set the container's security level label
stop_grace_period
Specifies how long to wait before sending a SIGKILL signal to shut down the container if it cannot handle SIGTERM (or any stop_signal for that matter).
stop_grace_period: 1s # wait 1 second
stop_grace_period: 1m30s # wait 1 minute 30 seconds
The default wait time is 10 seconds.
stop_signal
Set an alternative signal for stopping the container. SIGTERM is used by default.
In the following example, SIGUSR1 is used instead of SIGTERM to stop the container.
stop_signal: SIGUSR1
sysctls
Set the kernel parameters in the container, either in array or dictionary format.
sysctls:
net.core.somaxconn: 1024
net.ipv4.tcp_syncookies: 0
sysctls:
- net.core.somaxconn=1024
- net.ipv4.tcp_syncookies=0
tmpfs
Installs a temporary filesystem inside the container. Can be a single value or a list of multiple values.
tmpfs: /run
tmpfs:
- /run
- /tmp
ulimits
Override the container’s default ulimit.
ulimits:
nproc: 65535
nofile:
soft: 20000
hard: 40000
volumes
Mount the host’s data volumes or files to the container.
version: "3.7"
services:
db:
image: postgres:latest
volumes:
- "/localhost/postgres.sock:/var/run/postgres/postgres.sock"
- "/localhost/data:/var/lib/postgresql/data"