Series on Docker in Practice & Production - Using Docker Compose to Start Multiple Services

Problem

docker run only allows starting one container at a time, which can be challenging and complex when your application needs to start multiple containers simultaneously. Additionally, manually executing individual commands can be error-prone. Imagine having a stack with multiple images - should you run docker run n times?

Docker Compose was created to address this problem. It stores a set of instructions on how to run multiple containers in a .yml/.yaml file (docker-compose.yml). With just one command, you can start all the containers together.

Docker Compose

Docker Compose used to be a plugin, but it has recently been integrated into Docker for Windows and macOS. However, it is still not available on Linux. If you are using Linux, you need to install Docker Compose separately.

Refer to this link for various ways to install Docker Compose.

For Linux users, simply use the following commands:

$ sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
$ sudo chmod +x /usr/local/bin/docker-compose
$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

# Check the version
$ docker-compose --version
docker-compose version 1.29.2, build 1110ad01

To get started, let's say you need to start 2 services: a Node.js API and a MySQL server. Create a docker-compose.yml file:

version: "3.9"
services:  
  node:  
    image: my-node-app
    environment:  
      - MYSQL_HOST: mysql
      - MYSQL_USER: hoaitx
      - MYSQL_PASSWORD: password
    ports:  
      - "3000:3000"
    networks:  
      - my_app
    depends_on:  
      - mysql

  mysql:  
    image: mysql
    environment:  
      - MYSQL_USER=hoaitx
      - MYSQL_PASSWORD=password
    networks:  
      - my_app
    volumes:  
      - ./data:/var/lib/mysql

networks:  
  my_app:  

Then use the docker-compose command to start them.

$ docker-compose up -d

To see the list of running services:

$ docker-compose ps

Note: the above docker-compose commands should be executed in the directory containing the docker-compose.yml file.

Similar to Dockerfile, a YAML file for Docker Compose also consists of a set of commands. Docker Compose continuously adds new commands in subsequent Docker versions, so make sure to check the version in the YAML file with your Docker version before using them.

Components

A YAML file typically contains version and services. The version specifies which version of the YAML file you are using, while services list the services that will be run. It also includes networks that contain the networks used in the file.

Here is an example of a YAML file version 3.9 that runs Node.js and MySQL concurrently:

version: "3.9"
services:  
  node:  
    image: my-node-app
    environment:  
      - MYSQL_HOST: mysql
      - MYSQL_USER: hoaitx
      - MYSQL_PASSWORD: password
    ports:  
      - "3000:3000"
    networks:  
      - my_app
    depends_on:  
      - mysql

  mysql:  
    image: mysql
    environment:  
      - MYSQL_USER=hoaitx
      - MYSQL_PASSWORD=password
    networks:  
      - my_app
    volumes:  
      - ./data:/var/lib/mysql

networks:  
  my_app:  

Tips: YAML files use spaces for indentation, so be careful when building this file. A misplaced line can cause docker-compose to fail.

• version specifies the version of the YAML file.
• services contains the services to be run.
• networks contains network configurations.

For each service, we will have a name, which is not only the service name but also the domain used for services to call each other. In the example above, Node.js calls MySQL using the name "mysql" (MYSQL_HOST: mysql).

• image specifies the image to be used in the container.
• environment sets the environment variables.
• ports maps the ports from inside the container to the host machine.
• volumes mounts a directory from the host machine to the Docker container.
• networks declares the networks that the container can join.
• depends_on sets the dependencies between services, meaning a service will only be initialized once the other service is running.

Apart from the above commands, Docker Compose also supports many other commands. It is not possible to list them all here, so you can find the documentation here. Here are a couple of notable commands:

• build replaces image. Typically, you need a specific image to start a container (service) using the docker build command. With build, you no longer need to build an image separately, as Docker Compose will build it when starting. See more details here.
• deploy is a useful command that is only applicable when using Docker Swarm. Generally, deploy is where you gather the instructions on how your services should be run. For example, you can configure the number of replicas, specify running on specific nodes, automatically restart the service if it fails, etc.
• There are many more commands that you can explore here.

Compose CLI

Knowing the docker-compose commands will help you easily deploy your applications. Below are some basic commands:

To view the help guide:

$ docker-compose --help

Check the Docker Compose version:

$ docker-compose --version
docker-compose version 1.29.2, build 1110ad01

To deploy, navigate to the directory containing the .yml file:

$ docker-compose up -d

To update changes from the .yml file, rerun the deploy command, which will check for changes and update accordingly.

View the list of services:

$ docker-compose ps

Check the logs of the services:

$ docker-compose logs

To clean up:

$ docker-compose down

Conclusion

Docker Compose allows us to run multiple services (containers) simultaneously. It stores the instructions in a .yml/.yaml file commonly named docker-compose.yml.

Docker Compose is suitable for small projects that do not require scaling across nodes. To scale, Docker provides a feature called swarm. I will discuss Docker swarm in more detail in the next article.