Docker Workshop

Learn the basics of Docker with this hands-on workshop

By William Guimont-Martin • 15 minutes read •


Table of Contents

  1. Setup
    1. Instructions
    2. Solutions
  2. Docker
    1. Why containerization?
    2. What is containerization?
  3. Containers manually
  4. Docker basics
    1. Exercise 01 – Running a prebuilt image
    2. Exercise 02 – Container management
    3. Exercise 03 – Inspecting a running container
    4. Exercise 04 – PostgreSQL database
  5. Dockerfile
    1. Exercise 05 – Haskell service Dockerfile
    2. Exercise 06 – Python application Dockerfile
  6. Advanced Docker
    1. Multi-stage image builds
    2. Exercise 07 – Rust multi-stage build
    3. Volumes and bind mounts
  7. Extras
    1. Cleanup
    2. Docker Compose
    3. Podman
    4. Kubernetes

Docker and containerization have revolutionized the way applications are developed, deployed, and managed. Containers provide a lightweight and efficient way to package applications and their dependencies, ensuring consistency across different environments.

Setup

The original workshop was designed to use with a virtual machine (willGuimont/IFT2001-Docker). While you can follow the instructions below to set up the virtual machine, you can also choose to follow the workshop using Docker.

Follow these steps to prepare your environment.

  1. Install Docker
  2. docker run -it --rm ubuntu:latest bash
  3. In the Docker container, run:
    1. apt update && apt install -y git
    2. mkdir .atelier && cd .atelier && git clone https://github.com/willGuimont/IFT2001-Docker ateliers-iftglo-2001
    3. cd ateliers-iftglo-2001
    4. ./helper/install.sh
    5. python3 ./helper/prepare_bashrc.py && source ~/.bashrc
    6. Copy the files in your home directory: cd ~/.atelier/ateliers-iftglo-2001/files && cp -r . ~/

Instructions

The instruction above sets up your environment for the workshop. The grading commands are of the form correction_nn, where nn is an integer representing the exercise number, for example correction_03 for exercise 3. Instructions for validating commands will be provided with the first question that has an expected result.

To complete this workshop, you must perform each exercise directly in the terminal. You will use the nano text editor to modify the requested scripts. To open a file with nano, run the following in the terminal.

nano test.sh

You can then edit the file; enter

echo 'Hello world'

To exit nano, press CTRL-X, then answer y to save the changes. Run the following commands to validate that everything works:

chmod +x test.sh
./test.sh

These commands should print Hello world.

Adventurers only

If you are comfortable with the command line, we encourage you to try completing this workshop using vim as your text editor. A brief introduction to vim commands is available on MIT’s The Missing Semester. vim offers many shortcuts to edit text and code very efficiently. As a programmer, you will spend a lot of time writing code, so investing in learning vim is worthwhile for the rest of your career. There are plugins emulating vim commands for most IDEs: Vim for VS Code or IdeaVim for JetBrains products.

Please note that no assistance regarding vim will be provided during this workshop.

Solutions

Proposed solutions to the exercises in these workshops are available in the following GitHub repository: ulavalIFTGLOateliers/IFT2001-Docker.

Docker

After your success with Bash, your manager has given you a new mission: find a solution to the application deployment problem. Your company frequently encounters outages due to incompatibilities between different dependency versions. Indeed, each developer can choose the operating system they wish to work on. Consequently, some developers use different Linux distributions (Ubuntu, Arch, Gentoo), while others prefer Windows.

This situation often leads to version issues, as dependencies vary from one OS to another. And worse, the versions used differ from those deployed on the production server. The recurring excuse “It worked on my machine” when a developer causes a production outage has finally exasperated your manager. He is asking you to find a solution.

That’s how you discover Docker, a platform that lets you build, deploy, and run applications in lightweight, isolated containers. With Docker, you can create a specific container for each microservice, including all required libraries and dependencies. This approach ensures that each application will run consistently, regardless of library versions used by individual developers. Moreover, Docker allows you to reproduce the development environment on the production server, thereby eliminating issues stemming from environment differences.

With Docker, you will tackle the challenge of running your company’s three main microservices. Here are the microservices you must get working:

  1. a web server health monitoring application written in Haskell;
  2. a document management API written in Rust, requiring a PostgreSQL database; and
  3. a Python application for visualizing web server health.

Why containerization?

Isolation
One of the main advantages of containerization is isolation. It often happens that different applications require specific dependency versions. With Docker, you can install different dependency versions in distinct containers, without risking conflicts or compromising other applications. Furthermore, if you need to run applications on different operating systems, such as Ubuntu and Arch Linux, you can simply wrap them in Docker containers, avoiding the need to rewrite applications for each system.

Portability
Portability is another key advantage of Docker. Once you have set up a development environment with all required dependencies, reproducing that configuration on other machines or for new team members becomes tedious. Thanks to Docker, you can create an image that contains all required dependencies and configuration, which can be easily distributed and run on different platforms, whether Linux, Windows, or others. This enables a new team member to start quickly without spending weeks configuring their environment. It’s also an advantage when deploying applications to a server.

Reproducibility
Reproducibility is also simplified with Docker. Docker images are versioned, which means it is easy to reproduce builds identically, ensuring that development, test, and production environments are consistent. You just need to specify the version of the Docker image used to guarantee consistency and avoid issues related to variations across environments.

Efficiency
In terms of efficiency, Docker offers a significant advantage over virtual machines (VMs). Unlike VMs, Docker does not need to run a full operating system for each container, which greatly reduces resource consumption. Docker containers share the host kernel, making them lightweight and quick to start.

Community
Docker’s surrounding community is very active and offers a multitude of preconfigured images for various popular tools and technologies such as Node.js, Python, and many others. These preconfigured images facilitate deployment and usage of these technologies, saving developers time by avoiding manual environment setup.

Industry
Finally, it’s important to emphasize that Docker has become a de facto standard in the software development industry. Many companies use Docker for development, testing, and deployment of their applications. Docker is now an essential tool for software developers.

What is containerization?

Docker is an open-source software platform that lets you create, deploy, and run applications in lightweight, isolated containers. A Docker container is a runtime unit that encapsulates an application together with all of its necessary elements, such as libraries, dependencies, and configuration files. These containers are self-contained and portable, meaning they can run consistently across different systems, whether a development, test, or production environment.

Docker is similar to a virtual machine that isolates an application from the host system. However, Docker is much more efficient than a VM, which requires a full operating system for each instance. Docker instead shares the host operating system’s kernel, saving resources and making containers faster to start and run.

Here is some Docker terminology. You do not have to read the advanced sections to complete this workshop. As a supplement, you can watch the video “Never install locally”.

Docker Image
An image contains the filesystem and user-space components required by the application: binaries, libraries, dependencies, configuration files, and often a Linux distribution’s userland. It does not contain its own Linux kernel. Docker images are typically created from files called Dockerfile that specify the steps to build the image.

Dockerfile
A Dockerfile is a text file that contains the instructions to build a Docker image. It specifies the image layers, dependencies to install, files to include, and commands to run during image construction.

Docker Container
A Docker container is a running instance of a Docker image. It is an isolated environment that runs the application with its dependencies. Containers are lightweight, portable, and self-contained, which makes them easy to deploy across different machines.

Docker Registry
A Docker registry is a centralized repository that stores and manages Docker images. The default public registry is Docker Hub, where you can find many ready-to-use images. You can also create and use your own private registry to store your own images.

Union filesystem (advanced concepts)
The Union Filesystem, also known as UnionFS or OverlayFS, is a technology used by Docker to manage images and container layers efficiently. The Union Filesystem allows you to overlay multiple file systems into a single logical view without physically merging them. This means Docker images and containers can share and reuse common file layers, saving storage space. It also speeds up image builds by caching layers already built. When a container starts, a new read-write layer is added on top of the image layers, enabling container-specific changes without affecting others.

cgroups (advanced concepts)
cgroups, or control groups, are a Linux kernel feature used by Docker to limit and manage system resources used by containers. cgroups allow control of resources such as CPU, memory, disk bandwidth, and network, ensuring balanced and fair resource usage among containers. Docker uses cgroups to set limits and quotas on resources allocated to each container, ensuring isolation and predictable performance.

Namespaces (advanced concepts)
Namespaces are a Linux kernel feature that isolates system resources among processes. Docker uses several types of namespaces to provide isolation between containers, including the PID namespace (process isolation), the network namespace (network isolation), the user namespace (user isolation), and the mount namespace (mount point isolation). These namespaces ensure that each container has its own isolated view of the system, preventing processes in one container from interfering with others or with the host system.

Chroot jail (advanced concepts)
Chroot, or change root, is a Unix/Linux feature that changes a process’s root directory and limits its access to the file system. Docker uses chroot to create an isolated environment inside the container, where the container’s root directory becomes the starting point for all file paths. This limits the container’s access to files and directories outside its isolated environment, thereby strengthening security and isolation.

Containers manually

To clearly illustrate what Docker does, we will manually build a simplified container without using Docker. This section is inspired by p8952/bocker: Docker implemented in around 100 lines of bash. In a terminal, enter the following commands:

# Verify that neofetch is not installed
# The following command should raise an error; do not install the package
neofetch

# Folder for containers
mkdir ~/containers; cd ~/containers

# Build the container's filesystem
sudo apt update; sudo apt install -y debootstrap
sudo debootstrap jammy ./ubuntu-container http://archive.ubuntu.com/ubuntu/

# Create an isolated environment (namespace)
sudo unshare --uts --pid --mount --ipc --fork

# Mount process, system, device, and temp folders
mount -t proc none ./ubuntu-container/proc/
mount -t sysfs none ./ubuntu-container/sys
mount -o bind /dev ./ubuntu-container/dev
mount -o bind /tmp ./ubuntu-container/tmp/
cp /etc/apt/sources.list ./ubuntu-container/etc/apt/sources.list

# Use chroot to launch a shell in the container
chroot ./ubuntu-container/ /bin/bash
# and there you go, you're in an isolated container

# Install neofetch
apt update
apt install -y neofetch

# Test neofetch; the command works!
neofetch

# You can run commands in the container here.

# Exit the container
exit

# Exit unshare
exit

# If we try neofetch again, the command is not found.
# We have indeed isolated the container!
neofetch

As you can see, there is no magic in containers. There is no magic in a container: at its core, Linux provides mechanisms such as namespaces, filesystem isolation, and cgroups. Docker packages these mechanisms behind a convenient image, networking, storage, and lifecycle-management interface.

Docker basics

Exercise 01 – Running a prebuilt image

Run the Docker image hello-world.

To run a Docker image, use the run command.

docker run image-name

If the command works, you will see a message displayed.

Solution 
docker run hello-world

To execute a command in the container, specify the command at the end of run. In the following examples, we pass bash -c 'echo "Hello world"' which interprets the string in the Bash interpreter.

# Prints 'Hello world' in an ubuntu container
docker run ubuntu bash -c 'echo "Hello world"'
# Prints information about the host system
cat /etc/os-release
# Prints information from a container running Arch Linux
docker run archlinux bash -c 'cat /etc/os-release'

To launch an interactive command (such as a shell), use the -it flag. The --rm flag removes the container once it exits.

docker run -it --rm archlinux bash

Exercise 02 – Container management

To see running containers: docker ps.

To stop a container: docker stop container_id.

Note that the command may take some time to run.

Tasks:

  1. Launch an interactive terminal with Docker;
  2. Open another terminal and inspect running containers;
  3. Stop the Docker container from the second terminal.
Solution 
# In the first terminal
docker run -it archlinux bash

# In the second terminal
docker ps
docker stop id
# or
docker rm -f id

Exercise 03 – Inspecting a running container

Containers often run in the background. Docker provides several commands to inspect and interact with running containers.

To display the output produced by a container:

docker logs container_name

To execute a command inside a running container:

docker exec container_name command

For example, to open an interactive shell:

docker exec -it container_name bash

Finally, docker inspect displays detailed information about a Docker object, including its network configuration, mounted filesystems, environment variables, and process configuration:

docker inspect container_name

Launch an nginx container in the background:

docker run --name web-server -d -p 8080:80 nginx

Tasks:

  1. Verify that the container is running.
  2. Display the container’s logs.
  3. Open an interactive shell inside the running container.
  4. From inside the container, inspect the files in /usr/share/nginx/html.
  5. Use docker inspect to find the container’s IP address.
  6. Stop and remove the container.
Solution 

# Verify that the container is running
docker ps

# Display its logs
docker logs web-server

# Execute Bash inside the existing container
docker exec -it web-server bash

# Inside the container
ls /usr/share/nginx/html
exit

# Inspect the container
docker inspect web-server
# Find the IP address
docker inspect \
    --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' \
    web-server

# Stop and remove it
docker stop web-server
docker rm web-server

docker run creates a new container, whereas docker exec executes another process inside an already running container. This distinction is useful when debugging services running in the background.

Exercise 04 – PostgreSQL database

To set environment variables in a container, use the -e flag.

docker run -it --rm -e HELLO=hello archlinux sh -c 'echo $HELLO'

To open a network port, use the -p host:docker flag, where host is the port number on the host and docker is the port number inside the container.

docker run -p 127.0.0.1:8080:80 nginx
# You can access the port via the URL http://localhost:8080/
curl http://localhost:8080/

Launch a PostgreSQL database with Docker.

Requirements:

Solution 
docker run -e POSTGRES_PASSWORD=postgres -p 5432:5432 --rm postgres
# or
docker run --name postgres -e POSTGRES_PASSWORD=postgres -p 5432:5432 --rm -d postgres

Dockerfile

Dockerfiles are configuration files used to create custom Docker images. They allow you to define, reproducibly and automatically, an application’s runtime environment inside a Docker container.

A Dockerfile contains a series of instructions that specify the steps needed to build a Docker image. These instructions include actions such as selecting the base image, installing dependencies, configuring environment variables, copying files, running commands, and more.

Here is an annotated example Dockerfile for a Python application.

# Use the Python 3.9 slim base image
FROM python:3.9-slim

# Set the working directory inside the container
# Specifies the directory for RUN, CMD, ENTRYPOINT, COPY, and ADD
WORKDIR /app

# Copy requirements.txt from the host into the Docker container
COPY requirements.txt .

# Run a Bash command to install Python dependencies
RUN pip install -r requirements.txt

# Copy the source code into the Docker container
COPY . .

# Set the default command when the container starts
CMD ["python", "app.py"]

To build the image, use docker build -t <tag> .. The -t <tag> argument gives the image the name <tag>. For the previous application, you could use docker build -t python-app .. Note that the order of instructions matters.

It is crucial to define the order of operations in a Dockerfile due to how Docker builds images. Each instruction in the Dockerfile creates a new layer in the Docker image, and the order can significantly impact build efficiency and performance.

Docker uses a cache to speed up image builds. When you run an instruction, Docker checks whether that instruction has already been executed in a previous layer. If so and the parameters are identical, Docker reuses the cached layer instead of rebuilding it. This saves build time. However, if you modify an instruction higher up in the Dockerfile, all subsequent instructions will have their cache invalidated and must be rebuilt. For example, if you perform resource-intensive operations such as compiling code that changes whenever you modify the source, it can be preferable to place those steps towards the end of the Dockerfile to benefit from caching as much as possible.

Exercise 05 – Haskell service Dockerfile

Use the README.md in the folder ~/Applications/status-checker to write a Dockerfile that runs the Haskell application.

Requirements:

Then, start the container exposing port 8080. If everything works, you should be able to run:

curl http://localhost:8080/endpoints
# Expected response
{}%
# Or
{}

Leave this container running.

Solution

Dockerfile:

FROM haskell:9.0-buster
RUN stack setup --install-ghc
WORKDIR /app
COPY . .
RUN stack build
CMD ["stack", "run"]

Execution:

docker build -t status-checker .
docker run --rm -p 8080:8080 status-checker

Exercise 06 – Python application Dockerfile

Use the README.md in the folder ~/Applications/python_app to write a Dockerfile that runs the Python application.

Then, launch the application interactively and with the correct network mode.

This application will need to access your computer’s local network to make requests to the container from the previous exercise; to keep this exercise simple, use --network="host" when running the container.

Host networking makes the container share the host’s network stack. This is convenient here, but it reduces network isolation and is not the usual way for containers to communicate with each other. Later, Docker Compose will place the containers on a shared Docker network, where services can communicate directly using their service names.

If everything works, you should be able to use the application to add URLs to monitor and see the results.

Solution

Dockerfile:

FROM python:3.9-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "main.py"]

Execution:

docker build -t python_app .
docker run --rm -it --network=host python_app

Advanced Docker

Sometimes, you may need to create more complex Docker images. Here, we will explore two advanced Docker features: multi-stage builds and volumes.

Multi-stage image builds

Multi-stage builds are an advanced Docker feature that enables optimized image creation using several distinct stages in the build process. This separates build and production stages, often resulting in final images that are lighter and more secure by minimizing what remains in the final image.

For example, for a JavaScript React application, we can split the build into two stages.

Create the application:

npx create-react-app my-app

Dockerfile:

# Stage 1: Build the application

# Use `as build` to name this build stage
FROM node:14-alpine as build
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build

# Stage 2: Prepare the server
# Here, we start a new image from scratch
FROM nginx:alpine
# Copy the compiled app from the previous stage
COPY --from=build /app/build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

To run:

docker build -t web-app .
docker run --rm -p 80:80 web-app

Exercise 07 – Rust multi-stage build

Use the README.md in the folder ~/Applications/rust_api to write a Dockerfile that runs the application.

Requirements:

Solution 
FROM rust:latest AS build
WORKDIR /app
COPY . .
RUN cargo build --release

FROM debian:bullseye-slim
RUN apt update; apt install -y libpq-dev
WORKDIR /app
COPY --from=build /app/target/release/rust_api ./rust_api
ENV DATABASE_URL=postgres://postgres:postgres@db
CMD ./rust_api

Execution

docker run -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres
docker build -t rust_api .
docker run --rm --network=host rust_api

# Validate that the server works
curl http://localhost:8081/documents

Volumes and bind mounts

Containers have a writable filesystem, but this filesystem belongs to the container. If the container is removed, the data written inside it is also removed. Docker provides mounts to store data outside the container’s filesystem.

There are two common types of mounts: volumes and bind mounts.

A volume is storage managed by Docker. You give the volume a name, and Docker decides where the data is stored on the host:

docker run -v database-data:/var/lib/postgresql/data postgres

Here, database-data is a Docker volume and /var/lib/postgresql/data is the directory where PostgreSQL sees it inside the container. The data remains available even if the container is removed. Volumes are therefore particularly useful for persistent application data such as databases.

A bind mount directly maps an existing file or directory from the host filesystem into the container:

docker run -v ./src:/app/src my-app

Here, ./src is a directory on the host and /app/src is where it appears inside the container. Changes made on either side are immediately visible on the other side.

Volumes are generally appropriate for persistent application data. Bind mounts are useful when the container must directly access files from the host, for example source code, configuration files, or datasets.

For example, we can persist PostgreSQL data using a named volume:

docker run \
    -e POSTGRES_PASSWORD=postgres \
    -p 5432:5432 \
    --rm \
    -v postgres-data:/var/lib/postgresql/data \
    postgres

Docker creates and manages postgres-data. Removing the container does not remove the volume.

During development, we can instead use a bind mount to expose the Python application’s source code directly inside the container. This avoids rebuilding the image whenever the source code changes.

A DockerfileDev can therefore omit the source code:

FROM python:3.9-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
# We removed the line COPY . .
CMD ["bash"]

Build and run it with:

docker build -t python_app -f DockerfileDev . 
docker run --rm -it -v .:/app python_app
# In the container
python main.py
# Modify the source code
python main.py
# Changes are reflected in the container

Thus, you can reuse the same image even if the source code has changed.

Extras

Some additional Docker features and tools that may be useful in your projects.

Cleanup

When using Docker, it is important to consider the storage used by images, containers, volumes, and other Docker artifacts. Poor storage management can lead to excessive disk usage and make Docker maintenance and resource management difficult. Docker provides several prune commands to remove unused containers and images. To clean up various Docker artifacts, the command docker system prune -a -f removes all unused resources.

Docker Compose

Docker Compose is a tool that allows you to easily define and manage multi-container applications. It simplifies deployment and orchestration of Docker containers using a simple, readable configuration file.

With Docker Compose, you can specify services, networks, volumes, and other configurations needed to run an application composed of multiple containers. You can also define dependencies among containers, environment variables, exposed ports, etc.

Docker Compose uses a YAML configuration file to describe the application’s infrastructure. This file contains sections such as services, networks, volumes, etc., where you can define the different parts of your application and their configurations.

Here is an example docker-compose.yml for the Rust application that launches both the server and the PostgreSQL database:

# List of containers to launch
services:
  # Database container
  # We find many of the same parameters as with `docker run`
  db:
    # Image to launch
    image: postgres:18
    # Restart the database if the container stops
    restart: always
    # Environment variables passed to PostgreSQL
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
    # Store the database outside the container using a named volume
    volumes:
      - db:/var/lib/postgresql/data
    # Test whether PostgreSQL is ready to accept connections
    # Starting a container does not necessarily mean that the
    # application inside it is ready yet.
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5
      start_period: 5s
  
  # Rust server
  api:
    # Build the Dockerfile in the directory containing compose.yaml
    build: .
    # Start the API only once the database healthcheck succeeds
    depends_on:
      db:
        condition: service_healthy
    # Map host port 8081 to container port 8081
    ports:
      - "8081:8081"
    # Containers in the same Compose application share a network.
    # Services can reach each other using their service name.
    # Therefore, the database is available at hostname `db`
    # rather than `localhost`.
    environment:
      DATABASE_URL: postgres://postgres:postgres@db

# Named volumes used by the application
volumes:
  db:
    driver: local

Run with:

docker compose up --build

Compose creates a default network for the application. The api container can therefore resolve db as a hostname and connect directly to the PostgreSQL container.

depends_on normally controls startup order only. By combining it with the database healthcheck and condition: service_healthy, Compose waits until PostgreSQL is actually ready to accept connections before starting the API.

Podman

Podman is an open-source alternative to Docker. Docker and Podman are two popular containerization tools that share similar features but differ in architecture and security approach. Docker uses a client-server architecture, where the Docker daemon runs as a separate process and Docker commands are executed via the CLI. In contrast, Podman uses a daemonless architecture, meaning it runs directly as a regular user and does not require a separate daemon process. Thus, Podman can run without special privileges, limiting potential risks associated with running as superuser. See What is podman? for more information.

Podman also offers tools to manage pods, a topic outside the scope of this workshop.

Kubernetes

Kubernetes is an open-source container orchestration system that facilitates the deployment, management, and scaling of containerized applications. Using Kubernetes offers many benefits, such as fault tolerance, easier deployments, and management and scaling of containerized applications.