docker

What is docker?

Docker is a tool for managing containers. Containers are somewhat like virtual machine instances except a lot more lightweight.

Think of an application e.g. a web app. You want to run the application, but you don't want to pollute your host OS with all the baggage that comes with it (libraries, dependencies, config files, services, …). So what do you do? You can run the application in a virtual machine.

What does it take to run an application in a virtual machine? First, you first need to set up a virtual machine. This means giving it a slice of your hardware (CPU, RAM, HDD, …). Then, you need to install an OS on the virtual machine (let's say, Ubuntu). Then, you also need to install everything the application needs (apt install ...). Finally, you need to copy your application inside the virtual machine. This works, but wastes a lot of resources + the application will obviously be a lot slower than it would have been if it was ran on the host.

What if instead of emulating hardware, we just run the application we wanted on the host, but somehow made the application “feel” like it's in a virtual machine? That's exactly what Docker does! “Containerizing” an application means making the application “feel” like it's in a virtual machine, even though it's not.

But actually, it's even better! Not only can we make an application “feel” like it's in a virtual machine, we can setup the environment inside the container such that everything the application expects is already there. Meaning, inside the container: every library that the application needs is already installed, every configuration file that the application reads is already configured, every environment variable that the application expects is already set, every directory that… you get the idea. All that remains is actually running the application inside that preconfigured container, and the application just works out of the box.

Therefore, it's actually more appropriate to think of Docker as an “environment manager” than anything! The fact Docker containers act like lightweight virtual machines is nice to think of as an analogy, but it's not where the true utility of Docker lies in.

Install

Follow the guide on https://docs.docker.com/engine/install/

On Debian and Ubuntu systems, this usually boils down to modifying /etc/apt/sourcesl.list then running:

 $ sudo apt update
 $ sudo apt install          \
       docker-ce             \
       docker-ce-cli         \
       containerd.io         \
       docker-buildx-plugin  \
       docker-compose-plugin

If you want to make a regular user be able to use docker command, just add them to the docker group:

 $ sudo gpasswd -a john docker
 $ sudo groups john
john : john docker

Remember that they will have to log out and then log back in, in order for their groups to update. Warning: Do not do this if you don't trust the user!

Practice

In order to see what Docker is in practice, it's best if you try it yourself:

docker run -it --rm ubuntu:latest

This puts you inside a container, running bash shell as root. You can now use apt to install anything you like. You can also break the system if you like (e.g. run rm -rf /usr). When you exit the shell, everything you ever did inside is lost. This includes anything you installed, but also anything you created, edited, changed, removed, or broke.

When you run docker run -it --rm ubuntu:latest again, you're put in the exact same environment again, as if you were running it for the first time. Anything you installed is no longer there, but also anything you broke before is no longer broken (e.g. if you did rm -rf /usr you now do have /usr). You get a blank slate. The power of Docker comes from the fact it's able to efficiently replicate the exact same image environment, every single time.

But what if you do want to preserve what you've installed? That's where docker commit and docker build commands come in, but that comes later.

Concepts

Images

Docker images are predefined environments. You run an application within these predefined environments with docker run -it --rm <image>. You list images with docker images. You remove an image with docker rmi <image>. An image is usually named <name>:<tag>. For example, ubuntu:latest. The tag is usually used for different versions of an image with the same name. You retag an image docker tag ubuntu:latest my:example. At this point, you can run 'ubuntu:latest' or 'my:example' – it's the same environment (makes no difference). You rename an image by retagging it and deleting the old one with docker rmi.

Docker images are pulled from https://hub.docker.com/. This is like GitHub for docker images. You use docker pull <image> to pull an image from the repository to your local filesystem. You use docker search <image> to search the repository for a particular image. Running the image will also pull the image if it doesn't exist on your filesystem.

Continuing on the virtual machine analogy, you can think of a docker image as .iso file, containing OS installation + system configuration. Running a container is then somewhat like starting a virtual machine, installing an OS with that .iso, then running a desired application within that OS.

Docker images are stored in /var/lib/docker directory. Docker manages images in an efficient way, using a filesystem engine called overlay2. Unlike .iso files, which are often ~4-6GB in size, docker images are usually pretty small (often several megabytes).

The following are some image related commands you should remember:

docker run -it --rm <image> - Run a docker image with image's default application (likely, the shell)
docker images - List docker images on your local filesystem
docker rmi <image> - Delete a docker image from your local filesystem
docker pull <image> - Pull a docker image from “docker.io” repository
docker search <image> - Try to find a particular image on docker.io repositories
docker tag <image1> <image2> - Retag an image e.g. 'ubuntu:latest' to 'my:image' (they refer to the same environment)

Containers

Docker containers are particular instances of an image. If an image is the .iso file, a container is the running virtual machine started from it. You can run many containers from the same image simultaneously, and they do not interfere with each other.

You run a container with docker run. The two flags you will use most are -it (attach an interactive terminal) and -d (run detached, in the background). The --rm flag from the Practice section deletes the container automatically when it exits; leave it off when you want the container to stick around.

docker run -it ubuntu:latest          # interactive, container stays after exit
docker run -d --name myapp nginx      # detached, named "myapp"

Containers that are not running still exist on disk until you delete them. docker ps shows only running containers; docker ps -a shows all of them, including stopped ones. This is a common source of confusion: you exit a container, it disappears from docker ps, but it is still there taking up disk space.

docker ps           # running containers
docker ps -a        # all containers, including stopped
docker stop myapp   # send SIGTERM, wait, then SIGKILL
docker start myapp  # restart a stopped container
docker rm myapp     # delete a stopped container
docker rm -f myapp  # force-delete a running container

To run a command inside an already-running container:

docker exec -it myapp bash     # open a shell in the running container
docker exec myapp ls /var/log  # run a one-off command

To see what a container printed to stdout/stderr:

docker logs myapp         # print all output so far
docker logs -f myapp      # follow output in real time (like tail -f)

The following are some container-related commands you should remember:

docker run -it <image> - Start a new container from an image, attach a terminal
docker run -d --name <name> <image> - Start a container in the background
docker ps - List running containers
docker ps -a - List all containers including stopped ones
docker stop <container> - Stop a running container gracefully
docker start <container> - Restart a stopped container
docker rm <container> - Delete a stopped container
docker exec -it <container> bash - Open a shell in a running container
docker logs <container> - Print the container's output

Volumes

By default everything inside a container is ephemeral: it exists only as long as the container exists, and disappears when you docker rm the container. Volumes are how you make data persist beyond the lifetime of a container.

The simplest form is a bind mount: you map a directory on your host into a directory inside the container. Whatever is written to that path inside the container is actually written to your host filesystem.

docker run -it --rm -v /home/ivan/data:/data ubuntu:latest

Now /data inside the container is actually /home/ivan/data on the host. Files written there survive after the container exits.

The other form is a named volume: Docker manages the storage location for you, and you refer to it by name.

docker volume create mydata
docker run -it --rm -v mydata:/data ubuntu:latest

Named volumes outlive any particular container. You can mount the same named volume into multiple containers (useful for sharing data between them, though you need to handle concurrent writes yourself).

docker volume ls                  # list named volumes
docker volume inspect mydata      # show where Docker actually stored it
docker volume rm mydata           # delete the volume and its data

The key commands:

docker run -v /host/path:/container/path <image> - Bind-mount a host directory
docker run -v <volume-name>:/container/path <image> - Mount a named volume
docker volume create <name> - Create a named volume
docker volume ls - List named volumes
docker volume rm <name> - Delete a named volume

Dockerfile

So far you have been running images that other people built. A Dockerfile is how you build your own. It is a plain text file that lists the steps to construct an image, starting from a base image and layering changes on top.

FROM ubuntu:22.04

RUN apt-get update && apt-get install -y gcc make

COPY . /app
WORKDIR /app

RUN make

CMD ["./myprogram"]

Each instruction (FROM, RUN, COPY, …) adds a layer on top of the previous one. Docker caches layers: if you rebuild and only the last RUN changed, Docker reuses all the cached layers up to that point and only re-executes from the change onward. This is why apt-get update && apt-get install should always be in one RUN instruction — splitting them means the install can run against a stale cached update layer.

You build the image with docker build:

docker build -t myimage:latest .    # build from Dockerfile in current directory
docker build -t myimage:latest -f path/to/Dockerfile .

The . at the end is the build context: the directory Docker sends to the daemon to use in COPY instructions. Keep large files out of it (or list them in .dockerignore) because the entire build context is transferred on every build.

Common Dockerfile instructions:

FROM <image> - Set the base image; every Dockerfile starts with this
RUN <command> - Execute a shell command during build; result is baked into the layer
COPY <src> <dst> - Copy files from the build context into the image
WORKDIR <path> - Set the working directory for subsequent instructions and the container's default shell
ENV <key>=<value> - Set an environment variable in the image
EXPOSE <port> - Document which port the container listens on (does not publish it)
CMD ["executable", "arg"] - The default command to run when the container starts

The following are some build-related commands:

docker build -t <name>:<tag> . - Build an image from a Dockerfile
docker commit <container> <name>:<tag> - Turn a modified container into a new image (quick and dirty alternative to a Dockerfile)
docker history <image> - Show the layers that make up an image

docker compose

Real applications rarely run as a single container. A web application might need a container for the app itself, one for the database, and one for a cache. Docker Compose lets you define all of them in a single docker-compose.yml file and bring them all up with one command.

# docker-compose.yml
services:
  web:
    build: .
    ports:
      - "8080:80"
    depends_on:
      - db
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

docker compose up        # build images (if needed) and start all containers
docker compose up -d     # same, but detached
docker compose down      # stop and remove all containers (volumes are kept)
docker compose down -v   # also delete volumes
docker compose logs -f   # follow logs from all services
docker compose exec web bash   # open a shell in the running "web" container

Compose automatically creates a shared network for all services, so the web container can reach the database simply by using the service name db as the hostname. Service names in docker-compose.yml are the DNS names inside the Compose network.