Ivan's wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
# Docker

## What is docker?
**Docker** is a tool for managing containers. Containers are somewhat like virtual machine instances except a lot more lightweight.

Think of an application e.g. a web app. You want to run the application, but you don't want to pollute your host OS with all the baggage that comes with it (libraries, dependencies, config files, services, ...). So what do you do? You can run the application in a virtual machine.

What does it take to run an application in a virtual machine? First, you first need to set up a virtual machine. This means giving it a slice of your hardware (CPU, RAM, HDD, ...). Then, you need to install an OS on the virtual machine (let's say, Ubuntu). Then, you also need to install everything the application needs (`apt install ...`). Finally, you need to copy your application inside the virtual machine. This works, but wastes a lot of resources + the application will obviously be a lot slower than it would have been if it was ran on the host.

What if instead of emulating hardware, we just run the application we wanted on the host, but somehow made the application "feel" like it's in a virtual machine? That's exactly what Docker does! "Containerizing" an application means making the application "feel" like it's in a virtual machine, even though it's not.

But actually, it's even better! Not only can we make an application "feel" like it's in a virtual machine, we can setup the environment inside the container such that everything the application expects is already there. Meaning, inside the container: every library that the application needs is already installed, every configuration file that the application reads is already configured, every environment variable that the application expects is already set, every directory that... you get the idea. All that remains is actually *running* the application inside that preconfigured container, and the application just works out of the box.

Therefore, it's actually more appropriate to think of Docker as an "environment manager" than anything! The fact Docker containers act like lightweight virtual machines is nice to think of as an analogy, but it's not where the *true* utility of Docker lies in.

## Install
Follow the guide on https://docs.docker.com/engine/install/

On Debian and Ubuntu systems, this usually boils down to modifying `/etc/apt/sourcesl.list` then running:
```bash
 $ sudo apt update
 $ sudo apt install          \
       docker-ce             \
       docker-ce-cli         \
       containerd.io         \
       docker-buildx-plugin  \
       docker-compose-plugin
```

If you want to make a regular user be able to use `docker` command, just add them to the `docker` group:
```
 $ sudo gpasswd -a john docker
 $ sudo groups john
john : john docker
```
Remember that they will have to log out and then log back in, in order for their groups to update.   Warning: Do not do this if you don't trust the user! 

## Practice
In order to see what Docker is in practice, it's best if you try it yourself:
```
docker run -it --rm ubuntu:latest
```
This puts you inside a container, running bash shell as root. You can now use `apt` to install anything you like. You can also break the system if you like (e.g. run `rm -rf /usr`). When you exit the shell, everything you ever did inside is lost. This includes anything you installed, but also anything you created, edited, changed, removed, or broke.

When you run `docker run -it --rm ubuntu:latest` again, you're put in the exact same environment again, as if you were running it for the first time. Anything you installed is no longer there, but also anything you broke before is no longer broken (e.g. if you did `rm -rf /usr` you now do have `/usr`). You get a blank slate. The power of Docker comes from the fact it's able to efficiently replicate the exact same image environment, every single time. 

But what if you **do** want to preserve what you've installed? That's where `docker commit` and `docker build` commands come in, but that comes later.

## Concepts
### Images
Docker images are predefined environments. You run an application within these predefined environments with `docker run -it --rm <image>`. You list images with `docker images`. You remove an image with `docker rmi <image>`.  An image is usually named `<name>:<tag>`. For example, `ubuntu:latest`. The tag is usually used for different versions of an image with the same name. You retag an image `docker tag ubuntu:latest my:example`. At this point, you can run 'ubuntu:latest' or 'my:example' -- it's the same environment (makes no difference). You rename an image by retagging it and deleting the old one with `docker rmi`.

Docker images are pulled from https://hub.docker.com/. This is like GitHub for docker images.  You use `docker pull <image>` to pull an image from the repository to your local filesystem. You use `docker search <image>` to search the repository for a particular image.  Running the image will also pull the image if it doesn't exist on your filesystem.

Continuing on the virtual machine analogy, you can think of a docker image as `.iso` file, containing OS installation + system configuration. Running a container is then somewhat like starting a virtual machine, installing an OS with that `.iso`, then running a desired application within that OS.

Docker images are stored in `/var/lib/docker` directory. Docker manages images in an efficient way, using a filesystem engine called `overlay2`. Unlike `.iso` files, which are often ~4-6GB in size, docker images are usually pretty small (often several megabytes).

The following are some image related commands you should remember:

 - `docker run -it --rm <image>` - Run a docker image with image's default application (likely, the shell)
 - `docker images` - List docker images on your local filesystem
 - `docker rmi <image>` - Delete a docker image from your local filesystem
 - `docker pull <image>` - Pull a docker image from "docker.io" repository
 - `docker search <image>` - Try to find a particular image on docker.io repositories
 - `docker tag <image1> <image2>` - Retag an image e.g. 'ubuntu:latest' to 'my:image' (they refer to the same environment)

### Containers
Docker containers are particular instances of an image. If an image is the `.iso` file, a container is the running virtual machine started from it. You can run many containers from the same image simultaneously, and they do not interfere with each other.

You run a container with `docker run`. The two flags you will use most are `-it` (attach an interactive terminal) and `-d` (run detached, in the background). The `--rm` flag from the Practice section deletes the container automatically when it exits; leave it off when you want the container to stick around.

```
docker run -it ubuntu:latest          # interactive, container stays after exit
docker run -d --name myapp nginx      # detached, named "myapp"
```

Containers that are not running still exist on disk until you delete them. `docker ps` shows only running containers; `docker ps -a` shows all of them, including stopped ones. This is a common source of confusion: you exit a container, it disappears from `docker ps`, but it is still there taking up disk space.

```
docker ps           # running containers
docker ps -a        # all containers, including stopped
docker stop myapp   # send SIGTERM, wait, then SIGKILL
docker start myapp  # restart a stopped container
docker rm myapp     # delete a stopped container
docker rm -f myapp  # force-delete a running container
```

To run a command inside an already-running container:

```
docker exec -it myapp bash     # open a shell in the running container
docker exec myapp ls /var/log  # run a one-off command
```

To see what a container printed to stdout/stderr:

```
docker logs myapp         # print all output so far
docker logs -f myapp      # follow output in real time (like tail -f)
```

The following are some container-related commands you should remember:

 - `docker run -it <image>` - Start a new container from an image, attach a terminal
 - `docker run -d --name <name> <image>` - Start a container in the background
 - `docker ps` - List running containers
 - `docker ps -a` - List all containers including stopped ones
 - `docker stop <container>` - Stop a running container gracefully
 - `docker start <container>` - Restart a stopped container
 - `docker rm <container>` - Delete a stopped container
 - `docker exec -it <container> bash` - Open a shell in a running container
 - `docker logs <container>` - Print the container's output

### Volumes
By default everything inside a container is ephemeral: it exists only as long as the container exists, and disappears when you `docker rm` the container. Volumes are how you make data persist beyond the lifetime of a container.

The simplest form is a **bind mount**: you map a directory on your host into a directory inside the container. Whatever is written to that path inside the container is actually written to your host filesystem.

```
docker run -it --rm -v /home/ivan/data:/data ubuntu:latest
```

Now `/data` inside the container is actually `/home/ivan/data` on the host. Files written there survive after the container exits.

The other form is a **named volume**: Docker manages the storage location for you, and you refer to it by name.

```
docker volume create mydata
docker run -it --rm -v mydata:/data ubuntu:latest
```

Named volumes outlive any particular container. You can mount the same named volume into multiple containers (useful for sharing data between them, though you need to handle concurrent writes yourself).

```
docker volume ls                  # list named volumes
docker volume inspect mydata      # show where Docker actually stored it
docker volume rm mydata           # delete the volume and its data
```

The key commands:

 - `docker run -v /host/path:/container/path <image>` - Bind-mount a host directory
 - `docker run -v <volume-name>:/container/path <image>` - Mount a named volume
 - `docker volume create <name>` - Create a named volume
 - `docker volume ls` - List named volumes
 - `docker volume rm <name>` - Delete a named volume

### Dockerfile
So far you have been running images that other people built. A Dockerfile is how you build your own. It is a plain text file that lists the steps to construct an image, starting from a base image and layering changes on top.

```
FROM ubuntu:22.04

RUN apt-get update && apt-get install -y gcc make

COPY . /app
WORKDIR /app

RUN make

CMD ["./myprogram"]
```

Each instruction (`FROM`, `RUN`, `COPY`, ...) adds a layer on top of the previous one. Docker caches layers: if you rebuild and only the last `RUN` changed, Docker reuses all the cached layers up to that point and only re-executes from the change onward. This is why `apt-get update && apt-get install` should always be in one `RUN` instruction — splitting them means the install can run against a stale cached update layer.

You build the image with `docker build`:

```
docker build -t myimage:latest .    # build from Dockerfile in current directory
docker build -t myimage:latest -f path/to/Dockerfile .
```

The `.` at the end is the **build context**: the directory Docker sends to the daemon to use in `COPY` instructions. Keep large files out of it (or list them in `.dockerignore`) because the entire build context is transferred on every build.

Common Dockerfile instructions:

 - `FROM <image>` - Set the base image; every Dockerfile starts with this
 - `RUN <command>` - Execute a shell command during build; result is baked into the layer
 - `COPY <src> <dst>` - Copy files from the build context into the image
 - `WORKDIR <path>` - Set the working directory for subsequent instructions and the container's default shell
 - `ENV <key>=<value>` - Set an environment variable in the image
 - `EXPOSE <port>` - Document which port the container listens on (does not publish it)
 - `CMD ["executable", "arg"]` - The default command to run when the container starts

The following are some build-related commands:

 - `docker build -t <name>:<tag> .` - Build an image from a Dockerfile
 - `docker commit <container> <name>:<tag>` - Turn a modified container into a new image (quick and dirty alternative to a Dockerfile)
 - `docker history <image>` - Show the layers that make up an image

### docker compose
Real applications rarely run as a single container. A web application might need a container for the app itself, one for the database, and one for a cache. Docker Compose lets you define all of them in a single `docker-compose.yml` file and bring them all up with one command.

```yaml
# docker-compose.yml
services:
  web:
    build: .
    ports:
      - "8080:80"
    depends_on:
      - db
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: secret
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:
```

```
docker compose up        # build images (if needed) and start all containers
docker compose up -d     # same, but detached
docker compose down      # stop and remove all containers (volumes are kept)
docker compose down -v   # also delete volumes
docker compose logs -f   # follow logs from all services
docker compose exec web bash   # open a shell in the running "web" container
```

Compose automatically creates a shared network for all services, so the `web` container can reach the database simply by using the service name `db` as the hostname. Service names in `docker-compose.yml` are the DNS names inside the Compose network.

The key commands:

 - `docker compose up` - Start all services defined in `docker-compose.yml`
 - `docker compose down` - Stop and remove all services
 - `docker compose ps` - Show the status of all services
 - `docker compose logs -f` - Follow logs from all services
 - `docker compose exec <service> bash` - Open a shell in a running service container
 - `docker compose build` - Rebuild images without starting containers