Files not being written to bound host directory, data not persisting?

Docker Desktop
,
1

I am starting up a new Gitlab container, and I thought I understood how to use a volume with a bind mount to an existing directory on the host computer, but I’m having trouble figuring out where my files are actually going, because it’s not in the directory I thought I bound to.

  • Image: gitlab/gitlab-ce, latest (eb7f8158e9e3)
  • Docker Desktop 4.45.0
  • Ubuntu 22.04.5 LTS

Steps to reproduce

Set up in ~/docker/gitlab:

.env

GITLAB_HOME='/srv/gitlab'

Excerpt from docker-compose.yaml

services: gitlab: [...] volumes: - '$GITLAB_HOME/config:/etc/gitlab:rw' - '$GITLAB_HOME/logs:/var/log/gitlab:rw' - '$GITLAB_HOME/data:/var/opt/gitlab:rw' [...]

The directory /srv/gitlab was created manually before I started the container, but it was empty, and I was letting docker create the subdirectories.

I ran:

docker compose up -d

and got an error related to file permissions. After a bit of research, I realized that locations outside of the /home directory needed to be shared explicitly. So in Docker Desktop > Settings > Resources > File Sharing, under Virtual file shares, I added /srv/github, and ran the same up command again. This time, it worked, but docker does not seem to be actually putting the shared volume data into the bound directories, as I would expect. If I look in the host system, there is nothing inside of /srv/gitlab at all.

If I inspect the running container, I see

"Mounts": { { "Type": "bind", "Source": "/srv/gitlab/data", "Destination": "/var/opt/gitlab", "Mode": "rw", "RW": "true", "propagation": "rprivate" },

…and so on for the other shares, so the definition looks like I would expect, binding to the correct host directory. But again, if I look on the host, there’s nothing there.

What’s more worrying is, after setting up gitlab with some users and some sample data, I ran into an error where it seemed I needed to restart gitlab, and ran

docker restart gitlab

…and when it game back up, it seems as if it has started from scratch, as if it were a new container? Which, of course, is a little worrying if my data is not persisting.

Where is it actually putting my files? And why was my data not persisting?

2

Docker Desktop runs its backend inside a utility vm, regardless of the host os. The utility vm under macOS and Linux requires host folders to be mapped as virtual file share into the utility vm in order to be bound as a source “host path” (which is now the virtual share inside the utility vm) into a target container path.

I don’t use Docker Desktop anywhere else than Windows, where virtual shares are not required.

May I ask why you want to run a service that is potentially supposed to run 24/7 with a container runtime that is targeted as tool for developers? It seems docker-ce is the better suited runtime for this task, unless you need Docker Desktop extension during your everyday operations, and can not live without it.

1 Like
3

Thanks for the reply! The answer is probably that I’m not deeply experienced with Docker, and I am used to running it with a GUI (mostly with Synology Container Manager, instead of Docker Desktop, but Desktop seemed similar). I’ve overloaded the resources on my Synology unit a bit, and so I’ve installed Ubuntu on another machine that I had handy, to offload some of the container workload.

I am plenty capable of setting things up via the terminal, but I do prefer a bit of a GUI when available. But if docker-ce is the better way to go here, I appreciate the pointer, and I’ll investigate that route.

Regardless, would that solve my main data storage issue here? Does docker-ce not have the same issue of that backend vm layer?

4

docker-ce runs docker on directly on the host, so a bind of a host folder actually mounts the very same inode into the container path, so that both point to the exact same location on the physical disk. The bind will work the same way you are used to it from your Synology.

If you want a ui, you might want to take a look at portainer or dockge.

1 Like
5

Thanks very much! I’ve installed docker-ce and am starting up the gitlab container now. I appreciate the GUI recommendations.

6

In case the state is still lost after a restart: use docker diff <container name or id> to see in folders the container actually writes, and then make sure to bind host folders into container folders that persist data.

Furthermore, Is it safe to assume that you added all dependent services to you compose.yaml, like a database?

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.