Volume isn't created in Scylla when using docker-compose in Windows 10 - docker

Question

I just started learning Docker and docker-compose and I want to try out ScyllaDB (database). I want to start a single instance of ScyllaDB in Docker through docker-compose with persistent storage. The persistent storage should be saved in folder 'target' relative to my docker-compose file. The problem is that I don't see any folder being created, but docker-compose seems to persist the data, but I am not sure where I can locate the files that ScyllaDB created. Step by step reproduction path:
Create a docker-compose.yml with the following content (/var/lib/scylla should be correct according to https://docs.scylladb.com/operating-scylla/procedures/tips/best_practices_scylla_on_docker/):
docker-compose.yml
version: '3'
services:
b-scylla:
image: "scylladb/scylla:4.3.1"
container_name: b-scylla
volumes:
- ./target:/var/lib/scylla
ports:
- "127.0.0.1:9042:9042"
- "127.0.0.1:9160:9160"
This does not give any result: $ docker volume ls
Start up docker-compose and wait a minute for ScyllaDB to start up: $ docker-compose up -d
This does still not give any result: $ docker volume ls. I expect that Docker should created a volume (./target/).
Persist some data in ScyllaDB to verify that the data is saved somewhere:
Run the following commands:
$ docker exec -it b-scylla cqlsh
$ create keyspace somekeyspace with replication = {
'class': 'NetworkTopologyStrategy',
'replication_factor': 2
};
The created keyspace is saved somewhere, but I don't know where. I would expect it is just in the target folder, but that folder isn't even created. When I restart docker-compose, the keyspace is still present, so the data is saved somewhere, but where?

Answer 1

You are using the "short syntax" for data mounting (https://docs.docker.com/compose/compose-file/compose-file-v3/#short-syntax-3) that is creating a mount point binding. Bindings are not volumes. They can't be checked with the docker volume ls. You can find out about your mounts with docker inspect {container}.
However, Scylla image does not start for me correctly with the bind mounting. I saw constant file system errors for writing sstables in mounted directory:
version: '3'
services:
b-scylla:
image: "scylladb/scylla:4.3.1"
container_name: b-scylla
volumes:
- ./target:/var/lib/scylla
$ docker compose up -f .\test.yaml
b-scylla | INFO 2021-03-04 07:24:53,132 [shard 0] init - initializing batchlog manager
b-scylla | INFO 2021-03-04 07:24:53,135 [shard 0] legacy_schema_migrator - Moving 0 keyspaces from legacy schema tables to the new schema keyspace (system_schema)
b-scylla | INFO 2021-03-04 07:24:53,136 [shard 0] legacy_schema_migrator - Dropping legacy schema tables
b-scylla | ERROR 2021-03-04 07:24:53,168 [shard 0] table - failed to write sstable /var/lib/scylla/data/system/truncated-38c19fd0fb863310a4b70d0cc66628aa/mc-8-big-Data.db: std::system_error (error system:2, No such file or directory)
I did not find out what causes this, but the dir is writable and contains most of the normal initial data - reserved commitlog segments and system ks data folders.
What actually works is using Volumes:
version: '3'
services:
b-scylla:
image: "scylladb/scylla:4.3.1"
container_name: b-scylla
volumes:
- type: volume
source: target
target: /var/lib/scylla
volume:
nocopy: true
volumes:
target:
$ docker compose up -f .\test.yaml
$ docker volume ls
DRIVER VOLUME NAME
local 6b57922b3380d61b960110dacf8d180e663b1ce120494d7a005fc08cee475234
local ad220954e311ea4503eb3179de0d1162d2e75b73d1d9582605b4e5c0da37502d
local projects_target
$ docker volume inspect projects_target
[
{
"CreatedAt": "2021-03-04T07:20:40Z",
"Driver": "local",
"Labels": {
"com.docker.compose.project": "projects",
"com.docker.compose.version": "1.0-alpha",
"com.docker.compose.volume": "target"
},
"Mountpoint": "/var/lib/docker/volumes/projects_target/_data",
"Name": "projects_target",
"Options": null,
"Scope": "local"
}
]
And Scylla starts successfully in this mode.
You of course can mount this volume to any other container with:
$ docker run -it --mount source=projects_target,target=/app --entrypoint bash scylladb/scylla:4.3.1
or accessing it via WSL (Locating data volumes in Docker Desktop (Windows)):
$ \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\projects_target\_data

Answer 2

Turns out I needed to reset my credentials in Docker Desktop