Building docker image that downloads large file fails with error - docker

Question

I'm new to docker. I've been trying it for less than two weeks. We have a service that we'd like to migrate into a container. The service makes use of about 50G worth of data, so we expect the image to be very large. We've written a Dockerfile for it. When we run the build it fails with the ff:
ApplyLayer exit status 1 stdout: stderr: write /mnt/spine_features/spine_features_subset.lmdb/data.mdb: input/output error
When we check docker ps -a for containers, we can see the build container listed with status:
Exited (1) About a minute ago
When we try to commit the container, we get the same error:
Error response from daemon: ApplyLayer exit status 1 stdout: stderr: write /mnt/spine_features/spine_features_subset.lmdb/data.mdb: input/output error
We can also docker inspect the container. When we exclude downloading the largest files, we are able to complete building the service image. Is there some sort of configuration we can change succeed while still including the larger files?
docker version
Client:
Version: 1.12.1
API version: 1.24
Go version: go1.7
Git commit: 23cf638
Built: Fri Aug 19 02:03:02 2016
OS/Arch: linux/amd64
Server:
Version: 1.12.1
API version: 1.24
Go version: go1.7
Git commit: 23cf638
Built: Fri Aug 19 02:03:02 2016
OS/Arch: linux/amd64
docker info
Containers: 2
Running: 0
Paused: 0
Stopped: 2
Images: 22
Server Version: 1.12.1
Storage Driver: devicemapper
Pool Name: docker-8:2-7603782-pool
Pool Blocksize: 65.54 kB
Base Device Size: 214.7 GB
Backing Filesystem: xfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 67.78 GB
Data Space Total: 107.4 GB
Data Space Available: 39.59 GB
Metadata Space Used: 37.04 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.11 GB
Thin Pool Minimum Free Space: 10.74 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.134 (2016-09-07)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: null host bridge overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.7.4-1-ARCH
Operating System: Arch Linux
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 15.58 GiB
Name: mega-haro
ID: MDQ5:JIT3:BVQX:XYO6:YTXI:HTRE:N2UQ:ML4V:ENIE:DDCO:ZGYF:3P5F
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
misty:5000
127.0.0.0/8

Answer 1

As Haoming Zhang recommended, mounting the host directory into the container is an acceptable solution. We are also exploring the possibility of using FUSE to load the data into the container at runtime instead of baking it into the image during build or having the host pass it into the container when the container is run.