My Python Bazel binaries fail because there is no space left in my /tmp folder.
Unfortunately, the /tmp on my machine is mounted on a disk with very limited capacity, and for other reasons, I can not mount it on a larger disk.
--output_base is useful for the build time, but at the run-time, the binary will be unzipped on /tmp.
https://bazel.build/reference/command-line-reference#flag--output_base
Question Is there any flag similar to --output_base to tell where the binary should be unzipped when I run it in the shell?
Related
I'm trying to set up a Docker container (RHEL8) with Kaniko. IN the Dockerfile I specified to install Python3.8 and PIP3 to install Python libraries that were requested for the specific container. requirements.txt lists about 9 libraries (joblib, nltk, numpy, pandas, scikit-learn, scipy, spacy, torch, transformers), from some of which are quite large in size (for example Torch: 890M) but then, when I run
RUN python3.8 -m pip install -r requirements.txt
it runs through the requirements.txt from top to bottom, downloads them but then after the last line it also downloads a lot of other folders/packages too, some quite huge in size, like:
nvidia-cublas-cu11 : 317M
nvidia-cudnn-cu11 : 557M
It installs a lot of packages, like: MarkupSafe, blis, catalogue, certifi, charset-normalizer, click, confection, cymem, filelock, huggingface-hub, idna, jinja, langcodes, murmurash, etc.. and the list is quite impressive
I had to increase the disk size of the runner with 6G in order to even cope with the increased amount of downloaded stuff, but the build still fails upon Taking a snapshot of the full filesystem, due to running out of free disk space.
I have increased free disk space from 8G to 10G and then as a second attempt, to 14G, but the build still fails. I have also specified --single-snapshot option for Kaniko to only take one single snapshot at the end, and not creating separate snapshots at every step (RUN, COPY). I have installed an Nvidia driver to the container, for which I picked a quite lightweight one (450.102.04) which should not take up too much space either.
My question is: are the packages installed by pip3 after installing the list specified in requirements.txt basically dependencies, that I still must install, or are those optional?
Is there any option to overcome this excessing disk space issue? When I start the build process (via GitLab CI - Kaniko) the available free space on the xfs is 12G from 14G, so I should be enough, but the build fails with exit code 1 and message: "no space left on drive"
I'm trying to build chromium v8 inside a container as part of another program's build chain, and one of the binary build tools crashes because it's trying to rename/move files across the filesystem boundary that overlayfs creates for container layers.
I've opened a bug with the build tool hoping there's a way it can gracefully handle overlayfs without potentially introducing a lot of burden to the chromium team, but in the best case scenario it's going to be quite a while before that's helpful.
I have some workarounds in the meantime:
running everything that touches those files in the same layer, which means they end up being re-downloaded every time (many GBs), which is a non-starter for some maintainers on the project I'm contributing to who are on metered connections.
copying to temp, deleting original, then moving back so it's owned by the layer, which will double the size on disk, and make the build much slower
Both of the options are kind of awful.
So that's why I'm curious if there's a way to just tell Docker or overlayfs to remove or ignore the boundaries between a few layers, or to tell them to move the files into the current layer.
For example:
Moving (even after a copy) is crossing a layer boundary and and can fail with XDEV errors.
COPY ./some-file ./some-file
RUN mv ./some-file ./some-renamed-file
I can get around this, by copying it internal to a layer, so it's owned by that layer:
COPY ./some-file ./some-temporary-file-name
RUN cp ./some-temporary-file-name ./the-real-file-name \
&& rm ./some-temporary-file-name
This is significantly more problematic and resource intesive when you need to do this with huge directories of source files, and the failure is part of an external program, where the specific files that will need to be moved are non-deterministic (files named with random hashes)
RUN git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
WORKDIR /v8-temp
RUN fetch v8 \
&& mkdir /v8
# I have to duplicate the entire working directory
# since I can't know what the tool will need to move
WORKDIR /v8
RUN cp -r /v8-temp /v8 \
&& rm -rf /v8-temp \
&& gclient sync
# and if I don't clean up the image through other means
# I also have to wait for the old source files to be deleted,
# increasing the build time or image size of the result.
I have a docker file which has below command.
#Kafka log cleanup for log files older than 7 days
RUN find /opt/kafka/logs -name "*.log.*" -type f -mtime -7 -exec rm {} \;
While executing it gives an error opt/kafka/logs not found. But I can access to that directory. Any help on this is appreciated. Thank you.
Changing the contents of a directory defined with VOLUME in your Dockerfile using a RUN step will not work. The temporary container will be started with an anonymous volume and only changes to the container filesystem are saved to the image layer, not changes to the volume.
The RUN step, along with every other step in the Dockerfile, are used to build the image, and this image is the input to the container, it does not use your running containers or volumes for the build input, so it makes no sense to cleanup files that are not created as part of your image build.
If you do delete files created in your image build, you should make sure this is done within the same RUN step. Otherwise, files you delete are already written to an image layer, and are transferred and stored on disk, just not visible in containers based on the layer that includes the delete step.
I made a dockerfile like this
FROM hyeshik/tailseeker:latest
RUN rm /opt/tailseeker/conf/defaults.conf
COPY /Users/Downloads/defaults.conf /opt/tailseeker/conf/
COPY /Users/Downloads/level2/* /opt/tailseeker/refdb/level2/
COPY /Users/Downloads/level3/* /opt/tailseeker/refdb/level3/
My /Users/Downloads/ folder also has other folders named input
When I ran
docker build -f /Users/Downloads/Dockerfile /Users/Downloads/
I get an error saying
Sending build context to Docker daemon 126.8 GB
Error response from daemon: Error processing tar file(exit status 1): write /input/Logs/Log.00.xml: no space left on device
One strange thing here is why is it trying to write to the input folder? And the other one is why does it complain about no space left on device. I have a 1TB disk and only 210GB of it is used. I also used qemu-img and resized my Docker.qcow2. Here is the info of my Docker.qcow2
image:/Users/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
file format: qcow2
virtual size: 214G (229780750336 bytes)
disk size: 60G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: true
refcount bits: 16
corrupt: false
Can anyone please help me to copy the contents from my /Users/Downloads folder into the docker image by using that dockerfile above?
Thanks in advance.
build starts by creating a tarball from the context directory (in your case /Users/Downloads/) and sending that tarball to the server. The tarball is created in the tmp directory, which is probably why you're running out of space when trying to build.
When you're working with large datasets the recommended approach is to use a volume. You can use a bind mount volume to mount the files from the host.
If the files you're trying to add aren't that large, you might need to use a .dockerignore to ignore other files under /Users/Downloads.
You can also start the docker daemon with an alternative temp directory using $DOCKER_TMPDIR
I had a big file that I'm moving about. The normal protocol in the lab is to copy it somewhere and then delete it.
I decided to change it to mv.
My question is, why is mv so much faster than cp?
To test it out I generated a file 2.7 GB in size.
time cp test.txt copy.txt
Took real 0m20.113s
time mv test.txt copy.txt
Took real 0m12.403s.
TL;DR mv was almost twice as fast as copy. Any explanations? Is this an expected result?
EDIT-
I decided to move/copy the folder to a destination other than the current folder.
time cp test.txt ../copy.txt
and
time mv test.txt ../copy.txt
This time cp took 9.238s and mv took only 0.297s. So not what some of the answers were suggesting.
UPDATE
The answers are right. When I tried to mv the file to a different disk on the same system, mv and cp took almost the same time.
When you mv a file on the same filesystem, the system just has to change directory entries to reflect your renaming. Data in the file is not even read.
(same filesystem means: same directory or same directory tree/same drive, provided that source and destination directories do not traverse symlinks leading to another filesystem of course!)
When you mv a file across file systems, it has the same effect as cp + rm: no speed gain (apart from the fact that you only run one command, and consistency is guaranteed: you don't have to check if cp succeeded to perform the rm)
(older versions of mv refused to move directories across filesystems, because they only did the renaming)
Be careful, it is not equivalent. cp overwrites destination by default, whereas mv will fail renaming a file/dir into an existing file/dir.