Hello could someone pleas help me on copying docker(I'm starter) host file into my jupyter/pyspark-notebook images. I've pulled this notebook from docker as public available.
I've created a Dockerfile which contains this.
FROM jupyter/pyspark-notebook:latest
ADD /home/abdoulaye/Documents/M2BIGDATA/Jaziri /bin/bash
I've changed /bin/bash to . but nothing is visible.
when I execute docker built it's like it copy files as shown in output below.
when I go to my my notebook I did note found folders. I check my snapshot if I can found these copied folder but I'm very confused.
In clear, I've an running notebook in my docker I use it in y navigator but I can not load data. I like to copy data in place where I can access it in notebook.
You can not copy using absoult path, the path should be relative to Dockerfile, so /home/abdoulaye/Documents/M2BIGDATA/Jaziri this path inside Dockerfile is not correct. Copy file to Dockerfile context and then copy like
ADD M2BIGDATA/Jaziri /work
Now First thing, you should not copy files from host to executable files directory.
For instance,
FROM alpine
copy hello.txt /bin/sh
If you copy like this, it will create a problem to run command inside container as sh or bash will be replaced or corrupt.
2nd, while you are building the docker image with invalid context, it should be the same where your Dockerfile is, so better to run the directory where you place the Dockerfile.
docker build -t my-jupyter .
3rd, you should not run cp command inside container to copy files from host to container.
docker cp /home/abdoulaye/Documents/M2BIGDATA/Jaziri container_id:/work
it will copy your files to /work path of the container.
Related
I'm a new leaner of docker.I came a cross a problem while I'm trying to make my own docker image.
Here's the thing.I create a new DockerFile to build my own mysql image in which I declared MYSQL_ROOT_PASSWORD and put some init scripts in the container.
Here is my Docker
FROM mysql:5.7
MAINTAINER CarbonFace<553127022#qq.com>
ENV TZ Asia/Shanghai
ENV MYSQL_ROOT_PASSWORD Carbon#mysqlRoot7
ENV INIT_DATA_DIR /initData/sql
ENV INIT_SQL_FILE_0 privileges.sql
ENV INIT_SQL_FILE_1 carbon_user_sql.sql
ENV INIT_SQL_FILE_2 carbonface_sql.sql
COPY ./my.cnf /etc/mysql/donf.d/
RUN mkdir -p $INIT_DATA_DIR
COPY ./sqlscript/$INIT_SQL_FILE_0 $INIT_DATA_DIR/
COPY ./sqlscript/$INIT_SQL_FILE_1 $INIT_DATA_DIR/
COPY ./sqlscript/$INIT_SQL_FILE_2 $INIT_DATA_DIR/
COPY ./sqlscript/$INIT_SQL_FILE_0 /docker-entrypoint-initdb.d/
COPY ./sqlscript/$INIT_SQL_FILE_1 /docker-entrypoint-initdb.d/
COPY ./sqlscript/$INIT_SQL_FILE_2 /docker-entrypoint-initdb.d/
CMD ["mysqld"]
I'm trying to build a docker image which contains my own config file and when mounted it would be showed in the local directory and can be modified.
I'm really confused that when I start my container with this image like the official description and also here is my commands:
docker run -dp 3306:3306 \
-v /usr/local/mysql/data:/var/lib/mysql \
-v/usr/local/mysql/conf:/etc/mysql/conf.d \
--name mysql mysql:<my builded tag>
You know I'm trying to mounted the
/usr/local/mysql/conf to the /etc/mysql/conf.d in the container which is been told as the custom config file mounted location.
And I supposed that my custom config file my.cnf which has been copied into the image during docker build and would be show in my local direcroty /usr/local/mysql/conf
And since I already copied my custom config file into image which you can see in my DockerFile.
But it turns out that the directory is empty and the /etc/mysql/conf.d is also overwrite by local directory.
Before I run my container, both /usr/local/mysql/conf and /usr/local/mysql/data is empty at all.
OK fine, I've been told that the volume mounted directory would overwrite the file inside the container.
But how could the empty data directory shows the data files inside the container but the empty conf directory overwrite the conf.d directory in the container.
It make no sense.
I was very confused and I would be very appreciate it if someone can explain why it happens.
My OS is MacOS Big Sur and I used the latest docker.
A host-directory bind mount, -v /host/path:/container/path, always hides the contents of the image and replaces it with the host directory. If the host directory is empty, the container directory will be the same empty directory at container startup time.
The Docker Hub mysql container has an involved entrypoint script that checks to see if the data directory is empty, and if so, initializes the database; abstracted out
#!/bin/sh
# (actually in hundreds of lines of shell code, with more options)
if [ ! -d /var/lib/mysql/data/mysql ]; then
mysql_install_db
# (...and start a temporary database server and run the
# /docker-entrypoint-initdb.d scripts)
fi
# then run the main container command
exec "$#"
Simply the presence of a volume doesn't cause files to be copied (with one exception at one specific point in the lifecycle for named volumes), so if you need to copy content from a container to the host you either need to do it manually with docker cp or have a way in the container code to do it.
I am using COPY command in my docker file on top of ubuntu 16.04. I am getting error as no such file or directory eventhough the directory is present. In the below docker file I want to copy the directory "auth" present inside workspace directory to the docker image (at path /home/ubuntu) and then build the image.
FROM ubuntu:16.04
RUN apt-get update
COPY /home/ubuntu/authentication/workspace /home/ubuntu
WORKDIR /home/ubuntu/auth
a Dockerfile COPY command can only refer to files under the context - the current location of the Dockerfile, aka .
so you have a few options now:
if it is possible to copy the /home/ubuntu/authentication/workspace/ directory content to somewhere inside your project before the build (so now it will be included in your Dockerfile context and you can access it via COPY ./path/to/content /home/ubuntu) it can be great. but sometimes you dont want it.
instead of copying the directory, bind it to your container via a volume:
when you run the container, add a -v option:
docker run [....] -v /home/ubuntu/authentication/workspace:/home/ubuntu [...]
mind that a volume is designed so any change you made inside the container dir(/home/ubuntu) will affect the bound directory on your host side (/home/ubuntu/authentication/workspace) and vice versa.
i found a something over here: this guy is forcing the Dockerfile to accept his context- he is sitting inside the /home/ubuntu/authentication/workspace/ directory, and running there
docker build . -f /path/to/Dockerfile
so now inside his Dockerfile he can refer to /home/ubuntu/authentication/workspace as his context (.)
I want to copy my compiled war file to tomcat deployment folder in a Docker container. As COPY and ADD deals with moving files from host to container, I tried
RUN mv /tmp/projects/myproject/target/myproject.war /usr/local/tomcat/webapps/
as a modification to the answer for this question. But I am getting the error
mv: cannot stat ΓÇÿ/tmp/projects/myproject/target/myproject.warΓÇÖ: No such file or directory
How can I copy from one folder to another in the same container?
You can create a multi-stage build:
https://docs.docker.com/develop/develop-images/multistage-build/
Build the .war file in the first stage and name the stage e.g. build, like that:
FROM my-fancy-sdk as build
RUN my-fancy-build #result is your myproject.war
Then in the second stage:
FROM my-fancy-sdk as build2
COPY --from=build /tmp/projects/myproject/target/myproject.war /usr/local/tomcat/webapps/
A better solution would be to use volumes to bind individual war files inside docker container as done here.
Why your command fails
The command you are running tries to access files which are out of context to for the dockerfile. When you build the image using docker build . the daemon sends context to the builder and only those files are accessible during the build. In docker build . the context is ., the current directory. Therefore, it will not be able to access /tmp/projects/myproject/target/myproject.war.
Copying from inside the container
Another option would be to copy while you are inside the container. First use volumes to mount the local folder inside the container and then go inside the container using docker exec -it <container_name> bash and then copy the required files.
Recommendation
But still, I highly recommend to use
docker run -v "/tmp/projects/myproject/target/myproject.war:/usr/local/tomcat/webapps/myproject.war" <image_name>
I have a simple Dockerfile:
FROM php:7.1-apache
LABEL maintainer="rburton#agsource.com"
COPY C:/Users/rburton/code/MyAgsourceAPI /var/www
It is the last line that is giving me problems. I am copying from a Windows structure to a docker container (Linux I assume). When I build this image I get:
...
Step 3/3 : COPY C:/Users/rburton/code/MyAgsourceAPI /var/www
COPY failed: stat /var/lib/docker/tmp/dockerbuilder720374851/C:/Users/rburton/code/MyAgsourceAPI: no such file or directory
First, something is preventing the recognition that this is an absolute path and naturally if the path is pre-pended with /var/lib/docker/tmp/dockerbuilder720374851 then the file will not be found. Second, I have tried / and \ but all with the same result. Also the drive letter I suspect is confusing to docker. So the question is how do I copy files and folders (along with the contents) from a Windows folder to a docker container?
First, change your Dockerfile to:
FROM php:7.1-apache
LABEL maintainer="rburton#agsource.com"
COPY MyAgsourceAPI /var/www
Then, to go your code directory: cd Users/rburton/code.
Within that directory, run:
docker build -t <image_name> .
Another tip that might be helpful, I've seen same issue while running build from correct context, and issue remained until I've used all small caps on src folder that I wanted to copy from. eg:
COPY MyAgsourceAPI /var/www -> COPY myagsourceapi /var/www
The root of the path is relative to the Dockerfile and not your Windows filesystem.
If for example your filesystem is layed out like this:
+-+-MyProject
|
+---Dockerfile
|
+-+-build
|
+---MyAgsourceAPI
You can use:
COPY /build/MyAgsourceAPI /var/www
Note that "MyProject" (or anything above it) is excluded from the path.
RUN cp /data/ /data/db, this command does not copy the files in /data to /data/db.
Is there an alternate way to do this?
It depends where /data is for you: already in the image, or on your host disk.
A Dockerfile RUN command execute any commands in a new layer on top of the current image and commit the results.
That means /data is the one found in the image as built so far.
Not the /data on your disk.
If you want to copy from your disk to the image /data/db folder, you would need to use COPY or ADD.
At runtime, when you had an existing running container, you could also use docker cp to copy from or to a container.