How to copy folder from parent into current directory for Dockerfile using Makefile - docker

I have a makefile that looks like this:
push:
docker build -t dataengineering/dataloader .
docker tag dataengineering/dataloader:latest 127579856528.dkr.ecr.us-west-2.amazonaws.com/dataengineering/dataloader:latest
docker push 127579856528.dkr.ecr.us-west-2.amazonaws.com/dataengineering/dataloader:latest
deploy:
#if [ ! "$(environment)" ]; then echo "environment must be defined" && exit 1; fi
#if [ ! "$(target)" ]; then echo "target must be defined" && exit 1; fi
kubectl delete deploy dataloader-$(target) -n dataengineering|| continue
kubectl apply -f kube/$(environment)/deployment-$(target).yaml -n dataengineering
But I need a folder inside the dataloader in order for my dockerfile to actually work.
Does this work?
push:
cd ..; cp -r datastore/ dataloader/
docker build -t dataengineering/dataloader .
docker tag dataengineering/dataloader:latest 1111111111.dkr.ecr.us-west-2.amazonaws.com/dataengineering/dataloader:latest
docker push 11111111111.dkr.ecr.us-west-2.amazonaws.com/dataengineering/dataloader:latest
deploy:
#if [ ! "$(environment)" ]; then echo "environment must be defined" && exit 1; fi
#if [ ! "$(target)" ]; then echo "target must be defined" && exit 1; fi
kubectl delete deploy dataloader-$(target) -n dataengineering|| continue
kubectl apply -f kube/$(environment)/deployment-$(target).yaml -n dataengineering
My dockerfile:
FROM python:3.7
WORKDIR /var/dataloader
COPY assertions/ ./assertions/
...
COPY datastore/ ./datastore/
RUN pip3 install -r requirements.txt
ENTRYPOINT ["python", "dataloader.py"]

If all you need is to copy the directory into the current directory (which would server as your Docker context), you can use cp -r ../datastore/ dataloader/. Unless you want the dataloader directory to be in the same directory as the datastore directory, then you'd do cp -r ../datastore/ ../dataloader/.

Related

Docker shows me an error of COPY of to fix?

I'm using this container to set up X11 in GitPod.
ARG base
FROM ${base}
# Dazzle does not rebuild a layer until one of its lines are changed. Increase this counter to rebuild this layer.
ENV TRIGGER_REBUILD=1
# Install Xvfb, JavaFX-helpers and Openbox window manager
RUN sudo install-packages xvfb x11vnc xterm openjfx libopenjfx-java openbox
# Overwrite this env variable to use a different window manager
ENV WINDOW_MANAGER="openbox"
USER root
# Change the default number of virtual desktops from 4 to 1 (footgun)
RUN sed -ri "s/<number>4<\/number>/<number>1<\/number>/" /etc/xdg/openbox/rc.xml
# Install novnc
RUN git clone --depth 1 https://github.com/novnc/noVNC.git /opt/novnc \
&& git clone --depth 1 https://github.com/novnc/websockify /opt/novnc/utils/websockify
COPY novnc-index.html /opt/novnc/index.html
# Add VNC startup script
COPY start-vnc-session.sh /usr/bin/
RUN chmod +x /usr/bin/start-vnc-session.sh
USER gitpod
# This is a bit of a hack. At the moment we have no means of starting background
# tasks from a Dockerfile. This workaround checks, on each bashrc eval, if the X
# server is running on screen 0, and if not starts Xvfb, x11vnc and novnc.
RUN echo "export DISPLAY=:0" >> /home/gitpod/.bashrc.d/300-vnc
RUN echo "[ ! -e /tmp/.X0-lock ] && (/usr/bin/start-vnc-session.sh &> /tmp/display-\${DISPLAY}.log)" >> /home/gitpod/.bashrc.d/300-vnc
USER root
### checks ###
# no root-owned files in the home directory
RUN notOwnedFile=$(find . -not "(" -user gitpod -and -group gitpod ")" -print -quit) \
&& { [ -z "$notOwnedFile" ] \
|| { echo "Error: not all files/dirs in $HOME are owned by 'gitpod' user & group"; exit 1; } }
USER gitpod
This is where it gets sketchy :
# Install novnc
RUN git clone --depth 1 https://github.com/novnc/noVNC.git /opt/novnc \
&& git clone --depth 1 https://github.com/novnc/websockify /opt/novnc/utils/websockify
COPY novnc-index.html /opt/novnc/index.html
I get this output please help !
COPY failed: file not found in build context or excluded by .dockerignore: stat novnc-index.html: file does not exist
Knowing that my dockerfile is in /src and i'm building in /src . I tried to rebuild with the --no-cache flag and use export DOCKER_BUILDKIT=1 . But still I'm stuck with this problem .

How to run an sh script in docker file?

When running a sh script in docker file, i got the following error:
./upload.sh: 5: ./upload.sh: sudo: not found ./upload.sh: 21:
./upload.sh: Bad substitution
sudo chmod 755 upload.sh # line 5
version=$(git rev-parse --short HEAD)
echo "version $version"
echo "Uploading file"
for path in $(find public/files -name "*.txt"); do
echo "path $path"
WORDTOREMOVE="public/"
echo "WORDTOREMOVE $WORDTOREMOVE"
# cause of the error
newpath=${path//$WORDTOREMOVE/} # Line 21
echo "new path $path"
url=http://localhost:3000/${newpath}
...
echo "Uploading file"
...
done
DockerFile
FROM node:10-slim
EXPOSE 3000 4001
WORKDIR /prod/code
...
COPY . .
RUN ./upload.sh
RUN npm run build
CMD ./DockerRun.sh
Any idea?
If anyone faces the same issue, here how I fixed it
chmod +x upload.sh
git update-index --chmod=+x upload.sh (mandatory if you pushed the file to remote branch before changing its permission)
The docker image you are using (node:10-slim) has no sudo installed on it because this docker image runs processes as user root:
docker run -it node:10-slim bash
root#68dcffceb88c:/# id
uid=0(root) gid=0(root) groups=0(root)
root#68dcffceb88c:/# which sudo
root#68dcffceb88c:/#
When your Dockerfile runs RUN ./upload.sh it will run:
sudo chmod 755 upload.sh
Using sudo inside the docker fails because sudo is not installed, there is no need to use sudo inside the docker because all of the commands inside the docker run as user root.
Simply remove the sudo from line number 5.
If you wish to update the running PATH variable run:
PATH=$PATH:/directorytoadd/bin
This will append the directory "/directorytoadd/bin" to the current path.

docker - pass arguments to the script during build

I would like to pass argument (from the docker command) to the shell script inside the Dockerfile.
This is the docker command line.
docker build --file=DockerfileTest --build-arg docker_image=PX-release-migration --tag=test-image:latest --rm=true .
This is a script that is called inside the Dockerfile.
#!/bin/sh -e
image_name=$1
echo "docker image is $image_name"
if [[ ($image_name == '') || ($image_name == *"-dev-"*) ]]; then
echo "This is development"
cp src/main/resources/application-dev.properties src/main/resources/application.properties
elif [[ $image_name == *"-preprod-"* ]]; then
echo "This is preprod"
cp src/main/resources/application-stg.properties src/main/resources/application.properties
elif [[ $image_name == *"-release-"* ]]; then
echo "This is production"
cp src/main/resources/application-prod.properties src/main/resources/application.properties
fi
When I execute separately the script, it works, but it doe
This is docker file.
ARG spring_env=local
ARG docker_image=-local-
FROM maven:3.6.1-jdk-8
COPY . /apps/demo
WORKDIR /apps/demo
RUN chmod +x /apps/demo/initialize_env.sh
RUN ./initialize_env.sh $docker_image
RUN echo "spring_env is ${spring_env}"
So basically, i would like to use a different spring application properties file during the build depending on the docker_image name. If a docker image name contains 'release', i would like to package application-prod.properties during the build.
This is the error message that I am getting.
Step 1/8 : ARG spring_env=local
Step 2/8 : ARG docker_image=-local-
Step 3/8 : FROM maven:3.6.1-jdk-8
---> 4c81be38db66
Step 4/8 : COPY . /apps/demo
---> 41439197c465
Step 5/8 : WORKDIR /apps/demo
---> Running in 56bd408c2eb1
Removing intermediate container 56bd408c2eb1
---> 4c4025bf5f64
Step 6/8 : RUN chmod +x /apps/demo/initialize_env.sh
---> Running in 18dc3a5c1a54
Removing intermediate container 18dc3a5c1a54
---> 60d2037a0209
Step 7/8 : RUN ./initialize_env.sh $docker_image
---> Running in 2e049b2cf630
docker image is
./initialize_env.sh: 5: ./initialize_env.sh: Syntax error: word unexpected (expecting ")")
The command '/bin/sh -c ./initialize_env.sh $docker_image' returned a non-zero code: 2
When I execute separately the script, it works, but it doesn't inside the docker container.
Tip: Use ShellCheck to check scripts for syntax errors.
#!/bin/sh -e
if [[ ($image_name == '') || ($image_name == *"-dev-"*) ]]; then
[[ is bash syntax but your script is declared to use plain sh. It works on your machine presumably because sh is really symlinked to bash, but inside the container that's not the case. maven:3.6.1-jdk-8 is based on debian:stretch which uses dash instead of bash.
Change the shebang line. You can also delete the parentheses; they're superfluous.
#!/bin/bash -e
if [[ $image_name == '' || $image_name == *"-dev-"* ]]; then
You could also use a case block to simplify the repetitive checks.
case "$image_name" in
''|*-dev-*)
echo "This is development"
cp src/main/resources/application-dev.properties src/main/resources/application.properties
;;
*-preprod-*)
echo "This is preprod"
cp src/main/resources/application-stg.properties src/main/resources/application.properties
;;
*-release-*)
echo "This is production"
cp src/main/resources/application-prod.properties src/main/resources/application.properties
;;
esac

Docker - make ADD command conditional on command line argument

Can ADD command be customised to work with command line argument.
Basically i have different versions of a file let's say file1 and file2, and based on some condition passed through command line.
The following command works correctly and transfers the file from host to docker, but i didn't find any references to do it conditionally.
ADD target/file.yml file.yml
No, ADD does not support the conditional way of copying files.
But there is a way to deal with such configuration while coping from Host.
Copy all configuration to some temp location in your case copy all file1 file2 to some /temp location, then base on ARG pass to docker build, move the file to target.
Or do the above using docker entrypoint based on ENV, instead of base on ARG
FROM alpine
ADD target/ /temp_target
RUN mkdir /target
#filename to be copied to the target
ARG file_name
# pass update_file true or false if true file wil be update to new that is passed to build-args
ARG update_file
ENV file_name=$file_name
ARG update_file=$update_file
#check if update_file is set and its value is true then copy the file from temp to target, else copy file1 to target
RUN if [ ! -z "$update_file" ] && [ "${update_file}" == true ];then \
echo "$update_file"; \
echo "echo file in temp_target"; \
ls /temp_target ;\
echo "updating file target" ;\
cp /temp_target/"${file_name}" /target/ ; \
echo "list of updated files in target"; \
ls /target ; \
else \
echo "copy with default file that is ${file_name}" ;\
cp /temp_target/file1 /target ;\
fi
Build:
This will not updated file, will copy with default filename that is file1
docker build --no-cache --build-arg file_name=file1 --build-arg update_file=false -t abc .
This will update file in the target, so the new file will be in the target is file2.
docker build --no-cache --build-arg file_name=file2 --build-arg update_file=true -t abc .

Docker: Jar: not found

I am new to Docker and working on developing the Docker image for our application in Ubuntu environment.
However, the below command is not working when executed from within the Dockerfile/ from within the docker-entrypoint file.
command: “jar xf ./abc.ear”
_/docker-entrypoint.sh: 69: /docker-entrypoint.sh: jar: not found**
I verified and ear file is present in the directory.
I tried passing the full path to ear or passed the full path to Jar command, however, no success.
Please help.
...............
#!/bin/sh
set -e
start=$(date +'%s')
# Setting Environment Variables
DEPLOY_DIR=/home/docker/xyz
SCRIPT_DIR=/usr/local/src
if [ "$(ls -A $DEPLOY_DIR/Install 2> /dev/null)" = "" ]; then
echo "The directory $DEPLOY_DIR/Install is empty."
# Fetch Installable from Artifactory
echo "[INFO] Extracting files from Artifactory"
mkdir -p $DEPLOY_DIR
cd $DEPLOY_DIR
wget -nv ArtifactoryPath
unzip "123.zip" -d $DEPLOY_DIR
# Install
cd $DEPLOY_DIR/
Install command
# Configure JAVA
echo "[INFO] Linking java folder"
ln -s /usr/lib/jvm/java-8-oracle $DEPLOY_DIR/Install/jdk
# Explode ear and war files
echo "[INFO] Explode ear and war files\n"
cd $DEPLOY_DIR/Install/jboss/deployments
ls -al
mv "$WFC_DEPLOY_DIR/Install/jboss/deployments/abc.ear" "$WFC_DEPLOY_DIR/Install/jboss/deployments/abc-old.ear"
mkdir -p abc.ear
cd $DEPLOY_DIR/Install/jboss/deployments/abc.ear
echo $PWD
mv "$DEPLOY_DIR/Install/jboss/deployments/abc-old.ear" ./
ls -al
jar xvf "$DEPLOY_DIR/Install/jboss/deployments/abc.ear/abc-old.ear"
rm -rf $DEPLOY_DIR/Install/jboss/deployments/abc.ear/abc-old.ear
else
echo "$DEPLOY_DIR/Install is not empty."
fi

Resources