I would lIke to ask how to add (copy) the Maven project dependencies to maven docker image (https://hub.docker.com/_/maven) which is running during the CI/CD in the sandbox without access to the internet?
I tried the following approach but it seems not to be working.
The dependencies are stored in the ~/.m2 directory.
Is a better approach to copy dependency folders to Maven image or use the following: command for copy project .pom file in the Dockerfile?
RUN mvn -B -f /tmp/pom.xml -s /usr/share/maven/ref/settings-docker.xml dependency:resolve
Many Thanks for any advice.
# Pull base image.
FROM library/maven
############################
# Install Dependencies
#COPY pom.xml /tmp/pom.xml
RUN pwd
ADD repository /usr/share/maven/ref/repository
#RUN mvn -B -f /tmp/pom.xml -s /usr/share/maven/ref/settings-docker.xml dependency:resolve
#RUN mvn clean install -o
#RUN mvn dependency:go-offline
# Define default command.
#CMD ["bash"]
############################
You can run maven in offline mode mvn -o install
Related
My dockerfile is like.
FROM maven:3.6.0-jdk-8-alpine
RUN apk add curl jq
#copying src from my framework
COPY src /home/SeleniumFramework/src/
#copying pom.xml of my framework
COPY pom.xml /home/SeleniumFramework/
RUN mvn -f /home/SeleniumFramework dependency:go-offline
#copying testng.xml of my framework
COPY testng.xml /home/SeleniumFramework/
ADD healthcheck.sh healthcheck.sh
#Running the suite
CMD mvn -f /home/SeleniumFramework/pom.xml clean test
I build an image and run using docker run -it mayankluckym/selenium-5 , after execution Report and target folders are not generating.
but if I am using docker run -it --entrypoint sh mayankluckym/selenium-5
then I move into
/ # cd /home/SeleniumFramework/
and I run Dockerfile Entrypoint on / # cd /home/SeleniumFramework/ path
/ # cd /home/SeleniumFramework/mvn -f /home/SeleniumFramework/pom.xml clean test
in that case Reports and target folders got generated.
I started down this path: Slow gradle build in Docker. Caching gradle build
FROM gradle:5.6.4-jdk11 as cache
RUN mkdir -p /home/gradle/cache_home
ENV GRADLE_USER_HOME /home/gradle/cache_home
COPY build.gradle /home/gradle/java-code/
WORKDIR /home/gradle/java-code
RUN gradle clean build -i --stacktrace
....
But I'm using a container for building and running tests. It seems like the gradle test command wants to re download all dependencies even though I already built and copied over the dependencies folder.
I tried this:
ARG GRADLE_IMAGE=gradle:6.3.0-jdk11
FROM ${GRADLE_IMAGE} as cached_deps
RUN mkdir -p /home/gradle/cache_home
ENV GRADLE_USER_HOME /home/gradle/cache_home
ADD gradle /home/gradle/project/gradle
COPY build.gradle gradlew settings.gradle /home/gradle/project/
WORKDIR /home/gradle/project
RUN ./gradlew clean build test -i --stacktrace
FROM ${GRADLE_IMAGE} as build_scripts
COPY --from=cached_deps /home/gradle/cache_home /home/gradle/.gradle
COPY . /usr/src/project/
WORKDIR /usr/src/project
Now if I run and exec into this container and run ./gradlew test -i --stacktrace I see all of this:
Downloading https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar to /tmp/gradle_download18403789671596645805bin
Downloading https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar to /tmp/gradle_download1760192958465004068bin
Downloading https://repo.jenkins-ci.org/releases/org/jenkins-ci/commons-jexl/1.1-jenkins-20111212/commons-jexl-1.1-jenkins-20111212.jar to /tmp/gradle_download3145068843648248657bin
Downloading https://repo.maven.apache.org/maven2/net/sf/ezmorph/ezmorph/1.0.6/ezmorph-1.0.6.jar to /tmp/gradle_download17358018284924937151bin
Downloading https://repo.maven.apache.org/maven2/commons-lang/commons-lang/2.5/commons-lang-2.5.jar to /tmp/gradle_download13077847091207454426bin
Downloading https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/workflow/workflow-step-api/625.vd896b_f445a_f8/workflow-step-api-625.vd896b_f445a_f8.jar to /tmp/gradle_download10404796874240754081bin
Downloading https://repo.maven.apache.org/maven2/org/apache/ant/ant-launcher/1.10.12/ant-launcher-1.10.12.pom to /tmp/gradle_download4624045316930218421bin
Downloading https://repo.maven.apache.org/maven2/org/apache/ant/ant-junit/1.10.12/ant-junit-1.10.12.pom to /tmp/gradle_download7473104223985039780bin
Downloading https://repo.maven.apache.org/maven2/org/apache/ant/ant-antlr/1.10.12/ant-antlr-1.10.12.pom to /tmp/gradle_download7676119557444436230bin
Why does it have to download anything? I already ran build and test so it should have copied over the build and test compile dependencies.
I'm using clusters of my corporation by ICM. It provides a convenient way to configure the remote by docker:
So, I want to build a docker image of my developing environment (python packages, cuda, some utility scripts like screen and rsync, also some necessary data) to deploy on the remote machine. Here is my Dockerfile:
# syntax=docker/dockerfile:1
FROM pytorch/pytorch:1.7.1-cuda11.0-cudnn8-runtime
WORKDIR /app
RUN sudo apt-get install rsync
RUN sudo apt-get install screen
RUN conda create --prefix /data/vxxx/nn python=3.8
RUN conda init
# shotcut for activating my environment
RUN echo 'alias nn="conda activate /data/xxx/nn"' >> ~/.bashrc
RUN source ~/.bashrc
RUN nn
RUN pip3 install torchtext==0.8.1 pandas scipy scikit-learn transformers tensroboard -f https://download.pytorch.org/whl/torch_stable.html
# copy files from windows
COPY /mnt/c/test .
RUN cd /data
RUN mkdir xxx
RUN cd xxx
RUN mkdir Data
RUN mkdir Code
RUN cd Code
RUN git clone https://github.com/namespace-Pt/Document-Reduction.git
RUN git config --global user.name 'xxx'
RUN git config --global user.email 'xxx#1.com'
CMD [ "sleep", "infinity"]
I'm new to docker and I followed the official python image tutorial, I have the following questions:
what is WORKDIR, does it mean to create a new directory where all files will be stored?
why my COPY command is not working?
how to publish my image to make it usable for the cluster?
Beginner here, from what i understood.
WORKDIR Is work directory for other commands as RUN, CMD, COPY or ADD.
Don't know. I will double check directories paths.
Don't know. Normally i'am using dockerhub
I have created a docker file in which I have installed golang dep tool which will be used to install the dependencies required by golang project. I have been able to install the tool. But unable to install the dependencies using that tool. I am not sure how to configure dep tool to be able to run dep command in docker image which will install all the dependencies required by golang project
I am using below command to run dep tool and it works in local machine
# initialize the project and install dependencies
RUN dep init
I am always getting an error:
init failed: unable to determine the import path for the root project
/go: /go is not within any GOPATH/src
Now I do not know if I need to set path to o binary files or how I can achieve that. There are tutorials to build a docker file to build golang project but nothing is there on internet to install dependencies using golang dep tool.
Here is an example of Dockerfile with dep:
FROM golang:latest
LABEL version="1.0"
RUN mkdir /go/src/app
RUN go get -u github.com/golang/dep/cmd/dep
ADD ./main.go /go/src/app
COPY ./Gopkg.toml /go/src/app
WORKDIR /go/src/app
RUN dep ensure
RUN go test -v
RUN go build
CMD ["./app"]
You need to change the directory to that of your project. Also, in order to get dependencies, you will usually already have a Gopkg.toml and Gopkg.lock - dep init is ONLY used when you're moving from a project which was using another vendoring tool, no vendoring at all or you're starting a project from scratch.
To sum up, I'd do something like this:
FROM golang:latest
RUN go get -u github.com/golang/dep/cmd/dep \
&& mkdir /go/src/github.com/you \
&& git clone https://github.com/you/yourproject /go/src/github.com/you/yourproject
WORKDIR /go/src/github.com/you/yourproject
RUN dep ensure -v
&& go build
CMD ["./yourproject"]
How can I mount a volume to store my .m2 repo so I don't have to download the internet on every build?
My build is a Multi stage build:
FROM maven:3.5-jdk-8 as BUILD
COPY . /usr/src/app
RUN mvn --batch-mode -f /usr/src/app/pom.xml clean package
FROM openjdk:8-jdk
COPY --from=BUILD /usr/src/app/target /opt/target
WORKDIR /opt/target
CMD ["/bin/bash", "-c", "find -type f -name '*.jar' | xargs java -jar"]
You can do that with Docker >18.09 and BuildKit. You need to enable BuildKit:
export DOCKER_BUILDKIT=1
Then you need to enable experimental dockerfile frontend features, by adding as first line do Dockerfile:
# syntax=docker/dockerfile:experimental
Afterwards you can call the RUN command with cache mount. Cache mounts stay persistent during builds:
RUN --mount=type=cache,target=/root/.m2 \
mvn --batch-mode -f /usr/src/app/pom.xml clean package
Although the anwer from #Marek Obuchowicz is still valid, here is a small update.
First add this line to Dockerfile:
# syntax=docker/dockerfile:1
You can set the DOCKER_BUILDKIT inline like this:
DOCKER_BUILDKIT=1 docker build -t mytag .
I would also suggest to split the dependency resolution and packagin phase, so you can take the full advantage from Docker layer caching (if nothing changes in pom.xml it will use the cached layer with already downloaded dependencies). The full Dockerfile could look like this:
# syntax=docker/dockerfile:1
FROM maven:3.6.3-openjdk-17 AS MAVEN_BUILD
COPY ./pom.xml ./pom.xml
RUN --mount=type=cache,target=/root/.m2 mvn dependency:go-offline -B
COPY ./src ./src
RUN --mount=type=cache,target=/root/.m2 mvn package
FROM openjdk:17-slim-buster
EXPOSE 8080
COPY --from=MAVEN_BUILD /target/myapp-*.jar /app.jar
ENTRYPOINT ["java","-jar","/app.jar","-Xms512M","-Xmx2G","-Djava.security.egd=file:/dev/./urandom"]