CircleCI with Maven Exec Plugin - circleci

I am trying to run maven exec plugin that needs to run npm executable in CircleCI environment. Circle CI fails the build with an error npm command not found. It looks like CircleCI's environment does not have npm installed by default. Tried the config samples in https://circleci.com/developer/orbs/orb/circleci/node#usage-npm_install but does not seem to work.
Here is my plugin configuration:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.6.0</version>
<executions>
<execution>
<id>npm install (initialize)</id>
<goals>
<goal>exec</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<skip>${npm.exec.skip}</skip>
<executable>npm</executable>
<arguments>
<argument>install</argument>
</arguments>
</configuration>
</execution>
<execution>
<id>copy to swagger</id>
<goals>
<goal>exec</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<skip>${npm.exec.skip}</skip>
<executable>sh</executable>
<arguments>
<argument>-c</argument>
<argument>mkdir -p ${project.basedir}/../src/swagger && cp ${project.basedir}/../node_modules/#okta/openapi/dist/spec.yaml ${project.basedir}/../src/swagger/api.yaml</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
Here is my CircleCI config:
version: 2.1
aliases:
- &build_steps
- checkout
- run: java -version
- run: ./mvnw clean install -Pci
jobs:
jdk8:
docker:
- image: cimg/openjdk:8.0.322
environment:
JVM_OPTS: -Xmx3200m
steps: *build_steps
jdk11:
docker:
- image: cimg/openjdk:11.0.13
environment:
JVM_OPTS: -Xmx3200m
steps: *build_steps
workflows:
build_and_test:
jobs:
- jdk8
- jdk11
Any pointers on what I am missing here?

Related

GITLAB CI dependency:get does not find pom file cause of TIMESTAMPLE

I deploy maven packages on gitlab artifact. Maven called them xxx-api-model-2.0.2-20221223.095542-17.pom
My mvn dependency:get from docker file in another folder search for xxx-api-model-2.0.2.pom
Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.2.0:get (default-cli) on project xxx-action-batch: Couldn't download artifact: org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact xxx:xxx-api-model:jar:2.0.2 in gitlab-maven
Here is settings file:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
https://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server> <!-- serveur Gitlab :
<id>gitlab-maven</id>
<username>${GITLAB_REPO_USER}</username>
<password>${GITLAB_REPO_USER}</password>
<configuration>
<httpHeaders>
<property>
<name>Job-Token</name>
<value>${env.CI_JOB_TOKEN}</value>
</property>
</httpHeaders>
</configuration>
</server>
</servers>
</settings>
pom file :
<repositories>
<repository>
<id>gitlab-maven</id>
<url>${GITLAB_REPO_URL}</url>
</repository>
</repositories>
<distributionManagement>
<repository>
<id>gitlab-maven</id>
<url>${GITLAB_REPO_URL}</url>
</repository>
<snapshotRepository>
<id>gitlab-maven</id>
<url>${GITLAB_REPO_URL}</url>
</snapshotRepository>
</distributionManagement>
GITLAB_REPO_URL = https://example.com/api/v4/projects/2/packages/maven
dockerfile :
FROM eclipse-temurin:11-jre-focal as base
FROM maven:3.6.2-jdk-11 as build
WORKDIR /app
COPY pom.xml .
COPY src src
COPY .m2 .m2
ARG GITLAB_REPO_URL
ARG GITLAB_REPO_USER
ARG GITLAB_REPO_PASS
ENV GITLAB_REPO_USER=$GITLAB_REPO_USER
ENV GITLAB_REPO_PASS=$GITLAB_REPO_PASS
RUN bash -c "mvn dependency:get -Dartifact=xxx:xxx-api-model:2.0.2 -DremoteRepositories=gitlab-maven::::$GITLAB_REPO_URL -DGITLAB_REPO_USER=$GITLAB_REPO_USER -DGITLAB_REPO_PASS=$GITLAB_REPO_PASS"
RUN bash -c "mvn -B -f pom.xml -s .m2/settings.xml install -DskipTests -DGITLAB_REPO_URL=$GITLAB_REPO_URL -DGITLAB_REPO_USER=$GITLAB_REPO_USER -DGITLAB_REPO_PASS=$GITLAB_REPO_PASS"
WORKDIR /workspace/app/target/
ENTRYPOINT exec java $JAVA_OPTS -jar xxx-action-batch-1.0.0.jar
ci :
stages:
- action-batch-build&push
build&push:action-batch:
stage: action-batch-build&push
image: docker:stable
services:
- docker:19.03-dind
script:
- >
docker build
--pull
--build-arg VCS_REF=$CI_COMMIT_SHA
--build-arg VCS_URL=$CI_PROJECT_URL
--build-arg GITLAB_REPO_URL=${GITLAB_REPO_URL}
--build-arg GITLAB_REPO_USER=${GITLAB_REPO_USER}
--build-arg GITLAB_REPO_PASS=${GITLAB_REPO_PASS}
--tag xxx-action-batch:$CI_COMMIT_SHA
action-batch/
allow_failure: false
I tried to put some
-DuniqueVersion=false
to solve the problem with names of pom and jar file put nothing change

Gitlab: How to pass CI_JOB_TOKEN to Docker to be used with maven?

I have a ci_settings.xml. With it im building artifacts and deploy them to the
gitlab package repository:
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
<servers>
<server>
<id>gitlab-maven</id>
<configuration>
<httpHeaders>
<property>
<name>Job-Token</name>
<value>${env.CI_JOB_TOKEN}</value>
</property>
</httpHeaders>
</configuration>
</server>
</servers>
</settings>
But if I also want to be able do this in a docker container I need to pass the CI_JOB_TOKEN
to the docker container. Yet I have not been able to find out how to do that.
My dockerfile:
FROM maven:3.8.1-jdk-11
COPY pom.xml /
COPY ci_settings.xml /build/
COPY lombok.config /build/
COPY app/pom.xml /build/
COPY app/src /build/src
WORKDIR /build/
ARG CI_JOB_TOKEN
ENV CI_JOB_TOKEN=$CI_JOB_TOKEN
RUN mvn package -B -s ci_settings.xml
RUN cp /build/target/*.jar /app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]
My gitlab-ci.yml:
stages:
- build
variables:
IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG # TODO semantic versioning
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
build:
stage: build
# variables:
# CI_JOB_TOKEN: "${CI_JOB_TOKEN}" # does not work
script:
# - echo "CI_JOB_TOKEN=$CI_JOB_TOKEN" >> .env # does not work
- docker info
# setting it as build-arg also does not work
- docker build --build-arg CI_JOB_TOKEN=${CI_JOB_TOKEN} -f app/Dockerfile --pull -t $IMAGE .
- docker push $IMAGE
What do I need to do to pass the CI_JOB_TOKEN to the docker container so that
${env.CI_JOB_TOKEN} inside the ci_settings.xml which I copy to the container
is getting resolved correctly?
Use something like
RUN sed -i "s/CI_JOB_TOKEN/$CI_JOB_TOKEN/" ci_settings.xml
after the file is copied, and replace ${env.CI_JOB_TOKEN} by CI_JOB_TOKEN in your xml file to simplify the substitution.

Cypress cannot run in Docker container

I have a web application running in a Docker container. From my understanding, using the ‘cypress/base’ image should provide the necessary dependencies. However, trying to start the Cypress tests from an attached shell (run with headless), results in the following output:
Unhandled rejection Error: Your system is missing the dependency: Xvfb
Install Xvfb and run Cypress again.
Read our documentation on dependencies for more information:
https://on.cypress.io/required-dependencies
If you are using Docker, we provide containers with all required dependencies installed.
----------
Error: spawn Xvfb ENOENT
----------
Platform: linux (Debian - 10.11)
Cypress Version: 8.5.0
at /app/node_modules/cypress/lib/errors.js:328:17
at tryCatcher (/app/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/app/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/app/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/app/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/app/node_modules/bluebird/js/release/promise.js:729:18)
at Promise._fulfill (/app/node_modules/bluebird/js/release/promise.js:673:18)
at Promise._resolveCallback (/app/node_modules/bluebird/js/release/promise.js:466:57)
at Promise._settlePromiseFromHandler (/app/node_modules/bluebird/js/release/promise.js:559:17)
at Promise._settlePromise (/app/node_modules/bluebird/js/release/promise.js:604:18)
Installing Xvfb manually does not resolve the issue; it only results in another error saying Cypress’s dependencies could not be resolved.
Dockerfile:
FROM node:16
RUN mkdir /app
WORKDIR /app
COPY package.json .
COPY yarn.lock .
RUN yarn
RUN npx browserslist#latest --update-db
COPY . .
CMD ["yarn", "start"]
docker-compose.yml:
version: "3"
services:
ponder:
image: cypress/base:16
container_name: myApplication
build: ./
volumes:
- ./src:/app/src
- ./public:/app/public
- ./package.json:/app/package.json
- /app/node_modules
ports:
- 3001:3000
stdin_open: true
Thoughts?
Edit:
Xvfb was installed manually by running apt-get update, then apt-get install xvfb. Attempting to run Cypress after this, gives:
Unhandled rejection Error: Cypress failed to start.
This may be due to a missing library or dependency. https://on.cypress.io/required-dependencies
Please refer to the error below for more details.
----------
/root/.cache/Cypress/8.5.0/Cypress/Cypress: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory
----------
Platform: linux (Debian - 10.11)
Cypress Version: 8.5.0
at /app/node_modules/cypress/lib/errors.js:328:17
at tryCatcher (/app/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/app/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/app/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/app/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/app/node_modules/bluebird/js/release/promise.js:729:18)
at Promise._fulfill (/app/node_modules/bluebird/js/release/promise.js:673:18)
at Promise._resolveCallback (/app/node_modules/bluebird/js/release/promise.js:466:57)
at Promise._settlePromiseFromHandler (/app/node_modules/bluebird/js/release/promise.js:559:17)
at Promise._settlePromise (/app/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/app/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/app/node_modules/bluebird/js/release/promise.js:729:18)
at Promise._fulfill (/app/node_modules/bluebird/js/release/promise.js:673:18)
at Promise._resolveCallback (/app/node_modules/bluebird/js/release/promise.js:466:57)
at Promise._settlePromiseFromHandler (/app/node_modules/bluebird/js/release/promise.js:559:17)
at Promise._settlePromise (/app/node_modules/bluebird/js/release/promise.js:604:18)
Instead of FROM node:16, try using FROM cypress/included . that image comes with node 16 already and browsers pre-installed. NOTE: the image might be a lot larger.
Nice attempt but you are overdoing it a little.
You need to install dependencies. https://docs.cypress.io/guides/continuous-integration/introduction#Dependencies
apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2 libxtst6 xauth xvfb

Can't connect to the docker daemon while running Maven jib plugin in Gitlab CI

I'm trying to build a docker image with my application in Gitlab continuous integration using the jib Maven plugin, but I can't get the plugin connect to the docker daemon using the dind (docker-in-docker) service. Currently I'm using this configuration in my gitlab-ci.yml file:
deploy:mvn:
image: maven:3.6.3-jdk-8-openj9
stage: deploy
services:
- docker:dind
script:
- mvn compile jib:dockerBuild
This is the error I get:
[ERROR] Failed to execute goal
com.google.cloud.tools:jib-maven-plugin:0.9.11:dockerBuild
(default-cli) on project my-application: Build to Docker daemon failed,
perhaps you should make sure Docker is installed and you have correct
privileges to run it -> [Help 1]
UPDATE
I updated to 2.2.0 and it's running locally. I added the docker images command before and the plugin seems not to be able to find the docker command in Gitlab CI:
$ docker images && mvn compile jib:dockerBuild /bin/bash: line 97:
docker: command not found
This is the configuration for the jib plugin:
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<version>2.2.0</version>
<configuration>
<from>
<image>adoptopenjdk/openjdk11:alpine-jre</image>
</from>
<to>
<image>my-application:latest</image>
</to>
<container>
<entrypoint>
<shell>sh</shell>
<option>-c</option>
<arg>chmod +x /entrypoint.sh && sync && /entrypoint.sh</arg>
</entrypoint>
<ports>
<port>8080</port>
</ports>
<environment>
<SPRING_OUTPUT_ANSI_ENABLED>ALWAYS</SPRING_OUTPUT_ANSI_ENABLED>
<JHIPSTER_SLEEP>0</JHIPSTER_SLEEP>
</environment>
<creationTime>USE_CURRENT_TIMESTAMP</creationTime>
</container>
</configuration>
</plugin>
the plugin seems not to be able to find the docker command in Gitlab CI:
No, it is not Jib but /bin/bash that cannot find the docker command. Even before using Jib, you don't have docker available. Take a closer look at the error message.
$ docker images && mvn compile jib:dockerBuild /bin/bash: line 97: docker: command not found
For example, on my Linux, if I try a command foo that doesn't exist in a shell script, it outputs the same message.
$ /bin/bash -c "foo && mvn -v"
/bin/bash: line 1: foo: command not found
Therefore, the following command without mvn will fail with the same error.
script:
- docker images
This proves that either docker doesn't exist in your GitLab runtime environment or is not on the PATH environment variable.
UPDATE
Updating my answer, since you answered that you now use jib:build instead of jib:dockerBuild.
If you use jib:build, you don't even need Docker. Jib doesn't require Docker when building and pushing an image to a remote registry with jib:build. Therefore, you can completely forget about setting up Docker and remove docker:dind and export DOCKER_HOST:
mvn compile jib:build -Djib.to.auth.username=$DOCKER_REGISTRY_USER -Djib.to.auth.password=$DOCKER_REGISTRY_PWD
Finally this is the configuration I got it working with:
services:
- docker:dind
deploy:mvn:
image: maven:3.6.3-jdk-8-openj9
stage: deploy
script:
- export DOCKER_HOST=tcp://docker:2375
- mvn compile jib:build -Djib.to.auth.username=$DOCKER_REGISTRY_USER -Djib.to.auth.password=$DOCKER_REGISTRY_PWD
only:
- tags
Apart from using the Docker in Docker service I needed to establish the DOCKER_HOST environment variable and also pass the credentials to my mvn jib:build command. I stored the credentials in CI settings, as environment variables. Thanks #Chanseok Oh for your help.
See also:
Using docker build
Jib docs

Run SpringBoot-based docker image return error message:Invalid or corrupt jarfile /app.jar

I followed the Spring official tutorial(https://spring.io/guides/gs/spring-boot-docker/) to build springBoot-based app to docker image. The docker image was built successfully, but when I wanted to execute docker run command to start a container, I got the following error message:
Error: Invalid or corrupt jarfile /app.jar
and the containner couldn't start running normally.
Has anybody gotten the same error message before? I really need your help. Thank you very much!
Structure of java aplication
Demo
└── src
| ├── main
| │ ├── java
| │ │ └── org
| │ │ └── demo
| │ │ └── Application.java
| │ └── resources
| │ └── application.properties
| └── test
| └── java
| └── org
| └── demo
| └── Application.java
├──── Dockerfile
├──── pom.xml
Content of pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.executablejar</groupId>
<artifactId>demo</artifactId>
<packaging>jar</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>demo Maven Webapp</name>
<url>http://maven.apache.org</url>
<properties>
<java-version>1.8</java-version>
<docker.image.prefix>springDemo</docker.image.prefix>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.3.3.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</dependency>
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
<finalName>demo</finalName>
</build>
Content of Dockerfile
FROM java:8
EXPOSE 8080
ADD /target/app.jar demo.jar
ENTRYPOINT ["java","-jar","demo.jar"]
Commands to build and run image
Go to the directory of project.Lets say D:/Demo
$ cd D/demo
$ mvn clean install
$ docker build demo .
$ docker run -p 8080:8080 -t demo
You have to add the JAR_FILE build arg to the dockerfile-maven-plugin, since it is missing in the code sample provided by the spring tutorial:
<plugin>
<groupId>com.spotify</groupId>
<artifactId>dockerfile-maven-plugin</artifactId>
<version>1.4.9</version>
<configuration>
<repository>${docker.image.prefix}/${project.artifactId}</repository>
<buildArgs>
<JAR_FILE>target/${project.build.finalName}.jar</JAR_FILE>
</buildArgs>
</configuration>
</plugin>
I had the same problem, check the name of jar application in dockerFile

Resources