Dockerfile:
FROM node:8.14.1-alpine
RUN apk update && apk add --no-cache openjdk8-jre
RUN apk update && apk add python py-pip curl unzip libexif udev nano
ENV CHROME_BIN=/usr/bin/chromium-browser
RUN echo #v3.8 http://nl.alpinelinux.org/alpine/v3.8/community >> /etc/apk/repositories && \
echo #v3.8 http://nl.alpinelinux.org/alpine/v3.8/main >> /etc/apk/repositories && \
apk add --no-cache \
chromium#v3.8 \
nss#v3.8
# Tell Puppeteer to skip installing Chrome. We'll be using the installed package.
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
WORKDIR /project_compile/vue_app
COPY vue_app /project_compile/vue_app
RUN rm -rf node_modules
RUN npm config set registry http://registry.npmjs.org/ && npm install
CMD npm run test:e2e:headless
When it is run, this error happens:
Error retrieving a new session from the selenium server
Connection refused! Is selenium server started?
{ value:
{ message: 'Timed out waiting for driver server to start.\nBuild info: version: \'3.141.59\', revision: \'e82be7d358\', time: \'2018-11-14T08:25:53\'\nSystem info: host: \'a3f993cb3a2b\', ip: \'172.22.0.2\', os.name: \'Linux\', os.arch: \'amd64\', os.version: \'4.15.0-33-generic\', java.version: \'1.8.0_191\'\nDriver info: driver.version: unknown',
error: 'unknown error' },
status: 13 }
ERROR Error: Command failed: /project_compile/vue_app/node_modules/nightwatch/bin/nightwatch --config nightwatch.conf.js --env chrome
Error: Command failed: /project_compile/vue_app/node_modules/nightwatch/bin/nightwatch --config nightwatch.conf.js --env chrome
at makeError (/project_compile/vue_app/node_modules/execa/index.js:174:9)
at Promise.all.then.arr (/project_compile/vue_app/node_modules/execa/index.js:278:16)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:189:7)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! vue_app#0.1.0 test:e2e:headless: `vue-cli-service test:e2e --config nightwatch.conf.js --env chrome`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the vue_app#0.1.0 test:e2e:headless script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2019-01-14T12_14_57_744Z-debug.log
Seems that chrome works correctly (testable running something like chromium-browser --headless --no-sandbox --dump-dom https://www.chromestatus.com/) and also selenium seems to be working fine.
Why does the test not run? The error is not explainable.
All code is contained in this repo https://github.com/nicolalandro/vue-selenium-docker. So reproducing it is easy - clone the repo and run two commands:
docker-compose -f docker/test/js/docker-compose-js-e2e.yml build
docker-compose -f docker/test/js/docker-compose-js-e2e.yml up --exit-code-from e2e
Maybe you have to wait for selenium container to be up?
check out this guide:
https://github.com/SeleniumHQ/docker-selenium#waiting-for-the-grid-to-be-ready
Write down a script to check for "ready": true,
The problem was that the selenium version was not aligned to chrome version.
Related
How can I activate the --verbose mode for the node npm install run started by Vaadin 23.1.15?
I use Java 17, Gradle 7.4.2, Eclipse 22-3 and Windows. I have the problem that node npm install often hangs during vaadinBuildFrontend Gradle task execution. I use Gradle with --no-daemon option.
2022-11-15 09:59:05,566 INFO [ForkJoinPool.commonPool-worker-2] TaskUpdatePackages - using 'C:\Users \ --- \ .vaadin\node\node.exe C:\Users \ --- \ .vaadin\node\node_modules\npm\bin\npm-cli.js --no-update-notifier --no-audit --scripts-prepend-node-path=true --ignore-scripts install' for frontend package installation
While installing hadoop 3.2 with docker, I encounter the following error:
Setting up npm (3.5.2-0ubuntu4) ...
Processing triggers for libc-bin (2.23-0ubuntu11.2) ...
WARN engine npm#7.13.0: wanted: {"node":">=10"} (current: {"node":"4.2.6","npm":"3.5.2"})
WARN engine npm#7.13.0: wanted: {"node":">=10"} (current: {"node":"4.2.6","npm":"3.5.2"})
/usr/local/lib
`-- (empty)
npm ERR! Linux 4.15.0-29-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install" "npm#latest" "-g"
npm ERR! node v4.2.6
npm ERR! npm v3.5.2
npm ERR! path /usr/local/lib/node_modules/.staging/#npmcli/ci-detect-c7bf9552
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall rename
npm ERR! enoent ENOENT: no such file or directory, rename '/usr/local/lib/node_modules/.staging/#npmcli/ci-detect-c7bf9552' -> '/usr/local/lib/node_modules/npm/node_modules/#npmcli/ci-detect'
npm ERR! enoent ENOENT: no such file or directory, rename '/usr/local/lib/node_modules/.staging/#npmcli/ci-detect-c7bf9552' -> '/usr/local/lib/node_modules/npm/node_modules/#npmcli/ci-detect'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! Please include the following file with any support request:
npm ERR! /root/npm-debug.log
npm ERR! code 1
How can I solve this problem?
The bash script, supplied with Hadoop 3.2 source file and contains the docker commands, is as follows:
#!/usr/bin/env bash
set -e # exit on error
cd "$(dirname "$0")" # connect to root
docker build -t hadoop-build dev-support/docker
if [ "$(uname -s)" = "Linux" ]; then
USER_NAME=${SUDO_USER:=$USER}
USER_ID=$(id -u "${USER_NAME}")
GROUP_ID=$(id -g "${USER_NAME}")
# man docker-run
# When using SELinux, mounted directories may not be accessible
# to the container. To work around this, with Docker prior to 1.7
# one needs to run the "chcon -Rt svirt_sandbox_file_t" command on
# the directories. With Docker 1.7 and later the z mount option
# does this automatically.
if command -v selinuxenabled >/dev/null && selinuxenabled; then
DCKR_VER=$(docker -v|
awk '$1 == "Docker" && $2 == "version" {split($3,ver,".");print ver[1]"."ver[2]}')
DCKR_MAJ=${DCKR_VER%.*}
DCKR_MIN=${DCKR_VER#*.}
if [ "${DCKR_MAJ}" -eq 1 ] && [ "${DCKR_MIN}" -ge 7 ] ||
[ "${DCKR_MAJ}" -gt 1 ]; then
V_OPTS=:z
else
for d in "${PWD}" "${HOME}/.m2"; do
ctx=$(stat --printf='%C' "$d"|cut -d':' -f3)
if [ "$ctx" != svirt_sandbox_file_t ] && [ "$ctx" != container_file_t ]; then
printf 'INFO: SELinux is enabled.\n'
printf '\tMounted %s may not be accessible to the container.\n' "$d"
printf 'INFO: If so, on the host, run the following command:\n'
printf '\t# chcon -Rt svirt_sandbox_file_t %s\n' "$d"
fi
done
fi
fi
else # boot2docker uid and gid
USER_NAME=$USER
USER_ID=1000
GROUP_ID=50
fi
docker build -t "hadoop-build-${USER_ID}" - <<UserSpecificDocker
FROM hadoop-build
RUN groupadd --non-unique -g ${GROUP_ID} ${USER_NAME}
RUN useradd -g ${GROUP_ID} -u ${USER_ID} -k /root -m ${USER_NAME}
RUN echo "${USER_NAME} ALL=NOPASSWD: ALL" > "/etc/sudoers.d/hadoop-build-${USER_ID}"
ENV HOME /home/${USER_NAME}
UserSpecificDocker
#If this env varible is empty, docker will be started
# in non interactive mode
DOCKER_INTERACTIVE_RUN=${DOCKER_INTERACTIVE_RUN-"-i -t"}
# By mapping the .m2 directory you can do an mvn install from
# within the container and use the result on your normal
# system. And this also is a significant speedup in subsequent
# builds because the dependencies are downloaded only once.
docker run --rm=true $DOCKER_INTERACTIVE_RUN \
-v "${PWD}:/home/${USER_NAME}/hadoop${V_OPTS:-}" \
-w "/home/${USER_NAME}/hadoop" \
-v "${HOME}/.m2:/home/${USER_NAME}/.m2${V_OPTS:-}" \
-u "${USER_ID}" \
"hadoop-build-${USER_ID}" "$#"
and I used the following command to run the file:
sudo ./start-build-env.sh
Edit: tried to install hadoop with marven over two days, but I have problem installing the pre-existing packages, and don't know what do about it. Thus I tried the docker version, which consider to be easier, but it does not works...
Why is this build stage in my gitlab pipeline failing with
npm Err! 401: Unable to authenticate, need: Basic realm="Artifactory Realm"
When I run the command $ npm-cli-login -u $USERNAME -p $API_KEY -e $EMAIL -r $REPOSITORY it seems like I get correcly logged in. My correct username gets displayed and the global .npmrc file gets created in my home directory. But when I run npm install or npm i --registry=https://<my_private_repo>.jfrog.io/<my_private_repo> it fails with the 401.
Following output am I seeing in the logs of my failed pipeline stage:
Pulling docker image node:14.15.4 ...
Using docker image sha256:924763541c... for node:14.15.4 with digest node#sha256:cb01e9d98a...
Preparing environment 00:01
Running on runner-ffeacb89-project-...-concurrent-0 via ....r.gitlab.host...
Getting source from Git repository 00:01
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/gravity/my_test_project/.git/
Checking out 5f7... as dev...
Removing .gradle/
Skipping Git submodules setup
Restoring cache 00:06
Checking cache for my_test_project...
cache.zip is up to date
Successfully extracted cache
Executing "step_script" stage of the job script 01:07
Using docker image sha256:924763541c0c8b3839132... for node:14.15.4 with digest node#sha256:cb01e9d9... ...
$ export GRADLE_USER_HOME=`pwd`/.gradle
$ pwd
/builds/my_test_project
$ cd ~
$ npm install -g npm-cli-login
npm WARN deprecated request#2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator#5.1.5: this library is no longer supported
/usr/local/bin/npm-cli-login -> /usr/local/lib/node_modules/npm-cli-login/bin/npm-cli-login.js
> core-js#3.10.1 postinstall /usr/local/lib/node_modules/npm-cli-login/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"
+ npm-cli-login#0.1.1
added 634 packages from 451 contributors in 34.086s
$ npm-cli-login -u $USERNAME -p $API_KEY -e $EMAIL -r $REPOSITORY
info attempt registry request try #1 at 2:32:54 PM
http request PUT https://<my_private_repo>.jfrog.io/<my_private_repo>/api/npm/npm/-/user/org.couchdb.user:<my_correct_username>
http 201 https://<my_private_repo>.jfrog.io/<my_private_repo>/api/npm/npm/-/user/org.couchdb.user:<my_correct_username>
$ pwd
/root
$ cat .npmrc
//<my_private_repo>.jfrog.io/<my_private_repo>/api/npm/npm//:_authToken=eyJ2...(very long token, looks correct)
$ echo 'always-auth = true' >> .npmrc
$ cat .npmrc
//<my_private_repo>.jfrog.io/<my_private_repo>/api/npm/npm//:_authToken=eyJ2...(very long token, looks correct)
always-auth = true
$ npm i -g #angular/cli
npm WARN deprecated request#2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator#5.1.5: this library is no longer supported
/usr/local/bin/ng -> /usr/local/lib/node_modules/#angular/cli/bin/ng
> #angular/cli#11.2.9 postinstall /usr/local/lib/node_modules/#angular/cli
> node ./bin/postinstall/script.js
+ #angular/cli#11.2.9
added 242 packages from 181 contributors in 12.287s
$ cd /builds/my_test_project/ui
$ printenv NPM_TOKEN
eyJ2Z...(my long token)
$ npm config set //registry.npmjs.org/:_authToken ${NPM_TOKEN}
$ npm install
npm WARN deprecated debug#4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated axios#0.20.0: Critical security vulnerability fixed in v0.21.1. For more information, see https://github.com/axios/axios/pull/3410
npm WARN deprecated request#2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated fsevents#2.1.3: "Please update to latest v2.3 or v2.2"
npm WARN deprecated chokidar#2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated har-validator#5.1.5: this library is no longer supported
npm WARN deprecated fsevents#1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated urix#0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated resolve-url#0.2.1: https://github.com/lydell/resolve-url#deprecated
npm ERR! code E401
npm ERR! Unable to authenticate, need: Basic realm="Artifactory Realm"
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2021-04-16T14_33_25_786Z-debug.log
Cleaning up file based variables 00:00
ERROR: Job failed: exit code 1
This is my build stage in the .gitlab-ci.yml file (Spring Boot Project)
build:
stage: build
image: node:14.15.4
script:
- pwd
- cd ~
- npm install -g npm-cli-login
- npm-cli-login -u $USERNAME -p $API_KEY -e $EMAIL -r $REPOSITORY
- cat .npmrc
- echo 'always-auth = true' >> .npmrc
- cat .npmrc
- npm i -g #angular/cli
- cd /builds/myProjectFolder
- printenv NPM_TOKEN
- npm config set //myrepo.jfrog.io/myrepo/api/npm/npm//:_authToken ${NPM_TOKEN}
- npm install --registry=https://myrepo.jfrog.io/<my_private_repo>/api/npm/npm/
artifacts:
paths:
- app/src/ui/dist/dm-consent-page
expire_in: 5 days
I solved it by using this method: https://gruchalski.com/posts/2020-09-09-authenticate-to-private-jfrog-npm-registry/
After running this curl command I received everything that I needed to put into my global .npmrc file:
curl -u ${JFROG_USER}:${JFROG_ENCRYPTED_PASSWORD} https://${JFROG_ORG}.jfrog.io/${JFROG_ORG}/api/npm/auth
For anyone who's interested, the full script in my gitlab ci pipeline stage now looks like this:
script:
- npm -v
6.14.10
- node -v
v14.15.4
- cd ~
- pwd
/root
# install angular globally
- npm i -g #angular/cli
# create the config file '.npmrc' for authenticating at jFrog when running 'npm install'.
- cat > .npmrc
- echo _auth = ${NPM_AUTH_TOKEN} >> .npmrc <- This is the token that I received after running the curl command from the tutorial / link above
- echo always-auth = true >> .npmrc
- echo email = ${EMAIL} >> .npmrc
# the next line makes npm look for the packages that are annotated with #<my-private-repo> at the JFrog Repo.
- echo #<my-private-repo>:registry=${UI_JFROG_REGESTRY} >> .npmrc
# change back to the project folder.
- cd /builds/<my-project-folder>/ui
# install all packages + the <my-private-repo> package from JFrog.
- npm install
Create a pipeline to do the jenkins cypress test, but I always run it, make a mistake, and try various solutions, including Jenkins Xvfb, but I still didn't succeed.
Error
+ npm run exec:e2e
> projectname#0.1.0 exec:e2e /var/lib/jenkins/workspace/project/myproject
> npx cypress run
It looks like this is your first time using Cypress: 3.4.0
[?25l[01:56:21] Verifying Cypress can run /var/lib/jenkins/.cache/Cypress/3.4.0/Cypress [started]
[01:56:21] Verifying Cypress can run /var/lib/jenkins/.cache/Cypress/3.4.0/Cypress [failed]
[?25hYour system is missing the dependency: Xvfb
Install Xvfb and run Cypress again.
Read our documentation on dependencies for more information:
https://on.cypress.io/required-dependencies
If you are using Docker, we provide containers with all required dependencies installed.
----------
Error: spawn Xvfb ENOENT
----------
Platform: linux (Ubuntu Linux - 19.04)
Cypress Version: 3.4.0
[?25hnpm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! projectname#0.1.0 exec:e2e: `npx cypress run`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the projectname#0.1.0 exec:e2e script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /var/lib/jenkins/.npm/_logs/2019-07-24T01_56_21_795Z-debug.log
I get resolved this question with other solution.
First, i created a pipeline
stage('E2E Tests') {
sh 'docker run -v $PWD:/e2e -w /e2e cypress/included:3.4.0'
}
What you can do is having a lock on your stage
stage('Run E2E tests') {
dir("cypress-e2e-tests") {
lock("cypress-e2e-tests-${env.NODE_NAME}") {
sh 'docker run -v $PWD:/e2e -w /e2e cypress/included:3.4.0'
}
}
}
I have been trying to set up a gitlab ci using shared runners. My .gitlab-ci.yml looks like this:
image: docker:latest
# When using dind, it's wise to use the overlayfs driver for
# improved performance.
variables:
DOCKER_DRIVER: overlay2
GCP_PROJECT_ID: example-12345
IMAGE_NAME: test
services:
- docker:dind
stages:
- publish
publish-image:
stage: publish
script:
# Create our image. Expected to create an image 'image_id'
- docker build -t $IMAGE_NAME .
# Tag our image for container registry
- docker tag $IMAGE_NAME eu.gcr.io/$GCP_PROJECT_ID/$IMAGE_NAME
# Install CA certs, openssl to https downloads, python for gcloud sdk
- apk add --update make ca-certificates openssl python
- update-ca-certificates
# Download and install Google Cloud SDK
- wget https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz
- tar zxvf google-cloud-sdk.tar.gz && ./google-cloud-sdk/install.sh --usage-reporting=false --path-update=true
- google-cloud-sdk/bin/gcloud --quiet components install kubectl
- google-cloud-sdk/bin/gcloud --quiet components update
- google-cloud-sdk/bin/gcloud auth activate-service-account --key-file g.json
# Optionally tag the image with the commit short-sha
- docker tag $IMAGE_NAME eu.gcr.io/$GCP_PROJECT_ID/$IMAGE_NAME:$(echo $CI_COMMIT_SHA | cut -c1-8)
#- google-cloud-sdk/bin/gcloud docker -- push eu.gcr.io/$GCP_PROJECT_ID/$IMAGE_NAME:latest
- google-cloud-sdk/bin/gcloud docker -- push eu.gcr.io/$GCP_PROJECT_ID/$IMAGE_NAME:$(echo $CI_COMMIT_SHA | cut -c1-8)
# Only run builds for these refs
only:
- master
The project I am trying to build is Meteor application (Meteor version 1.6.1) thats supposed to be run in a kubernetes cluster. For creating containers I am using https://github.com/jshimko/meteor-launchpad.
When I use docker build on my local machine everything is working, but when in the CI I get this error (after build in stage Running npm install in the server bundle...):
> node-pre-gyp install --fallback-to-build
module.js:540
throw err;
^
Error: Cannot find module '../'
at Function.Module._resolveFilename (module.js:538:15)
at Function.Module._load (module.js:468:25)
at Module.require (module.js:587:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/opt/meteor/dist/bundle/programs/server/npm/node_modules/.bin/node-pre-gyp:15:20)
at Module._compile (module.js:643:30)
at Object.Module._extensions..js (module.js:654:10)
at Module.load (module.js:556:32)
at tryModuleLoad (module.js:499:12)
at Function.Module._load (module.js:491:3)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! bcrypt#1.0.3 install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the bcrypt#1.0.3 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-03-08T13_29_09_532Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! meteor-dev-bundle# install: `node npm-rebuild.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the meteor-dev-bundle# install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-03-08T13_29_09_574Z-debug.log
But I dont believe it is an actual meteor problem since I know the code works.
Another weird thing is that the build actually is successful in an older version of the code running meteor 1.5.2.2 (New version is 1.6.1), however it then breaks when run in kubernetes. The error from the pod reads:
=> Starting app on port 3000...
## There is an issue with `node-fibers` ##
`/opt/meteor/dist/bundle/programs/server/node_modules/fibers/bin/linux-x64-57/fibers.node` is missing.
Try running this to fix the issue: /opt/nodejs/bin/node /opt/meteor/dist/bundle/programs/server/node_modules/fibers/build
/opt/meteor/dist/bundle/programs/server/node_modules/fibers/fibers.js:20
throw new Error('Missing binary. See message above.');
^
Error: Missing binary. See message above.
at Object.<anonymous> (/opt/meteor/dist/bundle/programs/server/node_modules/fibers/fibers.js:20:8)
at Module._compile (module.js:635:30)
at Object.Module._extensions..js (module.js:646:10)
at Module.load (module.js:554:32)
Dispite the build log from the runner states:
> fibers#1.0.15 install /opt/meteor/dist/bundle/programs/server/node_modules/fibers
> node build.js || nodejs build.js
`linux-x64-46` exists; testing
Binary is fine; exiting
Since I know that the build can succeed I assume that the docker-in-docker is run with the right settings on the shared runners. Either by binding the outer docker socket to the inner och by running privileged (As described here and here)
So why do my build get corrupted/fail on the CI?
EDIT: My local builds are done on ubuntu 16