How can I activate the --verbose mode for the node npm install run started by Vaadin 23.1.15?
I use Java 17, Gradle 7.4.2, Eclipse 22-3 and Windows. I have the problem that node npm install often hangs during vaadinBuildFrontend Gradle task execution. I use Gradle with --no-daemon option.
2022-11-15 09:59:05,566 INFO [ForkJoinPool.commonPool-worker-2] TaskUpdatePackages - using 'C:\Users \ --- \ .vaadin\node\node.exe C:\Users \ --- \ .vaadin\node\node_modules\npm\bin\npm-cli.js --no-update-notifier --no-audit --scripts-prepend-node-path=true --ignore-scripts install' for frontend package installation
Related
I am trying to use packer with GCP in gitlab-ci but every time I get to the packer build it will fail with the following error:
starting remote command: chmod +x /tmp/script_5147.sh; DEBIAN_FRONTEND='noninteractive' PACKER_BUILDER_TYPE='googlecompute' PACKER_BUILD_NAME='my_vm' /tmp/script_5147.sh
==> googlecompute.my_vm: /tmp/script_5147.sh: line 1: {message:401 Unauthorized}: command not found
2022/11/13 13:26:05 [INFO] 0 bytes written for 'stdout'
2022/11/13 13:26:05 packer-plugin-googlecompute_v1.0.16_x5.0_linux_amd64 plugin: 2022/11/13 13:26:05 [ERROR] Remote command exited with '127': chmod +x /tmp/script_5147.sh; DEBIAN_FRONTEND='noninteractive' PACKER_BUILDER_TYPE='googlecompute' PACKER_BUILD_NAME='my_vm' /tmp/script_5147.sh
2022/11/13 13:26:05 packer-plugin-googlecompute_v1.0.16_x5.0_linux_amd64 plugin: 2022/11/13 13:26:05 [INFO] RPC endpoint: Communicator ended with: 127
The script
#!/bin/bash
set -e
if [ "$EUID" -ne 0 ]
then echo "Please run as root"
exit
fi
apt update
apt install -y curl
curl -fsSL https://deb.nodesource.com/setup_18.x | bash -
apt upgrade -y iptables
# The iptables-persistent must be installed in order to create the /etc/iptables/rules.v4 file
apt install -y nginx libzmq3-dev nodejs ipset iptables-persistent net-tools libre2-dev
npm install -g yarn
rm /etc/nginx/sites-enabled/default
The packer is working locally (also with the image) and I created a VM in GCP all of them are working, only in gitlab it fails.
I created a custom image of packer and it still failed for me on gitlab.
I will try to move it to my own runner but it will take a few days for me.
Would love if someone could help me figure it out.
Ok, I figured out the problem.
I curled some files and the token was not good so when it got to packer it failed because the files were not as expected.
I printing the 401 of the curl.
Why is this build stage in my gitlab pipeline failing with
npm Err! 401: Unable to authenticate, need: Basic realm="Artifactory Realm"
When I run the command $ npm-cli-login -u $USERNAME -p $API_KEY -e $EMAIL -r $REPOSITORY it seems like I get correcly logged in. My correct username gets displayed and the global .npmrc file gets created in my home directory. But when I run npm install or npm i --registry=https://<my_private_repo>.jfrog.io/<my_private_repo> it fails with the 401.
Following output am I seeing in the logs of my failed pipeline stage:
Pulling docker image node:14.15.4 ...
Using docker image sha256:924763541c... for node:14.15.4 with digest node#sha256:cb01e9d98a...
Preparing environment 00:01
Running on runner-ffeacb89-project-...-concurrent-0 via ....r.gitlab.host...
Getting source from Git repository 00:01
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/gravity/my_test_project/.git/
Checking out 5f7... as dev...
Removing .gradle/
Skipping Git submodules setup
Restoring cache 00:06
Checking cache for my_test_project...
cache.zip is up to date
Successfully extracted cache
Executing "step_script" stage of the job script 01:07
Using docker image sha256:924763541c0c8b3839132... for node:14.15.4 with digest node#sha256:cb01e9d9... ...
$ export GRADLE_USER_HOME=`pwd`/.gradle
$ pwd
/builds/my_test_project
$ cd ~
$ npm install -g npm-cli-login
npm WARN deprecated request#2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator#5.1.5: this library is no longer supported
/usr/local/bin/npm-cli-login -> /usr/local/lib/node_modules/npm-cli-login/bin/npm-cli-login.js
> core-js#3.10.1 postinstall /usr/local/lib/node_modules/npm-cli-login/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"
+ npm-cli-login#0.1.1
added 634 packages from 451 contributors in 34.086s
$ npm-cli-login -u $USERNAME -p $API_KEY -e $EMAIL -r $REPOSITORY
info attempt registry request try #1 at 2:32:54 PM
http request PUT https://<my_private_repo>.jfrog.io/<my_private_repo>/api/npm/npm/-/user/org.couchdb.user:<my_correct_username>
http 201 https://<my_private_repo>.jfrog.io/<my_private_repo>/api/npm/npm/-/user/org.couchdb.user:<my_correct_username>
$ pwd
/root
$ cat .npmrc
//<my_private_repo>.jfrog.io/<my_private_repo>/api/npm/npm//:_authToken=eyJ2...(very long token, looks correct)
$ echo 'always-auth = true' >> .npmrc
$ cat .npmrc
//<my_private_repo>.jfrog.io/<my_private_repo>/api/npm/npm//:_authToken=eyJ2...(very long token, looks correct)
always-auth = true
$ npm i -g #angular/cli
npm WARN deprecated request#2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator#5.1.5: this library is no longer supported
/usr/local/bin/ng -> /usr/local/lib/node_modules/#angular/cli/bin/ng
> #angular/cli#11.2.9 postinstall /usr/local/lib/node_modules/#angular/cli
> node ./bin/postinstall/script.js
+ #angular/cli#11.2.9
added 242 packages from 181 contributors in 12.287s
$ cd /builds/my_test_project/ui
$ printenv NPM_TOKEN
eyJ2Z...(my long token)
$ npm config set //registry.npmjs.org/:_authToken ${NPM_TOKEN}
$ npm install
npm WARN deprecated debug#4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated axios#0.20.0: Critical security vulnerability fixed in v0.21.1. For more information, see https://github.com/axios/axios/pull/3410
npm WARN deprecated request#2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated fsevents#2.1.3: "Please update to latest v2.3 or v2.2"
npm WARN deprecated chokidar#2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated har-validator#5.1.5: this library is no longer supported
npm WARN deprecated fsevents#1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated urix#0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated resolve-url#0.2.1: https://github.com/lydell/resolve-url#deprecated
npm ERR! code E401
npm ERR! Unable to authenticate, need: Basic realm="Artifactory Realm"
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2021-04-16T14_33_25_786Z-debug.log
Cleaning up file based variables 00:00
ERROR: Job failed: exit code 1
This is my build stage in the .gitlab-ci.yml file (Spring Boot Project)
build:
stage: build
image: node:14.15.4
script:
- pwd
- cd ~
- npm install -g npm-cli-login
- npm-cli-login -u $USERNAME -p $API_KEY -e $EMAIL -r $REPOSITORY
- cat .npmrc
- echo 'always-auth = true' >> .npmrc
- cat .npmrc
- npm i -g #angular/cli
- cd /builds/myProjectFolder
- printenv NPM_TOKEN
- npm config set //myrepo.jfrog.io/myrepo/api/npm/npm//:_authToken ${NPM_TOKEN}
- npm install --registry=https://myrepo.jfrog.io/<my_private_repo>/api/npm/npm/
artifacts:
paths:
- app/src/ui/dist/dm-consent-page
expire_in: 5 days
I solved it by using this method: https://gruchalski.com/posts/2020-09-09-authenticate-to-private-jfrog-npm-registry/
After running this curl command I received everything that I needed to put into my global .npmrc file:
curl -u ${JFROG_USER}:${JFROG_ENCRYPTED_PASSWORD} https://${JFROG_ORG}.jfrog.io/${JFROG_ORG}/api/npm/auth
For anyone who's interested, the full script in my gitlab ci pipeline stage now looks like this:
script:
- npm -v
6.14.10
- node -v
v14.15.4
- cd ~
- pwd
/root
# install angular globally
- npm i -g #angular/cli
# create the config file '.npmrc' for authenticating at jFrog when running 'npm install'.
- cat > .npmrc
- echo _auth = ${NPM_AUTH_TOKEN} >> .npmrc <- This is the token that I received after running the curl command from the tutorial / link above
- echo always-auth = true >> .npmrc
- echo email = ${EMAIL} >> .npmrc
# the next line makes npm look for the packages that are annotated with #<my-private-repo> at the JFrog Repo.
- echo #<my-private-repo>:registry=${UI_JFROG_REGESTRY} >> .npmrc
# change back to the project folder.
- cd /builds/<my-project-folder>/ui
# install all packages + the <my-private-repo> package from JFrog.
- npm install
I need to install an SDK package inside an Ubuntu 18.04 Docker container, but am constantly running into this problem:
theuser#e9fa4f39e0f0:/src/spinnaker$ sudo dpkg -i libspinnaker_2.2.0.48_arm64.deb
(Reading database ... 52013 files and directories currently installed.)
Preparing to unpack libspinnaker_2.2.0.48_arm64.deb ...
Unpacking libspinnaker (2.2.0.48) over (2.2.0.48) ...
logname: no login name
dpkg: warning: old libspinnaker package post-removal script subprocess returned error exit status 1
dpkg: trying script from the new package instead ...
logname: no login name
dpkg: error processing archive libspinnaker_2.2.0.48_arm64.deb (--install):
new libspinnaker package post-removal script subprocess returned error exit status 1
logname: no login name
dpkg: error while cleaning up:
new libspinnaker package post-removal script subprocess returned error exit status 1
Errors were encountered while processing:
libspinnaker_2.2.0.48_arm64.deb
I've tried all manner of workarounds, setting USER, SUDO_USER, LOGNAME, running the container with the "-u" switch to my uid/gid and all get the same logname error. Is there a work around for this?
I had the same problem with the latest spinnaker api release.
The issue is that postinst call logname to find out where your home directory is, to install some config files. In the docker build context, there is no logged in user.
My egregious hack was to overwrite the logname executable with "echo root".
e.g.:
# Install spinnaker sdk https://www.flir.com/support-center/iis/machine-vision/downloads/spinnaker-sdk-and-firmware-download/
COPY external/spinnaker/* spinnaker/
# Pre-answer the apt install prompts
COPY spinnaker.dat .
RUN cat spinnaker.dat >> /var/cache/debconf/config.dat
# Fake out logname (no login context in docker build)
RUN echo "echo root" > /usr/bin/logname
# Install other postinst dependencies
RUN DEBIAN_FRONTEND=noninteractive apt install -y iputils-ping wget
RUN DEBIAN_FRONTEND=noninteractive apt install -y --no-install-recommends ./spinnaker/lib*.deb && rm -rv spinnaker
The contents of spinnaker.dat (to avoid being prompted from the preinst script) are:
Name: libspinnaker/accepted-flir-eula
Template: libspinnaker/accepted-flir-eula
Value: true
Owners: libspinnaker
Flags: seen
Name: libspinnaker/error-flir-eula
Template: libspinnaker/error-flir-eula
Owners: libspinnaker
Name: libspinnaker/present-flir-eula
Template: libspinnaker/present-flir-eula
Value:
Owners: libspinnaker
Flags: seen
Dockerfile:
FROM node:8.14.1-alpine
RUN apk update && apk add --no-cache openjdk8-jre
RUN apk update && apk add python py-pip curl unzip libexif udev nano
ENV CHROME_BIN=/usr/bin/chromium-browser
RUN echo #v3.8 http://nl.alpinelinux.org/alpine/v3.8/community >> /etc/apk/repositories && \
echo #v3.8 http://nl.alpinelinux.org/alpine/v3.8/main >> /etc/apk/repositories && \
apk add --no-cache \
chromium#v3.8 \
nss#v3.8
# Tell Puppeteer to skip installing Chrome. We'll be using the installed package.
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
WORKDIR /project_compile/vue_app
COPY vue_app /project_compile/vue_app
RUN rm -rf node_modules
RUN npm config set registry http://registry.npmjs.org/ && npm install
CMD npm run test:e2e:headless
When it is run, this error happens:
Error retrieving a new session from the selenium server
Connection refused! Is selenium server started?
{ value:
{ message: 'Timed out waiting for driver server to start.\nBuild info: version: \'3.141.59\', revision: \'e82be7d358\', time: \'2018-11-14T08:25:53\'\nSystem info: host: \'a3f993cb3a2b\', ip: \'172.22.0.2\', os.name: \'Linux\', os.arch: \'amd64\', os.version: \'4.15.0-33-generic\', java.version: \'1.8.0_191\'\nDriver info: driver.version: unknown',
error: 'unknown error' },
status: 13 }
ERROR Error: Command failed: /project_compile/vue_app/node_modules/nightwatch/bin/nightwatch --config nightwatch.conf.js --env chrome
Error: Command failed: /project_compile/vue_app/node_modules/nightwatch/bin/nightwatch --config nightwatch.conf.js --env chrome
at makeError (/project_compile/vue_app/node_modules/execa/index.js:174:9)
at Promise.all.then.arr (/project_compile/vue_app/node_modules/execa/index.js:278:16)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:189:7)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! vue_app#0.1.0 test:e2e:headless: `vue-cli-service test:e2e --config nightwatch.conf.js --env chrome`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the vue_app#0.1.0 test:e2e:headless script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2019-01-14T12_14_57_744Z-debug.log
Seems that chrome works correctly (testable running something like chromium-browser --headless --no-sandbox --dump-dom https://www.chromestatus.com/) and also selenium seems to be working fine.
Why does the test not run? The error is not explainable.
All code is contained in this repo https://github.com/nicolalandro/vue-selenium-docker. So reproducing it is easy - clone the repo and run two commands:
docker-compose -f docker/test/js/docker-compose-js-e2e.yml build
docker-compose -f docker/test/js/docker-compose-js-e2e.yml up --exit-code-from e2e
Maybe you have to wait for selenium container to be up?
check out this guide:
https://github.com/SeleniumHQ/docker-selenium#waiting-for-the-grid-to-be-ready
Write down a script to check for "ready": true,
The problem was that the selenium version was not aligned to chrome version.
Trying to create a docker image that has sbt installed and can build sbt projects but, when building, will not be running as the root user (this is all in the context of running Jenkins inside docker).
Dockerfile sets up sbt
ENV SBT_VERSION=1.1.6
RUN \
curl -L -o sbt-$SBT_VERSION.deb http://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb && \
apt-get update && \
apt-get install sbt && \
sbt sbtVersion
And if I then run sbt as the root user, all works ok
docker exec -u root myjenkins sbt sbtVersion
produces
[warn] No sbt.version set in project/build.properties, base directory: /
[info] Set current project to root (in build file:/)
[info] 1.1.6
But when I run sbt as the jenkins user, it tries to download sbt 1.1.6 again and eventually fails when it tries to modify an apt system file.
docker exec -u jenkins myjenkins sbt sbtVersion
produces:
Getting org.scala-sbt sbt 1.1.6 (this may take some time)...
downloading https://repo1.maven.org/maven2/org/scala-sbt/sbt/1.1.6/sbt-1.1.6.jar ...
[SUCCESSFUL ] org.scala-sbt#sbt;1.1.6!sbt.jar (68ms)
.
.
.
[warn] No sbt.version set in project/build.properties, base directory: /
[error] java.io.FileNotFoundException: /var/cache/apt/archives/lock (Permission denied)
I understand that all of the "RUN" commands in your Dockerfile are as a root user.
SBT downloading Scala: Check where it is downloading. SBT by default downloads dependencies on ~/.ivy2 (and/or ~/.m2). If you change user, your home also changes, so it will look for dependencies in /home/jenkins/.ivy2, then on .ivy2 (double-check on this), which do not have those dependencies downloaded already, so it tries to download them.
About the var/cache/apt/archives/lock, it is trying to install via SBT via apt with your jenkins user, when you need to be privileged user to use apt. Your app-user should not need to install anything (or anything that requires root access), but rather build an image with all required installs and then use it as a separate user. Also, if apt gives you headaches, you can just install via download into folder, something like:
RUN \
curl -fsL http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz | tar xfz - -C /usr/local && \
ln -s /usr/local/scala-$SCALA_VERSION/bin/* /usr/local/bin/
PS: You may want to run your container always as jenkins user, in that case you can use USER jenkins after you finished installations and do any additional unprivileged operations there.