I'm trying to define and build a docker container via a Dockerfile that is pulling in an npm dependency that is privately hosted in GCP Artifactory
The .npmrc file works for publishing the dep. But when the docker container runs npm install it struggles to access the private npm registry.
How do I grant a Dockerfile the permission to do an npm install where one of the dependancies is hosted on GCP's artifactory?
Inject the credentials? Copy them in? (doesn't seem safe)
I get an error like this:
---> Running in 058536ed8d33
npm ERR! code E403
npm ERR! 403 403 Forbidden - GET https://us-central1-npm.pkg.dev/<.....>
Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/<....>/locations/us-central1/repositories/<...>" (or it may not exist)
Related
I've got a mono-repo setup with lerna and yarn workspaces.
To build the docker images for servers in the project, I'm using verdaccio as a GitLab service.
In the build phase, I want to publish packages and when running the server I'm planing to install them from the verdaccio registry.
With GitLab CI services we can not pass config files to the service. (APMK)
Currently the error I'm having is, error with auth, since I'm trying to publish a scoped package.
Step 14/24 : RUN lerna publish from-package --yes --no-git-reset
---> Running in 9c2db92f90f0
info cli using local version of lerna
lerna notice cli v4.0.0
lerna notice FYI Unable to verify working tree, proceed at your own risk
lerna WARN Unable to determine published version, assuming "#scope/package-1" unpublished.
lerna WARN Unable to determine published version, assuming "#scope/package-2" unpublished.
Found 2 packages to publish:
- #scope/package-1 => 1.0.0
- #scope/package-2 => 1.0.0
lerna info auto-confirmed
lerna info publish Publishing packages to npm...
lerna notice Skipping all user and access validation due to third-party registry
lerna notice Make sure you're authenticated properly ¯\_(ツ)_/¯
lerna notice FYI Unable to set temporary gitHead property, it will be missing from registry metadata
lerna info lifecycle root#undefined~prepare: root#undefined
lerna WARN lifecycle root#undefined~prepare: cannot run in wd root#undefined husky install (wd=/home/app)
lerna http fetch PUT 401 http://verdaccio:4873/#scope%2fpackage-1 26ms
lerna ERR! E401 authorization required to publish package #scope/package-1
The command '/bin/sh -c lerna publish from-package --yes --no-git-reset' returned a non-zero code: 1
In the CI environment I can not run npm adduser since it's an interactive prompt. I tried few way to bypass is seems they doesn't work.
Some like
RUN npm adduser <<! \
auser \
'1234' \
user#domain.tld \
!
Can I create a user inside a Dockerfile or can I bypass auth for scoped packages?
I'm running a docker build in windows and npm has failed in a step called array-slice. I installed the dependency manually and re-ran the build. Then it failed on buffer-from.
It says it cannot access the dependency url in the error - but it works when I test it in the browser - which seems to me that my docker build process is having trouble to access some urls, for some reason...
npm WARN server#1.0.0 No description
npm WARN server#1.0.0 No repository field.
npm ERR! network timeout at: https://registry.npmjs.org/array-unique/-/array-unique-0.3.2.tgz
Is there any necessary additional config to make sure npm can access those dependencies?
This is my Dockerfile content:
FROM node:12-alpine
WORKDIR /usr/app
COPY package.json package-lock.json ./
RUN npm install
COPY . .
EXPOSE 3333
CMD ["npm", "run", "dev"]
It was just an installation issue on windows... Removed everything, including virtualbox, and installed it all again... seems to work fine now...
The only problem is that my app is still not exposed to the port I've requested... but that's another question.
I am running docker on ubuntu 16.04. While trying to make my nodejs app inside docker container accessible by https, I made a few changes to port mapping like so:-
docker run -p 127.0.0.1:443:8443/tcp bookshelf-server:0.1 bash
docker run -p 127.0.0.1:80:8443/tcp bookshelf-server:0.1 bash
because I was trying to map port 8443 as I was listening to it for https
This did not work, but it broke my npm install inside docker container. Or maybe something else might have broken it as I was not testing it while trying experiments.
Nevertheless, now when npm run install runs from the docker container, it gives the following error :-
Step 5/13 : RUN npm install
---> Running in 7c78745fb509
npm ERR! code ECONNREFUSED
npm ERR! errno ECONNREFUSED
npm ERR! FetchError: request to https://registry.npmjs.org/body-parser failed, reason: connect ECONNREFUSED 104.16.22.35:443
npm ERR! at ClientRequest.req.on.err (/usr/local/lib/node_modules/npm/node_modules/pacote/node_modules/make-fetch-happen/node_modules/node-fetch-npm/src/index.js:68:14)
I reckon I might have messed up some port connections or something, can anyone give me a direction of how to debug the issue.
Thanks
I've a frustrating issue when "npm install" is executed inside a Jenkins Groovy pipeline using the NodeJS plugin, the process hangs with the following error -
npm install --ddd ng-cli
npm info it worked if it ends with ok
npm verb cli [ '/var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/nodejs893-v2/bin/node',
npm verb cli '/var/jenkins_home/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/nodejs893-v2/bin/npm',
npm verb cli 'install',
npm verb cli '--ddd',
npm verb cli 'ng-cli' ]
npm info using npm#5.5.1
npm info using node#v8.9.3
npm verb npm-session e522ad0a36f1c038
npm sill install loadCurrentTree
npm sill install readLocalPackageData
npm http fetch GET 503 https://registry.npmjs.org/ng-cli 70252ms attempt #3
npm sill fetchPackageMetaData error for ng-cli#latest 503 Service Unavailable: ng-cli#latest
npm verb stack Error: 503 Service Unavailable: ng-cli#latest
When the command is executed directly on the EC2, the package installs without issue as the Jenkins user.
Also when the command is executed inside the Jenkins Docker, the package installs without issue as the Jenkins user using the same Node installation.
The Docker instance is not limited by CPU or RAM.
The setup is Jenkins v2.138.1 running inside a Docker container, which in turn is hosted on an EC2 v2018.03. Jenkins home is mounted as an EFS volume. The JVM is running on Java v1.8.0_181. NPM is v5.1.1.
Any pointers would be much appreciated.
Reply to first suggestion
Yes there is direct internet connectivity, without any proxy. If a single package is installed such as
npm install ng-cli
The npm install works without issue.
The issue consisted of two parts -
First off - the EFS mount point directory (/var/jenkins_home) required permissions of 777, it doesn't need to be a recursive permission.
The new EFS disk had content migrated from the old Jenkins EFS and this was also contributing to this issue. The fix was to not transfer any content from the old EFS to the new EFS via the backup.tar.gz feature. The new Jenkins is working as expected with npm install.
Dockerfile pulled from - https://hub.docker.com/r/jenkins/jenkins/
I'm trying to build a docker image with a global install of firebase-tools and angular-cli.
I'm building the same image for two versions of node: 6.x (LTS boron) and v8.x (latest alpine). Locally, both images build fine, but when I try to build in docker hub only the v6.x builds successfully. With the v8.x it gets trapped in the access permissions for undefined and nobody user. I'm already using the root user (USER root), since without this setting (or using USER node) both images fail to build.
This is my Dockerfile:
FROM node:latest
USER root
RUN npm install --quiet --no-progress -g #angular/cli#latest firebase-tools
RUN npm cache clean --force
And this is the output:
Step 4/5 : RUN npm install --quiet --no-progress -g #angular/cli#latest firebase-tools
---> Running in fce3da11b04e
npm WARN deprecated node-uuid#1.4.8: Use uuid module instead
/usr/local/bin/firebase -> /usr/local/lib/node_modules/firebase-tools/bin/firebase
/usr/local/bin/ng -> /usr/local/lib/node_modules/#angular/cli/bin/ng
> node-sass#4.5.3 install /usr/local/lib/node_modules/#angular/cli/node_modules/node-sass
> node scripts/install.js
Unable to save binary /usr/local/lib/node_modules/#angular/cli/node_modules/node-sass/vendor/linux-x64-57 : { Error: EACCES: permission denied, mkdir '/usr/local/lib/node_modules/#angular/cli/node_modules/node-sass/vendor'
at Object.fs.mkdirSync (fs.js:890:18)
at sync (/usr/local/lib/node_modules/#angular/cli/node_modules/mkdirp/index.js:71:13)
at Function.sync (/usr/local/lib/node_modules/#angular/cli/node_modules/mkdirp/index.js:77:24)
at checkAndDownloadBinary (/usr/local/lib/node_modules/#angular/cli/node_modules/node-sass/scripts/install.js:111:11)
at Object.<anonymous> (/usr/local/lib/node_modules/#angular/cli/node_modules/node-sass/scripts/install.js:154:1)
at Module._compile (module.js:569:30)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:503:32)
at tryModuleLoad (module.js:466:12)
at Function.Module._load (module.js:458:3)
errno: -13,
code: 'EACCES',
syscall: 'mkdir',
path: '/usr/local/lib/node_modules/#angular/cli/node_modules/node-sass/vendor' }
> grpc#1.3.8 install /usr/local/lib/node_modules/firebase-tools/node_modules/grpc > node-pre-gyp install --fallback-to-build --library=static_library
node-pre-gyp ERR! Tried to download(undefined): https://storage.googleapis.com/grpc-precompiled-binaries/node/grpc/v1.3.8/node-v57-linux-x64.tar.gz node-pre-gyp ERR! Pre-built binaries not found for grpc#1.3.8 and node#8.1.2 (node-v57 ABI) (falling back to source compile with node-gyp)
gyp WARN EACCES user "undefined" does not have permission to access the dev dir "/root/.node-gyp/8.1.2" gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp"
gyp WARN EACCES user "nobody" does not have permission to access the dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp/8.1.2" gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp"
gyp WARN EACCES user "nobody" does not have permission to access the dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp/8.1.2" gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp"
gyp WARN EACCES user "nobody" does not have permission to access the dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp/8.1.2" gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp"
gyp WARN EACCES user "nobody" does not have permission to access the dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp/8.1.2" gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/local/lib/node_modules/firebase-tools/node_modules/grpc/.node-gyp"
(infinite loop)
The problem is because while NPM runs globally installed module scripts as the nobody user, which kinds of makes sense, recent versions of NPM started setting the file permissions for node modules to root. As a result module scripts are no longer allowed to create files and directories in their module.
See discussion in NPM issue #3849, for some references.
A simple workaround, which makes sense in a docker environment, is to set the NPM default global user back to root, like so:
npm -g config set user root
After which you shouldn't have any more EACCES errors.
Please DO NOT set the user to root or use --unsafe-perm.
Simply use the node user, provided with the current official (e.g. alpine) images.
Commented Dockerfile below:
FROM node:15.5-alpine
# 👉 Security: do not use the `root` user.
ENV USER=node
# You can not use `${USER}` here, but reference `/home/node`.
ENV PATH="/home/node/.npm-global/bin:${PATH}"
# 👉 The `--global` install dir
ENV NPM_CONFIG_PREFIX="/home/node/.npm-global"
# All subsequent commands are run as the `node` user.
USER "${USER}"
# Pre-create the target dir for global install.
RUN mkdir -p "${NPM_CONFIG_PREFIX}/lib"
WORKDIR /usr/src/app
COPY package.json package-lock.json ./
# 👉 Configure NPM, so pkg get installed with correct credentials.
# Avoids `chmod u+x $DIR` and other workarounds.
RUN npm --global config set user "${USER}" \
&& npm --global --quiet --no-progress install \
&& npm cache clean --force
Extended version of #gabriel-araujo answer.
You can replace setting the user via the CLI flags (npm --global config set user "${USER}") by configuring this in the .npmrc file. Place user=node in the project roots .npmrc file or just set it directly from the Dockerfile.
RUN echo "user=node" > "${NPM_CONFIG_PREFIX}/etc/.npmrc"
If you use docker-compose.yml, you can add it as environment: - … variable.
Hope that helps you running a more safe NodeJS container somewhere and pull up some awesome stuff.
Use the --unsafe-perm flag:
npm install --quiet --no-progress --unsafe-perm -g #angular/cli#latest firebase-tools
I think that's still better than setting the npm user permanently to root. You can use --unsafe-perm only with the packages which cause problem
I was able to get it working by changing the default npm-global directory.
This is my dockerfile now:
FROM node:latest
USER node
RUN mkdir /home/node/.npm-global
ENV PATH=/home/node/.npm-global/bin:$PATH
ENV NPM_CONFIG_PREFIX=/home/node/.npm-global
RUN npm install --quiet --no-progress -g #angular/cli#latest firebase-tools
RUN npm cache clean --force
instead of forcing NPM's hand to install the package inside the container I bypassed this issue by mapping/referncing a host folder for the missing module
this also prevents future headaches for when I replace the docker image with the latest version, I won't have to repeat the stages to reinstall the missing module inside the container:
the steps i took:
create an empty folder on the host environment (will be used as a target for the node js modules). call it node_modules
when launching running the docker container use the --volume and --env switches
-- volume to pass map the new host folder (from step 1) to a folder accessible inside docker
--env to define/set an environment variable NPM_CONFIG_PREFIX from inside the docker to the /node_modules folder we created in step 1
access the container using :
sudo docker exec -i -t sh
and go to the folder above the folder right above the local /node_modules folder (this not the folder we mapped to the environment variable, but rather the preexisting folder that came with the docker image)
then run the command:
> npm install -g <mdule-name>
example
> npm install -g request
this will install the module in the host folder we've created. this module will also be accessible from both docker and host.