Using cypress behind proxy in Jenkins pipeline - jenkins

I have seen this on github but I was still not able to get cypress to configure/download correctly. In my jenkins pipeline I run npm install but this runs into a timeout because of the proxy. It downloads all other dependencies expect cypress.
What I did was download cypress and put the zip file in the project. I then run sh "setCYPRESS_INSTALL_BINARY=cypress.zip npm i cypress" but this still fails.
Part that fails in Jenkins pipeline:
sh "npm config set proxy http://<proxy>"
sh "npm config set registry http://<proxy>/"
sh "setCYPRESS_INSTALL_BINARY=cypress.zip npm i cypress"
sh "npm i"
sh "npm run build
How can I have npm i run without download cypress. I currently cannot get passed this line sh "setCYPRESS_INSTALL_BINARY=cypress.zip npm i cypress" but I am also concerned npm i will still try to download cypress after setCYPRESS_INSTALL_BINARY actually works
-------------------------Update One-----------------
I updated the Jenkinsfile to have CYPRESS_INSTALL_BINARY=cypress.zip npm i but now I get the below error. Cypress.zip is in the project now but would really like to have it uploaded to Jenkins and just reference the file path within my Jenkinsfile.
I am not sure that is possible and the most simple solution is the one I am doing I think however it fails.
New error:
[view] Running shell script
+ CYPRESS_INSTALL_BINARY=cypress.zip
+ npm install
npm WARN locking Error: EIO: i/o error, open '/home/jenkins/.npm/_locks/staging-c21b8f081f002623.lock'
npm WARN locking /home/jenkins/.npm/_locks/staging-c21b8f081f002623.lock failed { Error: EIO: i/o error, open '/home/jenkins/.npm/_locks/staging-c21b8f081f002623.lock'
npm WARN locking stack: 'Error: EIO: i/o error, open \'/home/jenkins/.npm/_locks/staging-c21b8f081f002623.lock\'',
npm WARN locking errno: -5,
npm WARN locking code: 'EIO',
npm WARN locking syscall: 'open',
npm WARN locking path: '/home/jenkins/.npm/_locks/staging-c21b8f081f002623.lock' }
npm ERR! path /home/jenkins/.npm/_locks/staging-c21b8f081f002623.lock
npm ERR! code EIO
npm ERR! errno -5
npm ERR! syscall open
npm ERR! EIO: i/o error, open '/home/jenkins/.npm/_locks/staging-c21b8f081f002623.lock'
script returned exit code 251

I had similar issues on Windows environment. I placed cypress.zip into project's root folder.
set CYPRESS_INSTALL_BINARY=cypress.zip
echo %CYPRESS_INSTALL_BINARY%
npm install cypress --save-dev
So installation did not work, because npm executable looks for cypress in <your project folder>\node_modules\cypress\
Try to set CYPRESS_INSTALL_BINARY to absolute path.

I'm assuming you're on linux, so the way to set an environment variable is CYPRESS_INSTALL_BINARY='cypress.zip' ("set " is only used on Windows)
Cypress will not try to download the binary if it's found on the system, even when running npm install again. By default Cypress downloads the binary to ~/.cache/Cypress, so on subsequent builds Cypress will not have to extract the zip if this directory persists between builds.
The following should work:
CYPRESS_INSTALL_BINARY='cypress.zip` npm install
no need to explicitly run npm install cypress
Then, ensure the ~/.cache directory is saved after and restored before npm install across builds.
or
Pre-extract the cypress.zip onto the build system to /Cypress, and use the following:
CYPRESS_INSTALL_BINARY=0 npm install ## Don't install the binary!
CYPRESS_RUN_BINARY='Cypress/Cypress' ## this is the Cypress executable!
again, make sure /Cypress persists across builds
What if I don't cache ~/.cache ?
Well, your build will still run, but take significantly longer as Cypress will have to extract its binary on every build. Read more here about caching the binary in CI
...
Edit: For Question Update
You're still not setting environment variables correctly. This will not work:
CYPRESS_INSTALL_BINARY='cypress.zip'
npm install
You need to do:
CYPRESS_INSTALL_BINARY='cypress.zip' npm install
In one line.
or
export CYPRESS_INSTALL_BINARY='cypress.zip'
npm install
-> Why here

I created a tool to deal with this problem:
https://github.com/tomasbjerre/dictator-cypress-example
You would need to clone this repository and publish your own cypress-dictator to your private npm repository:
https://github.com/tomasbjerre/dictator-cypress
It uses dictator builder to, before npm install, copy platform specific zip-files to root of repository and make sure it is pointed at by .npmrc.
{
"message": "Copy linux cypress to cypress.zip",
"triggers": [
{
"runningOnPlatform": ["linux"]
}
],
"actions": [
{
"copyFrom": "linux-x64.zip",
"target": "cypress.zip"
}
]
}

Related

npm install failing when it tries to download dependencies from bitbucket server

When i tried to run npm install command from freestyle job in jenkins It is failing with below error
npm ERR! command-line line 0: unsupported option "accept-new"
I found it is a bug -
[BUG] StrictHostKeyChecking=accept-new causes install failure on OpenSSH <7.6 #31 as per https://github.com/npm/git/issues/31
As a workaround they have mentioned to add
GIT_SSH_COMMAND="ssh" npm install
But after that i'm getting the below error
From detailed log
Could you please suggest how to fix this
freestyle job command as below
From the error code - EMFILE - hitting ulimit conditions. After increased ulimit using ulimit -n issue got fixed.

Gitlab CI /CD: Cannot override private project npm package registry URL for dependencies

I was wondering if someone could help me with my CI/CD configuration for a multi-JS project setup.
Project js-core has some core JS libaries, and project vue-core has some reusable vue components. They are both published as npm packages to the "js-core" project's repository (so that we can use a deploy token with npm publish, as opposed to using a group deploy token, where you have to script the package creation and push directly to the API).
vue-core has a dependency of js-core, so it needs to access the npm package registry of the js-core project to download it inside of the Docker CI/CD instance.
However, gitlab does not allow me to override the package registry's URL. According to some google research, setting the values with yarn config set #myorg:registry <gitlab-url> / yarn config set //gitlab.com/api/v4/... <deploy token> or npm config set #myorg:registry <gitlab-url> / npm config set //gitlab.com/api/v4/... <deploy token> should work. However I can see that it is still trying to download the packages from the vue-core package registry, even though it is disabled as a feature there.
This is the part of my .gitlab-ci.yml that runs before it fails:
image: node:17-alpine
stages:
- build
- test
before_script:
- yarn config set #myorg:registry https://gitlab.com/api/v4/projects/${NPM_PACKAGES_PROJECT_ID}/packages/npm/
- yarn config set //gitlab.com/api/v4/projects/${NPM_PACKAGES_PROJECT_ID}/packages/npm/:_authToken ${NPM_PACKAGES_PROJECT_TOKEN}
- yarn install
It fails at yarn install with:
[2/4] Fetching packages...
error An unexpected error occurred: "https://gitlab.com/api/v4/projects/<vue-core_project_id>/packages/npm/#myorg/js-core/-/#myorg/js-core-1.0.0.tgz: Request failed \"404 Not Found\"".
Where the project ID should be the value of <js-core_project_id>.
I have tried writing to all possible .npmrc file paths (~/.npmrc, ./.npmrc, ${CI_BUILD_DIR}/.npmrc, setting the values with npm config set and yarn config set, deactivating the packages feature in the gitlab project itself. I have also not found any predefined environment variables that would override my configs (see https://docs.gitlab.com/ee/ci/variables/predefined_variables.html). #myorg is correct and matches the gitlab url, as the actual value for "myorg" is a single word...
I am pretty much out of ideas at this point, any help is appreciated!
Update 1: I don't think it is a npm / yarn issue, maybe a gitlab caching problem? If I execute npm config ls -l and yarn config list the correct URLs are output in a format that works locally. I will attempt to clear the yarn cache (globally and locally) and pray that that works.

Jenkins Slave (service) , cannot detect protractor

We are using slave as service and we r trying to run protractor with simple batch file after calling to npm install , but from some reason protractor was not detected, do u know what could be the reason/problem?
if i use web option (slave) for running job - everything went fine,
BTW - I try to set the service with user that allowed to run the test, and also set node in PATH, but nothing help
Appreciate your comments,
Thanks
Eyal
because you install protractor as a global package, So you should use webdriver-manager from global package install folder. The current folder where you execute npm install -g protractor has no webdriver-manager this cmd/binary. So jenkins report can't find webdriver-manager in current folder or PATH.
For best practice, you should add protractor as your nodejs project's dependency through npm install -S protractor before you write script, after do that, you will found 'protractor' will be add into package.json.
When others who get your source code, he only need to execute npm install under folder where package.json insides to get all dependencies installed.
After npm install done, the webdriver-manager will be found <package.json file inside folder>\node_modules\.bin\webriver-manager
So your cmd should write down as following:
pwd
ls -l "${WORKSPACE}"
cd /d <package.json file inside folder>
npm install
node_modules\.bin\webdriver-manager update
node_modules\.bin\protractor conf.js

Grunt integration with Jenkins on windows issue

I am running Jenkins on Tomcat7 - Windows 7. I have provided the node bin path in my jenkins configuration. Then running a shell script as follows:
echo $PATH
node --version
npm install -g grunt-cli
npm install
grunt cssmin
As suggested in other post Jenkins integration with Grunt, I have restarted my jenkins several times, and tried to work on all the answers written in that post, but still it shows error, grunt: command not found.
Error stack trace from jenkins console output:
/c/apache-maven-3.2.5/bin:/c/Program Files/Java/jdk1.7.0_79/bin:/c/Program Files/nodejs/bin:/c/Program Files/Java/jdk1.7.0_79/bin:/c/Program Files/nodejs/:
+ node --version
v0.10.30
+ npm install -g grunt-cli
C:\Windows\system32\config\systemprofile\AppData\Roaming\npm\grunt -> C:\Windows\system32\config\systemprofile\AppData\Roaming\npm\node_modules\grunt-cli\bin\grunt
grunt-cli#0.1.13 C:\Windows\system32\config\systemprofile\AppData\Roaming\npm\node_modules\grunt-cli
├── resolve#0.3.1
├── nopt#1.0.10 (abbrev#1.0.7)
└── findup-sync#0.1.3 (lodash#2.4.2, glob#3.2.11)
+ npm install
npm WARN package.json Trademust#1.0.0 No repository field.
+ grunt cssmin
C:\Program Files\Apache Software Foundation\Tomcat 7.0\temp\hudson2968878175697925824.sh: line 6: grunt: command not found
Build step 'Execute shell' marked build as failure
Finished: FAILURE
I have also followed the steps mentioned on this site grunt-on-jenkins
package.json and Gruntfile.js are in root directory the very first time when I executed the jenkins build, grunt installed all modules from my gruntfile.js, and after that in all other build's its showing the above output.
Can anyone please check what's going on wrong here.
After searching a lot, I found where my grunt is installed. As far as jenkins build is concerned it installs in drive:/.jenkins....../workspace/node_modules/.bin.
After providing this path in jenkins using shell script export path=$PATH:drive:/.jenkins....../workspace/node_modules/.bin, grunt started executing.
Also what I learnt in this process was checking where the executable's are available on system path or which path jenkins refers to is using which "executable_name" without qoutes. you can use this command both on windows as well as linux. Ex: which grunt will show the path where grunt executable file is present.
From the error message it seems, sheel is not able to find grunt. Could you please check if it is present in the $PATH variable. On which node this shell script is running? You cab check the $PATH of the particular node. You can also add grunt installation path to $PATH variable during the shell script.
grunt_path="grunt_installtion_path"
export PATH=${PATH}:${grunt_path}

new grunt task added to Jenkins is not found

I am running Jenkins on Windows and grunt v0.4.1. I added a new task grunt-replace to the grunt.js however jenkins is having an issue seeing it:
C:\Jenkins\workspace\xxx>grunt
Local Npm module "grunt-replace" not found. Is it installed?
I've tried several things...
npm install -g grunt-replace
Restarting the jenkins service
npm install grunt-replace
I have installed these as the service jenkins is running under..
What am i missing here? What is the correct way to get jenkins to see the new task?
whats the way you start your task? i would add grunt-replace to the devDependencies in your package.json , add your test-command to the scripts.test-property in your package.json, and then let jenkins execute that.
package.json
{
...some other configs
"scripts": {
"test": "grunt test"
},
"devDependencies": {
"grunt-replace": "*",
... all the grunt-modules you need to be installed
}
}
jenkins excute
$ npm install && npm test

Resources