I am running a TFS build for Fortify security results. The web app has a npm/gulp task dependency so those tasks run correctly which then creates the node_modules folder. I then push the scan results to Fortify which is flagging over 600 issues within the node_modules folder. Is there a task I can add to the TFS-Fortify build to run the gulp/npm tasks and then remove or exclude the node_modules from the Fortify scan results?
I have tried adding a windows batch task to the build with the path set to where the node_modules folder will be added and set the arguments to rmdir "node_modules" which causes ##[error]Process completed with exit code 1. Any suggestions?
Related
We have a build pipeline that's been working for a year or two. This builds and creates a few artifacts - ASP.Net web application, windows services, etc. The issue here is with the ASP.Net web application.
We've added some REACT components to the build. There are now an enormous \React\node_modules\ folder with tens of thousands of files. I want to exclude the \React\node_modules folder from the "Publish Artifact" step. I've edited this step, by adding this under the File copy options setting:
/XD react/node_modules
I have a "Publish Artifact" task that has "/XD react/node_modules" set as the File Copy options. However, when I run the build pipeline, it still copies everything in the react/node_modules folder.
When I view the raw logs, I do not see that being passed as an argument in the Robocopy command. I only see this in the logfile for the Publish Artifact step:
2022-04-13T00:24:47.8605686Z ##[command]robocopy.exe /E /COPY:DA /NP /R:3 /MT:8 "C:\agent_agentwork\1\s\Website" "c:\builds\HF\HF_3.8.1.75\Website" "*"
Any idea why it's not working here?
This is a bug in the Publish artifact task. A github issue has been logged for this in 2019 (https://github.com/microsoft/azure-pipelines-tasks/issues/11451). Sadly, it is not fixed yet.
Awaiting a fix, you could work around it by putting a build step that explicitly calls robocopy to copy the build output to a temporary location (with the /XD switch). And then put the 'Publish artifact' step that copies from that temporary location.
We're having issues with our automated deployment system.
On our own computers, running ng build generates the dist folder. Within the folder, it has the assets as expected.
I have replicated this, on the build server, manually pulling the git repository, and running the "build file" (the build server runs on Windows Server. The build and deploy process is managed via a PowerShell script for convenience).
When our Jenkins server runs the build script, the assets folder is missing from the /dist/ folder, as well as some other files configured in angular.json.
It is also not properly compiling the stylesheets, which I've believe is due to the same root cause.
The issue persists when running the PowerShell script directly from the Jenkins workspace when the shell is run as a system administrator.
The CLI does not produce any errors.
I'm attaching a verbose log, in case this could be helpful.
https://gist.github.com/cf-jola/6cc6cff138da5105f3b10adffb72895f#file-output-txt
Running the script as the system administrator I've ruled out it being a permissions issue. Jenkins is also managing to create other files, such as the
.js files, and the index.html fine.
My workaround right now is to, via the deploy powershell script, to manually copy the assets folder, how-ever I'd love to get rid of this workaround as we're starting to get multiple files in our angular.json > assets section.
For references:
angular.json https://gist.github.com/cf-jola/6cc6cff138da5105f3b10adffb72895f#file-angular-json
deploy script: https://gist.github.com/cf-jola/6cc6cff138da5105f3b10adffb72895f#file-deploy-ps1
Its a bug, in either Node or Angular CLI.
Because we have brackets, ( & ), in the build path, they get encapsulated in square brackets.
This causes the path: C:\Program Files (x86)\Jenkins\... to become this C:\Program Files [(]x86[)]\Jenkins\... and thereby become invalid.
We discovered the issue by using Process Monitor and looking over the events generated during the build process.
I am trying to create a jenkins slave for building gradle lambda projects. Jenkins slave is throwing the below error while building the project.
Exception in thread "main" java.lang.RuntimeException: Could not create parent directory for lock file /gradle/wrapper/dists/gradle-4.2.1-bin/dajvke9o8kmaxbu0kc5gcgeju/gradle-4.2.1-bin.zip.lck
at org.gradle.wrapper.ExclusiveFileAccessManager.access(ExclusiveFileAccessManager.java:43)
at org.gradle.wrapper.Install.createDist(Install.java:48)
at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:107)
at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)
/home/jenkins/workspace/ddoa-subprod/lf-security-gateway2/lf-security-gateway2
FAILURE: Build failed with an exception.
* What went wrong:
Failed to load native library 'libnative-platform.so' for Linux amd64.
Please help me in understanding the issue and let me know how to fix the same.
To fix this error: What went wrong: Failed to load native library 'libnative-platform.so' for Linux amd64. do the following:
Check if your Gradle cache (**~user/.gradle/**native folder exist at all or not).
Check if your Gradle cache (~user/.gradle/native folder exist and the file in question i.e. libnative-platform.so exists in that directory or not).
Check if the above folder ~user/.gradle or ~/.gradle/native or file: ~/.gradle/native/libnative-platform.so has valid permissions (should not be read-only. Running chmod -R 755 ~/.gradle is enough).
IF you don't see native folder at all or if your native folder seems corrupted, run your Gradle task ex: gradle clean build using -g or --gradle-user-home option and pass it's value.
Ex: If I ran mkdir /tmp/newG_H_Folder; gradle clean build -g /tmp/newG_H_Folder, you'll see Gradle will populate all the required folder/files (that it needs to run even before running any task or any option) are now in this new Gradle Home folder (i.e. /tmp/newG_H_Folder/.gradle directory).
From this folder, you can copy - just the native folder to your user's ~/.gradle folder (take backup of existing native folder in ~/.gradle first if you want to) if it already exists -or copy the whole .gradle folder to your ~ (home directory).
Then rerun your Gradle task and it won't error out anymore.
Gradle docs says:
https://docs.gradle.org/current/userguide/command_line_interface.html
-g, --gradle-user-home
Specifies the Gradle user home directory. The default is the .gradle directory in the user’s home directory.
Note: using gradle <sometask> -g <a_dynamic_folder_ex_jenkins_workspace> will always work as Gradle will create fresh .gradle cache in that -g defined folder, but doing this, it'll not reap the true benefit of Gradle's cache concept.
If you are using a version 3.4 if Gradle, then it could possibly be this issue.
To fix it, you can try to update your Gradle distribution to version 3.5 or higher, where this issue was solved.
I ran the command as sudo and it went through fine
I'm running latest version of TFS 2015 release manager. I have a simple release that includes a "CopyFiles" task. I need to have the "Clean Target Folder" option of the task enabled.
I have three target environments configured. Both are newly created identical Windows servers. The CopyFiles task works fine against the first two servers but for the next environment in the pipeline, when copying to the second server I get the following error:
2016-11-11T21:41:07.4324714Z found 2 files
2016-11-11T21:41:07.4324714Z ##[debug]file:D:/Build/_work/df9f2c4cf/(master) Orchard BTDF/drop/20161111.5.zip will be copied.
2016-11-11T21:41:07.4324714Z ##[debug]file:D:/Build/_work/df9f2c4cf/(master) Orchard BTDF/drop/Unzip.ps1 will be copied.
2016-11-11T21:41:07.4324714Z Cleaning target folder: \\Steatbt02\BizTalkDeployments2\Orchard
2016-11-11T21:41:07.4324714Z ##[debug]rm -rf \\Steatbt02\BizTalkDeployments2\Orchard
2016-11-11T21:41:07.4480915Z rm: could not remove directory (code EPERM): \\Steatbt02\BizTalkDeployments2\Orchard
2016-11-11T21:41:07.4480915Z ##[debug]task result: Failed
There's no sign of errors in the event log.
I created the share "\Steatbt02\BizTalkDeployments2" and granted "full-access" permissions to the account running the build agent. I even tried logging onto the build server and run as for the command prompt using the credentials of the build agent. From this command window I was able to remove the target folder
"\Steatbt02\BizTalkDeployments2\Orchard" - so I can't think why the build agent would have trouble doing the same!
Any ideas what could be wrong, or what extra steps I could take to track down the root cause of the problem?
The problem was the agent queue selected for the problem environment.
I've setup a .Net TFS Build vNext build to run the following:
npm install
gulp
visual studio solution build
The build is confgured to clean automatically before checking for sources like so:
I have two problems with this build but this question is about problem #2.
The npm install step fails to install phantomjs because the command node install.js can't find node, even though node is in the System path.
Running builds configured to Clean fails because of the path too long error below.
[error]The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
I would like for the Clean setting to just work and clean up these longs paths, but I don't know how to make it work properly.
When I tried to re-create this same build in TeamCity the Clean option works just fine. Also, I know that if I had access to the script that is running the clean that it could just use a *rmdir /S /Q to remove these paths. Hopefully the VSO build can just run this or something that works.
[EDIT] - I have confirmed that this is fixed in the latest Build agent version (1.85.1)
I've been told by MS folks that this is currently a bug in the vNext builds and will be fixed in the next release of the product. I'll update this answer once I see this fix released.
From an admin console, run: npm install npm -g
Run npm version or npm -v to make sure it worked. Also note if you have node installed, you want to browse to the node directory and update it from there.
Also note vso agents sometimes store a local version. Browse to agent/tasks/npm.