In our bower.json we have this dependency:
"ourpackage": "git+https://xxx.visualstudio.com/DefaultCollection/_git/ourpackage#1.2.3"
I'm using alternate credentials and this works fine on my machine.
When I run this on Visual Studio Team Services it fails as it doesn't have my credentials. I don't want to add my personal alternate credentials to the bower.json file. Another solution would be to just add the files to our project without using bower.
Is there another solution to this?
I ran across this same issue and fixed it like this:
Make sure to select Options > Allow Scripts to Access OAuth Token in the build definition.
Add a powershell script task to your build definition task, before your bower install command task
$file = "path\to\bower.json"
(Get-Content $file| ForEach-Object { $_ -replace "https://your.visualstudio.com", "https://un:$env:SYSTEM_ACCESSTOKEN#your.visualstudio.com" }) | Set-Content $file
The one caveat I found is that if you have a dependency tree of bower packages hosted on VSTS, you'll have to add all of the dependencies in the tree to your root bower.json file, i.e. package1 includes package2 in its bower.json - you'll need to include both package1 and package2 in your build project's bower.json.
Try using build access oauth token:
Check this link for reference: Use the OAuth token to access the REST API
Related
I'm trying to get our TFS2015 Build (TFSBuild vnext) going along with VS Team Services.
Thus far most blogs and documentation has been helpful, except for when trying to build a project that uses custom package sources for NuGet automatic package restore.
I've successfully deployed an a Team Services Build Agent (the new mechanism for builds) and all seemed to go well until the build task to Restore NuGet packages failed.
The failure was due to custom/private package source not being picked up from the installed VS configuration, so those packages could not be found. The traditional packages from the default NuGet package feed did restore without issue.
How do I specify the additional package sources for NuGet to use when using an agent?
EDIT: vcastro asked about the NuGet Installer build step and defining paths in the configuration of that build step. The above is using the Visual Studio Build step and the integrated option to have NuGet restore packages. Refer this image for reference:
Alternatively you could also add a NuGet Installer build step before the Visual Studio Build step in your build configuration to restore all NuGet packages.
There you can pass the location of your private repo as argument to nuget.exe:
-source "https://www.nuget.org/api/v2/;http://mynugetserver"
I've scrounged the web with little success, but after fiddling the following will help:
OK It seems that the package sources configured for NuGet.config is stored per user account, e.g.
c:\Users\<<username>>\AppData\Roaming\NuGet\NuGet.config
My issue was harder to resolve, because the build agent was running as a Windows Service under the Local System account. So to get NuGet configuration to for the build, I had to use the following path instead:
64-bit Windows C:\Windows\SysWOW64\config\systemprofile\AppData\Roaming\NuGet\NuGet.Config
32-bit Windows C:\Windows\System32\config\systemprofile\AppData\Roaming\NuGet\NuGet.Config
You may need to have elevated permissions in order to create the NuGet subfolder and NuGet.Config file.
Note: I have no solution for using the Local Service account. The above only works for the Local System (or an actual user) account.
Add a NuGet.config to your project that specifies an alternate package location. The rules for resolution are well-defined and explained in the official documentation.
There is a new VSTS Task called "NuGet Installer" this allows you to check in your NuGet.config file and specify the different package sources. Run this task before you run MSBuild.
If you are using the VSTS NuGet Feed you will need to add the build service account to the feed to enable downloading of packages https://www.visualstudio.com/get-started/package/use/common-identities
One solution (works for me) is change account for tfs 2015 build agent service (on my build machine VSO Agent tsf.Agent-PC) to tfsagent, for example, and add Nuget.config to the C:\Users\tfsagent\AppData\Roaming\Nuget. That's all!
Specify your custom NuGet feed URL’s in the solution’s nuget.config file. Do not store any usernames & passwords in this file.
<add key="nuget.org" value="https://www.nuget.org/api/v2/" />
<add key="MyCompany" value="https://nuget.mycompany.com:443/nuget" />
Create username & password variables in your build definition in VSTS. Variables can be encrypted and will not be displayed in any of the build log outputs. Here I'll create MyCompanyNugetUser and MyCompanyNugetPwd variables.
In our build steps, we add a Powershell script as the first action, this will read the username & password variables and update the user level nuget.config file on the build machine. Below is the code snippet from my inline Powershell script:
Arguments:
$(MyCompanyNugetUser) $(MyCompanyNugetPwd)
Script:
param($user, $pwd)
$nugetFile = "$ENV:AGENT_HOMEDIRECTORY\agent\worker\tools\nuget.exe"
Write-Output "Looking for nuget.exe in $nugetFile"
if (-not (Test-Path $nugetFile))
{
Write-Error "nuget.exe could not be located."
return
}
Write-Output "nuget.exe located"
$cmd = "$nugetFile sources add -name MyCompany -source https://nuget.mycompany.com:443/nuget -username $user -password $pwd -StorePasswordInClearText"
Write-Output $cmd
iex $cmd
Next, we just continue to execute the default NuGet Restore step from Microsoft’s templates
More here: https://codingcase.com/2016/07/27/vsts-build-setup-custom-nuget-feeds-with-authentication/
HTH
In the RTM of Team Foundation Server 2015 you have to add a build step of the type "NuGet Installer", and restore the packages of the Solution file before you run the actual build process. In this task you can pass the argument -ConfigFile path/to/nuget.config which contains your repository path.
For Example:
<configuration>
<packageSources>
<add key="Internal Nuget" value="\\srv-nuget\Repo" />
</packageSources>
</configuration>
If you are having trouble getting this to work on UWP ONLY, then ensure that you have the CASE of the package name spelt correctly. If the case is wrong, then (for UWP only) our build server fails the build.
for instance, if you have a package called Com.Company.Components and update the package using "install-package com.company.components" (note the case of the initial letter) then the UWP build on the build server may fail to find the package in your local store.
Installing a plugin from the Update center results in:
Checking internet connectivity Failed to connect to
http://www.google.com/. Perhaps you need to configure HTTP proxy? Deploy Plugin Failure - Details hudson.util.IOException2: Failed to download from
http://updates.jenkins-ci.org/download/plugins/deploy/1.9/deploy.hpi
Is it possible to download the plugin and install it manually into Jenkins?
Yes, you can. Download the plugin (*.hpi file) and put it in the following directory:
<jenkinsHome>/plugins/
Afterwards you will need to restart Jenkins.
Download the plugin.
Inside Jenkins: Manage Jenkins → Manage Plugins → There is a tab called Advanced and on that page there is an option to upload a plugin (the extension of the file must be hpi).
Sometimes, when you download plugins you may get (.zip) files then just rename with (.hpi) and use the UI to install the plugin.
If you use Docker, you should read this file: https://github.com/cloudbees/jenkins-ci.org-docker/blob/master/plugins.sh
Example of a parent Dockerfile:
FROM jenkins
COPY plugins.txt /plugins.txt
RUN /usr/local/bin/plugins.sh /plugins.txt
plugins.txt
<name>:<version>
<name2>:<version2>
I have created a simple script that does the following:
Download one or more plugins to the plugin directory
Scan all plugins in that directory for missing dependencies
download this dependencies as well
loop until no open dependencies are left
The script requires no running jenkins - I use it to provision a docker box.
https://gist.github.com/micw/e80d739c6099078ce0f3
Sometimes when you download plugins you may get (.zip) files then just rename with (.hpi) and then extract all the plugins and move to <jenkinsHome>/plugins/ directory.
Update for Docker: use the install-plugins.sh script. It takes a list of plugin names minus the '-plugin' extension. See the description here.
install-plugins.sh replaces the deprecated plugins.sh which now warns :
WARN: plugins.sh is deprecated, please switch to install-plugins.sh
To use a plugins.txt as per plugins.sh see this issue and this workaround:
RUN /usr/local/bin/install-plugins.sh $(cat /usr/share/jenkins/plugins.txt | tr '\n' ' ')
Use https://updates.jenkins-ci.org/download/plugins/. Download it from this central update repository for Jenkins.
The accepted answer is accurate, but make sure that you also install all necessary dependencies as well. Installing using the CLI or web seems to take care of this, but my plugins were not showing up in the browser or using java -jar jenkins-cli.jar -s http://localhost:8080 list-plugins until I also installed the dependencies.
The answers given work, with added plugins.
If you want to replace/update a built-in plugin like the credentials plugin, that has dependencies, then you have to use the frontend. To automate I use:
curl -i -F file=#pluginfilename.hpi http://jenkinshost/jenkins/pluginManager/uploadPlugin
In my case, I needed to install a plugin to an offline build server that's running a Windows Server (version won't matter here). I already installed Jenkins on my laptop to test out changes in advance and it is running on localhost:8080 as a windows service.
So if you are willing to take the time to setup Jenkins on a machine with Internet connection and carry these changes to the offline server Jenkins (it works, confirmed by me!), these are steps you could follow:
Jenkins on my laptop: Open up Jenkins, http://localhost:8080
Navigator: Manage Jenkins | Download plugin without install option
Windows Explorer: Copy the downloaded plugin file that is located at "c:\program files (x86)\Jenkins\plugins" folder (i.e. role-strategy.jpi)
Paste it into a shared folder in the offline server
Stop the Jenkins Service (Offline Server Jenkins) through Component Services, Jenkins Service
Copy the plugin file (i.e. role-strategy.jpi) into "c:\program files (x86)\Jenkins\plugins" folder on the (Offline Jenkins) server
Restart Jenkins and voila! It should be installed.
This is a way to copy plugins from one Jenkins box to another.
Copy over the plugins directory:
scp -r jenkins-box.url.com:/var/lib/jenkins/plugins .
Compress the plugins:
tar cvfJ plugins.tar.xz plugins
Copy them over to the other Jenkins box:
scp plugins.tar.xz different-jenkins-box.url.com
ssh different-jenkins-box.url.com "tar xvfJ plugins.tar.xz -C /var/lib/jenkins"
Restart Jenkins.
use this link to download the lastest version of the plugins' hpi. https://updates.jenkins-ci.org/download/plugins/
Then upload the plugin through 'manage plugin' in Jenkins
To install plugin "git" with all its dependencies:
curl -XPOST http://localhost:8080/pluginManager/installNecessaryPlugins -d '<install plugin="git#current" />'
Here, the plugin installed is git ; the version, specified as #current is ignored by Jenkins. Jenkins is running on localhost port 8080, change this as needed. As far as I know, this is the simplest way to install a plugin with all its dependencies 'by hand'. Tested on Jenkins v1.644
RUN /usr/local/bin/install-plugins.sh amazon-ecs:1.37 configuration-as-code:1.47 workflow-aggregator:2.6 \
cloudbees-folder:6.15 antisamy-markup-formatter:2.1 build-timeout:1.20 credentials-binding:1.24
Cat out the plugins.txt and install in Dockerfile as above.
We have a solution with many Class Library projects. These produce a set of DLLs each.
The way we want to deploy these is to install them to the GAC. So we can have a script (BAT file) to run Gacutil /i on the DLL.
We currently use TFS to build our solution using the default TFS build template.
We need TFS to build the solution, and produce folders for each project (the release package) with the DLL and install script. So all we need to do after that is go to the drop folder, see a list of folders (release packages), and everything we need to install each class library in within the folder.
e.g.
-> Dropfolder
-> Proj1
-> Proj1.dll
-> Install.bat
-> Proj2
-> Proj2.dll
-> Install.bat
-> Proj3
-> Proj3.dll
-> Install.bat
Update:
For each project I have set up the following Post-Build event:
powershell -executionpolicy Bypass -file "$(ProjectDir)Deployment\CreateDeploymentPackage.ps1" -Bin $(TargetDir) -Source $(ProjectDir) -Name $(ProjectName)
In my source I have the said powershell script Deployment\CreateDeploymentPackage.ps1
The powershell scripts looks something like this:
param(
[string][parameter(mandatory=$true )] $Bin,
[string][parameter(mandatory=$true )] $Source,
[string][parameter(mandatory=$true )] $Name)
$packagePath = "$Bin$Name"
New-Item "$packagePath" -type directory
Copy-Item *.dll $packagePath -recurse
Copy-Item "$deploymentScripts\Install.ps1" "$packagePath" -recurse
Copy-Item "$deploymentScripts\Uninstall.ps1" "$packagePath" -recurse
Now this works when I build locally. It picks up all dlls in the bin directory and copies them into the folder.
This also works on the TFS build server, but only when building the project by itself.
The problem lies in how TFS builds all the dlls in my solution into a single bin output directory.
Question
Is there a way to copy only the dlls that are related to the project (including dependencies) without explicitly listing each dll?
Is doesn't matter in the slightest, you can setup your source control what ever best suits you needs.
Have a read of this MSDN Article that explains how to control where TFS places your assemblies. For your install.bat files you can check them into TFS and use a post build step to copy them with your output. If you really wanted you might be able to create a custom TFS Activity to generate them as part of the build - have a read of this Blog Series by Ewald Hofman, to get your head around customising TFS builds.
I would like to know, how can I install a plugin to Jenkins, using the Jenkins Remote
access API?
I found a way to install using jenkins CLI. But I need to know how to do the same using API.
I tried using jenkins-python library. But I did not find any way to
install plugin there.
Send (HTTP POST) the following xml data (with your plugin-id#version) to Jenkins plugin manager. Check out my jenkins install plugin script on gist.
This HTTP POST request install jenkins git plugin 2.0.
curl -X POST -d '<jenkins><install plugin="git#2.0" /></jenkins>' --header 'Content-Type: text/xml' http://localhost:8080/pluginManager/installNecessaryPlugins
Some plugins are hard to update on the file system because others depend on it (credentials is one example). For such plugins it is only possible to update them using the web interface.
Jenkins frontend has a page under 'Manage Jenkins' -> 'Manage Plugins'. Under the 'Advanced' tab is a form to 'uploadPlugin'. It allows web automation with curl, you might need to add authentication.
curl -i -F file=#pluginfilename.hpi http://jenkinshost/jenkins/pluginManager/uploadPlugin
In addition to the methods already mentioned (I personally used the "curl uploadPlugin" one provided by #bbaassssiiee), you need to consider that if you use pluginManager Jenkins will try to load your plugin dinamically, but in case you need to restart Jenkins to initialize the plugin properly (this was my case), you should add:
curl -kX POST https://${JENKINS_URL}/safeRestart
In case you copy the plugin directly to jenkins/plugin, the restart is mandatory for the plugin to be loaded.
As suggested by malenkiy_scot, we can create a job and use the Jenkins CLI. Here is the secret way I do for my automation in installing plugins. Jenkins plugins are available in the Jenkins mirror here: http://updates.jenkins-ci.org/latest This link might not list anything but you can download the plugin if you know the name of the plugin. For example, if you want to download the skype-notifier plugin, you can download it from http://updates.jenkins-ci.org/latest/skype-notifier.hpi The generic URL is "http://updates.jenkins-ci.org/latest/.hpi"
After downloading that plugin, it should go to the "plugins" directory in Jenkins home on the server. For linux machine, it will most likely be in "/var/lib/jenkins/plugins". Simple example
wget http://updates.jenkins-ci.org/latest/skype-notifier.hpi
mv skype-notifier.hpi /var/lib/jenkins/plugins
There are two things to note here:
If the plugin has any dependencies, those will not be installed by default. If you know what other plugins are required, those can be installed the same way. A bit of manual process is required here. But if a same set of plugins are required, the dependency can be resolved just once and script can be written to download and move them to the Jenkins home.
Downloaded plugins cannot be used right away. A reload of Jenkins is required.
After a lot of blood sweat and tears my suggested solution is:
Download the hpi files (plugin and dependencies) using plugin-installation-manager-tool
(requires java) or install-plugins.sh (requires bash only, but is officially deprecated, though still working 09/2021)
Note: Both are also contained in official docker image (see also Offline Installations)
Then install all downloaded files via
curl -i -F file=#plugin.hpi http://${JENKINS_URL}/pluginManager/uploadPlugin
Why?
POSTing to /pluginManager/installNecessaryPlugins always installs latest version (known bug or feature?) and seems to only install the requested plugin without proper dependency handling.
Simple example
Requires install-plugins.sh and its dependency jenkins-support from jenkinsci/docker.
You have do adapt install-plugins.sh line 27 to point to your jenkins-support file, e.g.
. jenkins-support if you have everything in one folder and execute it from there.
pluginFolder=$(mktemp -d)
# Download plugins
JENKINS_UC=https://updates.jenkins.io REF="${pluginFolder}" \
install-plugins.sh \
docker-workflow:1.26 docker-plugin:1.2.2
# add more plugins in here, pass a bash array or load from file
# (see Real-life example bellow)
# Install all downloaded plugin files via HTTP
for pluginFile in "${pluginFolder}/plugins"/*; do
curl -i -F "file=#${pluginFile}" http://${JENKINS_URL}/pluginManager/uploadPlugin
done
Real-life example
Taken from cloudogu/gitops-playground.
download-plugins.sh - loads all plugins declared in plugins.txt using install-plugins.sh to a directory passed as parameter.
init-jenkins.sh calls download-plugins.sh, then installs the plugins using jenkins-REST-client.sh
I do not think this is possible. However, as a workaround you may consider creating a job that would install plugins via Jenkins CLI; you then can invoke that job via the API with appropriate parameters.
I would like to know, how can I install a plugin to Jenkins, using the Jenkins Remote
access API?
I found a way to install using jenkins CLI. But I need to know how to do the same using API.
I tried using jenkins-python library. But I did not find any way to
install plugin there.
Send (HTTP POST) the following xml data (with your plugin-id#version) to Jenkins plugin manager. Check out my jenkins install plugin script on gist.
This HTTP POST request install jenkins git plugin 2.0.
curl -X POST -d '<jenkins><install plugin="git#2.0" /></jenkins>' --header 'Content-Type: text/xml' http://localhost:8080/pluginManager/installNecessaryPlugins
Some plugins are hard to update on the file system because others depend on it (credentials is one example). For such plugins it is only possible to update them using the web interface.
Jenkins frontend has a page under 'Manage Jenkins' -> 'Manage Plugins'. Under the 'Advanced' tab is a form to 'uploadPlugin'. It allows web automation with curl, you might need to add authentication.
curl -i -F file=#pluginfilename.hpi http://jenkinshost/jenkins/pluginManager/uploadPlugin
In addition to the methods already mentioned (I personally used the "curl uploadPlugin" one provided by #bbaassssiiee), you need to consider that if you use pluginManager Jenkins will try to load your plugin dinamically, but in case you need to restart Jenkins to initialize the plugin properly (this was my case), you should add:
curl -kX POST https://${JENKINS_URL}/safeRestart
In case you copy the plugin directly to jenkins/plugin, the restart is mandatory for the plugin to be loaded.
As suggested by malenkiy_scot, we can create a job and use the Jenkins CLI. Here is the secret way I do for my automation in installing plugins. Jenkins plugins are available in the Jenkins mirror here: http://updates.jenkins-ci.org/latest This link might not list anything but you can download the plugin if you know the name of the plugin. For example, if you want to download the skype-notifier plugin, you can download it from http://updates.jenkins-ci.org/latest/skype-notifier.hpi The generic URL is "http://updates.jenkins-ci.org/latest/.hpi"
After downloading that plugin, it should go to the "plugins" directory in Jenkins home on the server. For linux machine, it will most likely be in "/var/lib/jenkins/plugins". Simple example
wget http://updates.jenkins-ci.org/latest/skype-notifier.hpi
mv skype-notifier.hpi /var/lib/jenkins/plugins
There are two things to note here:
If the plugin has any dependencies, those will not be installed by default. If you know what other plugins are required, those can be installed the same way. A bit of manual process is required here. But if a same set of plugins are required, the dependency can be resolved just once and script can be written to download and move them to the Jenkins home.
Downloaded plugins cannot be used right away. A reload of Jenkins is required.
After a lot of blood sweat and tears my suggested solution is:
Download the hpi files (plugin and dependencies) using plugin-installation-manager-tool
(requires java) or install-plugins.sh (requires bash only, but is officially deprecated, though still working 09/2021)
Note: Both are also contained in official docker image (see also Offline Installations)
Then install all downloaded files via
curl -i -F file=#plugin.hpi http://${JENKINS_URL}/pluginManager/uploadPlugin
Why?
POSTing to /pluginManager/installNecessaryPlugins always installs latest version (known bug or feature?) and seems to only install the requested plugin without proper dependency handling.
Simple example
Requires install-plugins.sh and its dependency jenkins-support from jenkinsci/docker.
You have do adapt install-plugins.sh line 27 to point to your jenkins-support file, e.g.
. jenkins-support if you have everything in one folder and execute it from there.
pluginFolder=$(mktemp -d)
# Download plugins
JENKINS_UC=https://updates.jenkins.io REF="${pluginFolder}" \
install-plugins.sh \
docker-workflow:1.26 docker-plugin:1.2.2
# add more plugins in here, pass a bash array or load from file
# (see Real-life example bellow)
# Install all downloaded plugin files via HTTP
for pluginFile in "${pluginFolder}/plugins"/*; do
curl -i -F "file=#${pluginFile}" http://${JENKINS_URL}/pluginManager/uploadPlugin
done
Real-life example
Taken from cloudogu/gitops-playground.
download-plugins.sh - loads all plugins declared in plugins.txt using install-plugins.sh to a directory passed as parameter.
init-jenkins.sh calls download-plugins.sh, then installs the plugins using jenkins-REST-client.sh
I do not think this is possible. However, as a workaround you may consider creating a job that would install plugins via Jenkins CLI; you then can invoke that job via the API with appropriate parameters.