Azure DevOps Server 2020 with self hosted agents on a different server
I have a build, which is creating an artifact in Azure Artifacts, then a release pipeline is triggered which should download this artifact and do something with it. Previously, the output of a build was on a file share and it work as expected. When I switched to Azure Artifacts - the download artifact task (which is added automatically by Azure DevOps) fails with following:
All other build tasks work on this server without a problem, and also this release pipeline works as expected on other servers. How I can start to troubleshoot this issue as I don't see any meaningful error message?
According to your screenshot, I could reproduce the similar issue in Azure Devops Server.
But in the task, I could see the error message:
Failed in getBuildApi with error: Error: self signed certificate.
If you have the same issue, you could try the following methods:
1.You could re-config the agent with the self signed certificate.
.\config.cmd --sslcacert ca.pem xxxx
Here is a ticket about the detailed steps, you could refer to it.
2.You could check if you have set the firewall. Firewalls could block the download of artifacts
3.You could set the system environment :NODE_TLS_REJECT_UNAUTHORIZED=0 and restart the agent services.
We had similar problem with self-hosted agents but there was a minor difference - our agents were deployed in vnet.
If that's the case make sure that in vnet's subnet you have enabled Microsoft.Storage service endpoint.
Hope this will save someone tons of time.
Related
I have a Git repository on the Azure Dev-ops server and use Jenkins for continuous integration build.
I want to know that how a specific branch like master Jenkins can automatically run the build and then notify the user via a shell log that the build was successful or not?
Microsoft seems to have the thing pretty well documented, Create a service hook for Azure DevOps Services and TFS with Jenkins
Set up the Jenkins job, set up the TFS / Azure DrevOps ServiceHook, off to the races.
We have it working fine for Jenkins 2.x and AzureDevOps on-prem. Best to use service accounts with limited necessary permissions on both sides.
We have a Git repo on TFS and I am trying to create a pipeline using azure pipelines to connect to the TFS repos.
I get the following error:
fatal: unable to access 'http://tfs.****************': Could not resolve host: tfs.******
##[error]Git fetch failed with exit code: 128
I would suggest you first use "git clone" command line to clone remote repo.
Kindly check when you run it manually from the build agent, it work for that repo or not.
This will narrow down if the issue related to your environment or pipeline.
If you are able to use git command to connect and clone that repo.
This means there is something wrong with your build service
account. You should make sure build service account has access to that
repo. You could also directly change the service account to the one
you used to run git command.
If you are not able to do it. Then this may related to network
environment. Make sure your build agent are able to access TFS
on-premise server. Temporarily turn off firewall and any proxy. Also
try to directly use browser to login TFS web portal.
It seems that it is a self-deployment of TFS server then you need to make sure that the Server can be reached from Azure DevOps.
Based on URL in your post, I assume server is not reachable from public internet. So TFS server should be either on-prem or on a VM in Azure. So reach out to your infrastructure team to see where the server is, and how the connection could be established from build agent being used by Azure DevOps to the TFS server.
I have a local Jenkins server running on one of my spare computers (win10). Note that it is not behind any sort of a server and hence is only available within my local network. I have set it up so that it does the continuous fetch from my remote git repo and builds the artifacts and archives them for a successful build. I would like to publish these archives to my AzureDevops Release pipeline. How do I do this? (And yes I have looked through all the tutorials but they assume that I have Jenkins running on a VM somewhere on the cloud).
So far I have had no luck with the tutorials on the web since I donot really have a URL to this instance of Jenkins since it is only available on my local network. I cannot really build these artifacts on a remote Jenkins server, so I am really restricted to using this solution for running the builds.
I am looking to have these archives that Jenkins builds be directly available within my Azure DevOps release pipeline, on every successful build. Thanks for the help!
So since nobody else has answered this I am going to detail what I ended up doing (maybe not the best of the approaches but it works for my setup, suggestions are welcome!).
To interface with the Azure DevOps platform from a local machine you will need to configure a self-hosted agent (based on your specific OS), which will allow you to trigger builds, archive and upload the build artifacts to the Azure DevOps platform. This way you also donot have to poll for SCM changes too (which I think is not that elegant sometimes).
1. So you will need to go through the setup as outlined here for you local self-hosted agent:
Windows: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops
Linux: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-linux?view=azure-devops
MacOS: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-osx?view=azure-devops
NOTE: I have chosen to run the agent as service on windows for my setup
2. Next setup your Jenkins build job how you normally would, with your usual repo access setup. Things to keep in mind are following:
Under "Build Triggers", select the Poll SCM option, but make sure that the schedule is blank, this will make sure that the trigger from your post-commit hook from the agent works. Example setup shown below:
Under "Post-build Actions", make sure that you are archiving the artifacts as required. Example shown below:
3. Now time to setup your project's "Jenkins Service Connection", this can be accessed from the Project Settings tab on the bottom left of you project view in Azure DevOps. Note that this basically helps you self-hosted agent to locate and communicate with the Jenkins instance running locally (or an other network accessible location!). Go under Pipelines -> Service Connections and a new service connection for Jenkins. Note that the trick here is to use the URL for the connection as seen by you local self-hosted agent, which means it can be just any IP (including localhost) that the agent can access normally. Username and password are the same as the ones you setup in Jenkins. Example shown below:
NOTE: You can try to do "Verify and Save" but it will throw an error, so ignore the error or just go ahead and "Save without verification". Also you will have to do this per project, unlike the self-hosted agent setup which is per machine.
4. Now you just need to configure your build pipeline to give jobs to the right agent and pointing to the right service end-point. Now under you build pipeline settings use the agent pool that has the self-hosted agent(s) which can access your build servers. And choose the Jenkins connection that you just created in the above step. The rest of the setup is identical to how you would normally setup your project's build pipeline. An example would be as follows:
NOTE: The key here is the correct "Job name" (this should be the same as the one you have setup in you Jenkins build server instance) and the correct "Jenkins service connection".
5. The rest is straight forward in the sense that you just now need to make sure that you have a step to "Download artifacts" (NOT necessary if you donot want the artifacts on the DevOps platform) & "Publish Artifacts" (this is needed for your release pipeline to see that build artifact and to trigger it too if you want), after your jenkins queue job step. Make sure to setup the correct job directories for download from you local self-hosted agent. Example setup for both the steps:
NOTE: If you are having trouble with the paths for download and publish refer to this link for predefined variables for the self-hosted agents: https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml
6. Now in your release pipeline you should be able to add the artifact sources from you build pipeline. Example shown below:
Now you should be able to get the local artifacts in the cloud on the Azure DevOps platform, in case you cannot use the build agents provided by Microsoft for any reason!
I noticed that all of my builds suddenly got stuck. If I click the build I get the following error: All eligible agents are disabled or offline
When I check my agent pools I get the following:
How can I bring them online/determine why they are offline?
Try logging into the VM, and run the run.cmd from an elevated powershell prompt in your agent directory.
Source: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops
Login to powershell.Navigate to the agent folder where it contains run cmd prompt and config cmd prompt.
Execute the following command
.\run.cmd
Check the date and time on your VM. If it's wrong, correct it.
Then run powershell as administrator and cd into the azagent folder that contains the run.cmd file (most likely C:\azagent\A4).
Execute the following command:
.\run.cmd
For me, issue was suddenly agents were reporting offline on the portal.
Restarted the agent service in the VM - didn't help
Removed config file also didn't help because logs were showing that dev.azure.com wasn't connecting
Simply just restarting VM helped, but still couldn't able to find the cause.
The VM that host the build agents can have a auto shut down time. Make sure your build vm is running by going to Virtual machines in the azure portal
My agent stopped going live when I updated our Azure Devops to the latest version 2019.1.3 iirc.
To solve this I did the following:
Went to services.msc on the agent machine. Stopped the existing service.
Went to agent folder and removed the existing configuration via "config.cmd remove"
Downloaded latest agent code from the Azure Devops Agent Pools portal page.
Replaced contents of old agent with new agent files.
Went back to agent folder and configured a new agent via "config.cmd". Told it to start as a service.
Voila! Service back running again.
One of my Azure agents started showing as offline. As a test, I ran curl https://dev.azure.com/{organization} and got back an error "curl: (35) schannel: failed to receive handshake, SSL/TLS connection failed".
I fixed the issue by downloading IISCryptoCli.exe from https://www.nartac.com/Products/IISCrypto/Download and then running "IISCryptoCli.exe /template default /reboot".
I solved this by connecting to the agent via Remote Desktop Connection and running the "Run Puppet Agent" application.
Please check the agent from agent tab under releases.
There should be a server listed for the agent under agent tab.
Now connect to that server and check weather the agent batch file or cmd window is open or not.(this agent for is the same file which is downloaded while configuring the agent.if you are struggling to find the file then follow new agent steps and download zip again and search with that file name.)
Now open the(filename -run) file on that server with admin right if it is already not running. This will make agent online in devops portal.
Now check the agent tab again in devops portal under releases.
This
We are using Hudson as a CI tool. At present we are needed to use Jenkins, to deploy the build to Stage, Prod environment. What is the best aproach we should follow.
I know about promote buld plugin, but the issue is authentication. I want whevener we need to promote a build to deploy to Stage or Prod, it should ask for netqwork credential first. And then the promote job should execute the Batch command using the creadential supplied. At present, the promote plugin, runs using the credentials which the Tomcat server is configured to run.
Same issue with Build Pipeline plugin.
I want only dev or even hudson admin also should not be able to execute the promote build unless credential supplied. (We have windows 2008 r2 OS)
Can you please help me in resolving the issue. so that basically whenever a user click on Promote build to QA\Stage\Prod the plugin should ask for credential or should use the logged on users credential and execute the batch script using the logged users credential only and not by using the credentials of the account with which the tomcat server is configured.
Can you please help me?
Please suggests us the best aproach for making automated build on prod\stage.
For deployment I normally use SSH, Private/Public keys takes care of the authentication problems normally associated with running commands on other servers.
SSH is normally associated with unix based systems, but it does support windows.
Finally, I would recommend considering decoupling your build system (jenkins) from the system performing the deployment by using an intermediate repository. See the following answer for more details:
Jenkins : how to check out artifact from Nexus and Deploy on Tomcat-