I am using the Azure Devops CLI on one of my pipelines. In order to use the CLI I need first login (authenticate). Unlike using the REST API, I can't use the OAuth token that is available to me.
So here's my understanding of my options:
I can do an "az login" using a PAT that I map to this environment variable:
AZURE_DEVOPS_EXT_PAT
THIS IS THE WAY I'm doing it now.
Apparently you can use a Service Principal. I like this the most because I should theoretically be able to have this principal apply to everyone on my team. Is that correct?
Use "az login" with a user/password. This is least desirable way to doing it because it involves passing around credentials. Too messy.
Although my pipeline has the OAuth token expost (System.AccessToken), it cannot be use by the CLI. For example is I try to assign the value of the OAuth token to the AZURE_DEVOPS_EXT_PAT it fails (AZURE_DEVOPS_EXT_PAT=$System.AccessToken).
Questions:
Is it possible to use the OAuth token to log in to the CLI?
Is the Service Principal the best way to go?
Additional Info:
I do not have subscriptions only a tenant-id, we're not creating any Azure resources, we're an AWS shop that happens to be using ADO only for CICD.
Use az devops login instead of az login
From your pipeline use:
- script: echo $(AccessToken) | az devops login
env:
AccessToken: $(System.AccessToken)
Few interesting notes:
Secrets (like System.AccessToken) are available to scripts unless you pass them in explicitly as environment variables
the System.AccessToken variable is the default access token of the build agent
there is a project-specific build agent and a project-collection build agent. The one you use is actually controlled by the 'limit access to current project scope' flag in the Pipeline settings for the project.
you may need to elevate permissions for the build agent if you're trying to manipulate objects. For example, you could grant the Create Tag permission on a repository if you wanted the build agent to update the repository.
you can also create your own PAT token with permissions that you specify.
Related
I'm using Bitbucket as a source control service and I'm interested to start using its pipelines capability to build and deploy my app. I'm using AWS CodeArtifact to host my Java artifacts.
The thing I'm struggle with is how to authenticate AWS CodeArtifact from the Bitbucket pipelines.
How to run
aws sso login --profile XXXX
export CODEARTIFACT_AUTH_TOKEN=`aws codeartifact get-authorization-token ....
Is there a best practice to deal with this??
I think the exportation of the CODEARTIFACT_AUTH_TOKEN env var is quite fine. For the first authentication to AWS, you probably want to take a look into Bitbucket OIDC capabilities:
https://bitbucket.org/blog/bitbucket-pipelines-and-openid-connect-no-more-secret-management
https://support.atlassian.com/bitbucket-cloud/docs/deploy-on-aws-using-bitbucket-pipelines-openid-connect/
Essentially, setting up an identity provider in you AWS account that will let your pipelines assume a role by just declaring
- step:
name: My pipeline
oidc: true
...
(also exporting an AWS_ROLE_ARN somewhere)
Identities and the assumed roles can be set up to granular clearance levels per repository, deployment stage, etc
Setting up an OIDC identity provider can be cumbersome. You might be interested in giving https://registry.terraform.io/modules/calidae/bitbucket-oidc/aws/latest a look, even if you weren't using terraform.
We recently migrated from an on-premise TFS server to Azure DevOps. Our team uses TFVC for source control, and I'm getting the following exception when Jenkins polls for new check-ins:
FATAL: This server requires federated authentication but no mechanism was available to handle it.
com.microsoft.tfs.core.exceptions.TFSFederatedAuthException: This server requires federated authentication but no mechanism was available to handle it.
Given the exception class name is TFSFederatedAuthException I suspect Azure is expecting some sort of OAuth integration, but Jenkins doesn't appear to support that for TFVC.
All I did was change the Collection URL for that Jenkins build to https://dev.azure.com/MyCompany. The Project path remains the same, and I verified this, because I was able to re-map all of my TFVC branches in Visual Studio by just pointing to the different collection URL and keeping the same project path. A screenshot of the Jenkins source control config is below:
This Jenkins server is internal with no public facing IP address or host name.
How can I allow Jenkins to poll a TFVC repository hosted in Azure DevOps in order to trigger a CI build in Jenkins?
Why not use Azure pipelines? That's a much bigger migration effort at the moment, and I'm just trying to solve a short term problem.
Using Azure pipelines is my long term goal, but I need to figure out how our automated tests can use an Oracle database first, because all data is deleted before each test is executed using Selenium.
Azure DevOps uses OAuth to communicate by default, putting in your username and password won't work because of that. Instead, the trick is to generate a Personal Access Token (I suspect the Code|Read+Write scope should do it) and pass that in.
For the username pass in ., for the password your generated personal access token. Give the token a nice name so you know which one is about to expire once you get the email notification.
I have installed latest Jenkins on ubuntu server and Jenkinsfile in my project repo on Gitlab.
I am able to connect to private repo on Gitlab using username/password credential on Jenkins configuration for the project pipeline without using Jenkins Gitlab plugin. This does not seem safe to me. How can I use Gitlab API token instead of username/password for Jenkins to access remote private Gitlab repo without using Jenkins Gitlab plugin. Another option is to set ssh private key on Jenkins server to athenticate against Gitlab repo. Is this option possible?
Jenkins Gitlab plugin is not officially supported and not well maintained because Gitlab wants customers to user their own CI/CD solution in order to tie customers to their platform for marketing reasons.
A relatively safe way to handle this situation is to store your credentials is the credentials system in Jenkins (that way you do not have to include the credentials in the JenkinsFile), and using a deploy token (available for Gitlab 10.7 and later) for the relevant repository. That token allows you to provide read-only rights to the repository.
Step 1 - setup the deploy token in GitLab
From the GitLab documentation
You can create as many deploy tokens as you like from the settings of your project:
Log in to your GitLab account.
Go to the project you want to create Deploy Tokens for.
Go to Settings > Repository.
Click on “Expand” on Deploy Tokens section.
Choose a name and optionally an expiry date for the token.
Choose the desired scopes.
Click on Create deploy token.
Save the deploy token somewhere safe. Once you leave or refresh the page, you won’t be able to access it again.
Step 2 - Saving the deploy token in Jenkins' credentials system
Since the deploy tokens have a username and password, pick that as the type in the steps below. Write down the id you will use in this step (see below) as you will need it in your pipeline declaration.
From the Jenkins documentation
To add new global credentials to your Jenkins instance:
If required, ensure you are logged in to Jenkins (as a user with the Credentials > Create permission).
From the Jenkins home page (i.e. the Dashboard of the Jenkins classic UI), click Credentials > System on the left.
Under System, click the Global credentials (unrestricted) link to access this default domain.
Click Add Credentials on the left. Note: If there are no credentials in this default domain, you could also click the add some credentials link (which is the same as clicking the Add Credentials link).
From the Kind field, choose the type of credentials to add.
From the Scope field, choose either:
Global - if the credential/s to be added is/are for a Pipeline project/item. Choosing this option applies the scope of the credential/s to the Pipeline project/item "object" and all its descendent objects.
System - if the credential/s to be added is/are for the Jenkins instance itself to interact with system administration functions, such as email authentication, agent connection, etc. Choosing this option applies the scope of the credential/s to a single object only.
Add the credentials themselves into the appropriate fields for your chosen credential type:
(...)
Username and password - specify the credential’s Username and Password in their respective fields.
(...)
In the ID field, specify a meaningful credential ID value - for example, jenkins-user-for-xyz-artifact-repository. You can use upper- or lower-case letters for the credential ID, as well as any valid separator character. However, for the benefit of all users on your Jenkins instance, it is best to use a single and consistent convention for specifying credential IDs. Note: This field is optional. If you do not specify its value, Jenkins assigns a globally unique ID (GUID) value for the credential ID. Bear in mind that once a credential ID is set, it can no longer be changed.
Specify an optional Description for the credential/s.
Click OK to save the credentials.
Step 3 - Use the credentials in your pipeline declaration
You can use the credentials in your jenkinsFile like so:
pipeline {
stages {
stage('Clone stage') {
steps {
git url: 'https://gitlab.com/[username]/[my-repo].git', branch: 'master', credentialsId: 'my-gitlab-repo-creds'
}
}
}
}
In the above example I assume you picked the id my-gitlab-repo-creds in step 2.
I want to know how to Create the credentials that can be used by Jenkins and by jobs running in Jenkins to connect to 3rd party services.
You should specify which 3rd party service you will work on.
Below is an example of credentials with bitbucket
I am now working with Jenkins ver. 1.568.
By default, there's Credentials feature. So, if you want to add a credential, just click on Add Credentials. For example, I'd like to add SSH Username with password, so I can use it in checking out code from bitbucket
Credentials plugin - provides a centralized way to define credentials that can be used by your Jenkins instance, plugins and build jobs.
Credentials Binding plugin - allows you to configure your build jobs to inject credentials as environment variables.
The third party plugins need to be installed in your Jenkins instance. For example, Assembla Auth Plugin allows you to authenticate to an Assembla repository.
Which 3rd party services are you working with?
Instead of using SSH Username with private key you can simply use username with password option
How to start Jenkins build remotely with cloudbees authentication enabled?
Please let me know the command to start it from a script or throw commang like wget/curl
You can POST to https://accountname.ci.cloudbees.com/job/jobname/build passing your CloudBees username (typically an email address) and Jenkins API token (see /me/configure) for BASIC authentication.
But if you would rather not keep your full credentials in a script, you can pick an arbitrary token and configure that in the job definition, in which case POSTing to build?token=… without authentication will schedule a build. In the usual case that your Jenkins instance is not visible to anonymous users, this will not work (since the whole URL space is blocked), in which case the Build Token Root plugin may be used to accomplish the same thing.