I am trying to migrate a teamcity pipeline to a jenkins one.
I am used to teamcity, and I still finding my way around Jenkins, but I haven't found a way to do this.
I want to have a pipeline with 2 steps, one for compiling an application, and a second step to send it to a server using SSH with private keys.
I can easily do that in Teamcity (after some research) but with Jenkins I have researched it and haven't found a way to get there, nor examples over the web.
I have tried a freestyle project where I can configure SSH agent but I can't seem to find where to place the rest of the logic, and a multibranch pipeline which uses a Jenkinsfile within the repo to compile, but nowhere to set details for sending files over.
Should I use a multibranch pipeline and "send" the files from within the jenkinsfile? If so, how do I tell Jenkins to make the key available to the docker container?
Or should I use the freestyle project, and if so, how do I tell it to use the Jenkinsfile first, and then send the resulting file or files to the destination server?
Or should I use something completely different?
You can do that in a pipeline :
1/ Add your private key to Jenkins by creating a new credential (ssh Username with private key)
2/ In a jenkinsFile :
node {
try {
stage ("build") {
dir('Build') {
env.NODEJS_HOME = "${tool 'Node 12.12'}"
env.PATH="${env.NODEJS_HOME}/bin:${env.PATH}"
sh 'npm install'
sh 'npm pack'
}
}
stage ("Deploy") {
def remote = [:]
remote.name = 'name of your server'
remote.host = 'ip of your server'
remote.allowAnyHosts = true
dir ('Build') {
withCredentials([sshUserPrivateKey(
credentialsId: 'id_of_your_previously_created_credential',
keyFileVariable: 'identityKey',
passphraseVariable: 'passphraseV',
usernameVariable: 'userR')])
{
remote.user = userR
remote.passphrase = passphraseV
remote.identityFile = identityKey
sshPut remote: remote, from: "yourArchive.tgz", into: '.'
}
}
}
} catch (e) {
println "Caught: ${e}"
throw e
}
}
I've used npm for the exemple, with 'npm pack' creating an archive that we will upload to a server.
You might also need to install (if not already done) the following Plugin:
SSH Credentials Plugin
SSH Pipeline Steps
Credentials Binding Plugin
Credentials Plugin
Related
I am currently a jenkins user and is exploring the Teamcity.
In jenkins we have a concept of shared libraries, which basically extends a generic groovy code into different jenkins pipeline and avoid re-writing the same functionality in each jenkins file following the DRY (don't repeat yourself) , hide implementation complexity, keep pipelines short and easier to understand
Example:
There could be a repository having all the Groovy functions like:
Repo: http:://github.com/DEVOPS/Utilities.git (repo Utilities)
Sample Groovy Scipt ==>> GitUtils.groovy with below functions
public void setGitConfig(String userName, String email) {
sh "git config --global user.name ${userName}"
sh "git config --global user.mail ${email}"
}
public void gitPush(StringbranchName) {
sh "git push origin ${branchName}"
}
In jenkinsfile we can just call this function like below (of course we need to define config in jenkins for it to know the Repo url for Shared library and give it a name):
Pipeline
//name of shared library given in jenkins
#Library('utilities') _
pipeline {
agent any
stages {
stage ('Example') {
steps {
// log.info 'Starting'
script {
def gitutil = new GitUtils()
gitutils.setGitConfig("Ray", "Ray#rayban.com")
}
}
}
}
}
And that's it anyone wanting the same function has to just include the library in jenkinsfile and use it in pipeline
Questions:
Can we migrate over the same to Teamcity, if yes how can it be done? We do not want to spend lot of time to re-writing
Jenkins also support Stashing and unstashing of workspace between stages, is the similar concept present in teamcity?
Example:
pipeline {
agent any
stages {
stage('Git checkout'){
steps {
stash includes: '/root/hello-world/*', name: 'mysrc'
}
}
stage('maven build'){
agent { label 'slave-1' }
steps {
unstash 'mysrc'
sh label: '', script: 'mvn clean package'
}
}
}
}
As for reusing common TeamCity Kotlin DSL libraries, this can be done via maven dependencies. For that you have to mention it in the pom.xml file within your DSL code. You can also consider using JitPack if your DSL library code is hosted on GitHub for example and you do not want to handle building it separately and publishing its maven artifacts.
Although with migration from Jenkins to TeamCity you will most likely have to rewrite the common library (if you still need one at all), as TeamCity project model and DSL are quite different to what you have in Jenkins.
Speaking of stashing/unstashing workspaces, it may be covered by either artifact rules and artifact dependencies (as described here: https://www.jetbrains.com/help/teamcity/artifact-dependencies.html) or repository clone mirroring on agents.
I'm learning how to use Jenkins and working on configuring a Jenkins file instead of the build using the Jenkins UI.
The source code management step for building from Bitbucket:
The build step for building a Docker container:
The build is of type multi configuration project:
Reading the Jenkins file documentation at https://www.jenkins.io/doc/book/pipeline/jenkinsfile/index.html and creating a new build using Pipeline :
I'm unsure how to configure the steps I've configured via the UI: Source Code Management & Build. How to convert the config for Docker and Bitbucket that can be used with a Jenkinsfile ?
The SCM will not be changed, regardless if you are using UI configuration or a pipeline, although in theory you can do the git clone from the steps in the pipeline, if you really insist convert the SCM steps in pure pipeline steps.
The pipeline will can have multiple stages, and each of the stages can have different execution environment. You can use the Docker pipeline plug-in, or you can use plain sh to issue the docker commands on the build agent.
Here is small sample from one of my manual build pipelines:
pipeline {
agent none
stages {
stage('Init') {
agent { label 'docker-x86' }
steps {
checkout scm
sh 'docker stop demo-001c || true'
sh 'docker rm demo-001c || true'
}
}
stage('Build Back-end') {
agent { label 'docker-x86' }
steps {
sh 'docker build -t demo-001:latest ./docker'
}
}
stage('Test') {
agent {
docker {
label 'docker-x86'
}
}
steps {
sh 'docker run --name demo-001c demo-001:latest'
sh 'cd test && make test-back-end'
}
}
}
}
You need to create a Pipeline type of a project and specify the SCM configuration in the General tab. In the Pipeline tab, you will have option to select Pipeline script or Pipeline script from SCM. It's always better to start with the Pipeline script while you are building and modifying your workflow. Once it's stabilized, you can add it to the repository.
I'm trying to get the following features to work in Jenkins' Declarative Pipeline syntax:
Conditional execution of certain stages only on the master branch
input to ask for user confirmation to deploy to a staging environment
While waiting for confirmation, it doesn't block an executor
Here's what I've ended up with:
pipeline {
agent none
stages {
stage('1. Compile') {
agent any
steps {
echo 'compile'
}
}
stage('2. Build & push Docker image') {
agent any
when {
branch 'master'
}
steps {
echo "build & push docker image"
}
}
stage('3. Deploy to stage') {
when {
branch 'master'
}
input {
message "Deploy to stage?"
ok "Deploy"
}
agent any
steps {
echo 'Deploy to stage'
}
}
}
}
The problem is that stage 2 needs the output from 1, but this is not available when it runs. If I replace the various agent directives with a global agent any, then the output is available, but the executor is blocked waiting for user input at stage 3. And if I try and combine 1 & 2 into a single stage, then I lose the ability to conditionally run some steps only on master.
Is there any way to achieve all the behaviour I'm looking for?
You need to use the stash command at the end of your first step and then unstash when you need the files
I think these are available in the snippet generator
As per the documentation
Saves a set of files for use later in the same build, generally on
another node/workspace. Stashed files are not otherwise available and
are generally discarded at the end of the build. Note that the stash
and unstash steps are designed for use with small files. For large
data transfers, use the External Workspace Manager plugin, or use an
external repository manager such as Nexus or Artifactory
I have an external tool that should be called as build-step in one of my jenkins jobs. Unfortunately, this tool has some issues with quoting commands to avoid problems with whitespaces in the path that is called from.
Jenkins is installed in C:\Program Files (x86)\Jenkins. Hence I'm having trouble with jenkins calling the external tool.
What I tried is to set "Workspace Root Directory" in Jenkins->configuration to C:\jenkins_workspace in order to avoid any whitespaces. This works for Freestyle Projects but my Multibranch Pipeline Project is still checked out and built under C:\Program Files (x86)\Jenkins\workspace.
One solution would be to move the whole jenkins installation to e.g. C:\jenkins. This I would like to avoid. Is there a proper way to just tell Jenkins Pipeline jobs to use the "Workspace Root Directory" as well?
Thanks for any help
the ws instruction sets the workspace for the commands inside it. for declarative pipelines, it's like this:
ws("C:\jenkins") {
echo "awesome commands here instead of echo"
}
You can also call a script to build the customWorkspace to use:
# if the current branch is master, this helpfully sets your workspace to /tmp/ma
partOfBranch = sh(returnStdout: true, script: 'echo $BRANCH_NAME | sed -e "s/ster//g"')
path = "/tmp/${partOfBranch}"
sh "mkdir ${path}"
ws(path) {
sh "pwd"
}
you can also set it globally by using the agent block (generally at the top of the pipeline block), by applying it to a node at that level:
pipeline {
agent {
node {
label 'my-defined-label'
customWorkspace '/some/other/path'
}
}
stages {
stage('Example Build') {
steps {
sh 'mvn -B clean verify'
}
}
}
}
Another node instruction later on might override it. Search for customWorkspace at https://jenkins.io/doc/book/pipeline/syntax/. You can also it use it with the docker and dockerfile instructions.
Try this syntax instead:
pipeline {
agent {
label {
label 'EB_TEST_SEL'
customWorkspace "/home/jenkins/workspace/ReleaseBuild/${BUILD_NUMBER}/"
}
}
}
I am trying to create a build pipeline based upon the Gradle tasks. I have viewed JenkinsFile configuration Pipeline-as-code-demo but I am unable to create a pipeline for gradle tasks. Please suggest me a possible way so that I can use the Jenkinsfile to automatically show the build pipeline just by reading the configurations from the Jenkinsfile.
Thankyou
In case your project uses Gradle Wrapper you can use the following snippet in your Jenkinsfile:
stage('Gradle Build') {
if (isUnix()) {
sh './gradlew clean build'
} else {
bat 'gradlew.bat clean build'
}
}
If you checkout to subdirectory sub-dir you might want to use
stage('Gradle Build') {
if (isUnix()) {
dir('sub-dir') {sh './gradlew clean build'}
} else {
dir('sub-dir') {bat 'gradlew.bat clean build'}
}
}
In case you're using Artifactory to resolve your build dependencies or to deploy your build artifacts, it is recommended to use the Pipeline DSL for Gradle build with Artifactory.
Here's an example taken from the Jenkins Pipeline Examples page:
node {
// Get Artifactory server instance, defined in the Artifactory Plugin administration page.
def server = Artifactory.server "SERVER_ID"
// Create an Artifactory Gradle instance.
def rtGradle = Artifactory.newGradleBuild()
stage 'Clone sources'
git url: 'https://github.com/jfrogdev/project-examples.git'
stage 'Artifactory configuration'
// Tool name from Jenkins configuration
rtGradle.tool = "Gradle-2.4"
// Set Artifactory repositories for dependencies resolution and artifacts deployment.
rtGradle.deployer repo:'ext-release-local', server: server
rtGradle.resolver repo:'remote-repos', server: server
stage 'Gradle build'
def buildInfo = rtGradle.run rootDir: "gradle-examples/4/gradle-example-ci-server/", buildFile: 'build.gradle', tasks: 'clean artifactoryPublish'
stage 'Publish build info'
server.publishBuildInfo buildInfo
}
Otherwise, you can simply run the gradle command with the sh or bat Pipeline steps.
In jenkins you can creates a jenkins pipeline using a script which is written in Jenkinsfile.
We write a script using 'stages' and 'node' as building block, these building blocks allow you to specify instructions that should be executed as part of jenkins pipeline.
To execute gradle build using JenkinsFile first check for Operating system and call appropriate shell that can execute that gradle task, as below:
Jenkinsfile
stage 'build_Project'
node{
if(isUnix()){
sh 'gradle build --info'
}
else{
bat 'gradle build --info'
}
}
Above code snippet create a stage with name build_project and execute gradle build script of the current project.