I'm trying to set up a jenkins pipeline for publishing a zip file to jfrog artifactory.
I am using com.jfrog.artifactory plugin to do so. This works great from command line gradle and I can run the artifactoryPublish task to publish the artifacts and tie them back to the module, which then has a tie back to the artifacts.
The artifacts show up with the properties:
build.name = `projectname`
build.number = `some large number`
And I can click from them to the build/module and back to the artifact.
However, when I run this from a jenkinsfile pipeline, the artifacts get published and get tied back to the module, but then the module does not successfully tie the module back to the artifacts.
The artifacts do not receives the build.name and build.number properties and i cannot click from the module back to the artifacts, as the module cannot find or resolve the paths back to the artifacts(a zip file and a generated pom).
I am passing the params from jenkins like:
ORG_GRADLE_PROJECT_buildInfo.build.number=${env.BUILD_NUMBER} which seems to work on other projects... but for whatever reason I cannot shake it.
I can include more jenkinsfile if that would help debug, but i'm really just checking out a repository and trying to publish it.
I have been reading heavily the documentation here:
https://www.jfrog.com/confluence/display/RTF/Gradle+Artifactory+Plugin
and haven't been able to make it work through -Pproject stuff.
Does anyone have any idea what else I can try? i don't really want to use the jenkins pipeline artifactory plugin directly because it's so nice to be able to deploy from the command line too.
build.gradle:
publishing {
publications {
ManualUpdaterPackage(MavenPublication){
artifact assembleManualUpdaterPackage
}
}
}
artifactory {
contextUrl = "${artifactoryUrl}" //The base Artifactory URL if not overridden by the publisher/resolver
publish {
defaults {
publications('ManualUpdaterPackage')
}
repository {
repoKey = project.version.endsWith('-SNAPSHOT') ? snapshotRepo : releaseRepo
username = "${artifactory_user}"
password = "${artifactory_password}"
maven = true
}
}
}
task assembleManualUpdaterPackage (type: Zip){
dependsOn anotherTask
from (packageDir + "/")
include '**'
// archiveName "manualUpdaterPackage-${version}.zip"
destinationDir(file(manualUpdaterZipDir))
}
jenkinsfile snip:
withCredentials(
[
[
$class : 'UsernamePasswordMultiBinding',
credentialsId : 'validcreds',
passwordVariable: 'ORG_GRADLE_PROJECT_artifactory_password',
usernameVariable: 'ORG_GRADLE_PROJECT_artifactory_user'
]
]
) {
withEnv(
[
"ORG_GRADLE_PROJECT_buildInfo.build.number=${env.BUILD_NUMBER}",
"ORG_GRADLE_PROJECT_buildInfo.build.name=${artifactName}",
"ORG_GRADLE_PROJECT_buildInfo.build.url=${env.JOB_URL}"
]
) {
sh 'chmod +x gradlew'
sh "./gradlew --no-daemon clean artifactoryPublish"
}
}
https://www.jfrog.com/confluence/display/RTF/Working+With+Pipeline+Jobs+in+Jenkins#WorkingWithPipelineJobsinJenkins-GradleBuildswithArtifactory
Eventually my coworker recommended looking into the Artifactory Pipeline Gradle plugin instead. It is very nice to work with and we've had much quicker success with it.
Related
I am trying to write a pipeline script to publish *.war/*.jar file to JFrogArtifactory. I don't find any syntax for the same.
Anyone can help me out on the same.
please help me with a sample script.
JFrog has a dedicated GitHub repository with many examples for such cases.
There are Jenkins Pipelines examples there.
First, you must install Artifactory Plugin and config it in Jenkins server.
Refer: https://www.jfrog.com/confluence/display/JFROG/Configuring+Jenkins+Artifactory+Plug-in
And then try add below script to Jenkinsfile:
script {
def server = Artifactory.server '<artifactory id>'
def uploadSpec = '''{
"files": [{
"pattern": "<name of war or jar file>",
"target": "<artifactory repo>/path-to/war-or-jar/file/in-Artifactory"
}]
}'''
server.upload(uploadSpec)
}
Don't forget replace <artifactory id> <name of war or jar file> and <artifactory repo>/path-to/war-or-jar/file/in-Artifactory
More information: https://www.jfrog.com/confluence/display/JFROG/Declarative+Pipeline+Syntax
The scripted pipeline syntax for deploying war files to JFrog artifactory is :
env.ARTIFACTORY = 'True'
if(env.ARTIFACTORY == 'True')
{
stage('Deploying to Artifactory')
{
FAILED_STAGE = env.STAGE_NAME
bat 'mvn deploy'
}
}
Note :
1.) 'bat' command is for Windows batch file. If you're using Linux, replace 'bat' with 'sh'
2.) env.ARTIFACTORY is used to give you control over whether or not you want to execute this particular stage in your pipeline job. if you don't want this stage to execute, simply set env.ARTIFACTORY = 'False'
3.) Also note, you've to configure JFrog in : Manage Jenkins -> Configure system -> JFrog Platform Instances
4.) Include JFrog in your pom.xml file under distributionManagement tag.
I'm struggling with deploying the war file to Nexus repository using Jenkinsfile via Gradle task.
The war is being created successfully. I have also no problem with deploying JARs (since there are examples everywhere how to do it).
So I have this publishing section in my build.grade:
publishing {
repositories {
maven {
URI releasesUrl = new URI("${UploadURL}/repository/releases")
URI snapshotsUrl = new URI("${UploadURL}/repository/snapshots")
afterEvaluate {
url version.endsWith("SNAPSHOT") ? snapshotsUrl : releasesUrl
}
credentials {
username "${user}"
password "${password}"
}
}
}
publications {
mavenWeb(MavenPublication) {
from components.web
artifact war.archivePath
}
}
}
With pluggins:
apply plugin: 'java'
apply plugin: 'maven'
apply plugin: 'maven-publish'
apply plugin: 'org.springframework.boot'
apply plugin: 'war'
The URL for repositories is also specified in the build script correctly (test publish with the jar works just fine)
And the Jenkinsfile:
stage ('Publish war') {
steps {
sh "sh gradlew publish"
}
}
Currently I'm getting this error from jenkins build:
Task :publishMavenWebPublicationToMavenRepository FAILED
FAILURE: Build failed with an exception.
What went wrong:
Execution failed for task ':publishMavenWebPublicationToMavenRepository'.
Failed to publish publication 'mavenWeb' to repository 'maven'
Invalid publication 'mavenWeb': multiple artifacts with the identical extension and classifier ('war', 'null').
I'm quite sure that the problem is within "publications" part of Gradle task.
For publishing the Jars I have been using it like this:
[...]
publications {
mavenJava(MavenPublication) {
from components.java
artifact sourceJar
}
}
[...]
task sourceJar(type: Jar) {
classifier 'sources'
from sourceSets.main.java
}
I do not know how to configure from, artifact and classifier for this task. I do not even know if all of these parameters should be configured... Could anyone help me with that?
It turned out, that the origin of the problem was this section:
afterEvaluate {
url version.endsWith("SNAPSHOT") ? snapshotsUrl : releasesUrl
}
This feature works with Gradle 5.X version however, I was using Gradle 4.8. That lead to null instead of propper url value...
Unfortunately, it took a while since the exception message does not suggest where the problem was.
I want to create a Jenkins job which does following:
Git>Mvn build> copy jar to some location of server.
So this can be done using a single job or 2 jobs?
Or which is preferred way of doing this , is pipeline preferred over creating a maven job?
I have created this pipeline script, but this does not copy the current build jar to the server location, it copies the previous build artifact jar.
node {
def mvnHome
stage('Preparation') { // for display purposes
// Get some code from a GitHub repository
git 'git#github.pie.ABC.com:abcdef/BoltRepo.git'
mvnHome = tool 'M2'
}
stage('Build') {
// Run the maven build
if (isUnix()) {
sh "'${mvnHome}/bin/mvn' -Dmaven.test.failure.ignore clean package"
} else {
bat(/"${mvnHome}binmvn" -Dmaven.test.failure.ignore clean package/)
}
}
stage('Results') {
archiveArtifacts 'target/*/BoltRepo*.jar'
}
stage('Deploy Artifact') {
copyArtifacts(
projectName: currentBuild.projectName,
filter: 'target/*/BoltRepo*.jar',
fingerprintArtifacts: true,
target: '/ngs/app/boltd/bolt/bolt_components/bolt_provision/test',
flatten: true )
}
}
What is the best way of achieving this.
I haven't used the pipeline before, but I have done what you want using "ArtifactDeployer" from the "Post-build Actions" in the job's configurations
Note: you will need to install "Artifact Deployer Plug-in"
I'm trying to automate the creation of a Jenkins Pipeline build from within a pipeline.
I have a pipeline which creates a Bitbucket repository and commits some code to it, including a Jenkinsfile.
I need to add another step to this pipeline to then create the Pipeline build for it, which would run the steps in the Jenkinsfile.
I think the Jobs DSL should be able to handle this but the documentation I've found for it has been very sparse, and I'm still not entirely sure if it's possible or how to do it.
Any help would be appreciated. The generated Pipeline build I would imagine just needs to have a link to the repository and be told to run the Jenkinsfile there?
Yes, Job DSL is what you need for your use case.
See this and this to help you get started.
EDIT
pipeline {
agent {
label 'slave'
}
stages{
stage('stage'){
steps {
// some other steps
jobDsl scriptText: '''pipelineJob(\'new-job\') {
def repo = \'https://xxxxx#bitbucket.org/xxxx/dummyrepo.git\'
triggers {
scm(\'H/5 * * * *\')
}
definition {
cpsScm {
scm {
git {
remote {
url(repo)
credentials('bitbucket-jenkins-access')
}
branches(\'master\')
scriptPath(\'Jenkinsfile\')
extensions { }
}
}
}
}
}'''
}
}
}
}
Documentation - https://jenkinsci.github.io/job-dsl-plugin/#path/pipelineJob-scm-git
By using this python library jenins-job-builder you can easily create your expected pipeline or free-style job from another pipeline or from any other remote location.
Example:
steps-1
python3 -m venv .venv
source .venv/bin/activate
pip install --user jenkins-job-builder
steps-2
Once you have done above, Create 2 file, one with name config.ini and the other one is job.yml. Please note - there are no strict rules about the file name. It can be up to you.
The config.ini file contain can looks like
[job_builder]
allow_duplicates = False
keep_descriptions = False
ignore_cache = True
recursive = False
update = all
[jenkins]
password = jenkins-password
query_plugins_info = False
url = http://jenkins-url.net
user = jenkins-username
If you are creating a pipeline job , then your job.yml file can look like
- job:
name: pipeline01
display-name: 'pipeline01'
description: 'Do not edit this job through the web!'
project-type: pipeline
dsl: |
node(){
stage('hello') {
sh 'echo "Hellow World!"'
}
}
steps-3
after all the above. Invoke below command
jenkins-jobs --conf config.ini update job.yml
Note- jenkins-jobs command can only be available if you have followed steps-1
I'm trying to migrate my build pipelines to the "Pipeline plugin" using the groovy build scripts.
My pipelines are usually:
Test (gradle)
IntegrationTest (gradle)
Build (gradle)
Publish (artifactory)
I would like to use the gradle variables like version/group etc. in my jenkins build script to publish to the correct folders in artifactory. Something the artifactory plugin would take care of for me in the past. How can this be achieved?
For a single gradle project I use something like this:
node('master')
{
def version = 1.0
def gitUrl = 'some.git'
def projectRoot = ""
def group = "dashboard/frontend/"
def artifactName = "dashboard_ui"
def artifactRepo = "ext-release-local"
stage "git"
git branch: 'develop', poll: true, url: "${gitUrl}"
dir(projectRoot)
{
sh 'chmod +x gradlew'
stage "test"
sh './gradlew clean test'
stage "build"
sh './gradlew build createPom'
stage "artifact"
def server = Artifactory.server('artifactory_dev01')
def uploadSpec = """{
"files": [
{
"pattern": "build/**.jar",
"target": "${artifactRepo}/$group/${artifactName}/${version}/${artifactName}-${version}.jar"
},
{
"pattern": "pom.xml",
"target": "${artifactRepo}/$group/${artifactName}/${version}/${artifactName}.pom"
}
]
}"""
def buildInfo1 = server.upload spec: uploadSpec
server.publishBuildInfo buildInfo1
}
}
For future reference here an example with the more modern declarative pipeline:
pipeline {
agent any
stages {
stage('somestage') {
steps {
script {
def version = sh (
script: "./gradlew properties -q | grep \"version:\" | awk '{print \$2}'",
returnStdout: true
).trim()
sh "echo Building project in version: $version"
}
}
}
}
}
see also:
Gradle plugin project version number
How to do I get the output of a shell command executed using into a variable from Jenkinsfile (groovy)?
I think you actually have two different approaches to tackle this problem :
1. Get version/group from sh script
Find a way to get Gradle version from gradle build tool (e.g. gradle getVersion(), but I'm not familiar with Gradle) and then use shell script to get this version. If Gradle command to get the version is gradle getVersion(), you would do in your pipeline :
def projectVersion = sh script: "gradle getVersion()", returnStdout: true
def projectGroup= sh script: "gradle getGroup()", returnStdout: true
and then just inject your $projectVersion and $projectGroup variables in your current pipeline.
2. Configure your Gradle build script to publish to Artifactory
This is the reverse approach, which I personnaly prefer : instead of giving Artifactory all your Gradle project information, juste give Gradle your Artifactory settings and use Gradle goal to easily publish to Artifactory.
JFrog has a good documentation for this solution in their Working with Gradle section. Basically, you will follow the following steps :
Generate a compliant Gradle build script from Artifactory using Gradle Build Script Generator and include it to your project build script
Use Gradle goal gradle artifactoryPublish to simply publish your current artifact to Artifactory
For others who Google'd their way here, if you have the Pipeline Utility Steps plugin and store what you need in your gradle.properties file, you can do something like this in the environment block:
MY_PROPS = readProperties file:"${WORKSPACE}/gradle.properties"
MY_VERSION = MY_PROPS['version']