I have a jenkins Pipeline that results with a build artifact which is a deb file.
I was planning to use that deb file to fill in the Package option in the Bake Configuration Phase.
It doesn't work and results in
How should I go about doing this?
I guess I need to transfer artifact from the spinnaker pipeline's build trigger(the Jenkins pipeline) but I don't understand how to do that.
Is this of any use? I can't wrap my head around what they want me to do in order to send the file.
Any help is appreciated. Thanks.
What you have done so far looks correct, so we need to debug why it fails.
You have to look at the source of the execution that fails (press the "source" link). This json structure is the execution context (you probably want to install a json formatter plugin in your browser or something to read it). Navigate to trigger.artifacts and look for the artifact there. If you find it, please post the result here.
I think maybe the issue is the name of the deb file. It should have been named like simplenodeappinstaller_1.0-0_amd64.deb. A simple solution is to add the jenkins build number or a timestamp as the release number.
You can also try to activate the artifact decorator (set artifact.decorator.enabled: true in igor-local.yml. This will cause Igor to parse deb and rpm files in a more sensible way, IMHO. See below for more info.
Another thing is that Jenkins sometimes puts all test results and other stuff into the artifacts list, and I think the default maximum number of artifacts returned are 20. This is however configurable under the key BuildArtifactFilter.maxArtifacts (see https://github.com/spinnaker/igor/blob/master/igor-web/src/main/java/com/netflix/spinnaker/igor/build/BuildArtifactFilter.java). Reading the code, it also seems deb-files should already be prioritized over other kinds of artifacts, so I don't really think this is the issue.
Decorator primer
Enabling the artifact decorator will convert this artifact:
fileName: "openmotif22-libs-2.2.4-192.1.3.x86_64.rpm"
displayPath: "openmotif22-libs-2.2.4-192.1.3.x86_64.rpm"
relativePath: "openmotif22-libs-2.2.4-192.1.3.x86_64.rpm"
into
fileName: "openmotif22-libs-2.2.4-192.1.3.x86_64.rpm"
displayPath: "openmotif22-libs-2.2.4-192.1.3.x86_64.rpm"
relativePath: "openmotif22-libs-2.2.4-192.1.3.x86_64.rpm"
reference: "openmotif22-libs-2.2.4-192.1.3.x86_64.rpm"
name: "openmotif22-libs"
type: "rpm"
version: "2.2.4-192.1.3.x86_64"
decorated: "true"
The built in decorator supports deb and rpm, but it can be extended using config like this:
artifact:
# This is a feature toggle for decoration of artifacts.
decorator:
enabled: true
fileDecorators:
- type: docker/image
decoratorRegex: '(.+):(.+)'
identifierRegex: '(.+\/.+:.+)'
Related
first let me tell you I am a quite beginner with Circleci, so some of my questions might not make too much sense but bear with me.
Recently I published a project of mine in Github PoC Akka FSM, I heard from several friends good things about Circleci and I am trying to use for my project.
Circleci web page created a config.yml for me and it is looking good but I have a complication, my Project is a Gradle Project but it has a dependency to Eclipse .p2 Repositories, which Gradle can't resolve out of the box. So I have a small Maven Project that collects my dependencies from .p2 repository and convert those to a big fat dependency jar.
Now when I locally build, I build first maven project which would place the the dependency in local maven repository, so the Gradle can locate the Artifact from Maven repository.
Now for the reason you might guess this is not working in Github with Circleci's default config.yml, so my question would it be possible to combine a maven build and gradle build in Circleci. Or a define in a prestep for an extra Build.
Otherwise I will try to convert Maven project to Gradle project (I am not sure Gradle would be able to read .p2 repositories).
The following is the Maven Project.
Thx for answers
I solved my Problem and I like to explain here how, in case another complete beginners lands here.
First, if you are complete new to CircleCI please start reading this before anything else, life would be much easier.
CircleCI Concepts
Secondly, this link
CirceCI Sequential Jobs
explains perfectly how Sequential Jobs should work, at the beginning the wordings 'Steps' being under 'Job' and 'Workflow' confused me a lot, I was not sure I have to implement my Maven Step in Job or Workflow, but it seems it must be Workflow.
Third, I was not quite sure how to transfer the Artefacts of Maven Build to Gradle Build, it seems Cache mechanism is responsible for it. When first I read it, a cache in Build System didn't not made too much sense but I think (I am completely speculating) caches in CirceCI is like Java Maps, you put in Job a value with key, next job can pick the value for key (value being here the Artefacts?). So I in my Maven Job, I am saving my Cache with Key "v1-dependencies-{{ checksum "fsm-akka-eclipse-dependencies/pom.xml" }}" and in the next Job, I load it with same key (and additionally v1-dependencies-{{ checksum "build.gradle" }} while I have a feeling if the checksum of 'build-gradle' is not changed CircleCI optimize something) Someone with more experience may be clear that here
Fourth, I was not sure how to say CircleCI to install Maven or Gradle but it seems CircleCI doing it in a magic way. In my previous point, I mentioned the caches to transfer the Artefacts', but for Maven Job, I am telling to save the cache at '.m2' directory which normally Gradle will look for Maven Artefact, may be this is doing the trick or the Cache I can't tell.
If you like the see how my final config.yml looks, you can find it on the link.
I am creating jenkins pipeline for all our application where I wanted to build and deploy. I can able to achieve that but all the deployment paths are hard coded on the pipeline script.
We have around 8 application and 5 environments. it means I need to specify 40 different deployable path on the pipeline scripts.
I like to know, are they any best way to store the deployment path?. I thought about storing them in XML and reading that while doing the build, but not sure on implementation.
looking for some ideas.
script {
def msbuild tool name: 'Msbuila', type: 'msbuild'
def action "${msbuild}\\msbuild.exe"
def rootPath "${NORKSPACE}\\test\\test";
def sinPath "${rootPath}\\test.sin"
def binPath "${rootPath}\\test\\bin"
bat “nuget restore \"${sinPath}\""
bat "\"${action}\" \"${sinPath)\" "
robocopy("\"${binPath}\" \"\\\\t.test.com\\test\" /MIR /xF ")
}
What I would do is use a config repository, having it configured this way:
Each application is a different repository (example: app_config)
Each environment is a different file
The same enviroment file in each repository is called by the same name
Each enviroment file is a yaml (key:value)
Then on the jenkins pipeline I would get the repo, read the yaml using readYAML (check the command usage and name, theres is a while since I used it) and load it on a map.
Then you use the variables of the map and that should help you
The tricky part is how to match the code repositories and the config repositories. As I mentioned before, I would use the same name and append "_config"
I have a local instance of Jenkins. I have previously tried storing the jenkins.yml in my system and giving its path on http://localhost:8080/configuration-as-code. This worked but I want to use a Gitlab repository to store the jenkins.yml file.
I have already tried giving the gitlab link of my jenkins.yml in the path or URL textbox. Some weird things happened, like
1. jenkins broke or huge error console
2. It reapplies the previous configuration(from system path)
jenkins:
systemMessage: "Hello, world"
Your problem as described: you want the job configuration to be saved in GIT and, when a build is triggered, the job should get the current stand of its configuration from there and then, run the build.
Maybe there is a kind of plug-in that does it for you, but I am not aware of any. Maybe anyone?
My suggestion is to define a pipeline job and use a declarative pipeline. It is a file, normally named Jenkinsfile that can be stored in GIT. In the Job, you define the GIT address and when you trigger a build, the file is got from GIT and executed.
There are several flaws in this: pipelines learning curve is not small, you are confronted with groovy (not XML!) and your current XML file is barelly useful.
Maybe someone shows up and tells us about new (for me) plugin that solves your problem using the configuration XML file. In the other hand, pipelines are such a beautyful feature that I encourage you to give it a try
I want to update Jenkins plugin via Artifactory.
Create a remote repo named Jenkins-update
Create a local repo named jenkins-update-center
Get the update-center.json from repo Jenkins-update to local and modify the URL from 'http://updates.jenkins-ci.org/' to my own URL 'https://artifacts.xxx.com/artifactory/Jenkins-update/' in update-center.json, then put update-center.json into local repo.
#!/bin/sh
curl -L -o /tmp/update-center.json http://localhost:8081/artifactory/Jenkins-update-cache/update-center.json
sed -i 's#http://updates.jenkins-ci.org/#https://artifacts.xxx.com/artifactory/Jenkins-update/#g' /tmp/update-center.json
curl -L -uuser:pass -T /tmp/update-center.json "http://localhost:8081/artifactory/jenkins-update-center/update-center.json"
Change the default update site from 'http://updates.jenkins-ci.org/' to 'https://artifacts.xxx.com/artifactory/jenkins-update-center/update-center.json' in Jenkins
There is an error 'SHA-512 digest mismatch: expected=49a22dc23f739a76623d10128b6803f79e0489de3ded0f1d01f3dfba4557136c7f318baaf4749a7713ec4b3f56633f2ac3afc4703e87d423ede029d68f84c74d in 'update site 'default''' when I click 'check now' button.
What should I do to make Jenkins update plugins from Artifactory?
Tkx
As soon as the content of update-center.json changed you need to re-generate "signature" section of this file.
For that you need to generate your key pair (see more details in How to create a local mirror of public Jenkins update site?)
Also you may use the following proposed approach :
there is probably a better way, by having a sandbox Jenkins on a system that has access to the internet. You update the server using the UI and then you can test that updated Jenkins thoroughly. When done, you just need to copy the war and hpi files over to your 'production' Jenkins. now you have even a nice process and QA in place.
Another way is to setup a transparent https proxy between your Jenkins and Artifactory server - in that case update-center.json will not change and signature verification should work fine.
With best regards,
Dmytro Gorbunov
As of 2023-01-10 there is a problem with making a mirror of the jenkins plugins on artifactory.
Artifactory documentation decribes only how to create a mirror: https://jfrog.com/knowledge-base/how-to-configure-artifactory-as-a-mirror-for-jenkins-plugins/
But this is not a complete solution. Because this leads to the situation when every plugin shall be manually updated. Having plugins with bunch of dependencies it is huge effort.
There is a need to generate a file: update-center.json
There is an internal jenkins tool to do this: https://github.com/jenkins-infra/update-center2, but documentation is poor and contains vague statements like:
With a few modifications it could easily be used to generate your corporate update center as well.
Without clear description, what shall be done.
I tried to follow steps and completely failed. Tool require some special environment variables, which are also not documented and so on.
So as of my experience mirroring jenkins plugins on artifactory is practically not possible. And honestly spoken, I would like to be wrong here.
I've been asked to move some variable from a Groovy script out into a configuration file. I'm fine using something like :-
readFile('../xx-software.cfg').split('\n').each { fileName ->
sh "wget ${theURL}${fileName}"
}
However, even though I have added xx-software.cfg into the same directory as my Groovy script it does become available for use within that groovy script.
I hope this makes sense!?
How can I move my variables out into a config file to make it easier for the application support team to make future edits without changing the code?
There are a few approaches you could use.
Firstly, file format for the configuration and how to read the data into variables. You could use Java Properties format, YAML or JSON and these are all handled by the Pipeline Utility Steps plugin with steps here. You can read the file with these steps:
readProperties
readYaml
readJSON
Next problem, how to get the file available to your pipeline so it can be read from the workspace using these steps. Possibilities are:
In source control with your pipeline code. It can be fetched with the pipeline.
In a separate source control for configuration, your pipeline will need a step to fetch it.
Use the Jenkins Config File Provider plugin. It has a step to provide a config file managed in Jenkins.
Provide it as a Custom Tool zipped archive from a binary server like Artifactory. You can use custom tool definition pipeline steps to make this available to the pipeline.
The Config File Provider option might provide any easy way to have a file that can be updated, but there won't be any version control of it.