I'm working on a set of builds related to our online images (such as wordpress content). Overall it's a large workflow, so it's broken into several jobs.
A couple jobs need to copy a large number of artifacts from other jobs; I've been using the Copy Artifacts plugin but it's too slow for my case and rsync would be much better suited.
Is it possible to effectively get the source artifact directory for the upstream build so that I can pass it to rsync in place of using the Copy Artifacts plugin? I'd like to have a simple script like:
rsync -a --delete $UPSTREAM_ARTIFACT_DIR $WORKSPACE
The upstream artifacts are accessible via what appear to be well defined URLs. For example,
the following URL enables one to access the last good builds' artifacts:
http://jenkins/job/job_name/lastSuccessfulBuild/artifact/
Can even specify the axis on a multiconfiguration project, if that's required:
http://jenkins/job/job_name/label=foo,arch=x86/lastSuccessfulBuild/artifact/
Related
I want to update Jenkins plugin via Artifactory.
Create a remote repo named Jenkins-update
Create a local repo named jenkins-update-center
Get the update-center.json from repo Jenkins-update to local and modify the URL from 'http://updates.jenkins-ci.org/' to my own URL 'https://artifacts.xxx.com/artifactory/Jenkins-update/' in update-center.json, then put update-center.json into local repo.
#!/bin/sh
curl -L -o /tmp/update-center.json http://localhost:8081/artifactory/Jenkins-update-cache/update-center.json
sed -i 's#http://updates.jenkins-ci.org/#https://artifacts.xxx.com/artifactory/Jenkins-update/#g' /tmp/update-center.json
curl -L -uuser:pass -T /tmp/update-center.json "http://localhost:8081/artifactory/jenkins-update-center/update-center.json"
Change the default update site from 'http://updates.jenkins-ci.org/' to 'https://artifacts.xxx.com/artifactory/jenkins-update-center/update-center.json' in Jenkins
There is an error 'SHA-512 digest mismatch: expected=49a22dc23f739a76623d10128b6803f79e0489de3ded0f1d01f3dfba4557136c7f318baaf4749a7713ec4b3f56633f2ac3afc4703e87d423ede029d68f84c74d in 'update site 'default''' when I click 'check now' button.
What should I do to make Jenkins update plugins from Artifactory?
Tkx
As soon as the content of update-center.json changed you need to re-generate "signature" section of this file.
For that you need to generate your key pair (see more details in How to create a local mirror of public Jenkins update site?)
Also you may use the following proposed approach :
there is probably a better way, by having a sandbox Jenkins on a system that has access to the internet. You update the server using the UI and then you can test that updated Jenkins thoroughly. When done, you just need to copy the war and hpi files over to your 'production' Jenkins. now you have even a nice process and QA in place.
Another way is to setup a transparent https proxy between your Jenkins and Artifactory server - in that case update-center.json will not change and signature verification should work fine.
With best regards,
Dmytro Gorbunov
As of 2023-01-10 there is a problem with making a mirror of the jenkins plugins on artifactory.
Artifactory documentation decribes only how to create a mirror: https://jfrog.com/knowledge-base/how-to-configure-artifactory-as-a-mirror-for-jenkins-plugins/
But this is not a complete solution. Because this leads to the situation when every plugin shall be manually updated. Having plugins with bunch of dependencies it is huge effort.
There is a need to generate a file: update-center.json
There is an internal jenkins tool to do this: https://github.com/jenkins-infra/update-center2, but documentation is poor and contains vague statements like:
With a few modifications it could easily be used to generate your corporate update center as well.
Without clear description, what shall be done.
I tried to follow steps and completely failed. Tool require some special environment variables, which are also not documented and so on.
So as of my experience mirroring jenkins plugins on artifactory is practically not possible. And honestly spoken, I would like to be wrong here.
I have a structure of artifacts in another build:
/
/bundle/docs
/bundle/bin
/bundle/bin/scripts
I want to copy all files and sudirectories into the current job's workspace subfolder 'product1' from /bundle/bin. I expect to see in %WORKSAPCE%/product1 contents of /bundle/bin.
I've configured it like this:
Artifacts to copy: bundle/bin/**
But it creates %WORKSAPCE%/product1//bundle/bin instead.
Is it possible?
Seems like that's just how the plugin works. Your options are:
Keep the same configuration and manipulate directories later using sh mv (Linux) or cmd move (Windows) command. This is the workaround used in my environment.
Check the "Flatten directories" option (but this will mix together the /bundle/bin and /bundle/bin/scripts)
Improve the plugin and contribute your code to the community :-)
I wonder if it is possible to remove only one build (including artifacts) from job workspace.
I tried to "Delete Build" in Build History but all it does is remove build reference from Build History table. I know I can ssh to a server and delete files from the command line but I am looking for a way to do it from Jenkins web interface.
After installing Workspace Cleanup Plugin I am able to wipe out current workspace but I want to keep my other builds in the workspace.
In your Jenkins instance, to be able to have folder/per build - set flag "Use custom workspace" in your job's settings. Here is a brief help info from the setting description:
For each job on Jenkins, Jenkins allocates a unique "workspace directory."
This is the directory where the code is checked out and builds happen.
Normally you should let Jenkins allocate and clean up workspace directories,
but in several situations this is problematic, and in such case, this option
lets you specify the workspace location manually.
One such situation is where paths are hard-coded and the code needs to be
built on a specific location. While there's no doubt that such a build is
not ideal, this option allows you to get going in such a situation.
...
And your custom directory path would look like this:
workspace\$JOB_NAME\$BUILD_NUMBER ~> workspace\my-job-name\123
where $JOB_NAME will be "my-job-name" and $BUILD_NUMBER is the build number, eq. "123".
There is one nasty problem with this approach and this is why I wouldn't recommend to use it - Jenkins will not be able to reclaim disk space for outdated builds. You would have to handle cleanup of outdated builds manually and it is a lot of hassle.
Alternative approach, that gives you more control, tools and is able to keep disk space usage under control (without your supervision) is to use default workspace settings and archive your build output (files, original source code, libraries and etc.) as a post-build action. Very-very handy and gives you access to a whole bunch of great tools like, Copy Artifact Plugin or ArtifactDeployer Plugin in other jobs.
Hope that info helps you make a decision that fits your needs best.
I also use "General/Advanced/Use custom workspace" (as in #pabloduo's answer) on a Windows machine with something like:
C:\${JOB_NAME}\${BUILD_NUMBER}
Just wanted to add a solution for getting rid of the build job's workspaces.
I use Groovy Events Listener Plugin for this.
Using the plug-in's standard configuration I just use the following Groovy script:
if (event == Event.JOB_DELETED){
new File(env.WORKSPACE).deleteDir()
}
And now the custom workspace is deleted when the build job is deleted.
Just be aware that this would also delete non-custom workspaces (because the event is triggered for all jobs on your Jenkins server).
I'm working with Jenkins 2 and trying to copy artifacts between jobs and in turn to an S3 bucket.
I have a simple web build which produces artifacts in /dist/public which I'd like to upload into the S3 bucket.
So once the job completes, I have a folder /dist in the workspace root. Jenkins gives you the ability to copy artifacts between jobs which leverages Ant's fileset.
The issue I'm having is that this is a restricted subset of Ant and all you're provided is include & exclude paths.
I can use dist/public/**/** however this copies the parent directories across also.
What I would prefer is to only copy the content of public/ but after doing some reading it seems this may be difficult to do without a custom Ant tasks, etc.
If you copy files by Ant, you should set:
<fileset dir="/dist/public"/>
at you copy task, or you can use flatten attribute.
If you use Jenkins artefact collector (as I do), I think now you have to copy this files to workspace root (see: Copy Artifact Plugin).
I wonder are there features for jenkins to capture the result /data in a node and persist it in master.
I come up with the scenario that I need to check some folders in two machines to see whether they have same no of files & same size.
If hudson can save some result like "ls -ltR" in master , then I can gather at both node the results in two jobs then compare.
Are there any elegant solution to this simple problem?
currently I can connect two machines to each other via SSH and solve the problem, while this connection is not always available.
(With SSH I believe the best way is to use rsync -an /path/to/ hostB:/path/to/)
Simple problem, only slightly elegant solution :
Write a simple job listdir which does DIR > C:\logs\list1.txt .. list
Go to Post-build Actions
Add Archive the artifacts for example from above: C:\logs\*.*
Now run a build and go to http://jenkinsservername:8080/job/listdir/
You'll see the list1.txt which you can click on, and see the contents.
I have given a Windows example, you can of course replace DIR with ls -ltr
Or use archive artifacts in combination with the Copy Artifacts Plugin to pull the result of another job in the job where the comparison shall be done.