I'm attempting to use the archive functionality but I'm not able to download all the files. For example, in Jenkins I see:
However, after clicking the link to download all the files as a zip, then unzipping the files, I see:
I'm not sure why the .gitignore and .gitattributes are missing from the zip file when I download it but are present in the artifacts listed in Jenkins.
I use the following command to archive the artifacts:
archiveArtifacts artifacts: "portfolios-deployment_${PACKAGE_VERSION}/**/*", onlyIfSuccessful: true, defaultExcludes: false
The archiveArtifacts option:
Archives the build artifacts (for example, distribution zip files or jar files) so that they can be downloaded later. Archived files will be accessible from the Jenkins webpage.
By that, it means it takes a copy of the artifacts from the workspace and stores them alongside the build logs on the controller. It is expected the workspace is ephemeral or its contents will be overwritten on next iteration, so they are preserved along with the logs, until the logs are deleted.
archiveArifacts has an Advanced ... option:
[ X ] Use default excludes
If you expand the help, you'll see the details:
Artifact archiver uses Ant org.apache.tools.ant.DirectoryScanner which
excludes by default the following patterns:
/%*%,/.git/,/SCCS,/.bzr,/.hg/,/.bzrignore,/.git,/SCCS/,/.hg,/.#*,/vssver.scc,/.bzr/,/._*,/##,**/~,/CVS,/.hgtags,/.svn/,/.hgignore,/.svn,/.gitignore,/.gitmodules,/.hgsubstate,/.gitattributes,/CVS/,/.hgsub,/.DS_Store,**/.cvsignore
This option allows to enable or disable the default Ant exclusions.
Turn that off and it should pick them up, but may also include much more. Consider using a pattern in:
Files to archive [ **,.gitignore,.gitattributes ].
Downloading
However, the "(all files in zip)" is part of the Jenkins UI standard DirectoryBrowserSupport capability. Any directory displayed within the "Browse Workspace" has the option displayed to download the content of that page. That "(all files in zip)" is also provided by the org.apache.tools.zip capability which would use the same Ant FileSet excludes. This applies equally to both the Browse Workspace and Browse Artifacts pages. But, as this capability is built into the Jenkins UI layer, there does not seem to be any overrides for the Artifacts. You just need to download them manually individually.
It may be worth filing a ticket (for component:core, label archive) such that the download "all files in zip" respect the content as displayed within "Browse Artifacts". (That would not be desirable behaviour for the workspace itself).
More generally:
As the help says, it's based on default Ant exclusions (a list which is modifiable). See also Ant FileSet for other conventions.
Ant is in turn based in unix conventions, where files beginning with a dot "." are "hidden files", typically considered "configuration files' and not generally of interest. The Ant list is more specifically to exclude configuration files which are part of common source control systems (there'd be no need to continually archive .git for example). "dot files" do not show up by default in ls -l; you need to run ls -la (all). That logic also extend to other commands.
Also note that while unix is case sensitive, the default for Ant is "no" to case sensitivity for includes/excludes, which may cause issues if (for some silly reason) you have Foo.java and foo.java, but which is then enabled by default in the archiveArtifacts option.
Related
I'm working on a basic Jenkins pipeline. The build and testing are successful but I'm looking at how to archive the build. For context, this is a simple Rust webserver.
Under the pipeline steps documentation in the Basic Steps plugin, it has the archive function. But it says:
Archives build output artifacts for later use. As of Jenkins 2.x, you may use the more configurable archiveArtifacts.
I cannot find any documentation on archiveArtifacts. There are some examples, but I would like to look at the documentation for it, what parameters it accepts, i.e. what makes it more configurable than archive.
My question: is there a place where this documentation is best found? jenkins.io is incomplete and wiki.jenkins.io is missing this command.
I suggest archiveArtifacts: Archive the artifacts from the Pipeline Steps Reference.
Archives the build artifacts (for example, distribution zip files or
jar files) so that they can be downloaded later. Archived files will
be accessible from the Jenkins webpage. Normally, Jenkins keeps
artifacts for a build as long as a build log itself is kept, but if
you don't need old artifacts and would rather save disk space, you can
do so.
Note that the Maven job type automatically archives any produced Maven
artifacts. Any artifacts configured here will be archived on top of
that. Automatic artifact archiving can be disabled under the advanced
Maven options.
artifacts
You can use wildcards like 'module/dist/**/*.zip'. See the includes attribute of Ant fileset for the exact format. The base directory is the workspace. You can only archive files that are located in your workspace.
Type: String
allowEmptyArchive (optional)
Normally, a build fails if archiving returns zero artifacts. This option allows the archiving process to return nothing without failing the build. Instead, the build will simply throw a warning.
Type: boolean
excludes (optional)
Optionally specify the 'excludes' pattern, such as "foo/bar/**/*". A file that matches this mask will not be archived even if it matches the mask specified in 'files to archive' section.
Type: String
In my repository I have some files with the name "build" (automatically generated and/or imported, spread around elsewhere from where I have my bazel build files). These seem to be interpreted by Bazel as its BUILD files, and fail the full build I try to run with bazel build //...
Is there some way I can tell Bazel in a settings configuration file to ignore certain directories altogether? Or perhaps specify the build file names as something other than BUILD, like BUILD.bazel?
Or are my options:
To banish the name build from the entire repository.
To add a gigantic --deleted_packages=<...> to every run of build.
To not use full builds, instead specifying explicit targets.
I think this is a duplicate of the two questions you linked, but to expand on what you asked about in your comment:
You don't have to rename them BUILD.bazel, my suggestion is to add an empty BUILD.bazel to those directories. So you'd end up with:
my-project/
BUILD
src/
build/
stuff-bazel-shouldn't-mess-with
BUILD.bazel # Empty
Then Bazel will check for targets in BUILD.bazel, see that there are none, and won't try to parse the build/ directory.
And there is a distressing lack of documentation about BUILD vs. BUILD.bazel, at least that I could find.
We have the Jenkins folder plugin. When a new folder is created, and 'Add an item' is selected, none of the projects from the containing Jenkins database are available to be copied in the 'copy from' selection. What's the easiest way to copy an existing Jenkins project into a new folder?
You need to specify the absolute path.
For example if you create folder
Util
And inside Util you want to copy job "FooJob" from the base level in "copy from" specify
/FooJob
Or of FooJob is inside folder Bar specify
/Bar/FooJob
Select your project
Choose "Move"
Select your folder from the drop-down
The answer given by krtrego (use absolute paths) is correct -- but: you can also copy the job on filesystem level, which is preferable in many cases. That is, on Linux, do
cp -ar $JENKINS_HOME/jobs/jobXYZ $JENKINS_HOME/jobs/folderABC/jobs
Shutdown Jenkins beforehand and start it afterwards (or do not shutdown, and just "reload configuration from disk" afterwards).
Depending on the definition of "easy", this might be the best choice, since
If you're copying many jobs, then this is only way that jobs won't start before you completed moving all of them ("Folder copy" on GUI-level does not work in "quiet-down" mode).
The Folder copy operation sometimes takes ages (literally) for certain jobs.
Tagline: for anything but the most simple tasks, it will be good to avoid Folder Move/Copy.
I have two jenkins jobs which are chained. The first job will run some testNG tests , in which we may get some failures. (A testng-failed.xml file will be generated that has all the tests that failed). In my second job, I need to run using that testng-failed.xml as my suite file. So I am using jenkins copy artifact plugin and saving the testng-failed.xml file. I am able to retrive it in the second job , where it says "Copied 1 artifact from "rerun_exp" build number 7".
But the problem is I dont know where it is saved to be used as my suite file.
You can easily check where the file is by looking in the workspace.
This can even be done directly from the Jenkins web UI, via the "Workspace" link in the sidebar of the job page.
Files matched by the "Artifacts to copy" field are copied to the same directory structure as the source build had.
You can use the "Flatten directories" option if you want the file(s) to be copied to the root of the build workspace. If you click the (?) help icon to the right of that checkbox, you can see documentation for this and for your original question.
Currently I'm using the url with lastFailedBuild to display png (capturing page errors) in the browser (jenkins report) : http ://-jenkins-/job/jobName/lastFailedBuild/artifact/screenshots/Fail1.png/
But the problem is : I would be able to store as many artifacts as maximum build I set in the conf. So I would like to keep the artifacts for each build. We can already do it, but the url contains the build number (and I'd like to avoid manipulating the path to get back the build number). Is there a jenkins alias, a plugin, or can we use wildcards for that?
Have an alias like currentBuild wich returns the build number would be perfect (and simple). By 'current build', I mean to refer to the number build report I'm watching, not the last build report.
Also, where are physically stored the artifacts? I mean, I know where the files are stored, in the workspace, but for artifacts the url displays /job/, where is the folder contening artifacts in my jenkins server? Is it a sort of symbolic link to files in workspace?
I wonder : if I delete the image (in the workspace) before each new build, will it keep the previous artifacts ? I think it's yes because when I overwrite a png image the artifact is kept (it seems to me).
I think this topic : aliasing jenkins artifact URLs doesn't answer my question.
More details :
Here my current report, now I want to refer to http ://-jenkins-/job/JDN/55/artifact/screenshots/Fail1.png/ if I'm on build report #55, or http ://-jenkins-/job/JDN/50/artifact/screenshots/Fail1.png/ if I'm on build report #50.
I could do it in my script looking for the last number build but it's a little heavy. I'd like to know if Jenkins manages that, like lastFailedBuild, lastSuccessfulBuild alias. -> an alias which refers to the artifacts of the observed report. -> it could be something as : http ://-jenkins-/job/JDN/currentReportNumber/artifact/screenshots/Fail1.png/
There is permalink to /lastBuild, which was what I think you mean by "current build"
You can also add /buildNumber to any permalink to get just the value of the build number, for example /lastBuild/buildNumber will return the numeric value of the last executed build, while /lastFailedBuild/buildNumber will return the numeric value of the last failed build.
Physically, the artifacts are stored on the server alongside your WORKSPACE. Under $JENKINS_HOME (or %JENKINS_HOME% for Windows), look for /jobs/<jobname>. There you will see
- config.xml (your job configuration)
- nextBuildNumber (contains the next build number, don't modify this)
- workspace folder (this is the job's WORKSPACE that Jenkins uses during build
- builds folder (this is the history of all your retained builds)
Open the builds folder, and you will see all your saved builds (in time-stamped folders), including symlinks representing the permalinks (such as lastFailedBuild, etc). Under each time-stamped folder, you will see archive folder. This is where the archived artifacts are stored.
To access the WORKSPACE files through the URL use http://<jenkins-url>/job/<job-name>/ws/<path-to-files>
Jenkins has a list of variables you can use , see Jenkins Set Environment Variables. So in the shell script launched by the job; you can see the build number with echo $BUILD_NUMBER and use it. Or directly use BUILD_URL.
I obtained the specific url for each build with that.