I have uploaded some release artifacts from Jenkins to Nexus. I can see they have been created under Nexus as per the -Dversion I have provided.
Example: AppName-BuildNumber-Snapshot.zip
However, somehow the repository path is autogenerated, it has extra yyyymmdd.hhmmss appended to it.
Example: AppName-BuildNumber-Snapshot-yyyymmdd.hhmmss.zip
I would like to download the release artifact from Jenkins but due to the autogenerated Nexus repository path I am not able to download it in the separate job.
How can I force it to stop adding yyyymmdd.hhmmss to the repository path? Or is there anyway I can retrieve the repository path using AppName and BuildNumber?
If you are uploading a SNAPSHOT, then Nexus dynamically stores it with a timestamp. That's because you can replace -SNAPSHOTs with newer copy w/same GAV. This is unlike real # jars which cannot/should not be redeployed, thus preserving their integrity.
To download a -SNAPSHOT, you just reference it as -SNAPSHOT; maven, aether, etc will retrieve the latest.
Maven (by default) only checks the remote repository for versions newer than your local once daily, unless you use -U option or change the updatePolicy.
If you want "solid" number, then you must release jar.
btw, you should see it's -yyyymmdd.hhmmss-nn, where nn is an incremental number for that version.
Nexus supports many different repository formats. If you only require maven, use Nexus 2 as it better supports maven.
Repository types: Maven has two distinct types:Release and Snapshot Repositories.
If you want to "upload a zip file containing release artifacts and retrieve it back based on the build version number?", then that's what you should do.
You must configure a repository of type release, not snapshots (gleened from: -DrepositoryId=dsnexus-snapshots) (or in addition to type snapshots).
To pass the "build version number", then presumably, you have:
[ X ] Create a formatted version number
Environment Variable Name [ label ]
Make sure your maven step has:
[ X ] Inject build variables
Then, mvn deploy:deploy-file -DgroupId=com.my.gid -DartifactId=AppName -Dversion=${label} -DrepositoryId=dsnexus-release
ps: you'll also want to implement a cleanup strategy in your Nexus repository if you are pumping every build as a new artifact.
Related
I am using Jenkins CI(single pipeline Job) to push different files to JFrog Artifactory.
For each and every build I am publishing the build info to Artifactory. But every build has same Module ID.
for example: In Artifactory Repository Browser select any artifact then go to Build tab, then it is showing all builds instead of respective artifact build information
So I would like to know how to modify the Artifact Module ID while publishing the artifact information to Artifactory
The build information and Artifact storage are to different mechanisms. The build information is only associated with artifacts in the sense that you can define what artifacts are produced from a build and what artifacts are a dependency of a build. The module ID has no association with the build info. The module ID is defined by the repository layout where the Artifact is stored. See the jfrog documentation here: https://www.jfrog.com/confluence/display/RTF/Repository+Layouts
You cannot change the module ID. Artifactory assigns a module ID based on the repository layout. This means the path to the artifact matters for defining the module ID.
For example the repository layout
[org]/[module]/[module]_[base_rev].[ext] will give module IDs to any artifact stored following that pattern as [org]:[module]:[base_rev].
Artifacts are associated to builds by both a build.name, build.number property on the artifact and publishing a build info from a .json file. The .json must include a section denoting what artifacts were produced from the build.
In general, it sounds like you need to ensure each artifact is being deployed to a unique path in artifactory with the build.name and build.number property being set. The other thing to look at is the file hash. it must match in the build info being published. So if the file hash is not different that means there is no change to the artifact from the previous build.
I don't know how much this will help, but there is a lot of nuances that go into getting the artifacts and builds to line up. You mentioned using Jenkins, have you looked into using the Jenkins Artifactory plugin. It should take care of a lot of this work for you, although we use Bamboo, so I am not as familiar with how it differs.
Is it possible to get current build's workspace path using jenkins remote api? I can get a build details based on build number with api/json, but it doesn't return the workspace details.
curl http://jenkinsServer:8080/job/testing/1/api/json
As far as I know you can't do that using Jenkins remote API. However, you can probably infer the workspace from your project name. If, say, Jenkins base workspace is /var/lib/jenkins/workspace (which is the case with default install on Unix), the workspace for your project should simply be :
/var/lib/jenkins/workspace/your-project
That could be sufficient for your needs, but workspace may vary, in particular if you are checking out some other repo inside your pipeline (or loading some other pipeline script from your base script), you could notice folders such as :
/var/lib/jenkins/workspace/your-project#tmp
/var/lib/jenkins/workspace/your-project#script
Currently I'm using the url with lastFailedBuild to display png (capturing page errors) in the browser (jenkins report) : http ://-jenkins-/job/jobName/lastFailedBuild/artifact/screenshots/Fail1.png/
But the problem is : I would be able to store as many artifacts as maximum build I set in the conf. So I would like to keep the artifacts for each build. We can already do it, but the url contains the build number (and I'd like to avoid manipulating the path to get back the build number). Is there a jenkins alias, a plugin, or can we use wildcards for that?
Have an alias like currentBuild wich returns the build number would be perfect (and simple). By 'current build', I mean to refer to the number build report I'm watching, not the last build report.
Also, where are physically stored the artifacts? I mean, I know where the files are stored, in the workspace, but for artifacts the url displays /job/, where is the folder contening artifacts in my jenkins server? Is it a sort of symbolic link to files in workspace?
I wonder : if I delete the image (in the workspace) before each new build, will it keep the previous artifacts ? I think it's yes because when I overwrite a png image the artifact is kept (it seems to me).
I think this topic : aliasing jenkins artifact URLs doesn't answer my question.
More details :
Here my current report, now I want to refer to http ://-jenkins-/job/JDN/55/artifact/screenshots/Fail1.png/ if I'm on build report #55, or http ://-jenkins-/job/JDN/50/artifact/screenshots/Fail1.png/ if I'm on build report #50.
I could do it in my script looking for the last number build but it's a little heavy. I'd like to know if Jenkins manages that, like lastFailedBuild, lastSuccessfulBuild alias. -> an alias which refers to the artifacts of the observed report. -> it could be something as : http ://-jenkins-/job/JDN/currentReportNumber/artifact/screenshots/Fail1.png/
There is permalink to /lastBuild, which was what I think you mean by "current build"
You can also add /buildNumber to any permalink to get just the value of the build number, for example /lastBuild/buildNumber will return the numeric value of the last executed build, while /lastFailedBuild/buildNumber will return the numeric value of the last failed build.
Physically, the artifacts are stored on the server alongside your WORKSPACE. Under $JENKINS_HOME (or %JENKINS_HOME% for Windows), look for /jobs/<jobname>. There you will see
- config.xml (your job configuration)
- nextBuildNumber (contains the next build number, don't modify this)
- workspace folder (this is the job's WORKSPACE that Jenkins uses during build
- builds folder (this is the history of all your retained builds)
Open the builds folder, and you will see all your saved builds (in time-stamped folders), including symlinks representing the permalinks (such as lastFailedBuild, etc). Under each time-stamped folder, you will see archive folder. This is where the archived artifacts are stored.
To access the WORKSPACE files through the URL use http://<jenkins-url>/job/<job-name>/ws/<path-to-files>
Jenkins has a list of variables you can use , see Jenkins Set Environment Variables. So in the shell script launched by the job; you can see the build number with echo $BUILD_NUMBER and use it. Or directly use BUILD_URL.
I obtained the specific url for each build with that.
I am modifying the hadoop source code but when I do a compile in maven it downloads from the maven repository, which is causing problems as my changes are not always be used.
How can I get maven to not download some files from the repository at all, and just use what is in the local classpath?
I am using maven 3.0.4
The best way is to give to "hadoop source code" a different SNAPSHOT version or a different artifactId and refer to it.
However, I think it is a stupid answer.
You can set local settings to offline=true. http://maven.apache.org/ref/3.0.3/maven-settings/settings.html;
or use a repository manager (sucha as Nexus) http://maven.apache.org/repository-management.html and deploy to Repository
Update these properties inside of your POM file to disable updates of your dependencies.
enabled: true or false for whether this repository is enabled for the
respective type (releases or snapshots).
updatePolicy: This element specifies how often updates should
attempt to occur. Maven will compare the local POM's timestamp
(stored in a repository's maven-metadata file) to the remote. The
choices are: always, daily (default), interval:X (where X is an
integer in minutes) or never.
POM Reference - MAVEN Site
I am new to ant and Ivy. We are using Jenkins for CI with ant for builds, Ivy for dependency manger. We have several modules/projects which generate jars and wars, which can be independently versioned and released (not all modules will be released at the same time), so, need to maintain version number separately for each module. We want to use the version format A.B.C.D (ex: 1.2.1.2). I found I can use a property file to enter a version number and use ant BuildNumber task to increment the number for our nightly builds. So, once all the features are in and tested we move the last successful nightly build as new released version but we want to change the version number without rebuilding it. For example last successful build was 1.2.1.20 and it was tested thoroughly and has all the feature, we have to make this build from 1.2.1.20 to 1.3.0.0 without rebuilding the modules. How can I do that using ant? And also I need to publish them to my shared repository with the version 1.3.0.0. How do I do that?
Also, we want to create a zip file for each module with all dependency files along with the module's jar file for delivery. Is there any ivy or ant tasks that can help to create this?
I think you've asked two questions...
Generally, every build I create is releasable so I'm always incrementing the last digit in my release number scheme. For controlling the version number I prefer to use the ivy buildnumber task, which increments based on what has been previously pushed to your repository (very useful).
Creating a zip package is quite straight forward. Just alter your ivy file to publish more than one artifact.