There's a concept in Artifactory of "Modules". In the UI, when you go to the "Builds" tab, and enter some specific build, you get a list of Module IDs, which is, from what i've seen, just the name of the build.
So when I do this:
jfrog rt upload --build-name Kuku --build-number 1 File.bin Repo/path/
I get a Build with name "kuku", the number "1", and inside I have a list of Module IDs, with only 1 entry - Kuku, and if I click on it then I can see the list of artifacts that are part of this build.
So my question is, what are these Modules? How can I add more modules (Or why should I) to a specific build?
Just mention that I'm using a Generic repo
Module are part of the JFrog build info open integration layer for the CI servers and build tools which is used for capturing build metadata.
Build info can be captured using CI integrations, build tools, the JFrog CLI or the REST API. Once it is published to Artifactory, you can query it using the REST API, Artifactory Query Language (AQL) and GraphQL.
As part of build info, a module is a way to group artifacts which where produced by as build and dependencies which where used when building them. A module, for example, can represent a micro service.
The JFrog CLI, which you are already using, can be used for collecting build info.
You can add artifacts when uploading them in the following way:
jfrog rt upload --build-name Kuku --build-number 1 --module Momo File.bin Repo/path/
And dependencies can be added when you are downloading them:
jfrog rt dl "my-local-repo/yoyo.zip" kuku-dependencies/ --build-name Kuku --build-number 1 --module Momo
To publish build info, use the following command:
jfrog rt bp kuku 1
Related
I have uploaded some release artifacts from Jenkins to Nexus. I can see they have been created under Nexus as per the -Dversion I have provided.
Example: AppName-BuildNumber-Snapshot.zip
However, somehow the repository path is autogenerated, it has extra yyyymmdd.hhmmss appended to it.
Example: AppName-BuildNumber-Snapshot-yyyymmdd.hhmmss.zip
I would like to download the release artifact from Jenkins but due to the autogenerated Nexus repository path I am not able to download it in the separate job.
How can I force it to stop adding yyyymmdd.hhmmss to the repository path? Or is there anyway I can retrieve the repository path using AppName and BuildNumber?
If you are uploading a SNAPSHOT, then Nexus dynamically stores it with a timestamp. That's because you can replace -SNAPSHOTs with newer copy w/same GAV. This is unlike real # jars which cannot/should not be redeployed, thus preserving their integrity.
To download a -SNAPSHOT, you just reference it as -SNAPSHOT; maven, aether, etc will retrieve the latest.
Maven (by default) only checks the remote repository for versions newer than your local once daily, unless you use -U option or change the updatePolicy.
If you want "solid" number, then you must release jar.
btw, you should see it's -yyyymmdd.hhmmss-nn, where nn is an incremental number for that version.
Nexus supports many different repository formats. If you only require maven, use Nexus 2 as it better supports maven.
Repository types: Maven has two distinct types:Release and Snapshot Repositories.
If you want to "upload a zip file containing release artifacts and retrieve it back based on the build version number?", then that's what you should do.
You must configure a repository of type release, not snapshots (gleened from: -DrepositoryId=dsnexus-snapshots) (or in addition to type snapshots).
To pass the "build version number", then presumably, you have:
[ X ] Create a formatted version number
Environment Variable Name [ label ]
Make sure your maven step has:
[ X ] Inject build variables
Then, mvn deploy:deploy-file -DgroupId=com.my.gid -DartifactId=AppName -Dversion=${label} -DrepositoryId=dsnexus-release
ps: you'll also want to implement a cleanup strategy in your Nexus repository if you are pumping every build as a new artifact.
I'm working on a new CI proof of concept. I'm using TFS build and attempting to integrate jFrog Artifactory.
I'm trying to create a folder structure within my Artifactory repository like so:
[repository]/[sub-repository]/[Artifacts Folder]/[Versioned Artifact Folder]/[Versioned Artifact Zip Archive]
I've scripted the creation of the following correct structure in my Artifactory staging directory with PowerShell:
[Artifacts Folder]\[Versioned Artifact Folder]\[Versioned Artifact Zip Archive]
... and finally compressed my [Artifacts Folder] into a [Artifacts Folder].zip archive for deployment to Artifactory repository.
Now, although jFrog documentation indicates the introduction of an --explode option in jFrog 1.7 for this purpose, attempts to upload using this option returned an Incorrect Usage error:
2018-10-01T10:21:28.3168258Z running 'C:\jfrog\jfrog.exe' rt upload '[Artifactory Staging Directory]\[Artifacts Folder]\*' '[repository]/[sub-repository]/[Artifacts Folder]' --url=https://www.artifactrepository.xxx.net/artifactory --explode=true --user=******** --password=******** --props='build.number=[build_number];build.name=[build_name]'
2018-10-01T10:21:28.3168258Z
2018-10-01T10:21:28.3168258Z
2018-10-01T10:21:29.6761967Z Incorrect Usage.
2018-10-01T10:21:29.6761967Z
2018-10-01T10:21:29.6761967Z NAME:
2018-10-01T10:21:29.6761967Z jfrog rt upload - Upload files
2018-10-01T10:21:29.6761967Z
2018-10-01T10:21:29.6761967Z USAGE:
2018-10-01T10:21:29.6761967Z jfrog rt upload [command options] [arguments...]
2018-10-01T10:21:29.6761967Z
2018-10-01T10:21:29.6761967Z OPTIONS:
2018-10-01T10:21:29.6761967Z --url [Optional] Artifactory URL
2018-10-01T10:21:29.6761967Z --user [Optional] Artifactory username
2018-10-01T10:21:29.6761967Z --password [Optional] Artifactory password
2018-10-01T10:21:29.6761967Z --apikey [Optional] Artifactory API key
2018-10-01T10:21:29.6761967Z --ssh-key-path [Optional] SSH key file path
2018-10-01T10:21:29.6761967Z --props [Optional] List of properties in the form of "key1=value1;key2=value2,..." to be attached to the uploaded artifacts.
2018-10-01T10:21:29.6761967Z --deb [Optional] Used for Debian packages in the form of distribution/component/architecture.
2018-10-01T10:21:29.6917936Z --recursive [Default: true] Set to false if you do not wish to collect artifacts in sub-folders to be uploaded to Artifactory.
2018-10-01T10:21:29.6917936Z --flat [Default: true] If set to false, files are uploaded according to their file system hierarchy.
2018-10-01T10:21:29.6917936Z --regexp [Default: false] Set to true to use a regular expression instead of wildcards expression to collect files to upload.
2018-10-01T10:21:29.6917936Z --threads [Default: 3] Number of artifacts to upload in parallel.
2018-10-01T10:21:29.6917936Z --dry-run [Default: false] Set to true to disable communication with Artifactory.
2018-10-01T10:21:29.6917936Z
I using jFrog Artifactory Deployer 2.1.1 TFS build task.
This command line option is described here: https://www.jfrog.com/confluence/display/CLI/CLI+for+JFrog+Artifactory#CLIforJFrogArtifactory-UploadingFiles
However, it seems that jFrog.exe which is on our TFS servers doesn’t understand --explode command line option.
(Note: I am unsure what version of jFrog.exe is running on our build servers; currently awaiting details from responsible team, update to follow.)
Is the issue that the jFrog.exe version is older (pre 1.7) and does not support the --explode command option? If so, is there an alternative way to achieve multiple artifact upload while preserving staging folder structure?
(Note: I applied the --flat=false option but the staging folder hierarchy was preserved right back to the root; this is not what's required either).
insights appreciated, thanks for looking..
In the end, we were able to work around the absence of the '--explode' command option by using placeholders like so:
In the jFrog Artifactory Deployer task:
Path to the Artifacts: [Artifacts Folder]\(**)\(*)
Target Repository [repository]/[sub-repository]/[Artifacts Folder]/{1}/
The use of placeholders in this way accomplished the preservation of folder structure in the push to the Artifactory repository as required.
I am using Jenkins CI(single pipeline Job) to push different files to JFrog Artifactory.
For each and every build I am publishing the build info to Artifactory. But every build has same Module ID.
for example: In Artifactory Repository Browser select any artifact then go to Build tab, then it is showing all builds instead of respective artifact build information
So I would like to know how to modify the Artifact Module ID while publishing the artifact information to Artifactory
The build information and Artifact storage are to different mechanisms. The build information is only associated with artifacts in the sense that you can define what artifacts are produced from a build and what artifacts are a dependency of a build. The module ID has no association with the build info. The module ID is defined by the repository layout where the Artifact is stored. See the jfrog documentation here: https://www.jfrog.com/confluence/display/RTF/Repository+Layouts
You cannot change the module ID. Artifactory assigns a module ID based on the repository layout. This means the path to the artifact matters for defining the module ID.
For example the repository layout
[org]/[module]/[module]_[base_rev].[ext] will give module IDs to any artifact stored following that pattern as [org]:[module]:[base_rev].
Artifacts are associated to builds by both a build.name, build.number property on the artifact and publishing a build info from a .json file. The .json must include a section denoting what artifacts were produced from the build.
In general, it sounds like you need to ensure each artifact is being deployed to a unique path in artifactory with the build.name and build.number property being set. The other thing to look at is the file hash. it must match in the build info being published. So if the file hash is not different that means there is no change to the artifact from the previous build.
I don't know how much this will help, but there is a lot of nuances that go into getting the artifacts and builds to line up. You mentioned using Jenkins, have you looked into using the Jenkins Artifactory plugin. It should take care of a lot of this work for you, although we use Bamboo, so I am not as familiar with how it differs.
I want to publish a generic file to our Artifactory as a ivy artifact, and I have tried using the "Generic-Artifactory Integration" option in the job configuration.
The deployment works, but the file is deployed as a straight up file in the Artifactory repository root, e.g.
<artifactory url>/artifactory/ivy-repo/my.file
How can I the plugin publish the artifact as an Ivy dependency instead?
When working with Generic Artifactory Integration you should define deployment patterns (as described in the tool-tip in Jenkins:
Since there is no way for Artifactory to determine the org path for a file without metadata (ivy.xml or pom.xml), you have to provide it manually in the right-hand part of the pattern expression, e.g. my.file=>my/org/*
I'm using Jenkins and have the "Archive the Artifacts" step at the end of my builds to archive them into a zip file.
Instead of using this step, I'd like to use a script to push the artifacts to a remote server at the end of the build. The server I'm pushing to uses a REST API / HTTP PUT request in a script to upload files.
Note that I'm looking to access the artifact created in the same build. So if I'm on build #5, I want the artifacts from build #5, not build #4.
Is there any way to access this zip file with a script, in the same build that it was created in?
I need to upload this zip remotely and don't want to create another job to do so.
You can install one of the "Publish Over..." plugins to upload your artifacts at the end of a build.
The goal of the Publish Over plugins is to provide a consistent set of
features and behaviours when sending build artifacts ... somewhere.
See also the full list of "upload" plugins for other methods of publishing your artifacts.
Like #Christopher said, you can use any of the Publish Over plugins on the Jenkins Plugins page to upload the artifact to any of the
If you want to access the archived zip file from within the build itself, you can use the following link to access it:
http://<server>/job/${JOB_NAME}/lastSuccessfulBuild/artifact/<artifact name w/folder>
For example:
server = myserver.com
job name = myproject
artifact = del/project.zip
Your URL would be:
http://myserver.com/job/myproject/lastSuccessfulBuild/artifact/del/project.zip
EDIT: Question was changed. In any case, this would work for accessing the artifact of the previous build in the current one.
There is no way that I have found to access the "Archive the Artifacts" package of the build that generates it. This step always occurs last in the build. Accessing the URL prior to the build ending (during the build via script for example) results in a blank zip file. To get around this limitation, I'm making a second linked build job to grab the zip and run my script to deploy it.