I am working with Visual Studio Team Services Build definition triggers. I am trying to set a Continuous Integration and Delivery for one project on my branch (but not the entire branch with many projects). This will help me build and release individual microservice (when the contract did not change...).
I could not find the answer here or on in the official docs:
Build definition triggers
I did try to play with the Path filters but it has proven not to be a robust way. I might got it wrong...
Any guidance will be appreciated. Tnx in advance
To trigger CI build only when changes made on project3, you can only specify the path filter to the project directory.
Such as if the file structure in your repo as below:
Root repo
|___ project1
|___ ...
|___ project2
|___ ...
|___ project3
|___ ...
|___ project.sln
|___...
Then you can specify the path filter as below:
And to upload the build artifacts only about project3 for Continuous Deployment, you can copy related files from $(Build.SourcesDirectory)\project3 to $(Build.ArtifactStagingDirectory), and then upload build artifacts from $(Build.ArtifactStagingDirectory).
First, you have to specify the CI option and enable it.
Enabling the "Batch changes while a build is in progress" is optional.
Add the branches you want to "Branch filters".
Then add the "Include" and the list of paths that you want to trigger the build.
Since you added to exclude all in your configurations (as in your question), the build cannot include the other directory.
You also don't need to specify the "*" in the path.
Related
We have a build pipeline that's been working for a year or two. This builds and creates a few artifacts - ASP.Net web application, windows services, etc. The issue here is with the ASP.Net web application.
We've added some REACT components to the build. There are now an enormous \React\node_modules\ folder with tens of thousands of files. I want to exclude the \React\node_modules folder from the "Publish Artifact" step. I've edited this step, by adding this under the File copy options setting:
/XD react/node_modules
I have a "Publish Artifact" task that has "/XD react/node_modules" set as the File Copy options. However, when I run the build pipeline, it still copies everything in the react/node_modules folder.
When I view the raw logs, I do not see that being passed as an argument in the Robocopy command. I only see this in the logfile for the Publish Artifact step:
2022-04-13T00:24:47.8605686Z ##[command]robocopy.exe /E /COPY:DA /NP /R:3 /MT:8 "C:\agent_agentwork\1\s\Website" "c:\builds\HF\HF_3.8.1.75\Website" "*"
Any idea why it's not working here?
This is a bug in the Publish artifact task. A github issue has been logged for this in 2019 (https://github.com/microsoft/azure-pipelines-tasks/issues/11451). Sadly, it is not fixed yet.
Awaiting a fix, you could work around it by putting a build step that explicitly calls robocopy to copy the build output to a temporary location (with the /XD switch). And then put the 'Publish artifact' step that copies from that temporary location.
We are starting to migrate our old xaml build def's to VSTS web-based build def's. For each branch, we have a debug build def and a release build def. The debug build def is set up as a Continuous Integration build. As a test, I modified one source file and checked it in. The old xaml build def checked out the 1 file and seemed to have built only the project that changed (what we want and expect in a CI build). In the xaml build log I see the following:
<InformationField Name="Message" Value="1 file(s) were downloaded with a total size of 0.29 MB." />
and it ran the build in 3.3 minutes.
In the new VSTS build - I see that it does a "tf get /version:170936" and gets all the files in changeset id "170936":
2018-06-12T15:08:39.8409262Z Checking if artifacts directory exists: C:\BuildAgent\agent2\_work\1\a
2018-06-12T15:08:39.8409262Z Deleting artifacts directory.
2018-06-12T15:08:39.8409262Z Creating artifacts directory.
2018-06-12T15:08:39.8564882Z Checking if test results directory exists: C:\BuildAgent\agent2\_work\1\TestResults
2018-06-12T15:08:39.8564882Z Deleting test results directory.
2018-06-12T15:08:39.8564882Z Creating test results directory.
2018-06-12T15:08:39.8877401Z Starting: Get sources
2018-06-12T15:08:39.9033640Z Entering TfvcSourceProvider.PrepareRepositoryAsync
2018-06-12T15:08:39.9033640Z localPath=C:\BuildAgent\agent2\_work\1\s
2018-06-12T15:08:39.9033640Z clean=False
2018-06-12T15:08:39.9033640Z sourceVersion=170936
2018-06-12T15:08:39.9033640Z mappingJson={"mappings":[{"serverPath":"$/Path/To/Branch","mappingType":"map","localPath":"\\"}]}
2018-06-12T15:08:39.9033640Z Syncing repository: Project Name (TFVC)
2018-06-12T15:08:39.9033640Z workspaceName=ws_1_45
2018-06-12T15:09:06.7318304Z Workspace Name: ws_1_45;a6060273-b85e-4d4b-ac63-3fbbcafc308b
2018-06-12T15:09:06.7630780Z tf get /version:170936
2018-06-12T15:09:21.6070136Z Getting C:\BuildAgent\agent2\_work\1\s;C124440
2018-06-12T15:09:21.6070136Z Getting C:\BuildAgent\agent2\_work\1\s;C124440
2018-06-12T15:09:21.6226405Z Getting C:\BuildAgent\agent2\_work\1\s\.nuget;C158533
2018-06-12T15:09:21.6226405Z Getting C:\BuildAgent\agent2\_work\1\s\Build Scripts;C141602
2018-06-12T15:09:21.6226405Z Getting C:\BuildAgent\agent2\_work\1\s\Databases;C124440
~
~ The rest of branch...
~
and then seems to rebuild all projects taking 13.2 min to run, almost 10 minutes longer than old xaml build.
Am I missing something with the new build def? I do not have the "Clean" button checked in the VS Build task. I do have a build.clean variable but it is Blank by default - sometimes we want to clean so we just can set it to "all" at queue time.
Clicking about on the web shows the following MS VSTFS version: 15.105.25910.0
Any help is much appreciated.
For multiple build agents, even though you have not checked Clean checkbox of the VS Build task, and the clean option in the Repository is set to "false".
Since the agent is randomly picked, it may not contain your source code before. That's why you see that TFS does a tf get /version:170936 and gets all the files in changeset id "170936" and also build all projects.
For Multi-configuration in Options tab of build definition. Please refer the official Article: How do I build multiple configurations for multiple platforms?
After that, it will split configurations to multiple builds during the build.
And ow that your build definition successfully builds multiple configurations, you can go ahead and enable parallel builds to reduce the duration/feedback time of your build. You specify this as additional option on the Options page:
Take a look at this post Building multiple configurations and/or platforms in Build 2015
To narrow down if the issue is related to multiple build agents, you could also send TFS build to a specific agent by demand, and queue the build multiple times with clean=false in the same build agent, it should have built only the project that changed(CI).
I need to build two Maven projects which are available on Github. The first project produces a library which is needed by the second.
What is the recommended option to do it in Jenkins ?
Create a shell script which checkout and build both projects (in the correct order).
Use a pipeline ?
If you could provide any example/ relevant link that would be great. Thanks
You can achieve using 2 (upto my knowledge) options. Lets consider project A which builds libraries and project B is the dependent. I described below two options by considering two constraints.
Project B should be built whenever Project A is built
Project B can also be built alone if there is only changes with Project B but not with A
Option 1:
You have to create two "Maven Build jobs" for Project A & B. In Project B, you have to specify "Build whenever a SNAPSHOT dependency is built" under "Build Triggers".
This only applies when project B pom.xml has dependency of Project A built artifacts and the artifacts are SNAPSHOT. Below is the image of the same.
Options 2: You can create two Freestyle job for both Project A & B. Define Project B is downstream for Project A ( It can also be done vice versa ) using "Build Other Projects" under "Post-Build Options" of Project A.
And you can copy artifacts from Project A to Project B using Copy Artifact Plugin
Exception: Ideally the above step is not necessary, if your Project A artifacts are installed on $HOME/.m2/repository and Project B dependency defined in pom.xml (or) both project A & B are using Private Maven Repository option.
Assumption: I hope using single pipeline, it will be hard to manage this scenario. Though I expect somebody else will write with pipeline example to enlighten me as well :)
You can achieve this using Jenkins pipeline jobs.
First we need to create a folder for project A and in side that you can download the source code using GIT plugin and you can build the solution.
And create another folder and inside your download the source code of another project and you can build the second solution.
eg:
node{
stage('first project'){
dir('project1'){
git branch: '<Branch>', changelog: false, poll: false, url: '<First Repo URL>'
sh 'mvn clean install'
}
}
stage('second project'){
dir('project2'){
git branch: '<Branch>', changelog: false, poll: false, url: '<Second Repo URL>'
sh 'mvn clean install'
}
}
}
Sharing a method I used for building two changes.
Just add "Depends: xxxxxxx" before the Change-Id in your git commit message:
Depends: fe1a5effxxxxxxxxxxxxxxxxb8xxxx
Change-Id: Ig43j5cxxxxxxxxxxxxxxxxxxxxxxxxxxxx
I wonder if it is possible to remove only one build (including artifacts) from job workspace.
I tried to "Delete Build" in Build History but all it does is remove build reference from Build History table. I know I can ssh to a server and delete files from the command line but I am looking for a way to do it from Jenkins web interface.
After installing Workspace Cleanup Plugin I am able to wipe out current workspace but I want to keep my other builds in the workspace.
In your Jenkins instance, to be able to have folder/per build - set flag "Use custom workspace" in your job's settings. Here is a brief help info from the setting description:
For each job on Jenkins, Jenkins allocates a unique "workspace directory."
This is the directory where the code is checked out and builds happen.
Normally you should let Jenkins allocate and clean up workspace directories,
but in several situations this is problematic, and in such case, this option
lets you specify the workspace location manually.
One such situation is where paths are hard-coded and the code needs to be
built on a specific location. While there's no doubt that such a build is
not ideal, this option allows you to get going in such a situation.
...
And your custom directory path would look like this:
workspace\$JOB_NAME\$BUILD_NUMBER ~> workspace\my-job-name\123
where $JOB_NAME will be "my-job-name" and $BUILD_NUMBER is the build number, eq. "123".
There is one nasty problem with this approach and this is why I wouldn't recommend to use it - Jenkins will not be able to reclaim disk space for outdated builds. You would have to handle cleanup of outdated builds manually and it is a lot of hassle.
Alternative approach, that gives you more control, tools and is able to keep disk space usage under control (without your supervision) is to use default workspace settings and archive your build output (files, original source code, libraries and etc.) as a post-build action. Very-very handy and gives you access to a whole bunch of great tools like, Copy Artifact Plugin or ArtifactDeployer Plugin in other jobs.
Hope that info helps you make a decision that fits your needs best.
I also use "General/Advanced/Use custom workspace" (as in #pabloduo's answer) on a Windows machine with something like:
C:\${JOB_NAME}\${BUILD_NUMBER}
Just wanted to add a solution for getting rid of the build job's workspaces.
I use Groovy Events Listener Plugin for this.
Using the plug-in's standard configuration I just use the following Groovy script:
if (event == Event.JOB_DELETED){
new File(env.WORKSPACE).deleteDir()
}
And now the custom workspace is deleted when the build job is deleted.
Just be aware that this would also delete non-custom workspaces (because the event is triggered for all jobs on your Jenkins server).
I hope you guys can help me.
my problem is the following:
i have 2 projects that depend on each other each of them in different svn root.
what i do toady is build the first , take its artifact and copy it manually to where ever i want in second project and than run the second build.
i want to be able to run 1 build it can be either with a new ant build.xml file who will run the first build , copy its product to where ever i want it and than will run the second build.
second option run those 2 project with dependency in teamcity. i'm not sure how to do so and how to pass the first project artifact to be placed in the right folder i need it in the second project.
maybe someone know's how to do that.
thanks.
TeamCity's Artifact Dependencies seems to be the best option.
Suppose you have ProjectA and ProjectB:
Configure TeamCity to publish files from ProjectA:
Open General Settings configuration step of ProjectA.
Fill in Artifact paths field with relative paths to published files. TeamCity will collect these files after build finishes.
Build ProjectA to make sure artifacts are collected properly. Artifacts must become available on build results page (see Artifacts tab).
See documentation for artifacts for more details
Configure Artifact dependency for ProjectB:
Open Dependencies configuration step of ProjectB, add new artifact dependency
Depend on - select ProjectA
Get artifacts from: - use whatever you need. Possibly, last successful build.
Artifacts - enter artifacts paths from ProjectA. You can use popup to choose artifacts with UI.
Destination path - enter destination path, where artifacts will be copied to. Path is relative to checkout directory
See documentation for artifact dependencies for more details
Now each time you start build for ProjectB, it will download artifacts from last successful build of ProjectA. If you want these builds to start simultaneously, add Snapshot dependency (on ProjectA) to ProjectB. With snapshot dependency, each time you start ProjectB, it will first trigger ProjectA, wait for ProjectA build to finish and (if has artifact dependency on "last successful build" of ProjectA) download fresh artifacts.