We are migrating 50+ .net project from TFS to GitHub, at the same, we want to use Jenkins to automate the build. Currently all the builds are done inside the Visual Studio manually. I know how to automate this build using MSBuild and we already have a lot of these projects building inside Jenkins.
My question: is there a way to set up these 50+ project quickly w/o creating them one by one manually? Anyway to script them? e.g. a Jenkins project has everything inside a folder, I can copy a sample project/folder to create a new one and modify something. Or create a Jenkins project using a script reading a config file? Any idea can save some time is appreciated.
Not a direct answer but too long for a comment so here it goes anyway. Following the Joel test (which in no way is dogmatic for me but does make a lot of good points), and in my experience, you should already have an msbuild file now to build all those projects 'in one click'. Then, setting up a build server, in fact any build server, is just a matter of making it build that single parent project. This might not work for everyone, but for several projects I've worked on this had the following advantages:
the entire build process gets defined by developpers, working locally on their machine, using 'standard' tools
as such they don't need to spend hours in a web interface figuring out the appropriate build steps, dependencies and whatnot (also those hours would have been worthless in the end if switching to a different build server)
since a complete build is now just a matter of msbuild master.proj, possibly along with some options to define configuration/platform/output directories getting this running on any build server should be painless and quick
in the same manner this makes it easy to test different build servers with a minimum of time and migrate between them (also no need to ask SO questions on how to set everything up :)
this also makes it easy for other developpers to get complete builds as well without having to go round via a build server
Anecdote: we once had Jenkins running on multiple different projects as well. It took us days to get everything running, with the templates etc, and we found the web intercae slow and cumbersome (and getting to know the API would have taken even more days). Then one day I got sick of this and made a bunch of msbuild scripts which could build everything from one msbuild command. That took much less time than setting up Jenkins, a couple of hours or so. Then I took a TeamCity installation we already had and made it build the new master project. Took like an hour and everything worked. Just recently I took the same project and got it working on Visual Studio Online, again in no time.
If those projects are more or less similar to build, you will probably be interested in using the template plug-in for jenkins. There you configure a dummy project such that it does what is common to (most of) the 50+ projects.
Afterwards you create a separate project for each: Create the first project and make it use the template project for each of the steps which can be shared with the template project (use build step from other project). All subsequent projects can be created as slightly adopted copy of this first 'real' project.
I use it such that the variable $JOB_NAME (the actual project name in jenkins that is) is part of the repository path and I can thus clone from http://example.org/$JOB_NAME/
Configured that way, I can include the source code management step in the templating job and use it unmodified. Similar with the build step and post-build step: they are run by a script which is somewhat universal accross all my projects (mostly calling make and guessing deployment / publication paths upon $JOB_NAME again).
Related
I have a "SOA" styled application, with a single solution containing almost 100 individual projects (which are a solution within them, i.e. they can be run independently). I have created a build and release definition in TFS 2018 and everything works perfectly.
The issue is if I make changes to a single service (out of 100), and check-in my code, the build definition is triggered which builds the entire application and then the release definition deploys the entire thing(100+) each time.
I don't want it. I need it to be specific to the service in which changes are made. Is there any way to do it?
Creating multiple build/release definition which is tied to the specific path of each service should solve it but I don't want to go down that road, because I will end up creating 100's of definitions.
Is there any other way to do it?
This is for a .Net application, hosted in TFS 2018 (On-premise).
The solution structure is as below:
AllWCFService.sln
|_Service1.csproj
|_Service2.csproj
|..
|..
|_Service100.csproj
Each Service can also be run and hosted independently.
This is my first question here. I apologize for any confusion.
As you have mentioned, by using Path filters on the Build definition should be the easiest solution.
With the proper path to the project in the Path filter only the proper Builds spin up, and any projects untouched do not trigger a build. Each build has it's own release which then deploys the specified app to it's own destination. As a ugly workaround, you could set up a group each with 5 services which will reduce 100's of definitions to 20's.
Otherwise, you have to customize your build definition/pipeline. Use some scripts to determine or judge which part of your Servicex.csproj changed base on your name.
Then call msbuild with /t option to build a single or multiple projects.
msbuild test.sln /t:project;project2 /p:Configuration="Release" /p:Platform="x86" /p:BuildProjectReferences=false
specify project file of a solution using msbuild
It will only build specified changed project and generated corresponding artifacts. Then add scripts in release to specify path according build generated artifacts to deploy each service.
Hope this helps.
So I have a website
Website_RWD
That has three dependencies SRECLeads, InetLeads, and BusinessLayer
I am trying to improve the speeds of our builds and deployments. One thing I stumbled upon was that it seems the main website is building twice.
Poking around on the internet my understanding is that the .metaproj exist because there are project dependencies.
My question is, using TFS Builds or in the solution file is there a way so the project builds in this order.
BusinessLayer
SRECLeads
InetLeads
Website
As it would build on the local machines.
If the below log is in the correct order you can see that after it builds the website a second time, it builds each individual project. If that is correct what is the point of doing that and is there a setting to stop that.
In fact the Website_RWD project is not built twice, from what I can see in the log snippet.
The MSBuild log output is structured as tree view, and as you can see the Website_RWD.csproj.metaproj is one level above the Website_RWD.csproj. That means the Website_RWD.csproj.metaproj build somehow contains the Website_RWD.csproj build as a sub-step.
Currently I'm in the progress of updating TFS 2012 to 2013. My plan is to use the default build process template and stick to it, if possible. The existing build definitions use the old TFS method with TFSBuild.proj files. In thse proj files everything happens. Initializing the solution directory, build, clean, run unit test, drop files, etc. Compared to the 2013 build process template this is incorrect, since the workflow has activities for build, clean, run tests, drop files, etc. As well it seems that the targets file used in the TFSBuild.proj files is of a previous TFS version and hasn't been updated to a 2013 version.
The problem is that besides the build, clean, run test an drop files activities there are other activities needed. Version numbers are changed in certain files, obfuscation for dlls, check procedures if the source doesn't contain any unwanted files, zipping of pdb files, etc.
Of course it is possible to execute these tasks/activities with PowerShell scripts. On the other hand working with tasks in the a project file also seems logical. My concern with performing extra tasks in a project files is that TFS is running the activities for test, dropping files, etc. after calling the MSBuild activity.
Can someone point me in the right direction? Do I need custom developed activities that I can use in the build process template? Or is working with PowerShell scripts best practice?
I think you've answered your own question! Poweshell is the way forward!
I would say that modifying the XAML and writing a custom activity should only be considered if powershell isn't working for you, or if your scripts are becoming so big and difficult to debug that it makes sense to turn them in to code. As you mention, some of these tasks will need to be performed well after the MSbuild activity has completed so that's not going to work.
It looks like you want to do some fairly simple steps at defined points in the process and the powershell extensibility points are perfect for this.
A simple script for versioning in the pre-build step. Checking the source for unwanted files could happen here as well.
The obfuscation, zipping and copying stuff can happen post test.
From a Maintainability point of view, I would say that powershell has a much bigger user base than Windows Workflow and MSBuild so that's another advantage to this approach.
I am new to TFSBuild but I have been able to create a build definition for my solution. I have a couple questions that help would be great with.
I have created 3 definitions - I wonder if this is the correct way to this.
A definition that fires for every check in, builds the code and runs unit tests only
A definition runs every night, builds everything, runs all unit and integration tests
A definition that I specifically use for deployments - so I specify the environment via a parameter and it builds the code, runs unit and integration tests and ms deploys it to specified environment, again via parameter
When I branch my code etc, I will have to create 3 definitions for each and this could become unmanageable. Feedback on this please?
Is it true that each definition has its own set of build numbers? Can they be shared?
My application is an MVC4 app with VS2012 IDE.
Sadly TFS Build doesn't have very good support for branches, yes this typically means you duplicate your build definitions for each branch. There are a few custom build process template that I've seen in the past which try to get around this, but nothing built in.
You could replace #2 with a windows scheduler task to run #1 with custom parameters, not the nicest solution, but could be extended to queue every build definition at midnight with the integration test flag.
For #3 instead of using a build definition to deploy I use an external tool called TFS Deployer, this allows me to use any build to deploy simply by changing the build quality of the build. Reducing the number of builds that need to be setup by 1 for each branch.
Each build definition has its own build numbers, there's no built in way to share however I believe this is set by the build definition, so you may be able to hack around it somehow.
I am playing with TFS 2010, and am trying to setup a build process that will have some custom steps.
These include things like, stopping/starting IIS, search and replace files etc... across environments.
I have tried to look for examples online and have not found anything clear and meaningful on how to just run a script or something over the source files. Looking at the default build process template (DefaultTemplate.xml) I cant make much sense of it.
How do I go about doing this ?
For info on customising the TFS2010 workflow build templates have a look at Ewald Hoffman's series. Start with Part 1 (archived here).
I should also mention that since it looks like you're doing deployment then you may want to break deployment automation away from build automation.
This is almost exactly what I'd say for this question (Split build and deplyment stages, investigate TFSDeployer). One additional element is more generic - for deployment tasks you can't find an easy integrated tool you should create a custom deployment script. You can call any script by adding an "InvokeProcess" step in your Build workflow. TFSDeployer also has locations where you can insert custom PowerShell Scripts. (If you don't like PowerShell you can have PowerShell or "InvokeProcess" call a different script engine.)