We store various design documents within TFS in multimarkdown format. We also have an EXE process that can run to take those MMD files and generate PDF's from them - but just by getting the files from a local folder.
What we'd like to do is to have a process run "on-checkin", just as if you'd run an automatic build on checkin (i.e., ultimately calling msbuild to compile an application) but in our case we'd like it to be able to get a list of the checked in files and to process and generate an output of them. The result doesn't need to be in TFS because they're a build output, not the source.
I'm sure this should be somehow possible by taking the same approach as must be taken by the workflow for a "normal" build.
Has anybody done anything like this or can point me in a suitable direction please ?
You could use the exec task in MSBuild to invoke the exe and "build" your output. Create a file called something like buildDocs.proj and check it in to TFS possibly in a folder under the things you want to build. Use the MSbuild below as a guide.
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Build">
<Exec Command='"My.exe" -My Paramiters' />
<ItemGroup>
<CopyItems Include="[path to output]\*.*" />
</ItemGroup>
<Copy SourceFiles="#(CopyItems)" DestinationFolder="$(OutDir)\SomeDir" />
</Target>
</Project>
The trick will be in identifying the various paths involved.
Use the default template to build the proj, just as you would a c# project. If you need to pass in additional Parameters to MSBuild you can do this from within the advanced section of the build definition process tab.
Related
I have a azure webjob project that uses config transforms to create dev/test/release configuration. We are using TFS for CI/CD deployment to Azure. I want to have MSBuild apply the transforms for dev so we can debug locally. However, when we are building in TFS in the CI/CD pipeline I need to disable the config transforms during the build step.
TFS has an "apply XML transformations" checkbox in the release step, which is where we want the transforms applied since we have the environment variable set during release. Unfortunately, this is not working because the transforms are already applied during build so the release artifact only has the finished output file, not the separate transform files.
I have tried editing the .csproj file to disable the transforms. I assume the transforms are being performed by the following section of the project file:
<UsingTask TaskName="TransformXml" AssemblyFile="$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)\Web\Microsoft.Web.Publishing.Tasks.dll" />
<Target Name="AfterCompile" Condition="Exists('App.$(Configuration).config')">
<!--Generate transformed app config in the intermediate directory-->
<TransformXml Source="App.config" Destination="$(IntermediateOutputPath)$(TargetFileName).config" Transform="App.$(Configuration).config" />
<!--Force build process to use the transformed configuration file from now on.-->
<ItemGroup>
<AppConfigWithTargetPath Remove="App.config" />
<AppConfigWithTargetPath Include="$(IntermediateOutputPath)$(TargetFileName).config">
<TargetPath>$(TargetFileName).config</TargetPath>
</AppConfigWithTargetPath>
</ItemGroup>
</Target>
<!--Override After Publish to support ClickOnce AfterPublish. Target replaces the untransformed config file copied to the deployment directory with the transformed one.-->
<Target Name="AfterPublish">
<PropertyGroup>
<DeployedConfig>$(_DeploymentApplicationDir)$(TargetName)$(TargetExt).config$(_DeploymentFileMappingExtension)</DeployedConfig>
</PropertyGroup>
<!--Publish copies the untransformed App.config to deployment directory so overwrite it-->
<Copy Condition="Exists('$(DeployedConfig)')" SourceFiles="$(IntermediateOutputPath)$(TargetFileName).config" DestinationFiles="$(DeployedConfig)" />
</Target>
I tried adding conditions like "$(Configuration)|$(Platform)' == 'Debug|AnyCPU'" to these sections, and it did not help (the transforms still got applied in all three environments). I even commented this section out completely, and I still got the transforms. This leaves me with three questions:
How can the config transforms be disabled?
How can I conditionally
disable them so they are still applied when debugging in VS?
Is this
the correct approach, or is there a better way to get the correct
transform applied when using CI/CD in TFS 2017?
To disable the config transform during the build, you just need to add argument /p:TransformWebConfigEnabled=False in MSBuild Arguments section of your Build task. You also need to add /p:AutoParameterizationWebConfigConnectionStrings=False if you want to update the connection string during the release.
Besides, you need to update your project file so that the Web.XXX.Config file will be included in the package if you are generating msdeploy package for deployment.
Change the "Build Action" of the config file from "None" to "Content".
Unload your project file and remove <DependentUpon> tag for the config file.
With these "MSBuild Arguments" it works for me:
/p:AutoParameterizationWebConfigConnectionStrings=false
/p:DeployOnBuild=true
/p:MarkWebConfigAssistFilesAsExclude=false
/p:PackageAsSingleFile=false
/p:PackageLocation="$(build.artifactstagingdirectory)\\"
/p:ProfileTransformWebConfigEnabled=false
/p:SkipInvalidConfigurations=true
/p:TransformWebConfigEnabled=false
/p:WebPublishMethod=FileSystem
The important ones:
/p:AutoParameterizationWebConfigConnectionStrings=false
/p:MarkWebConfigAssistFilesAsExclude=false
/p:ProfileTransformWebConfigEnabled=false
/p:TransformWebConfigEnabled=false
It started to work when I also added:
/p:ProfileTransformWebConfigEnabled=false
Thanks to #Eddie Chen - MSFT & #Wessel T.
Regards Hans
I have found other posts here on StackOverflow that deal with my issue I am experiencing, for example:
MSBuild: Deploying files that are not included in the project as well as Include Files in MSBuild that are not a part of the project
I wanted to share the code that I was able to create after reading these posts and ask for some help as to why it might not be working?
To elaborate on what exactly is not wrong and what I intend to do. I am using Visual Studio 2012, and TFS 2012.
I have a batch file called CreateMyFiles.bat, and what I would like to do is execute this and then take the files it outputs (it outputs them to my Includes/Javascript/Bundled folder) and include them in part of the build in MSBuild (so that they are deployed to the target IIS server).
When I edited my local .csproj in my local Visual Studio and added the code below to the bottom of the file and reloaded, I was able to right click on my web projbect, select 'publish', and then select my local file-based publishing profile which did indeed deploy my files to the correct location. It worked!
I then checked in my code to TFS, and went to 'builds' on TFS, and queued a new build. Sure enough, I was able to see the files output to the same directory on the build server. Now, i'm not 100% sure about MSBuild but I noticed that just like when I hit publish locally, it created a _publishedWebsite folder on the build server as well (a directory above the source). The thing is, within this publishedwebsite folder, my manually created files were not present. Furthermore, going to the target web server after the build was done unfortunately did not have the files I wanted.
So it seems like if I were to manually select publish, the code below works, but if I were to queue a build with TFS, it does not work. Does MSBuild use publish? Could that be the reason it does not work below?
Here is the code I've placed in my .csproj file:
<Target Name="CustomCollectFiles">
<Exec Command="CreateMyFiles.bat" /> <!-- Generate Files -->
<ItemGroup>
<!-- Create an identity called _CustomFiles, and associate it to the files I created -->
<_CustomFiles Include="Includes\JavaScript\Bundled\*" />
<FilesForPackagingFromProject Include="%(_CustomFiles.Identity)">
<DestinationRelativePath>Includes\JavaScript\Bundled\*%(Filename)%(Extension) </DestinationRelativePath>
</FilesForPackagingFromProject>
</ItemGroup>
</Target>
<!-- Hook into the pipeline responsible for gathering files and tell it to add my files -->
<PropertyGroup>
<CopyAllFilesToSingleFolderForPackageDependsOn>
CustomCollectFiles;
$(CopyAllFilesToSingleFolderForPackageDependsOn);
</CopyAllFilesToSingleFolderForPackageDependsOn>
<CopyAllFilesToSingleFolderForMsdeployDependsOn>
CustomCollectFiles;
$(CopyAllFilesToSingleFolderForPackageDependsOn);
</CopyAllFilesToSingleFolderForMsdeployDependsOn>
</PropertyGroup>
I'm really stuck on this and wanted to ask for some help as to why the files might not be going. I suspect MSBuild doesn't use publish and that's why it works locally (because i'm selecting publish)?
Thanks so much for your help
UPDATE
Tried this as per comments below, but this time the files didn't even appear (so it seemed to not even run the tasks now). Any idea why? Did I type this right?
<Target Name="CustomCollectFiles">
<Exec Command="CreateMyFiles.bat" />
<!-- Generate Files -->
<ItemGroup>
<!-- Create an identity called _CustomFiles, and associate it to the files I created -->
<_CustomFiles Include="Includes\JavaScript\Bundled\*" />
<FilesForPackagingFromProject Include="%(_CustomFiles.Identity)">
<DestinationRelativePath>Includes\JavaScript\Bundled\*%(Filename)%(Extension) </DestinationRelativePath>
</FilesForPackagingFromProject>
</ItemGroup>
</Target>
<!-- Hook into the pipeline responsible for gathering files and tell it to add my files -->
<PropertyGroup>
<PipelineCollectFilesPhaseDependsOn>
CustomCollectFiles;
$(PipelineCollectFilesPhaseDependsOn);
</PipelineCollectFilesPhaseDependsOn>
</PropertyGroup>
UPDATE 2
When I take the above code, and place it into my pubxml file and then execute an actual publish, it works, but as far as I know our process is to just queue a build from TFS. Is it possible to hook into the above code block when simply queuing a build? Or do I need to publish?
to do a publish from TFS build you need to add the following arguments
/p:DeployOnBuild=true;PublishProfile=Release
obviously using your own PublishProfile name
In VS2012, the target was renamed from:
CopyAllFilesToSingleFolderForPackageDependsOn
to:
CopyAllFilesToSingleFolderForMsdeployDependsOn
Update: looks like the above Targets are not getting called from within VS2012 targets, can you replace it with a call to the Target "PipelineCollectFilesPhaseDependsOn"? That should fix it.
<PropertyGroup>
<PipelineCollectFilesPhaseDependsOn>
CustomCollectFiles;
$(PipelineCollectFilesPhaseDependsOn);
</PipelineCollectFilesPhaseDependsOn>
</PropertyGroup>
I've been searching for a possibility to generate ANT targets from top-level macro.
Details:
We have heterogenic build system. ANT+IVY is used as top-level (inherited solutin, can't be changed). Some projects are built via MSBuild, called from ANT via exec task. For each of these projects, there's at least two distinct calls to msbuild (wrapped with macro for brevity), one in "build" target, and one in "clean". Two of them are different only by "target" parameter. So I was guessing, if there's possibility for something like this:
Extension nodes:
<extensionpoint name="build-ext-point" />
<extensionpoint name="clean-ext-point" />
<target name="build" depends="build-ext-point" />
<target name="clean" depends="clean-ext-point" />
My magic macro:
<macrodef name="msbuild-proj" />
<attribute name="project" />
<sequential>
<target name="#{project}-build" >
<msbuild project="#{project}" target="Build" />
</target>
<target name="#{project}-clean" >
<msbuild project="#{project}" target="Clean" />
</target>
</sequential>
</macrodef>
How it would be used:
<msbuild-proj project="CPP-proj" />
Thanks!
P.S: Yeah I know that I can define those build and clean overridden, or via ext point, or whatever. The question is actually whether I can remove some code duplication.
UPD: I'd answer this by myself. At the point, there's no such possibility. Mainly, because Target class is a task container, but not a task. So, it cannot be placed into container. So I guess I'll write some kind of extensible task.
ANT has a couple of mechanisms for building modular builds.
First of all I think your main question was on how to build "extension points" to your build? The following ANT tasks are designed to import common build logic from another build file:
import
include
Since you're already planning to extend your build using macrodefs, I'd recommend packaging these as a reusable ANTlib. The ANTlib can live within your project, but it's really designed to be packaged within a jarfile which another build can pickup, for example by installing it in the standard ANT lib directory:
$ANT_HOME/lib
$HOME/.ant/lib
Finally, if you're already using ivy and you package your taskdefs as ANT libs, you could version your build logic by installing it in a Maven repository manager like Nexus. This addresses one of the key problems with large ANT builds. Over time they become so big it's impossible to change the common logic without impacting older builds (Demonstrating that the builds are not properly isolated from each other).
Actually done this.
Does its job, although some caveats are present.
For those interested: https://bitbucket.org/targetsan/ant-events
I've tried a number of different configurations with this and I haven't achieved my result.
TL;DR
I'm trying to add config transforms into my build process and am looking for the right way to do it from MSBuild so that it shows up in my deployments via MSDeploy.
Background
I have an WebApp (MVC3), a Core app (CS Class Lib), and two test class libs, one for each.
I have a build script in my solution that uses MSBuild to compile.
One of those MSBuild targets deploys to an IIS server using MSDeploy
This process is working so far both manually and via CruiseControl.NET
Goal
I would like to add Web.Config transforms to this process. I figured I would do something simple at first, like an app setting called "PEAppsEnvironmentName", which I would make Dev, Test, or Prod based on the current environment.
Theory So Far
To me, it appears that when packaging with MSDeploy, I'm not transforming the config file.
When I run MSBuild with the DeployOnBuild option set to true, it creates another package that has the appropriately transformed config. It just seems like somehow I can't get it all to match up. The end result is that the web page displays "None" (the initial setting) instead of the transformed "Development" string.
I think if I could find out how to use MSDeploy during the packaging phase to transform the MSConfig, I'd be good to go.
Code
My web.config file
<appSettings>
<add key ="PEAppsEnvironmentName" value="None"/>
...
</appSettings>
My Web.Dev.config file
<appSettings>
<add key ="PEAppsEnvironmentName" xdt:Transform="Replace" xdt:Locator="Match(key)" value="Development" />
</appSettings>
My MSBuild Targets
Property group showing default config is "Dev"
<PropertyGroup>
<Configuration Condition="'$(Configuration)' == ''">Dev</Configuration>
</PropertyGroup>
My MSBuild "Compile" Target
<Target Name="Compile" DependsOnTargets="Init">
<MSBuild Projects="#(SolutionFile)" Targets="Rebuild" Properties="OutDir=%(BuildArtifacts.FullPath);DeployOnBuild=True"/>
</Target>
My MSBuild "Package" Target
<Target Name="Package" DependsOnTargets="Compile;Test">
<PropertyGroup>
<PackageDir>%(PackageFile.RootDir)%(PackageFile.Directory)</PackageDir>
<Source>%(WebSite.FullPath)</Source>
<Destination>%(PackageFile.FullPath)</Destination>
</PropertyGroup>
<MakeDir Directories="$(PackageDir)"/>
<Exec Command='"#(MSDeploy)" -verb:sync -source:iisApp="$(Source)" -dest:package="$(Destination)" '/>
</Target>
My MSBuild "Deploy" Target
(scrubbed for PWs, etc.)
<Target Name='Deploy' DependsOnTargets='Package'>
<PropertyGroup>
<Source>%(PackageFile.FullPath)</Source>
</PropertyGroup>
<Exec Command ='"#(MsDeploy)" -verb:sync -source:package="$(Source)" -dest:iisApp=PEApps,computerName=$(WebServerName),username=[User],password=[Password]'/>
</Target>
There was a lot to this question, I'm not sure if I'm fully on the same page as you but I'll summarize my impression of what you are asking. You have an existing web project which is in a solution with other projects. You need to be able to package the web project so that you can publish it to multiple destinations.
I have created a NuGet package which can be used for this exact purpose. It's called package-web. When you add it to your web project it will update the packaging process. When you create a package a few additional files will be included in the package, including all the web.config transform files. A .ps1 file will be created next to the package as well. You can use this script to publish the package. It will prompt you for which transform to run and for all the Web Deploy parameters. You can also save the responses to a file and then just pass them to the .ps1 file so that you can perform non-interactive publishes. I created a 5 minute video on it at http://nuget.org/packages/PackageWeb
package web: http://sedodream.com/2012/03/14/PackageWebUpdatedAndVideoBelow.aspx. FYI this is not yet working with VS 2012 but I'm working on the fix and should have it updated by the time VS 2012 is released.
If you don't find that useful you can see how I implemented the solution at https://github.com/sayedihashimi/package-web and you should see examples of everything that you need to do to roll your own.
FYI if you need to transform any files besides web.config on package create then you should take a look at my VS extension SlowCheetah. Here is a blog about how to integrate it into a build server.
I want to use the FTP task in ant, and I have found the appropriate jar files and got everything working fine. I have put the jar files in a "libs" directory alongside the other files used in the build. The only problem is that the user must run "ant -lib commons-net-ftp-2.0.jar" to make a build; I would really prefer that it were possible to just run "ant" with no arguments.
Reading the ant optional tasks intallation page, I see that there are five ways one can load up extra libraries in ant, and none of them are really what I'm looking for. I do not want to force the user to make any modifications to their system to run this task; it should be possible to just load it from the "libs" directory inside of our product's source folder. So that means setting the global CLASSPATH is also out (which is a bad idea anyways).
The last option, as noted in the documentation, is the preferred approach... loading the jarfiles individually from the build script itself. I have done this in the past with the ant-contrib tasks and JUnit, and would like to do that here, but I don't see how I can accomplish this. The FTP task doesn't support a nested classpath element, and I don't know the XML resource I would need to load this library via a taskdef. How can I load the libraries from within ant?
Edit: In response to the answers and questions which have been posted here so far, I'm using ant 1.7.1. Making an ftp taskdef definitely does not work; that throws the following error:
BUILD FAILED
/my/path/build.xml:13: taskdef class org.apache.tools.ant.taskdefs.optional.net.FTP cannot be found
Perhaps this is because the classname is wrong. How exactly do I find the classname I'm supposed to use if I only have a jarfile? It's not documented anywhere, and I couldn't find anything in the jar itself resembling that path.
The problem you are having is due to the different class-loaders in use. The Commons Net classes must be loaded by the same class-loader that loads the FTP task. Because the FTP task is loaded by Ant on start-up, you need to add the Commons Net to Ant's classpath so that it is loaded by the same class-loader. That's why the documentation gives you 4 different ways to do this.
I agree that none of them are ideal (the CLASSPATH environment variable being the worst). One way around this is to supply a shell script with your project that invokes Ant and passes the apporpriate -lib argument. You then get people to use this rather than invoking Ant directly. In fact, you could deviously name it 'ant' so that it gets run instead of the existing 'ant' on the path (this only works if the current directory is on the path, ahead of other directories).
The fifth option in the documentation is great in theory. They finally fixed the class-loading problems in 1.7.0. Unfortunately, as you mention, nobody retro-fitted the FTP task to take a classpath. You could try submitting an enhancement request, but this won't help in the short term.
There is one other option, which isn't any better than the others. Instead of making sure that the Commons Net classes are loaded by the class-loader that loads the FTP task, you could make sure that the FTP task is loaded by the class-loader that loads the Commons Net classes. To do this you have to remove the ant-commons-lib.jar file from the 'lib' directory of the Ant installation. This means that the FTP task won't get loaded on start-up. This is actually why the optional tasks are broken up into so many separate JARs - so that they can be individually removed. Put this JAR file alongside the Commons Net JAR file so that it can be loaded at the same time. Then you can do something like this (I tried this and it works):
<taskdef name="ftp"
classname="org.apache.tools.ant.taskdefs.optional.net.FTP">
<classpath>
<pathelement location="${basedir}/lib/ant-commons-net.jar"/>
<pathelement location="${basedir}/lib/commons-net-2.0.jar"/>
</classpath>
</taskdef>
<ftp server="yourserver.com"
userid="anonymous"
password="blah">
<fileset dir="somedirectory"/>
</ftp>
But this is probably a worse option than just using the -lib switch (with or without a wrapper script). The only other thing I can think of is to try to find a third-party FTP task to use instead of the default one.
I have a solution:
you can download a new "classloader" task from http://enitsys.sourceforge.net/ant-classloadertask/ and load it whith:
<taskdef resource="net/jtools/classloadertask/antlib.xml"
classpath="XXX/ant-classloadertask.jar"/>
Naw can do things like loading classes with the same classloader that ant use for his task:
<classloader loader="system" classpath="XXX/commons-net-2.0.jar"/>
or "loader="project""
Then you definde your task:
<taskdef name="ftp" classname="org.apache.tools.ant.taskdefs.optional.net.FTP"/>
and go :-)
So I succeeded in doing this for the ant-salesforce.jar that you get when trying to do salesforce work (fun...)
Check to see if the jar has an xml file in it that looks something like this:
<antlib>
<typedef name="compileAndTest" classname="com.salesforce.ant.CompileAndTest"/>
....
</antlib>
Then in ant give it a taskdev that reads that file from inside the given jar, like this:
<taskdef resource="com/salesforce/antlib.xml" classpath="lib/ant-salesforce.jar" />
Hope that helps some.
Ah, man, this is just so nasty. I run ant from eclipse. I don't want to reconfigure ant in eclipse for new workspaces, so here's what I decided to do, to decouple running the task and configuring ant. I extracted the ftp task to a separate build file. Next I added a native call to the command line to start a completely new ant process with the required libraries on the path:
<target name="deploy-ftp">
<exec command="ant">
<arg line="-buildfile ftp.xml deploy-ftp -lib lib/ant"/>
</exec>
</target>
Now the master build file can be run without any special arguments and no modifications are required to the ant installation. It's nasty though, since the ftp task runs in a completely clean environment. None of the properties and paths from the master build file are available. Luckily I had all of these in a separate property file anyway, so I only needed a single import.
I would like to add a big thanks to Dan Dyer. Without your extensive explanation of what's going on behind the scenes, I wouldn't have found this solution.
Will this work assuming libs is directly under you project's base directory
<taskdef name="ftp" classname="org.apache.tools.ant.taskdefs.optional.net.FTP">
<classpath>
<pathelement location="${basedir}\libs\commons-net-1.4.0.jar"/>
</classpath>
</taskdef>
Your users all have ant installed on their machines but you can't / don't want to make them add the FTP jar? Can you bundle ant with your project make tasks that call YOUR ant bundle, with the jars placed so it'll work as follows?
<taskdef name="ftp" classname="org.apache.tools.ant.taskdefs.optional.net.FTP">
<classpath>
<pathelement location="\lib\commons-net-1.4.0.jar"/>
</classpath>
</taskdef>
<target name="testFtp">
<ftp server="blah" userid="foo" password="bar">
<fileset file="test.file" />
</ftp>
</target>