I am completely new to vNext Tasks & need some help. I have some Custom Configuration values defined in my Solutions, as follows:
DEV
DEV.HA (HA stands for High Availability)
QA
QA.HA
UAT
UAT.HA
PROD
PROD.HA
I have Service Account certificates in a set of folders separated by-environment. These need to be copied into a KEYS folder to replace the default files there.
FOR INSTANCE:
DEV & DEV.HA would get from one source-folder
QA & QA.HA would get from a different source-folder
...and so on.
How do you do conditional copying in vNext?
Which vNext task is best for that?
Either stick the "variable part" in a variable. This would be possible if you have naming logic that allows you to do configuration by convention.
Or put the logic in a powershell script or shell script and run that to set the values of the variables. There are special logging commands you can use to set the value of a variable from your script:
write-host ##vso[task.setvariable variable=YourVariableName;]QA
Then use the variable either in the standard Copy Files task or stick the copy files logic in Powershell as well.
Related
I am currently doing a POC on Jenkins pipeline to figure out how to configure my product in a CI environment. The requirements of the pipeline are:
Checkout code from SVN
Compile the program
Deploy to a predefined location on the server
Change DB configurations (& maybe even other configs not identified yet) to point to the appropriate DB
Execute the program
Execute QA process to validate the output
I am currently having difficulty in achieving Point 4 above. All DB-related configurations reside in a database.xml file per program & a program can connect to 1 or more DBs.
Given that developers are free to check-in any DB configurations, I would still like my CI environment to point to a predefined DB to test against. I am unsure on how to dynamically change these configuration files to achieve this.
Please let me know if there are standard methods that others are also using to achieve the same.
TIA
Some approaches:
Properties using Advanced Platforms
Use some web platform like :
zookeeper
http://www.therore.net/java/2015/05/03/distributed-configuration-with-zookeeper-curator-and-spring-cloud-config.html
Spring Cloud
https://www.baeldung.com/spring-cloud-configuration
This is a java spring framework functionality in wich you can create properties file with configurations and configure your applications to read them.
magi-properties-management
This is a java web system in which you can create environments and any key:value in each one. You just need configure your application in any language to read this values.
cyber-properties-management
This is a nodejs application that allows you to store properties files (.properties .yml or .json) and then just consume them as rest endpoint from your applications.
With this approaches , when a change of configurations is required, you just need update the value in the system and restart your application. It is even possible a hot reload in java applications.
Properties from Environment variables
You can export your key:value properties as environment vars before starting the application :
export DATABASE_HOST=10.100.200.300
export LOG_DIR_LOCATION=/logs
And read it after after the application has started:
Java >> System.getEnv("DATABASE_HOST");
node.js >> process.evn.LOG_DIR_LOCATION
php >> getenv('DATABASE_HOST')
Properties from SCM
Create some svn repositoty called development-configurations
Upload your database.xml with development values
In your application, put a database.xml with dummy values : localhost, etc
Create a jenkins job and put the environment as an argument.
In the same job download svn source code of your application.
download svn repository called $environment-configurations. $environment will be your argument
replace the database.xml inside of your application with database.xml of $environment-configurations repository.
Just create another repositories for testing, uat and production. Job must be receive environment as an argument to choose the right database.xml
Properties from Database
Modify your applications to read configurations from some database instead of xml file
Properties from File System
Modify your application to read an external database.xml instead of the database.xml inside of your source code. With this approach you just need put the database.xml in some path of your server and delete it from your application source code.
Note
You can use these approaches not only for backend apps. You can use them for frontends applications:
Devops Variable Substitution for Frontend js applications
I'm setting up some TFS build definitions. I see that once I run a build, the work is done within a subfolder of a subdirectory of /Agent/_work. It appears that the subfolder is the same between different executions of the same build definition, but I see no way to know in advance where I should expect TFS to set its working directory, nor do I see any way of specifying the name of the directory manually.
How does TFS determine what directory to use for the output from a build?
Refer to the documentation on build variables. The base folder can be accessed via the $(Agent.BuildDirectory) variable. The source folder is $(Build.SourcesDirectory).
You can't set this. It's determined by the agent's configuration.
We have a version number which is retrieved from a file during the build and needs to be passed to the deployment script. What is the best way to do this in TFS 2017?
Right now, we are thinking of embedding this version number in the artifact file name and parsing it during deployment, which feels a bit clumsy.
BTW, is there a way to get the artifact file name easily? Looks like TFS already knows this as shown in the log below:
No path specified for search pattern: *.zip defaulting to: F:\TFS2017_Release_Agent\_work\r1\a
2017-12-08T16:38:36.8519067Z Searching for: *.zip under directory: F:\TFS2017_Release_Agent\_work\r1\a
2017-12-08T16:38:36.8519067Z Found: 1 files to extract:
2017-12-08T16:38:36.8519067Z F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip
2017-12-08T16:38:36.8519067Z Creating destination folder: F:\TFS2017_Release_Agent\_work\r1\a\48
2017-12-08T16:38:36.8519067Z Extracting file: F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip
2017-12-08T16:38:36.8519067Z [command]F:\TFS2017_Release_Agent\_work\_tasks\ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988\1.112.1\7zip\7z.exe x -oF:\TFS2017_Release_Agent\_work\r1\a\48 F:\TFS2017_Release_Agent\_work\r1\a\ZFJ0_ServiceSearchPlusBuildDefn\BETALink.Service.SearchPlus\ZFJ0_BETALink10.4 DevOps_BETALink.Service.SearchPlus_1.0.0_886.zip
2017-12-08T16:38:36.8987827Z
First, custom build variable couldn't be used directly in your release definition with TFS 2017.
There had also been a related user voice:
Project level build/release variables
https://visualstudio.uservoice.com/forums/330519-visual-studio-team-services/suggestions/14515326-project-level-build-release-variables#
As a workaround, you could use some 3-rd party extension such as Variable (de|re)Hydration Tasks that help you make use of your build variables in your Release workflows.
Variable Dehydration Task - During your build you can save the variables to a json file stored with your other build assets
Variable Rehydration Task - During your Release you can load the saved variables and gain access to them.
It does this by prefixes, so you can save/restore all default build variables that are prefixed with 'BUILD-', but if you prefix your own variables with i.e. 'ABC_' you can also save them from your build and restore them in your release.
Another way is do it through powershell scripts, how to please refer: TFS 2015 Release management access build variables
About build artifact name, there is not a related predefined system variable, it's just entered as an argument in Publish Build Artifacts task.
I am converting our CI platform from CruiseControl to Jenkins, and can't seem to figure something out that seems like it should be relatively simple to do (Disclaimer - I'm no CI or build automation expert, but this was dumped into my lap and I find it interesting)
In CruiseControl, I am able to declare variables like this:
<cb:define rootdir="J:\SOURCES\" />
<cb:define logdir="J:\SOURCES\buildlogs" />
<cb:define iisdir="J:\IIS\" />
<cb:define artifacts="artifacts\" />
Then use them as part of an MSBuild task
<msbuild>
<executable>C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe</executable>
<workingDirectory>$(rootdir)$(ProjectName)</workingDirectory>
<projectFile>$(ProjectName).sln</projectFile>
<buildArgs>/p:BuildDate="1";OutDir="$(iisdir)$(ProjectName)\bin\\";WebProjectOutputDir="$(iisdir)$(ProjectName)\\"</buildArgs>
<targets>Rebuild;$(ProjectName)</targets>
<timeout>180</timeout>
<logger>C:\Program Files (x86)\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MsBuild.dll</logger>
</msbuild>
If the root or IIS directories change, it can easily be applied to all projects at once. We have ~60 projects setup, so doing this project by project would be very time consuming. Migrating this to Jenkins, the MSBuild command line arguments now look like this (partial sample but includes what is relevant):
OutDir="J:\IIS\ProjectName\bin\\";WebProjectOutputDir="J:\IIS\ProjectName\\"
The IIS directory is hard coded. I need that to be something more like this:
OutDir="${IIS_DIR}\ProjectName\bin\\";WebProjectOutputDir="${ITEM_ROOTDIR}\ProjectName\\"
Is there a way to do that? I tried the configuration slicing plugin, which is useful, but doesn't fit this need from what I see
You can do this with built-in Jenkins' functionality:
Then you need to expand your variable. This, actually, depends on where you would use it.
For example: %MSBuild% and %IIS_DIR% for "Execute windows batch command" build step. Other build steps (and plugins) may use it differently.
For global variables, you need EnvInject plugin. This allows you (among other things) to setup variables at the Global (node) level, at job level or as a step. You can set variables directly, or from properties file, or from scripts.
Once set, the variables are available as environment variables to the rest of Jenkins and its steps (within scope).
For passing arguments to MSBuild, when you configure an MSBuild step, there is an option to pass "Command line arguments" in the format /p:Param=Value.
The "value" could be an environment variable. On Windows environment you would reference it as %myvar%
So, if you configure a global GLOBAL_IIS_DIR=C:\path\to\IIS using EnvInject, you can then reference it on command line with /p:IIS_DIR=%GLOBAL_IIS_DIR%
I'm trying to create a generic Jenkins job for deploying different projects from different GIT repos and branches to different application servers (in any combination).
I have 2 string build parameters for the repo and for the branch, and a small shell script in a pre-build step which based on the build params creates a deploy.properties file with properties URL and PROFILE.
Another pre-build step is the Inject environment variables which uses the deploy.properties file previously created.
I'm to use the URL property in the Jenkins Deploy Plugin in the following way: Tomcat URL field - $URL.
Also, in the build section, I'm using the PROFILE property: clean install -P$PROFILE .
The problem is that the placeholders or not replaced by the values I've set in the shell script. Not that is I do another post-build action and I'm echoing the same placeholders, the values are replaced and it seems to work. Other check I've done is the Environment Variables section from a given Build and the variable values are there, so the injection works.
Any ideas?
Try the below and make sure you don't have the cmd in single quotes or anything.
clean install -P${PROFILE}