We are constantly doing changes to the ETL program (IBM Datastage 8.5) and mostly in the same project folder.
There are so many versions of code which confuses the developer at the point of implementation.
What I am looking for is a GitHub thingy for datastage codes..
Can the experts please assist me...
Thanks!
Datastage is pretty terrible when it comes to version control.
Different organizations use different tools to try and get some amount of version control in place.
for example
1. they would store the latest dsx of the code in a server.
2. and from there on use SVN to check in and check out the code.
3. So if you need to work on a code you would first check if anyone has checked it out if not then proceed to extract the latest version of the dsx from SVN ; check out the dsx in SVN ;
4. import and make the changes ;
5. take the export of the changed code ;
6. check it back in at SVN
I know its not a solution really but probably your best bet in large developer environment.
We have InfoSphere Information Server Manager and its available with DataStage installation .
Use InfoSphere Information Server Manager to create deployment packages for
DataStage Components .
and get any good version control Tool to maintain versions .
Example - you can use eclipse SVN Version utility
To start this process , always get latest code from LIVE Env and make the changes in Test Env Or other Env.
before deployment -
use InfoSphere Information Server Manager to create backup package of related DataStage component in Prod and get the version number for that from SVN
create the package for modified jobs and send it to Live Env with out replacing the existing or running code .
get version number for this package from SVN .
Deployment -
Use InfoSphere Information Server Manager replace the existing code with new one .
Rollback the code to previous version if any problem with new deplyment
Currently I'm using the option to export jobs as dsx files do versionate them.
So, when I need to rollback to an previous version, just use the import objects option and select the exported dsx file.
Those following this thread might be interested to know that my organisation's solution for implementing DevOps approaches with DataStage development (called 'MettleCI') has now been announced on the Atlassian Marketplace:
https://marketplace.atlassian.com/plugins/com.datamigrators.atlassian.bitbucket.mettleci/
MettleCI Website
Example of DataStage job in Git
Related
I have just begun using TFS command line version. I love it. However, I need to know how to find an App Release that is associated/assigned to a given Branch (A) and (B) how to retrieve that App Release using TFS Command Line.
Is there a way to do this?
Thank you!!
I am lost on this one, after having spent a fair amount of time search the TFS Help files (tf vc help, etc.), Google, DuckDuckgo, etc.
tf vc help
I would like to see the App Release using TFS Command Line.
Just as Daniel suggested in the comment.
We recommend to use Rest API to fetch such kind of information.
You could either use some 3rd-party extension to Auto App Version.
Or you could link work items to a release or use Rest API to query release info in an 3rd-party/powershell task.
Kindly refer this blog-- Auto-generating release notes in Azure DevOps pipelines
tf.exe command which is used to run version control commands. Also have some limitations, it only work with TFVC not work with git.
We currently use TFS 2010 for controlling some of our .NET projects.
I'd like to know whether it's possible to provide some kind of versioning within TFS for a specific folder on a network share for example.
The idea being is to control release packages (zips) for every release we do. As this bit is done by some other person (not technical and he doesn't use Visual Studio or any tech tool), it'd be great to streamline the process of versioning the zip files for every release.
Technically speaking:
We've setup this folder \servername\releasezips and every time I drop a file named release.zip, it would automatically commit (push) this file to the TFS server (no comments needed) and furthermore, once the file contents change (meaning that someone dropped another version of the file in there), the system would again push the file to TFS, but with a following version.
Is this possible somehow? I've seen somewhere that I could programatically have some extra control over TFS, using REST API.
Thanks in advance!
Thanks for your tips, Daniel and PatrickLu-MSFT.
As I said, I did want something straighforward, some kind of control similar to what box.com provides, where you associate a local folder on your machine with the cloud. So, once you drop files in the local folder, the small box client synchronizes it along with the cloud. If access the box.com, I can see the different versions of that specific file.
So what I did was, I've created a small .NET app to monitor the folder and any new files dropped in there would get checked in to our TFS server, by using the tf command line (of course ignoring work items or comments).
So, progamatically within the .NET app, it builds up the tf required commands for processing and versioning the recently dropped files and therefore, executes them.
By the way, I could have used the Team Foundation API in order to do the same job, but it would demand way more effort.
Cheers
According to your description, you want to commit/check in files during the build/release pipeline.
It's not a recommend way to check-in generated build files and modify source code during a build pipeline. If you really want to do this, you could edit the build workspace files and use tf commands in custom activity and call the powershell eq:
cd $env:BUILD_SOURCESDIRECTORY
$TFFile = Get-Item "C:\Program Files (x86)\Microsoft Visual Studio 1x.0\Common7\IDE\TF.exe"
$tfOutput = [string]( & $TFFile.FullName checkin /noprompt /override:"***NO_CI*** New version is $newVersion." /comment:"***NO_CI*** New version is $newVersion." 2>&1)
Another way is installing TFS Power Tool and use the Windows PowerShell Cmdlets to check in the files. Refer to this link for more details: PowerShell and TFS: The Basics and Beyond
For version the dropped files, you could take a look this similar question: TFS Build Copy to Versioned Folder
Basically, you have to customize build definition with custom activity and based on build.buildnumber variable to generate/create .zip file.
Technologies:
Proget – Nuget Package management server
TFS – On premise 2017 Update 1
Issue:
When re-releasing a build from TFS release, to re-package a CI Nuget package that has already gone to my Proget development feed, there does not appear to be a way to get automatic Semantic versioning.
The help dialog that appears in regards to setting the version within the Nuget packager setup is as follows.
Use Date and Time If you choose 'Use the date and time', this will
generate a SemVer-compliant version formatted as X.Y.Z-ci-datetime
where you choose X, Y, and Z.
Use an Environment Variable If you choose 'Use an environment
variable', you must select an environment variable and ensure it
contains the version number you want to use.
Use the Build Number If you choose 'Use the build number', this will
use the build number to version you package. Note: Under General set
the build format to be
'$(BuildDefinitionName)_$(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r)
I’d like to be able to re-release a Nuget package that has gone from my CI build in TFS to my Proget development feed, over into my production Proget feed. Microsoft has a great article on Versioning NuGet packages in a continuous delivery world. In that article they elude to the fact that they are doing something similar, but they aren’t providing any real direction for how it was accomplished.
Question:
How would you configure the Nuget packager so that upon creating the package you would input a build variable? Or is there a way that you could set the major version and just have the minor increment each time? How are others handling the promotion of packages from development to production?
Have tried the following:
Tried $(Version) as a build & release variable, and it doesn’t seem to work. The package gets tagged with the date. Also, this only seems to be really functional in the Build portion of TFS where the modal window contains a spot to modify this value.
Tried using the date & time method, and it sticks CI into the build number. This is almost exactly what we want minus the CI definition. Because it automatically inserts CI, this is not suitable for production.
Turned it off and it pulls the version from the Nuspec, but then this would assume that in your CI build you are always upping the version number to one more than current after you have pushed your last release version. This is because the nuspec is in the build files that you are re-releasing through the TFS release chain. Confusing to say the least.
Use the build number set to $(BuildDefinitionName)$(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r) What I’d like here is $(Major).$(Minor).$(Patch). Trying $(Version)$ with a version of 1.0.0 gets you a file named with that has 2017.11.3.1 as the output, seemingly ignoring the $(Version) variable.
Not sure if I totally got your point, seems you would like to create a semantically versioned nupkg after ci process on TFS.
Usually the nupkg should be as shown MSVersioningSample: 1.0.8-ci-20171106-156033.nupkg
However you would like to rename nupgk and republish it to nuget server as the release version simply MSVersioningSample: 1.0.8.nupkg The same as $(Major).$(Minor).$(Patch).
You need to edit NuGetPackager.ps1 in the build agent, change the $VersionRegex value, details you could have a look at the answer in this question: How do I get TFS 2015 to parse 3 digit versioning for NuGet packaging
Also give a try with some 3-rd party extension to handle with Semantic Versioning in TFS build, release task, nuget package, a sample for your reference: Semantic Versioning Build and Release Tasks
Besides just a note: Semantic Versioning 2.0.0 is only supports with NuGet 4.3.0+ and Visual Studio 2017 version 15.3+.
We are using TFS on Visual Studio 2013. When our developers Get Latest Version (GLV) and there is a new db script file received, it is stored under a specific folder, to be run using our custom update app.
What I want is that upon doing a GLV, they get a notification (in Visual Studio) that there are new scripts to run to update the db (generically speaking, that a new file has been added under a certain path).
Is there a way to achieve this with TFS?
It will not give you exactly what you want, but you can use built in TFS alerts to notify you or the team when a file is checkedin under a folder with a specific name/specified path/file extension.
You could write a visual studio extension, to be triggered on Get Latest, which would check a certain path within source control, you would have to roll this to all of your developers and would either have to store the lookup paths centrally or redeploy the app if the look up paths changed.
Alternatively you could add a bat / powershell script to your source control. within this script you could do the get latest and also run any scripts that you would like to run. you would then get the developers to get the latest against this script and then run it, which would get the rest of the files and would also run the db scripts.
If I understand correctly, you want your users to be running their locally built solutions against the latest database version to keep everyone in sync. Why not just use the usual workflow to procure 'notifications' in the form of build output?
I've dealt with this in the past, and the best solution I've come up with is to write a custom MSBuild target for 'BeforeBuild' into each of the projects that rely on the database being updated. The MSBuild target checks the version of the database installed (You would have to come up with a means for doing so, it can be tricky!).
If the currently deployed database does not match the version in the scripts you have just synced to, a build event could be raised. When the versions match the target would output a success message (or maybe nothing at all), and if the versions do not match the target would issue a build Warning or build Error depending on the severity of change observed (it might depend for you insofar as Major/Minor version variance is considered).
Can I import the changes b/w the local version and the checkin version of the code.I am using TFS 2010.The reason I am asking this question is that ,I want send my code to an external reviewer who will not have access to TFS source control .
My current thinking is that if can I some import the changes then reviewer can review the changes through windiff or beyond compare tool.
Why not just:
Get latest
Copy it to a directory called <<filename>>-new.cs
Get specific version
Copy it to the same directory call it <<filename>>-old.cs
Send both files to the reviewer and have him use windiff (or whatever) to review?
If you want to import only the changed code, you can first check out all the files, then copy the files the reviewer send to you over the local files.
Then use the tfpt uu command from the TFS Power Tools to undo all unchanged files. Now you only have the list of files that are modified.
For a (very) different approach you could use git with TFS and push updates to your reviewers remote repository using git. You can then just pull their changes when you're ready and push them to TFS when you've looked at them.
For an idea on how this might be done have a look at http://www.richard-banks.org/2010/04/git-tfs-working-together-version-2.html