Our application is using continuous integration with Jenkins. We have problem at our hand in deploying incremental db changes to oracle server.
Current mechanism followed is having rollback scripts and alter or incremental scripts (both ddl and dml).
In jenkins pipeline, we are calling rollback first and then incremental changes every time when build runs along with our java code changes. This is not ideal way to solve this problem.
I am looking for some best practices which will allow incremental db scripts to run only once.
I mentioned that best practice before, and it is not tied to JenkinsCI: you need to record the execution of your db script in a dedicated table of your database.
That is what a product like Flyway does, but you can implement that "record" part yourself too. That way, when your JenkinsCI pipeline job re-execute those scripts (through a wrapper of yours), said wrapper will detect they were already executed.
Related
What are the persistence options for fitnesse files? So far it seems like a file system is the only thing supported. There does appear to be an out of date database plugin. Is there anything else that is supported (S3, database, etc.)? Is there a way to control where files are persisted if using the filesystem?
I believe there is very little in that area. The location of the files can be controlled using a command line option. See http://fitnesse.org/FitNesse.FullReferenceGuide.UserGuide.QuickReferenceGuide#FitNesseCommandLINE
-d /path/to/fitnesse/root
How I've used the FitNesse wiki is as a local development tool, with the pages on the file system. Once I'm satisfied with the tests I commit them to version control (e.g. git) so that they become part of the (integration) test pipeline setup (e.g. they are run as part of the CI/CD pipeline of the project).
There is a plugin I believe that will automatically commit any save actions to Git, but I've never used that. Saving each edit action just pollutes version control in my opinion. I only want to see tests after they have been checked/completed, and that tends not to be each save.
Working on a shared wiki environment (where I would expect a non-file system approach would fit in) you run into the same problem, I expect. Developing automated tests is a development task that requires some iterations before it is 'done', and not all attempts reach that 'done' state. So using shared storage for wiki persistence creates 'noise' in the test-set: which are the tests that form the current reference set that should pass and what is work in-progress.
If you are working on a larger project where new features are developed together with their automated tests it becomes even more important to know which test changes belong to which features/changes. Having tests on the file system, in version control, allows you to develop test in sync with code changes in the same branch. This is what I would recommend.
We use visual svn for version control. I have few cloud web servers where my websites are running.
I would like to create some repositories for the websites content. I checkout them in local editors (notepad ++), edit them and checkin to SVN. But when check-in to visualSVN, I would like them to get deployed to the webservers docroot. In some cases I would like to restart the webserver too.
Is it possible using Jenkins+deployment plugins. I am very new to jenkins, can somebody help me with some information how we can achieve this.
It is one of the scenarios Jenkins is designed for (Continuous Delivery, aka. CD). Your perfect plan might look like this:
Get a new instance of Jenkins up/running (for experiments) (if you're familiar with Docker it is one of the best ways to experiment with Jenkins);
Configure Subversion Plugin in Jenkins (integration with SVN);
Setup your first FreeStyle job in Jenkins that polls your Visual SVN server for changes (things you check-in to SVN) and learn how that works (* * * * * <~-
this polls changes from your source control an every minute, great for experiments);
Setup your second FreeStyle job that connects to one of your webservers (probably via SSH) and creates a file (simple "touch hello_world.log" is great to start with) in a special folder dedicated for that kind of tests (DO NOT MESS WITH YOUR PRODUCTUION CONTENT FOLDER(s));
Setup your third FreeStyle job that combines your experiences acquired in #1 and #2, and still writes to a test folder;
Compare results of the job output with your production deployment expectations (eq. files in place, content is processed the right way, configuration files are looking good and etc.);
Try it out on one of the production web servers, one folder/site at a time;
Apply your newly crafted delivery pipeline to the rest of servers/sites;
Learn how backup your Jenkins instance and actually make your first backup;
Try to restore your Jenkins instance from the backup made in the previous step;
Decide whether it is okay for you to maintain your own Jenkins instance or you will be better off with a hosted version of it (CloudBees Inc.);
Learn more about Pipeline in Jenkins and possibly (because it is not immediately obvious) migrate your FreeStyle job(s) to Pipeline DSL
and/or Jenkinsfile;
At times you might need to get back to "Get Started with Jenkins" manual and look up for the ideas or answers, it is okay - do not give up and feel free to post your questions here, at SO.
Hope these ideas will help you to get started.
I recently managed to convert several manually-created jobs to DSL scripts (inlined into temporary 'seed' jobs), and was pleasantly surprised how straightforward it was. Now I'd like to get rid of the multiple seed jobs and try to structure things more cleanly.
To that end, I created a new jenkins-ci repo and committed all the Groovy DSL scripts to it. Then I created a job-generator Jenkins job that pulls from the jenkins-ci repo and has a single Process Job DSLs step. This step has the Look on Filesystem box ticked, with the DSL Scripts field set to jobs/*.groovy. With global push notifications already in place, this works more-or-less as intended: if I make a change to the jenkins-ci repo, the job-generator job automatically runs and regenerates all the jobs—awesome!
What I don't like about this solution is that it has poor locality of reference: the DSL scripts for the job live in a completely separate repository from the code. What I'd really like is to keep the job DSL scripts in each individual code repository, in a jenkins subfolder, and have a single seed job that processes them all. That way, changes to CI setup could be code-reviewed right alongside the code. To me, that just feels like an ideal setup.
Unfortunately, I don't have a clear idea about how to make this happen. If I could figure out a way to make the seed job watch multiple repos, such that a commit to any one of them would trigger it, perhaps I could inject another build step before the Process Job DSLs step and (somehow) script my way to victory, but... I'm unsure how to even get to that point. (I certainly don't want to do full clones of each repo in the generator job just to pull in the DSL scripts!)
I suspect I'm not the first person to wish they could put the Job DSL scripts alongside the code, though perhaps I'm over-estimating the benefits. Any advice on this topic would be much appreciated—thanks!
Unfortunately there is no direct way of solving this. Several feature requests have been opened (JENKINS-33275, JENKINS-37220), but AFAIK no one is working on any of them.
As a workaround you can use the Pipeline Multibranch Plugin and create a multibranch project for each of your repositories. You must then add a simple Jenkinsfile to each repo/branch and use the Jenkinsfile to execute your Job DSL scripts. See Use Job DSL in Pipeline scripts for details. This would require minimal coding, but I think each repo must be cloned for this to work because the Job DSL files must be available on the file system.
You can use Job DSL to create the multibranch jobs, see multibranchPipelineJob in the API viewer. This would be your "root" seed job.
If your repos are hosted on GitHub, you can also checkout the GitHub Organization Folder Plugin. With that plugin you must only create one job for each organization instead of multiple multibranch jobs.
I have a scenario that requires me to revert to a clean snapshot before or after each Coded UI test method is executed. I have researched using the TFS Lab Management API (see http://blogs.microsoft.co.il/shair/2011/12/22/tfs-api-part-42-getting-started-with-lab-management-api/) to revert to a specific snapshot as part of the TestInitialize and/or TestCleanup method, but I can only get this to work when executed locally. When executed on a remote machine I get errors authenticating to the TFS service.
My other option is to somehow do a 'foreach test in testrun' into the build process template (LabDefaultTemplate.11.xaml). I have identified the area that I think this would fit best, but cannot find any documentation on running a loop on each test.
Is this something that is possible, or is there somehow a built in method to accomplish this that I have overlooked?
To do what you propose you should switch to Release Management and create a separate test run for each of your groupings, in your case each test. You can use RM to orchestrate looping through each of your runs and executing then.
http://nakedalm.com/execute-tests-release-management-visual-studio-2013/
However running a UI test should not break your application and I would suggest that either your tests are way too long, or there is some flaw in the design of your application.
While using the DB Migration plugin I came across an interesting question. In our regular war deployments, time and again, we need to run certain scripts for data updates to accommodate our changed code. While we can still run these externally, we were trying to find a way to add them as a part of DB Migration process.
Now one set of these scripts can be converted into migration scripts and added inside the grailsChange section and and they run pretty seamlessly. There is another set of scripts though, which are problematic because of a couple of reasons.
These scripts are run time and again so we would have to keep changing the id with every run as we don't want to duplicate the code, thus losing the original changes.
We pass params to these scripts from the command line and by the method above we have to add them to the scripts themselves just causing maintainability issues.
So my question would be, is there a more elegant way to trigger external grails or groovy scripts from within the DB migration scripts such that every time we need to run a script file, we can create the changelog with the updated call and tag it with the app.
I think there was a post on stackoverflow regarding this a while back, but I cannot for the love of my life, find it any more. Any help regarding this would be appreciated.
Thanks
Are the scripts something you could add into bootstrap.groovy? That would probably be the simplest. Just use groovy.sql.Sql to run the scripts.
Another more functional and flexible option would be to create a service to run the scripts (groovy.sql.Sql) and a domain class to track the scripts that have been run. You could trigger the service in the bootstrap.groovy file and the service could look at some migrations domain class you set up to see if the script has been run. You could even go as far to secure a front end to this mechanism to upload a script file to execute at runtime.
Let me know more details of what you want and I can try to be more detailed in my response.