Is there a way to start up two in-memory databases with grails? Specifically, I'd like to integration test my ETL process and allow reporting to be runnable in both development and test environments.
You should be able to do this with the Datasources Plugin
Related
just wanted to know, if the Aqueduct ORM supports a simple in memory database, for testing purposes. Looking for something easy and lightweight to write the backend, before actually connecting it to postgres.
I've used similar approach with H2 and Postgres with Java, but it is rather error-prone: While the SQL interface may be similar, you could be using a feature that is available in one, but not the other. Eventually, either your development is blocked, or it is OK, but then the real deployment will be hitting issues.
I've found that starting a Postgresql instance in a docker is much easier than I've first thought, and now I use the same principle for most external dependencies: run them inside docker. If there is an interest, I can open source a Dart package that starts the docker container and waits until a certain string pattern is present on the output (e.g. a report on successful start).
Aqueduct was built to be tested with a locally running instance of PostgreSQL. This avoids the class of errors that occur when using a different database engine in tests vs. deployment. It is a very important feature of Aqueduct.
The tl;dr is that you can use a local instance of PostgreSQL with the same efficiency as an in-memory database and there is documentation on the one-time setup process.
The Details
Aqueduct creates an intermediate representation of your data model at startup by reflecting on your application code. This representation drives database migrations, serialization, runtime reflection, and can even be exported as JSON to create data modeling tools on top of Aqueduct.
At the beginning of each test suite, your test harness uses this representation to generate temporary tables in a local database named dart_test. Temporary tables are destroyed as soon as the database connection is lost; which you can configure to happen between tests, groups of tests, or entire test suites depending on your needs. It turns out that this is very fast - on the order of milliseconds.
CI platforms like TravisCI and Appveyor both support local PostgreSQL processes. See this script and this travis config for an example.
I'm working asp.net web based application, I have deployed this application on server, Its getting response on port 80 from a outside client.
I want the to fix the bugs so I want to run this application in Debug mode so that I can attach the worker process with the application and this is making the Performance down and its disturbing the QA team.
So can I have two application one can run in release mode so that QA activity does not get disturbed and parallelly I can debug the build and fix the bugs or can do further development.
I'm facing the same problem during the development activity, If multiple developers are working paralley , only one is able to debug the application other one has to wait.
So please suggest me, If I can get rid of this situation.
I have only one server on which I can test this application.
This is a way too long discussion, but I will try to offer you a few ideas:
Each developer should develop on his own machine (sources and database should be local).
In order to sync your work you should use:
a. a source control solution like TFS or SVN (this is free) for your sources.
b. database changes can easily be synced by generating update scripts using SQL Schema Compare directly from Visual Studio (you will need SQL Server Data Tools for this), Redgate SQL Compare or another application that can compare the database strucure (there are many available online, some of them free).
You should have a separate server (DB and app) to testing.
You should have a separate server (DB anb app) for production.
You say you have one server to test the application. But I suppose each developer has his own computer, right? In this case you need to skip #3 and use the same server for testing and production, but with different databases and applications.
I suggest you check this website for similar answers (see Best practice for test and production environments for example) to find the best solution that applies in your case.
I'm sure someone has come across this scenario before. We're rolling out a product where each customer has a separate copy of a database and each customer requires quartz.net jobs.
Are there any recommendations on how to configure quartz to run against each copy of the database?
You'll have to run separate instances for each database. Each scheduler runs against a single database instance.
I keep reading about how important nightly builds (and automated builds in general) are to the development process, and I would love to implement them on my projects. The problem is, I develop MVC web applications. I cannot imagine how I would be able to easily run an arbitrary build. I would need to deploy it to a server including any dll, view page, and DB changes necessary, and possibly seed the DB with appropriate test data. That does not sound automatic or even useful.
Is there a useful way to do nightly builds in a web application? The best solution would be one where I can access the application at an address like nightlybuild.myapplication.com/buildnumber and the server would figure out what DLL, view pages, and DB to use.
I want to implement the following:
Web application in Grails going to MongoDB database
Long-running batch processes populating and updating that database in the background
I would like for both of them to reuse the same Grails services and same GORM domain classes (using mongodb plugin for Grails).
For the Web application everything should work fine, including the dynamic GORM finder methods.
But I cannot figure out how to implement the batch processes.
a. If I implement them as Grails service methods, their long-running nature will be a problem. Even wrapping them in some async executors will unnecessarily complicate everything, as I'd like them each to be a separate Java process so they can be monitored and stopped easily and separately.
b. If I implement them as src/groovy scripts and try to launch from command line, I cannot inject the Grails services properly (ApplicationHolder method throws NPE) or get the GORM finder methods to work. The standalone GORM guides all have Hibernate in mind and overall it seems not the right route to pursue.
c. I considered the 'batch-launcher' Grails plugin but it failed to install and seems a bit abandoned.
d. I considered the 'run-script' Grails command to run the scripts from src/groovy and it seems it might actually work in development, but seems not the right thing to do in production.
I cannot be the only person with such a problem - so how is it generally solved?
How do people run standalone scripts sharing the code base and DB with their Grails applications?
Since you want the jobs processing to be in a separate JVM from your front-end application, the easiest way to do that is to have two instances of Grails running, one for the front-end that serves web requests, and the other to deal with job processing.
Thankfully, the rich ecosystem of plugins for Grails makes this sort of thing quite easy, though perhaps not the most efficient, since running an entire Grails application just for processing is a bit overkill.
The way I tend to go about it is to write my application as one app, with services that take care of the job processing. These services are tied to the RabbitMQ plugin, so the general flow is that the web requests (or quartz scheduled jobs) put jobs into a work queue, and then the worker services take care of processing them.
The advantage with this is that, since it's one application, I have full access to all of the domain objects, etc., and I can leverage the dissconnected nature of a message queue to scale out my front- and back-ends seperately without needing more than one application. Instead, I can just install the same application multiple times and configure the number of threads dedicated to processing jobs and/or the queues that the job processors are looking at.
So, with this setup, for development, I will usually just set the number of job processing threads to whatever makes sense for the development work I'm doing, and then just a simple grails run-app, and I have a fully functional system (assuming I have a RabbitMQ server running to use as well).
Then, when I go to deploy into production, I deploy 2 instances of the application, one for the front-end work and the other for the back-end work. I just configure the front-end instance to have 1 or 0 threads for processing jobs, and the back-end instance I give many more threads. This lets me update either portion as needed or spin up more instances if I need to scale one part or the other.
I'm sure there are other ways to do this, but I've found this to be both really easy to develop (since it's all one application), and also really easy to deploy, scale, and maintain.