I was trying to deploy a Grails app(ver. 2.2.2) to Jenkins(ver. 1.638). It runs clean, test-app -Dgrails.env=XXX and war commands.
The clean and war worked great. For the test-app -Dgrails.env=XXX, the unit test part worked fine, every test case passed. However, most integration test cases failed. It seems that domain classes in the controller cannot get data. In the other words, the get() or list() methods didn't return correct records. I'm assuming the database is connected correctly because console output didn't complain the connection issue. The integration tests runs well on my local machine with the same datasource.
Test errors look like the following code, the size() is always 0. It is a controller integration test. The results is returned by get(), list() or createCriteria().list{} methods of domain classes in the controller.
results.size() == 9
| | |
[] 0 false
The following is the code in the DataSource.groovy. I have tried both testCI and testCI2. Both settings don't help. Honestly, I don't know much about the pool properties. I just copied it.
hibernate {
cache.use_second_level_cache = true
cache.use_query_cache = false
cache.region.factory_class = 'net.sf.ehcache.hibernate.EhCacheRegionFactory'
}
environments {
testCI {
dataSource {
dbCreate = "update" // one of 'create', 'create-drop', 'update', 'validate', ''
url = "jdbc:mysql://localhost:3306/build_db" // For non network testing
driverClassName = "com.mysql.jdbc.Driver"
username ="build"
password = "build"
pooled=true
properties {
maxActive = -1
minEvictableIdleTimeMillis=1800000
timeBetweenEvictionRunsMillis=1800000
numTestsPerEvictionRun=3
testOnBorrow=true
testWhileIdle=true
testOnReturn=true
validationQuery="SELECT 1"
}
}
}
testCI2 {
dataSource {
dbCreate = "update" // one of 'create', 'create-drop', 'update', 'validate', ''
url = "jdbc:mysql://localhost:3306/build_db" // For non network testing
driverClassName = "com.mysql.jdbc.Driver"
username ="build"
password = "build"
}
}
}
The most weirdest part is that the integration test sometimes works well after I push new commits to the git repository. It's not really often though.
Does anyone have any ideas about it? Thank you so much!!
IIRC, unit tests run on a memory-based GORM representation.
For integration tests, you'll need a real (from your example, MySQL) database.
Is MySQL installed on the Jenkins server -- that's the localhost when your integration tests are being run.
Related
I am using Grails 2.4.3 and the database-migration:1.4.0 plugin.
I have created a simple Domain class called Mod. I am able to create the groovy based changelog using dbm-generate-gorm-changelog changelog.groovy. This correctly generates the file. I then execute dbm-update which reports:
|Starting dbm-update for database sa # jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000
|Finished dbm-update
However, there are no table created in the database and running dvm-status returns:
|Starting dbm-status for database sa # jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE
1 change sets have not been applied to SA#jdbc:h2:mem:devDb
changelog.groovy::1413897188349-1::clarkrichey (generated)
|Finished dbm-status
My development environment configuration from DataSource.groovy is as follows:
development {
dataSource {
// dbCreate = "create-drop" // one of 'create', 'create-drop', 'update', 'validate', ''
url = "jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE"
}
}
The issue centers around the fact you are using an in-memory instance of h2. Thus, nothing is saved between application launches. If you want a persistent database then change your url to use a file based instance.
development {
dataSource {
dbCreate = "none" // one of 'create', 'create-drop', 'update', 'validate', ''
url = "jdbc:h2:file:/path/to/save/to/devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE"
}
}
It seems that Grails is trying to access my database when I first deploy it to my production tomcat server. I know this because I get the following error message in the stacktrace.log
invalid username/password; login denied
Now, I disabled database creation
dataSource {
pooled = false
driverClassName = "oracle.jdbc.OracleDriver"
dialect = org.hibernate.dialect.Oracle10gDialect
}
hibernate {
cache.use_second_level_cache = true
cache.use_query_cache = false
cache.region.factory_class = 'net.sf.ehcache.hibernate.EhCacheRegionFactory'
//Set jdbc metadata to false to not open a session
temp.use_jdbc_metadata_defaults = false
}
production {
dataSource {
dbCreate = "none"
url = "jdbc:oracle:thin:#1.1.1.1:1521:xe"
}
}
I don't supply the database password because we use database users for authentication and authorization (please don't criticize this decision, I know it's awful, but we have a legacy database). So the username/password is supplied when the user makes a request through the client. We used http://sergiosmind.wordpress.com/2013/03/14/grails-using-a-database-user-for-security-login/ to set this up.
It seems that the Grails app cannot start because of this. Why is Grails accessing the database? What is it trying to do?
Grails uses connections at startup to initialize GORM - there's one to detect the dialect, one to configure the LOB handler, and Hibernate connects to initialize its configuration also.
I discuss this in these two blog posts: http://burtbeckwith.com/blog/?p=312 and http://burtbeckwith.com/blog/?p=1565
I am trying to create a new domain object in the grails console with the help of this guide.
According to the console output the new object is created:
grails> shell
groovy:000> new foo.Book(title: 'bar').save(failOnError: true, flush: true)
groovy:000> foo.Book : 1
groovy:000> foo.Book.list()
groovy:000> [foo.Book : 1]
But this new book entity is not visible in the dbconsole
The table BOOK is present when I connect with the JDBC url for the dev environment as found in DataSource.groovy:
jdbc:h2:mem:devDb;MVCC=TRUE
username: sa
password: <blank>
but a select returns 0 rows
The relevant piece of DataSource.groovy config (the default)
dataSource {
pooled = true
driverClassName = "org.h2.Driver"
username = "sa"
password = ""
}
hibernate {
cache.use_second_level_cache = true
cache.use_query_cache = false
cache.region.factory_class = 'net.sf.ehcache.hibernate.EhCacheRegionFactory' // Hibernate 3
// cache.region.factory_class = 'org.hibernate.cache.ehcache.EhCacheRegionFactory' // Hibernate 4
}
// environment specific settings
environments {
development {
dataSource {
dbCreate = "create-drop" // one of 'create', 'create-drop', 'update', 'validate', ''
url = "jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000"
}
}
When the entity is created using the console, rather than the groovy shell, the issue remains.
I am using the newest grails build at this moment, which is 2.3.1
The embedded H2 database vrsion = H2 1.3.173 (2013-07-28)
I think the problem is that the database is getting locked. Let's try this one then (works on my experiment):
edit your grails-app/conf/spring/resources.groovy and make it looking like this:
// Place your Spring DSL code here
beans = {
h2Server(org.h2.tools.Server, "-tcp,-tcpPort,8043") { bean ->
bean.factoryMethod = "createTcpServer"
bean.initMethod = "start"
bean.destroyMethod = "stop"
}
}
Then, modify your grails-app/conf/DataSource.groovy to look like this:
test {
dataSource {
dbCreate = "update"
url = "jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000"
}
}
Now, you are ready to add some new objects as per the tutorial:
$ grails
grails> run-app
grails> shell
groovy:000> new test.Book(title: 'Book 1').save(failOnError: true)
===> test.Book : 1
groovy:000> new test.Book(title: 'Book 2').save(failOnError: true)
===> test.Book : 2
groovy:000> test.Book.list()
===> [test.Book : 1, test.Book : 2]
To view the H2 console, go to
http://localhost:8080/{project}/dbconsole
but select [Generic H2 Server] from the list and on the JDBC URL enter:
jdbc:h2:tcp://localhost:8043/mem:devDb
and connect. I hope that helps
======================
After a bit further experimentation, it appears that locking was your problem and you need to use a mixed mode approach when connecting to your H2. You can read more information here:
http://www.h2database.com/html/features.html#auto_mixed_mode
So, the simplest thing to do is use this jdbc connection URL:
url = "jdbc:h2:/tmp/myDB;MVCC=TRUE;LOCK_TIMEOUT=10000;AUTO_SERVER=TRUE"
for both your application and the H2 dbconsole (notice the AUTO_SERVER=TRUE) (no need to modify the spring bean)
I suggest to change the
dbCreate = "create-drop"
to
dbCreate = "update"
on your DataSource.groovy and try again
When I modified the spring bean as Nick suggested, I could not start run-app and at the same time start the grails console or shell. Here is the error I got:
Message: Error creating bean with name 'h2Server': Invocation of init method failed; nested exception is org.h2.jdbc.JdbcSQLException: Exception opening port "8043" (port may be in use), cause: "java.net.BindException: Address already in use" [90061-173]
The simple change to url worked, thanks Nick -:)
I have the following settings on a new grails project:
dataSource {
pooled = true
driverClassName = "com.mysql.jdbc.Driver"
dialect = "org.hibernate.dialect.MySQL5InnoDBDialect"
username = "sa"
password = ""
}
environments {
development {
dataSource {
dbCreate = "create-drop" // one of 'create', 'create-drop', 'update', 'validate', ''
url = "jdbc:mysql://localhost/myapp?useUnicode=yes&zeroDateTimeBehavior=convertToNull&characterEncoding=UTF-8"
username = "root"
password = ""
}
}
}
when I run my app it fails with an error: Error creating bean with name 'transactionManagerPostProcessor':
This error goes away when I manually go to my database and create a database called myapp
I thought the create-drop setting in dbCreate is suppose to create the db if it does not exist.
Question
How can I configure the settings so that the database gets created when it does not exist in the MySQL
Creating the database itself is impractical because it is very vendor-specific, even more so than the DDL to create tables, sequences, etc. You often need to specify access rules, storage options, etc.
Hibernate will generate and run the schama DDL but you have to start the process by creating the database itself except for simple databases like H2.
Grails makes it very easy to configure datasources for different environments (development, test, production) in its DataSources.groovy file, but there seems to be no facility for configuring multiple datasources in one environment. What to I do if I need to access several databases from the same Grails application?
Connecting different databases in different domain classes is very easy in Grails 2.x.x.
for example
development {
dataSource {//DEFAULT data source
.
.
}
dataSource_admin { //Convention is dataSource_name
url = "//db url"
driverClassName = "oracle.jdbc.driver.OracleDriver"
username = "test"
password = 'test123'
}
dataSource_users {
}
}
You can use any datasources in your domain classes by
class Role{
static mapping = {
datasource 'users'
}
}
class Product{
static mapping = {
datasource 'admin'
}
}
For more details look at this
If using Grails 2.0 or higher, there is no need for the plugin, it is supported natively.
http://www.grails.org/doc/latest/guide/single.html#multipleDatasources
There is now Grails plugin that enables the use of multiple datasources directly with Grails' GORM layer:
http://burtbeckwith.com/blog/?p=70
Grails 2.0 can handle multiple data sources without a plugin:
Example with a different datasource for the dev(h2 dataSource) and test(mysql dataSource_mysql) environments:
DataSource.groovy:
dataSource {
pooled = true
driverClassName = "org.h2.Driver"
username = "sa"
password = ""
}
dataSource_mysql {
dialect = org.hibernate.dialect.MySQLInnoDBDialect
driverClassName = 'com.mysql.jdbc.Driver'
username = "user"
password = "pass"
url = "jdbc:mysql://mysqldb.com/DBNAME"
}
hibernate {
cache.use_second_level_cache = true
cache.use_query_cache = false
cache.region.factory_class = 'net.sf.ehcache.hibernate.EhCacheRegionFactory'
}
// environment specific settings
environments {
development {
dataSource {
configClass = HibernateFilterDomainConfiguration.class
dbCreate = "update" // one of 'create', 'create-drop', 'update', 'validate', ''
url = "jdbc:h2:file:../devDb;MVCC=TRUE"
sqlLogging = true
}
}
test {
dataSource_mysql {
configClass = HibernateFilterDomainConfiguration.class
dbCreate = "create" // one of 'create', 'create-drop', 'update', 'validate', ''
sqlLogging = true
}
}
production {
dataSource {
dbCreate = "update"
url = "jdbc:h2:prodDb;MVCC=TRUE;LOCK_TIMEOUT=10000"
pooled = true
properties {
maxActive = -1
minEvictableIdleTimeMillis=1800000
timeBetweenEvictionRunsMillis=1800000
numTestsPerEvictionRun=3
testOnBorrow=true
testWhileIdle=true
testOnReturn=true
validationQuery="SELECT 1"
}
}
}
}
Do you really want to do this? In my experience, the usual scenario here is:
An application manages its own data in its own database schema
Often, the application will require data from other sources (for example, so reference data doesn't get copied and pasted)
I've normally always had the luxury of all the schemas residing on the one database instance. Therefore, my application:
only has one database connection - which is to the schema it owns and has read/write access
the other applications 'export' their data via views
my application has read access to those views, and has a synonym for that view making it appear local
The reason behind using views is so that the application that is exposing the data
knows explicitly that it is being exported and what is being exported
does not expose the internal structure of the schema (so if the internal structure changes, as long as the view is correct the consuming apps don't know)
I haven't actually had to do this with a Grails application, but the approach should work.
Another approach to sharing data across applications is to create a web service to expose the data. Grails makes this easy.
Hope that helps, but this approach may not be applicable for all situations.
The following post seems to be the best source of information on the subject:
How to get mutli-dataSource in grails
It boils down to:
Define datasource1 in DevelopmentDataSource
Define datasource2 in resources.xml
Write a DAO for CRUD of the domain objects using datasource2
In hibernate.cfg.xml, list all domain objects.
Only the first datasource will have dynamic finder methods.
If its a really simple query you are after and don't mind not having the ORM features you could use Groovy SQL or the native SQL features of Hibernate.