I'm facing an odd issue after migrating an app from Grails 2.5 to Grails 3.1 - database exceptions never bubble up to the controller.
The typical scenario looks like this: controller's delete action attempts to remove an entry, but a database-level constraint prevents the delete. This leads to an exception (see below) - in Grails 2.5, if uncaught, this would lead to "Error 500 internal server error". Now, we have a custom mapping for these, so even in case an uncaught error occurs, it is handled in a user-friendly way and the user knows something happened.
In Grails 3, this stopped working. The exception is logged, but would never bubble up to the controller. So in this specific case, our controller just goes on and returns "204" HTTP status code as if nothing happened, and the end user doesn't even know something went wrong.
I'm puzzled and I think this must be something stupid I'm missing, but can't figure it out. I have tested this Grails 2.5.5, 3.1.16, 3.2.12 and 3.3.2, but all tested 3.x releases behaved pretty much the same way. Also tested with both Hibernate 4 and 5, and tried H2 and PostgreSQL but I'm getting the same behavior regardless.
The easiest way to reproduce is to manually add a FK relationship (with delete no action) to the hibernate-generated database schema and attempt to delete via the controller (example #1 below) but the issue may also occur with some many-to-many relationships or other scenarios such as persisting a new entry (and failing at the database level).
UPDATE #1, April 25, 2018: To create a minimal reproducible test case using PostgreSQL as the database and Grails 3.3.x, just create a blank grails application via grails create-app testapp. Then adjust the application.yml to configure data source for your PostgreSQL instance, and use dbCreate: validate mode as the schema is provided below.
Add one minimal domain object:
class Book {
String name
}
And a controller to test object deletion, i.e. BookController:
class BookController {
#Transactional
def delete(Long id) {
Book.get(id).delete()
render "Book $id deleted"
}
}
The schema below creates 2 tables with a FK and a few entries. Book with id 1 will fail to be removed, books with id 2 and 3 will be removed successfully. Controller will return 200 OK in all 3 cases. You can test using the default UrlMappings via curl calls, i.e. curl http://localhost:8080/book/delete/1.
The database schema follows:
CREATE TABLE book (
id bigint NOT NULL,
version bigint NOT NULL,
name character varying(255) NOT NULL,
CONSTRAINT book_pkey PRIMARY KEY (id)
);
CREATE TABLE chapter (
id bigint NOT NULL,
book_id bigint NOT NULL,
name character varying(255),
CONSTRAINT chapter_pkey PRIMARY KEY (id),
CONSTRAINT fk_book FOREIGN KEY (book_id) REFERENCES book(id)
);
INSERT INTO book VALUES (1, 0, 'Book 1');
INSERT INTO book VALUES (2, 0, 'Book 2');
INSERT INTO book VALUES (3, 0, 'Book 3');
INSERT INTO chapter VALUES (1, 1, 'Book 3 Chapter');
CREATE SEQUENCE hibernate_sequence START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1;
SELECT pg_catalog.setval('hibernate_sequence', 4, true);
For the sake of completeness, here's the stack trace:
2018-04-25 16:23:22.462 ERROR --- [nio-8080-exec-1] o.h.engine.jdbc.spi.SqlExceptionHelper : ERROR: update or delete on table "book" violates foreign key constraint "fk_book" on table "chapter"
Detail: Key (id)=(1) is still referenced from table "chapter".
2018-04-25 16:23:22.473 ERROR --- [nio-8080-exec-1] org.hibernate.internal.SessionImpl : HHH000346: Error during managed flush [could not execute statement]
2018-04-25 16:23:22.720 ERROR --- [nio-8080-exec-1] o.g.web.errors.GrailsExceptionResolver : PSQLException occurred when processing request: [GET] /test/delete/1
ERROR: update or delete on table "book" violates foreign key constraint "fk_book" on table "chapter"
Detail: Key (id)=(1) is still referenced from table "chapter".. Stacktrace follows:
java.lang.reflect.InvocationTargetException: null
at org.grails.core.DefaultGrailsControllerClass$ReflectionInvoker.invoke(DefaultGrailsControllerClass.java:211)
at org.grails.core.DefaultGrailsControllerClass.invoke(DefaultGrailsControllerClass.java:188)
at org.grails.web.mapping.mvc.UrlMappingsInfoHandlerAdapter.handle(UrlMappingsInfoHandlerAdapter.groovy:90)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
at org.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:77)
at org.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:67)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [fk_book]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:241)
at org.springframework.orm.hibernate5.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:755)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:590)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:765)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:734)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:150)
at grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)
... 14 common frames omitted
Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:207)
at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:45)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3311)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3548)
at org.hibernate.action.internal.EntityDeleteAction.execute(EntityDeleteAction.java:98)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:586)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:460)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1295)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:468)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3135)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2352)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:491)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:147)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:231)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:65)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:582)
... 18 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "book" violates foreign key constraint "fk_book" on table "chapter"
Detail: Key (id)=(1) is still referenced from table "chapter".
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155)
at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:132)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:204)
... 36 common frames omitted
UPDATE #2, April 25, 2018: the fact that there is a database table that Hibernate isn't aware of was the point of the above example. It was an easy way to force a database-level exception that would only be logged, but wouldn't make it all the way up to the controller so on HTTP level it would appear as if no error occurred.
UPDATE #3, April 25, 2018: Another test case for Grails 3.3 that doesn't rely on any manual database schema "hacks" is below, with a many-to-many relationship between the same domain (User). It consists of 2 domain classes:
class User {
String name
static hasMany = [relationships: Relationship]
static mappedBy = [relationships: 'user']
static mapping = { table 'table_user' }
}
class Relationship implements Serializable {
User user
User related
static belongsTo = [ user: User, related: User ]
static mapping = {
table 'table_relationship'
id composite: ['user', 'related']
version false
}
}
And a minimal controller:
class TestController {
#Transactional
def delete(Long id) {
User.get(id).delete()
render "User $id deleted"
}
}
This can be reproduced with PostgreSQL or H2, just use the create-drop mode and the following BootStrap code to create 2 User entries:
def init = { servletContext ->
def user1 = new User(name: 'User 1').save()
new User(name: 'User 2', relationships: [new Relationship(related: user1)]).save()
}
Attempting to delete user 2 and then user 1 will work OK. Attempting to first remove user 1 will fail with an exception, yet the controller would still return "200 OK". Just try curl http://localhost/test/delete/1.
For convenience you can download the zipped archive with a runnable Grails 3.3.5 app here: https://www.dropbox.com/s/pycwuxm7r0wyxem/grails3_exception_issue_testapp.zip?dl=0
If you are going to use #Transactional you should use grails.gorm.transactions.Transactional instead of grails.transaction.Transactional.
Separate from that, the problem doesn't appear to be that the exception doesn't make its way up to the controller as much as what is really happening is the transaction isn't being committed until after you call render and the exception doesn't get thrown until the transaction is committed. Calling .delete(flush: true) will show different behavior but putting the transaction boundary in the controller is a bad idea anyway. A better plan is to put your database interactions in a transactional service.
I put your code in the project at https://github.com/jeffbrown/many33. The commit at https://github.com/jeffbrown/many33/commit/f8804c8793b399994c34043c1d340e4bf0d462cd shows a better way to organize this code and yields less surprising behavior.
I hope that helps.
Related
Does this mean we can not call some thing like this via Java API?
I get error - "Caused by: org.neo4j.graphdb.QueryExecutionException: Cannot perform schema updates in a transaction that has performed data updates."
This happens when I call schema update from a procedure call via neo4j console.
try (Transaction tx = db.beginTx()) {
String query = "CREATE INDEX ON :" + lbl + "(" + name + ")";
db.execute(query);
tx.success();
}
The Cypher query calling the procedure is already executed in a transaction, and there are no nested transactions in Neo4j: when you call db.beginTx(), you're getting the existing transaction, and it's not actually necessary unless you need the Transaction object (e.g. to create locks).
Anyway, even though it's not explicitly documented, it's apparently not possible to manipulate the schema from Neo4j procedures. You could say that it fails the use case of
To provide access to functionality that is not available in Cypher, such as manual indexes and schema introspection.
I created a test procedure similar to yours:
public class IndexProcedure {
#Context
public GraphDatabaseService db;
#Procedure
#PerformsWrites
public void index(#Name("label") String label, #Name("property") String property) {
db.schema().indexFor(Label.label(label)).on(property).create();
}
}
and ran it from the shell in the simplest Cypher query:
CALL my.package.index('Node', 'name');
Without the #PerformsWrite annotation, I get the following (expected) exception:
WARNING: Failed to invoke procedure my.package.index: Caused by: org.neo4j.graphdb.security.AuthorizationViolationException: Schema operations are not allowed for READ transactions.
With the annotation, I get the same exception as you:
WARNING: Failed to invoke procedure my.package.index: Caused by: org.neo4j.graphdb.QueryExecutionException: Cannot perform schema updates in a transaction that has performed data updates.
I guess the rationale is that setting up the schema is mostly a one-time operation that doesn't really need a procedure: if you're going to execute some Cypher query to call the procedure, you might as well run the script which creates the constraints and indices.
There could also be technical constraints: index creation is asynchronous and probably doesn't participate in the transaction (can you rollback the creation of an index?).
Or maybe it's just a bug? We should get someone from Neo to confirm.
Update: it will supposedly be fixed in Neo4j 3.1 when it's released, per a discussion on SlackHQ.
I'm using grails 1.3.7 together with Oracle 11g and trying to manage inner transactions.
I have a bean Person that is passed to a transactional (Propagation.REQUIRED) service method who makes some modification. Then it is passed to another transactional (propagation = Propagation.REQUIRES_NEW) method that makes some other modification and then throws an Exception.
What I expected to see is the rollback of all the modification of the second service but still valid those of the first one. This is the situation:
//outer transaction
class MyService {
def nestedService
#Transactional(propagation = Propagation.REQUIRED)
public void testRequiredWithError(Person person) {
person.name = 'Mark'
try {
nestedService.testRequiresNewWithError(person)
} catch (RuntimeException e) {
println person.age //this prints 15
println e
}
}
}//end MyService
//inner transaction
class NestedService{
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void testRequiresNewWithError(Person person) {
person.age = 15 //expected after Exception will be discarded
throw new RuntimeException("Rollback this transaction!")
}
}
then I run grails console and check on the DB after it ends.
...
def p = Person.get(671)
def myService = ctx.myService
println p.name //'John'...from DB
println p.age //25...from DB
myService .testRequiredWithError(p)
println p.name // 'Mark'....correct
println p.age // 15....UNEXPECTED..
//same result checking on the DB after console ends and transaction flushes
I tried to use Propagation.NESTED after activating it in the bootstrap by transactionManager.setNestedTransactionAllowed(true)and use savepoints like in this post
grails transaction set savepoint
but still got same result.
What am I missing????
Thank you in advance.
I'm a little late to the party but in case you haven't found your answer I think I know why you're having issues with transactions.
I looked at your link to the discussion about your issues with savepoints. According to that discussion you are using MySQL as your datasource. MySQL does not support transactions by default and in order to make it do so, you need to create your tables in a special way. I have provided a link below that explains how you should create your tables in MySQL when you want to use transactions:
http://www.tutorialspoint.com/mysql/mysql-transactions.htm
EDIT: This article suggests setting the type of your table to InnoDB in order to support transactions. Here is an example of what that looks like:
mysql> create table tcount_tbl
-> (
-> tutorial_author varchar(40) NOT NULL,
-> tutorial_count INT
-> ) TYPE=InnoDB;
It might be worth noting that this is not the only type that supports transaction but is the most common. MySQL 5.5 and greater should create tables as type InnoDB automatically.
Hope this helps!
I'd test for p.isAttached() as transaction rollback detaches the domain object from the Hibernate session. Also in the test I'd reload the domain object from database - effectively p = Person.get(671) again to reload data from database.
I think the reason the age property is set to 15 after the test is that after the exception the domain object and the database are out of sync (and the domain object is detached).
For more see: https://weblogs.java.net/blog/blog/davidvc/archive2007/04/jpa_and_rollbac.html
I am trying to follow the Grails application from the book Grails 2 - A Quick start guide. The grails version that i am using is 2.4.4 on Ubuntu 14.04 with the open jdk 7
I am getting the following error
Error 2015-03-09 20:05:02,117 [localhost-startStop-1] ERROR hbm2ddl.SchemaExport - HHH000389: Unsuccessful: alter table tek_event drop constraint FK_1xbf5b7edlnmgmrc90jhbyvg7 if exists
| Error 2015-03-09 20:05:02,118 [localhost-startStop-1] ERROR hbm2ddl.SchemaExport - Table "TEK_EVENT" not found; SQL statement:
There is nothing in my application except two domain classes and scaffolded controllers ...
Here are the domain classes TekUser and TekEvent
class TekEvent {
String city
String name
TekUser organizingUser
String venue
Date startDate
Date endDate
String description
String toString(){
"$name, $city"
}
static constraints = {
name()
city()
description maxSize:5000
organizingUser()
venue()
startDate ()
endDate()
}
}
And the TekUser Domain class
class TekUser {
String fullName
String userName
String password
String email
String website
String bio
String toString(){
fullName
}
static constraints = {
fullName()
userName()
email()
website()
bio maxSize:5000
}
}
The controllers for them are pretty barebones
class TekEventController {
def scaffold = TekEvent;
}
class TekUserController {
def scaffold = TekUser;
}
I am not able to follow what is going wrong here ...or is it a benign error that I could just ignore.
Also this started happening when I changed the datatype of organizingUser in the TekEvent class from String to TekUser
These are ignorable errors - there's nothing bad happening. Unfortunately the Hibernate developers changed the logging strategy in Hibernate 4, and that's what you're seeing.
In Hibernate 3, using "create-drop" means to drop all current mapped tables and associated objects (sequences, joins, etc.), then run the create statements, and if the jvm shuts down cleanly, drop all of the tables again at the end. "create" is slightly different in that it does all of the drops and all of the creates, but doesn't do anything at shutdown. This is one I use often since it lets me view data in disk-based databases after the app shuts down.
So the core issue is that the initial drop statements are missing the "if exists" clause, and sometimes not everything that Hibernate is trying to drop exists. So you get these scary looking errors, but they're harmless since they're just telling you that some of the things that were supposed to be deleted were never created.
In Hibernate 3, these errors were stored in a list of strings that you could access after running the script. In Hibernate 4 they changed it to log the errors, I assume since not all errors in that phase are ignorable, so it's better to have false positives in with the real issues than everything hidden in a list that is probably rarely checked.
You can choose to ignore these (at the risk of also ignoring unrelated real error messages), or use a custom Dialect that generates the proper SQL. One example is here
I'm having a little bit of trouble with console outputs for a delete statement.
I'm using Grails 2.3.11 for this project.
The idea ist quite simple and works, but there is an unwanted output to the console.
The project enables the user to delete a person, which has no ties to other domain objects. To achieve this, I use the following code:
def person = Person.get(params.id)
try {
person.delete(flush: true)
}
catch (org.springframework.dao.DataIntegrityViolationException e) {
def returnData = [:]
returnData.put("message", "Person is undeletable")
returnData.put("success", false)
render returnData as JSON
}
If it doesn't have any ties in the db, the object gets deleted. If it does, it doesn't get deleted, which is what I want!
If an object couldn't be deleted the server response is a status 200 with the JSON, so that works fine!
But it still sends a stacktrace to the console/log, which is in my opinion unnecessary.
Is there a way to get around that, that I havn't seen?
Thanks in advance!
Here's the relevant excerpt of the stacktrace:
| Error 2015-01-29 16:23:38,291 [http-bio-8080-exec-12] ERROR util.JDBCExceptionReporter - Cannot delete or update a parent row: a foreign key constraint fails (`db`.`table`, CONSTRAINT `FKFC41E6E036872635` FOREIGN KEY (`person_id`) REFERENCES `person` (`id`))
...
| Error 2015-01-29 16:23:38,294 [http-bio-8080-exec-12] ERROR events.PatchedDefaultFlushEventListener - Could not synchronize database state with session
Message: could not delete: [Person#13454]
Caused by MySQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`db`.`table`, CONSTRAINT `FKFC41E6E036872635` FOREIGN KEY (`person_id`) REFERENCES `person` (`id`))
I have been struggling with this error for a week now, and I am seriously losing my mind over this! I have tried multible implementations and work-arounds and hacks and what not, but I just keep stubling into just another exception.
I am using the Executor plugin to run a method asynchroniously:
runAsync{
run(...)
}
The method initially deletes some objects:
page.delete(flush:true)
And then later possibly recreating those objects:
def page = new Page(type : Page.TYPE_TABLE, domain : domainVersion.domain, identifier : tableName)
page.save(flush: true, failOnError: true)
But that fails with the following exception:
Caused by: org.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session: [com.ramboll.egs.ohs.domain.Domain#1]
The relationship between the Page and Domain is simply implemented by Page having a Domain attribute. No hasMany og belongsTo - as I was discouraged from this in an earlier post due to performance issues.
I think I have tried all thinkable combinations of save, merge, withTransachtion and PersistenceContextInterceptor...
How is this supposed to work? Examples please.
Thanks in advance!
It doesn't appear that working in a new thread is the issue, it looks like a standard validation problem. It's saying that the Page is null, which indicates a validation error since save() returns the instance if it was successful, or null if there's a one or more validation errors. There are a few options:
def page = new Page(type : Page.TYPE_TABLE,
domain: dbUpdate.domainVersion.domain, identifier: tableName)
page.save(flush:true)
if (page.hasErrors()) {
// handle errors
}
else {
def pageVersion = createPageVersion(page, dbUpdate.domainVersion,
con, tableName, dbUpdate.author).save(flush:true)
}
or use failOnError to throw an exception:
def page = new Page(type : Page.TYPE_TABLE, identifier: tableName,
domain: dbUpdate.domainVersion.domain).save(flush:true, failOnError: true)
def pageVersion = createPageVersion(page, dbUpdate.domainVersion,
con, tableName, dbUpdate.author).save(flush:true)