Serilog sql server sink not writing messages - serilog

-=-=-=-= Latest update
In my catch block, I brought up the exception variable in Quick Watch, and drilling down the inner exceptions showed me this error:
Unable to load DLL 'Microsoft.Data.SqlClient.SNI.x86.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
Problem is, in my bin\debug directory, that file is there!!! So why can it not be found? I tried but was not able to reference it directly.
-=-=-= Update
A suggested answer below had some links which I re-examined. I picked out useful code, such as assigning my log object to the static logger, followed by a call to CloseAndFlush.
That showed me Microsoft.Data.SqlClient.dll errors when the CloseAndFlush call was executed. I used NuGet to update SQLClient, but still getting those errors:
Exception thrown: 'System.DllNotFoundException' in Microsoft.Data.SqlClient.dll
Exception thrown: 'System.TypeInitializationException' in Microsoft.Data.SqlClient.dll
I am now investigating those SQL.Client errors.
-=-=-=-=-=-=
Working with Asp.NET, not core. Trying to get a basic sql server sink to work, having no luck. Something very simple must be the problem, but I can't see it.
Below is the table to write to, it already exists. With the seriuser account I can connect via SSMS, and perform all CRUD operations. I also wrote a method that uses SQL data objects that connects to the db and I can perform all CRUD operations on it that way also. But with Serilog, the log statement logs nothing. What could I be missing? The database is local on my machine. Breakpoint shows a valid logger object. Tried it also by passing in schemaname in sinkoptions, no luck. And no errors, had a try catch block around logger code previously.
A solution with a console app and a class library. Can't get much simpler than this.
The console app has this in program.cs:
class Program
{
static void Main(string[] args)
{
SLogger logger = new SLogger();
}
}
The class file and sql table:
public class SLogger
{
public SLogger()
{
var columnOption = new ColumnOptions();
columnOption.Store.Remove(StandardColumn.MessageTemplate);
ILogger log = new LoggerConfiguration()
.MinimumLevel.Verbose()
.WriteTo.MSSqlServer(#"Data Source=LW39\QA;Initial Catalog=DEV;user id=seriuser;Password=SERIUSER1;",
sinkOptions: new MSSqlServerSinkOptions { TableName = "Logs" },
columnOptions: columnOption)
.CreateLogger();
log.Information("Logger created.");
.....
.....
CREATE TABLE [dbo].[Logs](
[Id] [int] IDENTITY(1,1) NOT NULL,
[Message] [nvarchar](max) NULL,
[MessageTemplate] [nvarchar](max) NULL,
[Level] [nvarchar](128) NULL,
[TimeStamp] [datetime] NOT NULL,
[Exception] [nvarchar](max) NULL,
[Properties] [nvarchar](max) NULL,
CONSTRAINT [PK_Logs] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO

Mentioned in my last update that I was receiving this error, which I beleived was the root cause of my problem:
Unable to load DLL 'Microsoft.Data.SqlClient.SNI.x86.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
and my confusion because said file was in the class library directory.
As I also mentioned, I had two projects in my solution: the class library and the test console app. Turns out the dll need to be in the bin directory of the console app!! Specifically in the x86 directory, which I had to create.
Once I placed the file there, the problem was solved. Thanks for all who helped!

This is a very common question. Lots of answers here on SO:
Serilog MSSqlServer sink not writing to table
Serilog ms sql sink not writing to my table

Related

FS0039: The namespace is not defined

I'm building a Web API in F# to test some things out. I come from a C# background and I've structured things how I do there.
You can find the repo here: https://github.com/MrGodlike6/WebApiTest
I've referenced the WebApiTest.Repositories.SqlServer project in WebApiTest.WebApi but for some reason Visual studio says it can't find the WebApiTest.Repositories.SqlServer namespace.
The thing is that the solution compiles fine and I can run the application, it's just that I don't have IntelliSense for anything coming from the SqlServer project (e.g. in the CompositionRoot module).
I've tried a lot of things, from different DBs, to making a .NET Framework 4.7.2 project for the DB project. So far nothing works. Tried both Nuget & Paket and no difference. Tried VS 2019. In VS 2022 it gives some compile error that is unrelated.
I used dotPeek from JetBrains to look at the resulting .dll and the namespace & module are there as expected. Don't know what could be causing this.
Also can you indicate if the project/folder structure that I use is good for F#? Any other suggestions are also welcome.
EDIT: The SQL part is based of https://github.com/isaacabraham/get-programming-fsharp/tree/master/src/code-listings/lesson-35. Here everything works, although the SQL code is not put in a separate project.
I've commented the SqlEnumProvider and I no longer have problems in the composition root. I'll look a little closer at this provider and see if I can come up with something.
EDIT2: Created a issue on GitHub: https://github.com/fsprojects/FSharp.Data.SqlClient/issues/410
I've reproduced what you're seeing. The problem appears to be caused by a design-time error in the FSharp.Data type provider. In particular, SqlEnumProvider seems to be the source. When I remove the DbOperations type from AccountRepository.fs (using plain strings instead of enum values), I'm able to get Intellisense for WebApiTest.Repositories.SqlServer in CompositionRoot.fs.
Here's my kludgy workaround:
[<AutoOpen>]
module private DB =
let [<Literal>] Conn = #"Data Source=.;Database=BankAccountDb;Integrated Security=True;Connect Timeout=60"
type AccountsDb = SqlProgrammabilityProvider<Conn>
type GetAccountId = SqlCommandProvider<"SELECT TOP 1 AccountId FROM dbo.Account WHERE Owner = #owner", Conn, SingleRow = true>
type FindTransactions = SqlCommandProvider<"SELECT Timestamp, OperationId, Amount FROM dbo.AccountTransaction WHERE AccountId = #accountId", Conn>
type FindTransactionsByOwner = SqlCommandProvider<"SELECT a.AccountId, at.Timestamp, at.OperationId, at.Amount FROM dbo.Account a LEFT JOIN dbo.AccountTransaction at on a.AccountId = at.AccountId WHERE Owner = #owner", Conn>
// type DbOperations = SqlEnumProvider<"SELECT Description, OperationId FROM dbo.Operation", Conn>
let toBankOperation operationId =
match operationId with
| "Deposit" -> Deposit
| "Withdraw" -> Withdraw
| _ -> failwith "Unknown operation!"
let fromBankOperation bankOperation =
match bankOperation with
| Deposit -> "Deposit"
| Withdraw -> "Withdraw"
I searched for an existing issue on GitHub, but didn't find one, so you might want to open a new one yourself.
If anyone else wants to try this, here's the dummy database schema I used to get things to compile:
USE [BankAccountDb]
GO
/****** Object: Table [dbo].[Account] Script Date: 9/16/2021 11:40:48 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Account](
[Owner] [nchar](10) NOT NULL,
[AccountId] [uniqueidentifier] NOT NULL
) ON [PRIMARY]
GO
/****** Object: Table [dbo].[AccountTransaction] Script Date: 9/16/2021 11:40:48 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[AccountTransaction](
[AccountId] [uniqueidentifier] NOT NULL,
[Timestamp] [datetime] NOT NULL,
[OperationId] [nchar](10) NOT NULL,
[Amount] [money] NOT NULL
) ON [PRIMARY]
GO
/****** Object: Table [dbo].[Operation] Script Date: 9/16/2021 11:40:48 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Operation](
[Description] [nvarchar](100) NULL,
[OperationId] [nvarchar](100) NULL
) ON [PRIMARY]
GO
Thank you Brian for the script.
As the documentation says,
"An enumeration type (or enum type) is a value type defined by a set of named constants of the underlying integral numeric type."
I changed the OperationId column data type to int in dbo.Operation and dbo.AccountTransaction and populated the table dbo.Operation, because the SqlEnumProvider complained about lack of data during the compilation.
CREATE TABLE [dbo].[Operation] (
[Description] NVARCHAR (100) NULL,
[OperationId] INT NOT NULL
);
CREATE TABLE [dbo].[AccountTransaction] (
[AccountId] UNIQUEIDENTIFIER NOT NULL,
[Timestamp] DATETIME NOT NULL,
[OperationId] INT NOT NULL,
[Amount] MONEY NOT NULL
);
insert dbo.Operation (Description, OperationId) values ('Deposit', 1), ('Withdraw', 2)
Now I can compile the solution.

"Guid should contain 32 digits" serilog error with sql server sink

I am getting this error occasionally with the MSSQLServer sink. I can't see what's wrong with this guid. Any ideas? I've verified in every place I can find the data type of the source guid is "Guid" not a string. I'm just a bit mystified.
Guid should contain 32 digits with 4 dashes (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).Couldn't store <"7526f485-ec2d-4ec8-bd73-12a7d1c49a5d"> in UserId Column. Expected type is Guid.
The guid in this example is:
7526f485-ec2d-4ec8-bd73-12a7d1c49a5d
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
seems to match the template to me?
Further details:
This is an occasional issue, but when it arises it arises a lot. It seems to be tied to specific Guids. Most Guids are fine, but a small subset have this issue. Our app logs thousands of messages a day, but these messages are not logged (because of the issue) so it is difficult for me to track down exactly where the specific logs that are causing this error come from. However, we use a centralized logging method that is run something like this. This test passes for me, but it mirrors the setup and code we use for logging generally, which normally succeeds. As I said, this is an intermittent issue:
[Fact]
public void Foobar()
{
// arrange
var columnOptions = new ColumnOptions
{
AdditionalColumns = new Collection<SqlColumn>
{
new SqlColumn {DataType = SqlDbType.UniqueIdentifier, ColumnName = "UserId"},
},
};
columnOptions.Store.Remove(StandardColumn.MessageTemplate);
columnOptions.Store.Remove(StandardColumn.Properties);
columnOptions.Store.Remove(StandardColumn.LogEvent);
columnOptions.Properties.ExcludeAdditionalProperties = true;
var badGuid = new Guid("7526f485-ec2d-4ec8-bd73-12a7d1c49a5d");
var connectionString = "Server=(localdb)\\MSSQLLocalDB;Database=SomeDb;Trusted_Connection=True;MultipleActiveResultSets=true";
var logConfiguration = new LoggerConfiguration()
.MinimumLevel.Information()
.Enrich.FromLogContext()
.WriteTo.MSSqlServer(connectionString, "Logs",
restrictedToMinimumLevel: LogEventLevel.Information, autoCreateSqlTable: false,
columnOptions: columnOptions)
.WriteTo.Console(restrictedToMinimumLevel: LogEventLevel.Information);
Log.Logger = logConfiguration.CreateLogger();
// Suspect the issue is with this line
LogContext.PushProperty("UserId", badGuid);
// Best practice would be to do something like this:
// using (LogContext.PushProperty("UserId", badGuid)
// {
Log.Logger.Information(new FormatException("Foobar"),"This is a test");
// }
Log.CloseAndFlush();
}
One thing I have noticed since constructing this test code is that the "PushProperty" for the UserId property is not captured and disposed. Since behaviour is "undefined" in this case, I am inclined to fix it anyway and see if the problem goes away.
full stack:
2020-04-20T08:38:17.5145399Z Exception while emitting periodic batch from Serilog.Sinks.MSSqlServer.MSSqlServerSink: System.ArgumentException: Guid should contain 32 digits with 4 dashes (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).Couldn't store <"7526f485-ec2d-4ec8-bd73-12a7d1c49a5d"> in UserId Column. Expected type is Guid.
---> System.FormatException: Guid should contain 32 digits with 4 dashes (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).
at System.Guid.GuidResult.SetFailure(Boolean overflow, String failureMessageID)
at System.Guid.TryParseExactD(ReadOnlySpan`1 guidString, GuidResult& result)
at System.Guid.TryParseGuid(ReadOnlySpan`1 guidString, GuidResult& result)
at System.Guid..ctor(String g)
at System.Data.Common.ObjectStorage.Set(Int32 recordNo, Object value)
at System.Data.DataColumn.set_Item(Int32 record, Object value)
--- End of inner exception stack trace ---
at System.Data.DataColumn.set_Item(Int32 record, Object value)
at System.Data.DataRow.set_Item(DataColumn column, Object value)
at Serilog.Sinks.MSSqlServer.MSSqlServerSink.FillDataTable(IEnumerable`1 events)
at Serilog.Sinks.MSSqlServer.MSSqlServerSink.EmitBatchAsync(IEnumerable`1 events)
at Serilog.Sinks.PeriodicBatching.PeriodicBatchingSink.OnTick()
RESOLUTION
This issue was caused because someone created a log message with a placeholder that had the same name as our custom data column, but was passing in a string version of a guid instead of one typed as a guid.
Very simple example:
var badGuid = "7526f485-ec2d-4ec8-bd73-12a7d1c49a5d";
var badGuidConverted = Guid.Parse(badGuid); // just proving the guid is actually valid.
var goodGuid = Guid.NewGuid();
using (LogContext.PushProperty("UserId",goodGuid))
{
Log.Logger.Information("This is a problem with my other user {userid} that will crash serilog. This message will never end up in the database.", badGuid);
}
The quick fix is to edit the message template to change the placeholder from {userid} to something else.
Since our code was centralized around the place where the PushProperty occurs, I put some checks in there to monitor for this and throw a more useful error message in the future when someone does this again.
I don't see anything obvious in the specific code above that would cause the issue. The fact that you call PushProperty before setting up Serilog would be something I would change (i.e. set up Serilog first, then call PushProperty) but that doesn't seem to be the root cause of the issue you're having.
My guess, is that you have some code paths that are logging the UserId as a string, instead of a Guid. Serilog is expecting a Guid value type, so if you give it a string representation of a Guid it won't work and will give you that type of exception.
Maybe somewhere in the codebase you're calling .ToString on the UserId before logging? Or perhaps using string interpolation e.g. Log.Information("User is {UserId}", $"{UserId}");?
For example:
var badGuid = "7526f485-ec2d- 4ec8-bd73-12a7d1c49a5d";
LogContext.PushProperty("UserId", badGuid);
Log.Information(new FormatException("Foobar"), "This is a test");
Or even just logging a message with the UserId property directly:
var badGuid = "7526f485-ec2d-4ec8-bd73-12a7d1c49a5d";
Log.Information("The {UserId} is doing work", badGuid);
Both snippets above would throw the same exception you're having, because they use string values rather than real Guid values.

Neo4j Mapping Exception

I am a getting a MappingException error when executing the following piece of code:
Configuration configuration = new Configuration.Builder()
.uri("file:///var/lib/neo4j/data/databases/graph.db")
.credentials("neo4j","noor")
.build();
SessionFactory sessionFactory = new SessionFactory(configuration,"OntologyDescription","info.testNeo");
Session session = sessionFactory.openSession();
ResourceDescription classDescription = OntologyUtils.getClassDescription(classIRI,
dogont);
session.save(classDescription);
sessionFactory.close();
Full trace:
Exception in thread "main" org.neo4j.ogm.exception.core.MappingException: Field with primary id is null for entity state
at org.neo4j.ogm.context.MappingContext.nativeId(MappingContext.java:514)
at org.neo4j.ogm.context.EntityGraphMapper.newNodeBuilder(EntityGraphMapper.java:318)
at org.neo4j.ogm.context.EntityGraphMapper.mapEntity(EntityGraphMapper.java:257)
at org.neo4j.ogm.context.EntityGraphMapper.mapRelatedEntity(EntityGraphMapper.java:797)
at org.neo4j.ogm.context.EntityGraphMapper.link(EntityGraphMapper.java:501)
at org.neo4j.ogm.context.EntityGraphMapper.mapEntityReferences(EntityGraphMapper.java:414)
at org.neo4j.ogm.context.EntityGraphMapper.mapEntity(EntityGraphMapper.java:265)
at org.neo4j.ogm.context.EntityGraphMapper.map(EntityGraphMapper.java:149)
at org.neo4j.ogm.session.delegates.SaveDelegate.lambda$save$1(SaveDelegate.java:89)
at java.util.Collections$SingletonList.forEach(Collections.java:4822)
at org.neo4j.ogm.session.delegates.SaveDelegate.save(SaveDelegate.java:89)
at org.neo4j.ogm.session.delegates.SaveDelegate.save(SaveDelegate.java:51)
at org.neo4j.ogm.session.Neo4jSession.save(Neo4jSession.java:468)
at Examples.TestNeo4j.main(TestNeo4j.java:41)
I am getting the error only when getting classDescription from the OntologyUtils.getClassDescription method. However, if I instantiate a ResourceDescription newly, I don't get the error. In short, OntologyUtils.getClassDescription calls several other methods to add information to the new instance.
This problem was because the package names contain capital letters. As mentioned in this GitHub issue, package names should be completely in lower case

Unable to catch database exceptions in Grails 3

I'm facing an odd issue after migrating an app from Grails 2.5 to Grails 3.1 - database exceptions never bubble up to the controller.
The typical scenario looks like this: controller's delete action attempts to remove an entry, but a database-level constraint prevents the delete. This leads to an exception (see below) - in Grails 2.5, if uncaught, this would lead to "Error 500 internal server error". Now, we have a custom mapping for these, so even in case an uncaught error occurs, it is handled in a user-friendly way and the user knows something happened.
In Grails 3, this stopped working. The exception is logged, but would never bubble up to the controller. So in this specific case, our controller just goes on and returns "204" HTTP status code as if nothing happened, and the end user doesn't even know something went wrong.
I'm puzzled and I think this must be something stupid I'm missing, but can't figure it out. I have tested this Grails 2.5.5, 3.1.16, 3.2.12 and 3.3.2, but all tested 3.x releases behaved pretty much the same way. Also tested with both Hibernate 4 and 5, and tried H2 and PostgreSQL but I'm getting the same behavior regardless.
The easiest way to reproduce is to manually add a FK relationship (with delete no action) to the hibernate-generated database schema and attempt to delete via the controller (example #1 below) but the issue may also occur with some many-to-many relationships or other scenarios such as persisting a new entry (and failing at the database level).
UPDATE #1, April 25, 2018: To create a minimal reproducible test case using PostgreSQL as the database and Grails 3.3.x, just create a blank grails application via grails create-app testapp. Then adjust the application.yml to configure data source for your PostgreSQL instance, and use dbCreate: validate mode as the schema is provided below.
Add one minimal domain object:
class Book {
String name
}
And a controller to test object deletion, i.e. BookController:
class BookController {
#Transactional
def delete(Long id) {
Book.get(id).delete()
render "Book $id deleted"
}
}
The schema below creates 2 tables with a FK and a few entries. Book with id 1 will fail to be removed, books with id 2 and 3 will be removed successfully. Controller will return 200 OK in all 3 cases. You can test using the default UrlMappings via curl calls, i.e. curl http://localhost:8080/book/delete/1.
The database schema follows:
CREATE TABLE book (
id bigint NOT NULL,
version bigint NOT NULL,
name character varying(255) NOT NULL,
CONSTRAINT book_pkey PRIMARY KEY (id)
);
CREATE TABLE chapter (
id bigint NOT NULL,
book_id bigint NOT NULL,
name character varying(255),
CONSTRAINT chapter_pkey PRIMARY KEY (id),
CONSTRAINT fk_book FOREIGN KEY (book_id) REFERENCES book(id)
);
INSERT INTO book VALUES (1, 0, 'Book 1');
INSERT INTO book VALUES (2, 0, 'Book 2');
INSERT INTO book VALUES (3, 0, 'Book 3');
INSERT INTO chapter VALUES (1, 1, 'Book 3 Chapter');
CREATE SEQUENCE hibernate_sequence START WITH 1 INCREMENT BY 1 NO MINVALUE NO MAXVALUE CACHE 1;
SELECT pg_catalog.setval('hibernate_sequence', 4, true);
For the sake of completeness, here's the stack trace:
2018-04-25 16:23:22.462 ERROR --- [nio-8080-exec-1] o.h.engine.jdbc.spi.SqlExceptionHelper : ERROR: update or delete on table "book" violates foreign key constraint "fk_book" on table "chapter"
Detail: Key (id)=(1) is still referenced from table "chapter".
2018-04-25 16:23:22.473 ERROR --- [nio-8080-exec-1] org.hibernate.internal.SessionImpl : HHH000346: Error during managed flush [could not execute statement]
2018-04-25 16:23:22.720 ERROR --- [nio-8080-exec-1] o.g.web.errors.GrailsExceptionResolver : PSQLException occurred when processing request: [GET] /test/delete/1
ERROR: update or delete on table "book" violates foreign key constraint "fk_book" on table "chapter"
Detail: Key (id)=(1) is still referenced from table "chapter".. Stacktrace follows:
java.lang.reflect.InvocationTargetException: null
at org.grails.core.DefaultGrailsControllerClass$ReflectionInvoker.invoke(DefaultGrailsControllerClass.java:211)
at org.grails.core.DefaultGrailsControllerClass.invoke(DefaultGrailsControllerClass.java:188)
at org.grails.web.mapping.mvc.UrlMappingsInfoHandlerAdapter.handle(UrlMappingsInfoHandlerAdapter.groovy:90)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
at org.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:77)
at org.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:67)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [fk_book]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:241)
at org.springframework.orm.hibernate5.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:755)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:590)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:765)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:734)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:150)
at grails.gorm.transactions.GrailsTransactionTemplate.execute(GrailsTransactionTemplate.groovy:91)
... 14 common frames omitted
Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:207)
at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:45)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3311)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3548)
at org.hibernate.action.internal.EntityDeleteAction.execute(EntityDeleteAction.java:98)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:586)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:460)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1295)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:468)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3135)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2352)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:491)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:147)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:231)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:65)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:582)
... 18 common frames omitted
Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "book" violates foreign key constraint "fk_book" on table "chapter"
Detail: Key (id)=(1) is still referenced from table "chapter".
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155)
at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:132)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:204)
... 36 common frames omitted
UPDATE #2, April 25, 2018: the fact that there is a database table that Hibernate isn't aware of was the point of the above example. It was an easy way to force a database-level exception that would only be logged, but wouldn't make it all the way up to the controller so on HTTP level it would appear as if no error occurred.
UPDATE #3, April 25, 2018: Another test case for Grails 3.3 that doesn't rely on any manual database schema "hacks" is below, with a many-to-many relationship between the same domain (User). It consists of 2 domain classes:
class User {
String name
static hasMany = [relationships: Relationship]
static mappedBy = [relationships: 'user']
static mapping = { table 'table_user' }
}
class Relationship implements Serializable {
User user
User related
static belongsTo = [ user: User, related: User ]
static mapping = {
table 'table_relationship'
id composite: ['user', 'related']
version false
}
}
And a minimal controller:
class TestController {
#Transactional
def delete(Long id) {
User.get(id).delete()
render "User $id deleted"
}
}
This can be reproduced with PostgreSQL or H2, just use the create-drop mode and the following BootStrap code to create 2 User entries:
def init = { servletContext ->
def user1 = new User(name: 'User 1').save()
new User(name: 'User 2', relationships: [new Relationship(related: user1)]).save()
}
Attempting to delete user 2 and then user 1 will work OK. Attempting to first remove user 1 will fail with an exception, yet the controller would still return "200 OK". Just try curl http://localhost/test/delete/1.
For convenience you can download the zipped archive with a runnable Grails 3.3.5 app here: https://www.dropbox.com/s/pycwuxm7r0wyxem/grails3_exception_issue_testapp.zip?dl=0
If you are going to use #Transactional you should use grails.gorm.transactions.Transactional instead of grails.transaction.Transactional.
Separate from that, the problem doesn't appear to be that the exception doesn't make its way up to the controller as much as what is really happening is the transaction isn't being committed until after you call render and the exception doesn't get thrown until the transaction is committed. Calling .delete(flush: true) will show different behavior but putting the transaction boundary in the controller is a bad idea anyway. A better plan is to put your database interactions in a transactional service.
I put your code in the project at https://github.com/jeffbrown/many33. The commit at https://github.com/jeffbrown/many33/commit/f8804c8793b399994c34043c1d340e4bf0d462cd shows a better way to organize this code and yields less surprising behavior.
I hope that helps.

Entity Framework 4.3.1 Code First: database created but tables are not

Today I migrated an old application from EF 4.2 to EF 4.3.1.
In my app I was using CodeFirst, but after migration it stopped working, and couldn't find a reason for that.
To clear any other possible problem I decided to create a small console application and I used the Data Migration walk-through published by the ADO team:
http://blogs.msdn.com/b/adonet/archive/2012/02/09/ef-4-3-code-based-migrations-walkthrough.aspx
I copied exactly the code of the blog, but instead of working correctly (creating the DB, creating the schema, and inserting the blog) I get some errors:
only the DB is created, but no tables
I get this error Conversion failed when converting datetime from character string."
All of this is on SQL Server 2005 express.
I tried the same using SQL Compact, but same result (tho different error):
only the DB is created (in this case a sdf file in the bin folder), but no tables
I get the error The format of the specified date or time datepart is not valid. [ String = 2012-04-19T13.21.04.364 ]
I think in both cases the problem lies in the line that EF wants to enter as first migration:
INSERT INTO [__MigrationHistory] ([MigrationId], [CreatedOn], [Model], [ProductVersion])
VALUES ('201204191321184_init', '2012-04-19T13.21.04.364', ...., '4.3.1');
Apparently the format with the . is wrong, at least in my locale, it should be with :
Is this a bug or what? It always worked with other datetime before.
UPDATE
I tried running it as explicit migration, and applying the migration with the -verbose flag set, and here is what I get:
PM> Update-Database -Verbose
Using NuGet project 'ConsoleApplication2'.
Using StartUp project 'ConsoleApplication2'.
Target database is: '|DataDirectory|ConsoleApplication2.ConsoleApplication1.BlogContext.sdf' (DataSource: |DataDirectory|ConsoleApplication2.ConsoleApplication1.BlogContext.sdf, Provider: System.Data.SqlServerCe.4.0, Origin: Convention).
Applying explicit migrations: [201204191356197_Initial].
Applying explicit migration: 201204191356197_Initial.
CREATE TABLE [Blogs] (
[BlogId] [int] NOT NULL IDENTITY,
[Name] [nvarchar](4000),
CONSTRAINT [PK_Blogs] PRIMARY KEY ([BlogId])
)
CREATE TABLE [__MigrationHistory] (
[MigrationId] [nvarchar](255) NOT NULL,
[CreatedOn] [datetime] NOT NULL,
[Model] [image] NOT NULL,
[ProductVersion] [nvarchar](32) NOT NULL,
CONSTRAINT [PK___MigrationHistory] PRIMARY KEY ([MigrationId])
)
[Inserting migration history record]
System.Data.SqlServerCe.SqlCeException (0x80004005): The format of the specified date or time datepart is not valid. [ String = 2012-04-19T13.56.45.437 ]
at System.Data.SqlServerCe.SqlCeCommand.ProcessResults(Int32 hr)
at System.Data.SqlServerCe.SqlCeCommand.ExecuteCommandText(IntPtr& pCursor, Boolean& isBaseTableCursor)
at System.Data.SqlServerCe.SqlCeCommand.ExecuteCommand(CommandBehavior behavior, String method, ResultSetOptions options)
at System.Data.SqlServerCe.SqlCeCommand.ExecuteNonQuery()
at System.Data.Entity.Migrations.DbMigrator.ExecuteSql(DbTransaction transaction, MigrationStatement migrationStatement)
at System.Data.Entity.Migrations.Infrastructure.MigratorLoggingDecorator.ExecuteSql(DbTransaction transaction, MigrationStatement migrationStatement)
at System.Data.Entity.Migrations.DbMigrator.ExecuteStatements(IEnumerable`1 migrationStatements)
at System.Data.Entity.Migrations.Infrastructure.MigratorBase.ExecuteStatements(IEnumerable`1 migrationStatements)
at System.Data.Entity.Migrations.DbMigrator.ExecuteOperations(String migrationId, XDocument targetModel, IEnumerable`1 operations, Boolean downgrading)
at System.Data.Entity.Migrations.DbMigrator.ApplyMigration(DbMigration migration, DbMigration lastMigration)
at System.Data.Entity.Migrations.Infrastructure.MigratorLoggingDecorator.ApplyMigration(DbMigration migration, DbMigration lastMigration)
at System.Data.Entity.Migrations.DbMigrator.Upgrade(IEnumerable`1 pendingMigrations, String targetMigrationId, String lastMigrationId)
at System.Data.Entity.Migrations.Infrastructure.MigratorLoggingDecorator.Upgrade(IEnumerable`1 pendingMigrations, String targetMigrationId, String lastMigrationId)
at System.Data.Entity.Migrations.DbMigrator.Update(String targetMigration)
at System.Data.Entity.Migrations.Infrastructure.MigratorBase.Update(String targetMigration)
at System.Data.Entity.Migrations.Design.ToolingFacade.UpdateRunner.RunCore()
at System.Data.Entity.Migrations.Design.ToolingFacade.BaseRunner.Run()
The format of the specified date or time datepart is not valid. [ String = 2012-04-19T13.56.45.437 ]
Update 2
I installed SQL Server Profiler, and profiled what is happening over there.
I executed all the statements one by one via query analyzer and the one that fails is, as already stated above, the insertion of the migration.
INSERT INTO [__MigrationHistory] ([MigrationId], [CreatedOn], [Model], [ProductVersion]) VALUES ('201204231416585_InitialCreate', '2012-04-23T14.16.59.038Z', ...., '4.3.1')
When changing the format of the datatime string from 2012-04-23T14.16.59.038Z to 2012-04-23T14:16:59.038Z the command went through, so I guess somehow EF is sending the datatime in format that is not compatible with my locale.
Thank you
Simone
Thanks to the ADO.NET Team, this was a bug in the Migration code.
Apparently they forgot to specify InvariantCulture when they generate code for a DateTime field, so it works on EN locale, but doesn't in other locales.
To fix this problem, waiting for an official fix, you should specify a custom SqlGenerator that overrides the Generate(DateTime defaultValue) method:
class FixedSqlGenerator : SqlServerMigrationSqlGenerator
{
protected override string Generate(DateTime defaultValue)
{
return "'" + defaultValue.ToString("yyyy-MM-ddTHH:mm:ss.fffK", CultureInfo.InvariantCulture) + "'";
}
}
And then specify the new SqlGenerator in the Configuration class:
SetSqlGenerator("System.Data.SqlClient", new FixedSqlGenerator());
If you want to use it just in manual migrations that's enough, if you only need CodeFirst, you have to specify the configuration in the application startup code or in the DbContext.
Database.SetInitializer(new MigrateDatabaseToLatestVersion<BlogContext, Migrations.Configuration>());
HTH
I had the same problem with a brand new project, in my case I solved it by specifying the right culture in my web.config
<globalization enableClientBasedCulture="false" culture="en-US" />

Resources