This has been an issue for a few weeks and no one has managed to figure out the problem.
When I populate the database I do so by reading in and parsing an excel sheet.
For each line a create a product
Domains:
class Product {
String name
String comments
static hasMany = [components:Components]
}
class Components {
Product product
static hasMany = [alternatives:Alternatives]
}
class Alternatives {
Product product
}
I create an instance like this:
Product p = new Product(name: getCell(1).getStringVale(),
comments: getCell(2).getStringValue(),
components: [])
Components c = new Components(product: Product.get(getCell(3).getIntegerValue()),
alternatives = [])
c.alternatives.add(new Alternatives(product: Product.get(getCell(4).getIntegerValue())).save(failOnError:true,flush:true))
c.alternatives.add(new Alternatives(product: Product.get(getCell(5).getIntegerValue())).save(failOnError:true,flush:true))
p.components.add(c)
c.save(failOnError:true,flush:true)
p.save(failOnError:true,flush:true)
Now if I print this out right away like the following assuming the ID of that product was set to 1:
XML.use("deep")
Println Product.get(1) as XML
This works fine and seems to have saved to the database.
If I then look at the database there are no/or random relationships being set.
It is worth noting that the products are saved as part of a loop.
Has anyone experienced anything like this before?
Use c.addToAlterntives(...) and p.addToComponents(...) instead of c.alternatives.add(...) and p.components.add(...).
Most likely you have a validation issue. If you print it immediately you are just seeing the transient data stored in your domain object.
When you save do this:
if (!object.save()) {
println("ERRORS: ")
object.errors.allErrors.each {
println "\tERROR: ${it}"
}
}
See if that will show you what is going wrong. You can also call object.validate to see if they validate or not. Hope this helps! I have had similar problems before. Happy coding!
Related
I have two entities: Item (which keeps track of lists) and Tasks (which are task items within lists). In one view of the app, there is a swipe to delete feature which removes the list. This works with the following code:
offsets.map { items[$0] }.forEach(viewContext.delete)
I would like to delete now obsolete tasks for the list being deleted. I first tried this code:
tasks.filter{$0.listID! == listsID}.forEach(viewContext.delete)
I was proud of myself. What an elegant solution. Swift hated it. I get an error that says
Reference to member 'listID' cannot be resolved without a contextual type
I Googled and SOd and got nowhere. I don't know what that error means and can't figure out how to fix it in XCode 12 / iOS 14. So then I came up with the following not so elegant code:
let listsID = offsets.map {items[$0].id!}
let fetchRequest: NSFetchRequest<NSFetchRequestResult> = Tasks.fetchRequest()
fetchRequest.predicate = NSPredicate(format: "listID == %#", listsID)
let deleteRequest = NSBatchDeleteRequest(fetchRequest: fetchRequest)
deleteRequest.resultType = .resultTypeObjectIDs
do {
let result = try viewContext.execute(deleteRequest)
guard
let deleteResult = result as? NSBatchDeleteResult,
let ids = deleteResult.result as? [NSManagedObjectID]
else {return}
let changes = [NSDeletedObjectsKey: ids]
NSManagedObjectContext.mergeChanges(fromRemoteContextSave: changes,
into: [viewContext])
}
catch {
print(error as Any)
}
listsID captures the UUID of the list and stores it as a variable. However, you'll note that it is stored as an ARRAY (ugh). The fetchRequest.predicate code filters the tasks so that only those that have the attribute of 'listID' (which helps connect the task to the list it belongs to) matching the id of the list being deleted is pulled.
The code compiles (yay!). Then I get the following error when trying to delete a list:
Exception NSException * "Unexpected or improperly formatted UUID parameter with type Swift.__SwiftDeferredNSArray, expected NSUUID or well-formed NSString" 0x0000600003145b30
I'm sure there is a simple way to do this. I played with inverse relationships in Core Data but got nowhere. I didn't know how to tell it that 'id' (UUID) in 'Item' == 'listID' (UUID) in 'Tasks', so deleting the list didn't do anything to the tasks that belong to it.
I tried to create a ForEach loop, but ran into various errors that I couldn't resolve. I'd prefer to use the elegant code that I wrote at the top to make the deletion happen. Any ideas?
Thank you.
The credit for this goes to pbasdf for his super helpful responses. I used this link to learn about setting up relationships in Core Data: Seneca SDDS. I was missing one line in my persistence model:
newTask.list = newItem
How I solved this problem (in case it helps someone else) is that I created a one to many relationship for Item (the list of lists), and one to one relationship for Tasks (tasks are a subset of a list). Now, when I create a new task in the persistence script, I have a pointer back to the list it belongs to - hence the single line of code above.
The additional background that helped me understand Core Data relationships can be found in the link above. This snippet in particular was very helpful (in case the page goes 404):
Adding a new object, and setting the relationship
Assume that you have a reference to a Company object already; its variable name is c. How do you add a new Employee or Product? Create the new object and set its relationship. The relationship can be configured from either direction. In this section, we will configure it from the perspective of the just-added new object. For example:
// As noted above, assume that you have a reference
// "c" to an existing Company object...
// Create and configure a new employee
let peter = Employee(context: m.ds_context)
peter.name = "Peter McIntyre"
peter.age = 23
// etc.
// Now, set the relationship
peter.company = c
m.ds_save()
That’s it. If it seems too easy, well, it is easy.
I am importing a large amount of data from a csv file, (file size is over 100MB)
the code i'm using looks like this :
def errorLignes = []
def index = 1
csvFile.toCsvReader(['charset':'UTF-8']).eachLine { tokens ->
if (index % 100 == 0) cleanUpGorm()
index++
def order = Orders.findByReferenceAndOrganization(tokens[0],organization)
if (!order) {
order = new Orders()
}
if (tokens[1]){
def user = User.findByReferenceAndOrganization(tokens[1],organization)
if (user){
order.user = user
}else{
errorLignes.add(tokens)
}
}
if (tokens[2]){
def customer = Customer.findByCustomCodeAndOrganization(tokens[2],organization)
if (customer){
order.customer = customer
}else{
errorLignes.add(tokens)
}
}
if (tokens[3]){
order.orderType = Integer.parseInt(tokens[3])
}
// etc.....................
order.save()
}
and i'm using the cleanUpGorm method to clean session after each 100 entries
def cleanUpGorm() {
println "clean up gorm"
def session = sessionFactory.currentSession
session.flush()
session.clear()
propertyInstanceMap.get().clear()
}
I also turned 2nd level cache off
hibernate {
cache.use_second_level_cache = false
cache.use_query_cache = false
cache.provider_class = 'net.sf.ehcache.hibernate.EhCacheProvider'
}
the grails version of the project is 2.0.4 and as database i am using mysql
for every entry , i am doing 3 calls to a find
to check if the order already exists
to check if user is correct
to check if customer is correct
and finally i'm saving the order instance
the import process is too slow, i am wondering how can I speed up and optimise this code.
EDIT :
I found that the searchable plugin is also making it slower .
so , to get around this , I used the command :
searchableService.stopMirroring()
But it still not fast enough,I am finally changing the code to use groovy sql instead
This found this blog entry very useful:
http://naleid.com/blog/2009/10/01/batch-import-performance-with-grails-and-mysql/
You are already cleaning up GORM, but try cleaning every 100 entries:
def propertyInstanceMap = org.codehaus.groovy.grails.plugins.DomainClassGrailsPlugin.PROPERTY_INSTANCE_MAP
propertyInstanceMap.get().clear()
Creating database indexes might help aswell and use default-storage-engine=innodb instead of MyISAM.
I'm also in the process of writing a number of services that will accomplish loads of very large datasets (multiple files of up to ~17million rows each). I initially tried the cleanUpGorm method you use, but found that, whilst it did improve things, the loading was still slow. Here's what I did to make it much faster:
Investigate what it is that is causing the app to actually be slow. I installed the Grails Melody plugin, then did a run-app then opened a browser at /monitoring. I could then see which routines took time to execute and what the worst-performing queries actually were.
Many of the Grails GORM methods map to a SQL ... where ... clause. You need to ensure that you have an index for each item used in a where clause for each query that you want to make faster, otherwise the method will become considerably slower the bigger your dataset is. This includes putting indexes on the id and version columns that are injected into each of your domain classes.
Ensure you have indexes set up for all of your hasMany and belongsTo relationships.
If the performance is still too slow, use Spring Batch. Even if you've never used it before, it should take you no time at all to set up a batch parse of a CSV file to parse into Grails domain objects. I suggest you use the grails-spring-batch plugin to do this and use the examples here to get a working implementation going quickly. It's extremely fast, very configurable and you don't have to mess around with cleaning up the session.
I had used batch insert while insert records, this is much faster than gorm cleanup method. Below example describes you how to implement it.
Date startTime = new Date()
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
(1..50000).each {counter ->
Person person = new Person()
person.firstName = "abc"
person.middleName = "abc"
person.lastName = "abc"
person.address = "abc"
person.favouriteGame = "abc"
person.favouriteActor = "abc"
session.save(person)
if(counter.mod(100)==0) {
session.flush();
session.clear();
}
if(counter.mod(10000)==0) {
Date endTime =new Date()
println "Record inserted Counter =>"+counter+" Time =>"+TimeCategory.minus(endTime,startTime)
}
}
tx.commit();
session.close();
I have a problem similar to this. But I want a does not contain functionality.
Like I have a Post domain. A Post hasMany User.
What I'd like to do, using createCriteria, is something like this:
def c = Post.createCriteria()
def l = c.list (max: maxVar) {
notContains("users", thisUser)
}
I tried using ne But no luck.
def l = c.list (max: maxVar) {
users {
ne('id', thisUser.id)
}
}
To be clear, how can I get list of all the Post whose users field which is a collection does not contain thisUser ?
You can use HQL for this
List<Post> posts = Post.executeQuery("select distinct p from Post p where :myUser not member of p.users", [myUser: user, max: maxVar])
c.list (max: maxVar) { not { 'in'("users", thisUser) }}
Will return a list without thisUser. This works for grails version 2.*. You may need to override .equals for your User class.
There are also many other ways to this such as using .find. Please see this link http://groovy.codehaus.org/JN1015-Collections
I was not able to get a conventional solution to it, but I solved this problem using two separate queries.
First I got all the Posts whose users contain thisUser using
users {
eq('id', thisUser.id)
}
Then I got max number of Post whose id does not match any of the id from above.
Post.findAllByIdNotInList(usersList.id, [max:15])
where usersList is result from first query
P.S. : Please answer if anyone has a better solution. Thanks
I'm trying to use the repository pattern to save an entity using the Entity Framework. I'm unclear on how to save the Navigation Properties (like Account below). Can anyone shed some light on this. Especially how one would set the AccountId from an MVC controller all the way through to the repository where it's saved.
Thanks!
--- Sample Code ---
public void SavePerson(Person person)
{
if (person != null)
{
using (xxxxxxEntities bbEntities = new xxxxxxEntities())
{
//see if it's in the db
Person cPerson;
ObjectQuery<Person> persons = bbEntities.Person;
cPerson = (from p in persons
where p.PersonId == person.PersonId
select p).FirstOrDefault() ?? new Person();
//synch it
cPerson.Account.AccountId = person.Account.AccountId; //<=== ????
cPerson.Active = person.Active;
cPerson.BirthDay = person.BirthDay;
cPerson.BirthMonth = person.BirthMonth;
cPerson.BirthYear = person.BirthYear;
cPerson.CellPhone = person.CellPhone;
cPerson.CreatedBy = person.CreatedBy;
cPerson.CScore = person.CScore;
Etc....
I think you may be going about this the hard way. There are lots of posts on the repository pattern, the way that works best with MVC is to get the item, then update it with the form, then save it. What you're doing is passing the item through to your repository, getting it again and then updating it with the object.
But that's not the problem you asked about;
cPerson.Account = (from a in Account
where a.AccountId.Equals(person.Account.AccountId)
select a).FirstOrDefault();
You need to set the Account object to an instance of the account you're trying to reference like this. You could, and probably should, extract this code into a seperate repository for the account, just make sure they share the same Entity context.
edit #2: Question solved halfways. Look below
As a follow-up question, does anyone know of a non-intrusive way to solve what i'm trying to do below (namely, linking objects to each other without triggering infinite loops)?
I try to create a asp.net-mvc web application, and get a StackOverFlowException. A controller triggers the following command:
public ActionResult ShowCountry(int id)
{
Country country = _gameService.GetCountry(id);
return View(country);
}
The GameService handles it like this (WithCountryId is an extension):
public Country GetCountry(int id)
{
return _gameRepository.GetCountries().WithCountryId(id).SingleOrDefault();
}
The GameRepository handles it like this:
public IQueryable<Country> GetCountries()
{
var countries = from c in _db.Countries
select new Country
{
Id = c.Id,
Name = c.Name,
ShortDescription = c.ShortDescription,
FlagImage = c.FlagImage,
Game = GetGames().Where(g => g.Id == c.GameId).SingleOrDefault(),
SubRegion = GetSubRegions().Where(sr => sr.Id == c.SubRegionId).SingleOrDefault(),
};
return countries;
}
The GetGames() method causes the StackOverflowException:
public IQueryable<Game> GetGames()
{
var games = from g in _db.Games
select new Game
{
Id = g.Id,
Name = g.Name
};
return games;
}
My Business objects are different from the linq2sql classes, that's why I fill them with a select new.
An unhandled exception of type 'System.StackOverflowException' occurred in mscorlib.dll
edit #1: I have found the culprit, it's the following method, it triggers the GetCountries() method which in return triggers the GetSubRegions() again, ad nauseam:
public IQueryable<SubRegion> GetSubRegions()
{
return from sr in _db.SubRegions
select new SubRegion
{
Id = sr.Id,
Name = sr.Name,
ShortDescription = sr.ShortDescription,
Game = GetGames().Where(g => g.Id == sr.GameId).SingleOrDefault(),
Region = GetRegions().Where(r => r.Id == sr.RegionId).SingleOrDefault(),
Countries = new LazyList<Country>(GetCountries().Where(c => c.SubRegion.Id == sr.Id))
};
}
Might have to think of something else here :) That's what happens when you think in an OO mindset because of too much coffee
Hai! I think your models are recursively calling a method unintentionally, which results in the stack overflow. Like, for instance, your Subregion object is trying to get Country objects, which in turn have to get Subregions.
Anyhow, it always helps to check the stack in a StackOverflow exception. If you see a property being accessed over and over, its most likely because you're doing something like this:
public object MyProperty { set { MyProperty = value; }}
Its easier to spot situations like yours, where method A calls method B which calls method A, because you can see the same methods showing up two or more times in the call stack.
The problem might be this: countries have subregions and subregions have countries. I don't know how you implement the lazy list, but that might keep calling GetCountries and then GetSubRegions and so on. To find that out, I would launch the debugger en set breakpoints on the GetCountries and GetSubRegions method headers.
I tried similar patterns with LinqToSql, but it's hard to make bidirectional navigation work without affecting the performance to much. That's one of the reasons I'm using NHibernate right now.
To answer your edited question, namely: "linking objects to each other without triggering infinite loops":
Assuming you've got some sort of relation where both sides need to know about the other... get hold of all the relevant entities in both sides, then link them together, rather than trying to make the fetch of one side automatically fetch the other. Or just make one side fetch the other, and then fix up the remaining one. So in your case, the options would be:
Option 1:
Fetch all countries (leaving Subregions blank)
Fetch all Subregions (leaving Countries blank)
For each Subregion, look through the list of Countries and add the Subregion to the Country and the Country to the Subregion
Option 2:
Fetch all countries (leaving Subregions blank)
Fetch all Subregions, setting Subregion.Countries via the countries list fetched above
For each subregion, go through all its countries and add it to that country
(Or reverse country and subregion)
They're basically equialent answers, it just changes when you do some of the linking.