I'm wondering what the most efficient way of updating a single value in a domain class from a row in the database. Lets say the domain class has 20+ fields
def itemId = 1
def item = Item.get(itemId)
itemId.itemUsage += 1
Item.executeUpdate("update Item set itemUsage=(:itemUsage) where id =(:itemId)", [usageCount: itemId.itemUsage, itemId: itemId.id])
vs
def item = Item.get(itemId)
itemId.itemUsage += 1
itemId.save(flush:true)
executeUpdate is more efficient if the size and number of the un-updated fields is large (and this is subjective). It's how I often delete instances too, running 'delete from Foo where id=123' since it seems wasteful to me to load the instance fully just to call delete() on it.
If you have large strings in your domain class and use the get() and save() approach then you serialize all of that data from the database to the web server twice unnecessarily when all you need to change is one field.
The effect on the 2nd-level cache needs to be considered if you're using it (and if you edit instances a lot you probably shouldn't). With executeUpdate it will flush all instances previously loaded with get() but if you update with get + save if flushes just that one instance. This gets worse if you're clustered since after executeUpdate you'd clear all of the various cluster node caches vs flushing the one instance on all nodes.
Your best bet is to benchmark both approaches. If you're not overloading the database then you may be prematurely optimizing and using the standard approach might be best to keep things simple while you solve other problems.
If you use get/save, you'll get the maximum advantage of the hibernate cache. executeUpdate might force more selects and updates.
The way executeUpdate interacts with the hibernate cache makes a difference here. The hibernate cache gets invalidated on executeUpdate. The next access of that Item after the executeUpdate would have to go to the database (and possibly more, I think hibernate might invalidate all Items in the cache).
Your best bet is to turn on debug logging for 'org.hibernate' in your Config.groovy and examine the SQL calls.
I think they are equal. The both issue 2 sql calls.
More efficient would be just a single update
Item.executeUpdate("update Item set itemUsage=itemUsage+1 where id =(:itemId)", [ itemId: itemId.id])
You can use the dynamicUpdate mapping attribute in your Item class:
http://grails.org/doc/latest/ref/Database%20Mapping/dynamicUpdate.html
With this option enabled, your second way to update a single field using Gorm will be as efficient as the first one.
Related
Most of our tables have one or more columns which are set by the database, either by a trigger, or we want to use the database default value (which requires not sending the field at all in the insert or update)
This includes transaction dates set in the dB (so all the times are times stamped very accurately by a single source, not relying on the accuracy of the time on an arbitrary server or pc.)
The second VERY common use case is say if a customer record has his address and a last logged in field. the last logged in field (and number of failed logins) is set by another part of the system (e.g. by a web site server). The current overly simplistic CRUD system which GORM provides would overwrite such a field when an operator or customer edits their address for example. This is because GORM includes in its update and insert statements every field, even if it's null, or if it has not been changed.
We need a way to scrub the field from inserts and updates, but still have it used in the read calls.
I.e. a true "read only" attribute.
We tried this:
failedLogins editable: false, attributes: [readonly:true]
Which has no effect on the SQL generated (and doesn't even affect the scaffolded UIs - its still editable in create and edit, in grails 2.4.4 at least, but thats another story)
When we do want to explicitly write one of these fields, such as number of failed logins, we would resort to using embedded SQL.
I saw this post: Read-Only columns
Which asks exactly the same question, but only gives one solution, which is this plugin:
extended GORM mappings
Unfortunately, this plugin has not been updated since 2010, and only works with 1.3. We need something which works with 2.4.4.
Any grails app which has multiple systems which edits independent fields needs something like this, or to do extensive locking (Which is usually out of the question).
E.g. an operator opens the customer details for editing, edits something editable (e.g. address), then the operator fails a login on the website (a different grails or non-grails app), then the operator saves the player details. If the saving included the numberOfFailedLogins field, the system would fail. If opening the player details for editing locked the player, then the player would not be able to login, as updating the "lastLoggedIn" or "numFailedLogins" would fail to be able to write due to the lock. The solution is VERY simple - read only columns. Another way would be to put each read only type field in their own tables, but this would be untenable (and result in hundreds of one field tables)
Or we go back to using MyBatis, which has no such issues, and full control. Sadly, there is no good mybatis plugin for grails.
You can use derived properties for string and number properties:
class Batch {
String name
Integer timesRun
static mapping = {
timesRun formula: 'times_run' //times_run is a column in the "batch" table
}
}
In the code above, timesRun would be read in from the database but ignored in inserts and updates as Hibernate considers the column a calculated one.
Updated the example because the original one may have been misleading
This probably doesn't specifically answer your question, but you can use dynamicUpdates to tell GORM to only update the properties of the domain object that have changed during the current session. So as long as you don't change the "read-only" property in your code it won't be set in the SQL update statement generated by Grails. For added safety you could override (and noop) the setter so that your code can never change that property.
https://grails.github.io/grails-doc/latest/ref/Database%20Mapping/dynamicUpdate.html
One of the downsides of dynamicUpdates is that it might make the Hibernate query cache less useful. However, it seems that some Grails/Hibernate experts recommend that you disable the query cache anyway (at least in older versions of Grails). Not sure if that's true of Grails 2.4+
http://grails.github.io/grails-howtos/en/performanceTuning.html
http://www.anyware.co.uk/2005/2012/11/12/the-false-optimism-of-gorm-and-hibernate/
http://tech.puredanger.com/2009/07/10/hibernate-query-cache/
I ran into a problem which required me to have access to a bunch of variables which need to be changed every so often, so I made a Misc model and there is only ever one instance of it. This was my solution for having editable global variables.
It holds all types of stuff that didn't seem like they deserve their own models. Is this acceptable or does this violate some Rails-buliding principle I'm not aware of? It works, but I have doubts.
Is there a better alternative to this strategy (think fetching/editing (as an example) Misc.first.todays_specials).
If this is passable, then is there a way to prevent a creation of more than one item of a model in the database? The problem with the above approach as you can see is that if there are all of a sudden TWO entries for Misc, things will get wonky as it requests the .first under the assumption that there's ever only going to be one.
You can create a table for Settings storing key-value configs. It will be scalable and not depend on predefined keys. Also you won't have a table with one row this way.
If you need lots of read/writes you might also want to cache rails SQL Caching
you could use a singleton pattern.
a singleton class is a class that can only have one instance.
so you could do something like this:
initializers/config.rb
require 'singleton'
class MyConfig
include Singleton
attr_accessor :config1
def initialize
self.config1 = ["hello", "world"]
end
end
and use it in this way:
MyConfig.instance.config1
You can also consider global variables. Global variables are those which start with the $ sign, and are accessible in the whola application by all instances of your ws.
Using a singleton to hold global state is a very bad idea, especially in a web-server:
If you are using a multi-threaded environment - you will run into thread-safety issues.
More relevantly - if you run multi-process, or multi-server (as you would have to, if your web application ever succeeds...) - the state will be inconsistent, as changes in one process/machine will not be propagated to the other processes/machines.
A restart of your application will destroy the state of the application, since it is held only in memory.
You could use an SQL solution, as suggested by #Tala, but if you want something more light-weight and 'freestyle', you might want to look at some key-value stores like memcached or redis, where you could save your state in a central location, and fetch it when needed.
I've run into an apparent caching issue where what's being returned by NHibernate is not matching up with what's in the DB.
I believe this is level2 cache data. It looks like I can use the Evict to do this, but when should the Evict method actually be called? For my particular application, the data is going to be unique to the user and chances are the data provided will only be used once*.
Can I disable Level2 caching for these sets of objects completely?
UPDATE 10/31
My scenario is this: I have a shopping cart where the customer is going to be adding and removing items. What I am doing is the following: before the updates to the cart are processed, I evict the Cart and CartProduct entities. Once this is done, I retrieve the list of CartProducts from the provider and return the view (this is happening in a .NET MVC Controller).
UPDATE 11/3
The shopping cart has since been finished, and I ran into an issue that appeared to be related to the same NHibernate issue, but in fact was an MVC issue. Deeper digging revealed that the HTML Helper extensions were overriding the value I supposed and replacing with what was in the Model state. So a double whammy on this one. Hope this helps someone.
No, you cannot disable the cache for certain entities.
You do have several options:
Use ISession.Clear() to clear all entities from NHibernate's cache.
http://nhibernate.info/doc/nh/en/index.html#performance-sessioncache
Use ISession.Evict( obj ) to clear a specific entity from the cache.
http://nhibernate.info/doc/nh/en/index.html#performance-sessioncache
Use ISessionFactory.Evict( typeof( obj ) ) to evict all entities/objects of a particular type from the cache. This would be the closest to what you are wanting in my opinion.
http://nhibernate.info/doc/nh/en/index.html#performance-sessioncache
Use the IStatelessSession to fetch the objects/entities from the database as this completely bypasses all caches.
http://nhibernate.info/doc/nh/en/index.html#batch-statelesssession
With Grails there are several ways to do the same thing.
Finds all of domain class instances:
Book.findAll()
Book.getAll()
Book.list()
Retrieves an instance of the domain class for the specified id:
Book.findById(1)
Book.get(1)
When do you use each one? Are there significant differences in performance?
getAll is an enhanced version of get that takes multiple ids and returns a List of instances. The list size will be the same as the number of provided ids; any misses will result in a null at that slot. See http://grails.org/doc/latest/ref/Domain%20Classes/getAll.html
findAll lets you use HQL queries and supports pagination, but they're not limited to instances of the calling class so I use executeQuery instead. See http://grails.org/doc/latest/ref/Domain%20Classes/findAll.html
list finds all instances and supports pagination. See http://grails.org/doc/latest/ref/Domain%20Classes/list.html
get retrieves a single instance by id. It uses the instance cache, so multiple calls within the same Hibernate session will result in at most one database call (e.g. if the instance is in the 2nd-level cache and you've enabled it).
findById is a dynamic finder, like findByName, findByFoo, etc. As such it does not use the instance cache, but can be cached if you have query caching enabled (typically not a good idea). get should be preferred since its caching is a lot smarter; cached query results (even for a single instance like this) are pessimistically cleared more often than you would expect, but the instance cache doesn't need to be so pessimistic.
The one use case I would have for findById is as a security-related check, combined with another property. For example instead of retrieving a CreditCard instance using CreditCard.get(cardId), I'd find the currently logged-in user and use CreditCard.findByIdAndUser(cardId, user). This assumes that CreditCard has a User user property. That way both properties have to match, and this would block a hacker from accessing the card instance since the card id might match, but the user wouldn't.
Another difference between Domain.findByID(id) and Domain.get(id) is that if you're using a hibernate filter, you need to use Domain.findById(id). Domain.get(id) bypasses the filter.
AFAIK, these are all identical
Book.findAll()
Book.getAll()
Book.list()
These will return the same results
Book.findById(1)
Book.get(1)
but get(id) will use the cache (if enabled), so should be preferred to findById(1)
I have to calculate a "total" for each user based on individual actions -- that is, a User hasMany Actions and each Action has a point. So I need to basically get a sum of all the points on all the actions.
Actions are added and subtracted regularly. Since this can be a rather heavy operation, it's not feasible that I execute the "total" each time I need it. Therefore, I am thinking of running the operation once a day for each user and storing the total points on the User as an int.
Because I have thousands of users, I am trying to figure out the best way to do this. Should I basically iterate through all the users and call User.save() each time, or is there some kind of batch update mechanism I can use in Grails / GORM?
Is the model in your question your actual model, or a simplified version of it? If all you're doing is User hasMany Actions and each Action has a point value (an integer?) that you'd like to sum up, that's really the kind of thing that relational databases excel at. If you have the proper indexes on your database, I'd think this would be a very quick call, as long as you're doing a groupBy query (or using grails projection criteria).
Here's an example method that will take a list of users and will return an array that pairs user.id to the number of points that user currently has:
def calculateCurrentPoints(users) {
Action.executeQuery('select a.user.id, sum(points) from Action a where a.user.id in (:userIds) group by a.user.id', [userIds: users.id])
}
If desired, you could easily turn this into a map of user id to points for easier look-up:
def calculateCurrentPoints(users) {
def result = Action.executeQuery('select a.user.id, sum(points) from Action a where a.user.id in (:userIds) group by a.user.id', [userIds: users.id])
result.inject([:]) { map, userWithPoints ->
map[userWithPoints[0]] = userWithPoints[1]
return map
}
}
If this really is slower than I'm thinking it is, you could use the executeUpdate with HQL similiar to what I have above to update a "totalPoints" field on each user (or keep it updated as part of a service whenever you add a new Action to that user).
Calling User.save() will not actually write the change to the database, it is only when the Hibernate session is flushed that the change is written (docs)
You can flush the session manually be accessing the SessionFactory and calling flush on the resulting session, as shown in this FAQ.
I imagine you would want to load the users using a batching technique to ensure you don't have all of the thousands of users in memory at the same time.
As an aside if you wanted to cache the value but have it automatically updated whenever you add an action you could hook into active record's events to update the calculated value, here is an example from Ruby