I've got a domain class, Widget, that I need to delete all instances out of -- clear it out. After that, I will load in fresh data. What do you suggest as a mechanism to do this?
P.S. Note this is not at bootstrap time, but at "run-time".
The easiest way is to use HQL directly:
DomainClass.executeUpdate('delete from DomainClass')
DomainClass.findAll().each { it.delete() }
If you want to avoid any GORM gotchas, such as needing to delete the object immediately and checking to make sure it actually gets deleted, add some arguments.
DomainClass.findAll().each { it.delete(flush:true, failOnError:true) }
Fairly old post, but still actual.
If your table is very large (millions of entries), iterating using findall()*.delete() might not be the best option, as you can run into transaction timeouts (e.g. MySQL innodb_lock_wait_timeout setting) besides potential memory problems stated by GreenGiant.
So at least for MySQL Innodb, much faster is to use TRUNCATE TABLE:
sessionFactory.currentSession
.createSQLQuery("truncate table ${sessionFactory.getClassMetadata(MyDomainClass).tableName}")
.executeUpdate()
This is only useful if your table is not referenced by other objects as a foreign key.
From what I learnt, I agree with #ataylor the below code is fastest IF there are no associations in your domain object (Highly unlikely in any real application):
DomainClass.executeUpdate('delete from DomainClass')
But if you have assiciations with other domains, then the safest way to delete (and also a bit slower than the one mentioned above) would be the following:
def domainObjects = DomainClass.findAll()
domainObjects.each {
it.delete(flush:it==domainObjects.last, failOnError:true)
}
If you have a list of objects and want to delete all elements, you can use * operator.
'*' will split the list and pass its elements as separate arguments.
Example.
List<Book> books = Book.findAllByTitle('grails')
books*.delete()
Related
I have a method:
class Role
def currently_active
klass = roleable_type.constantize
actor = Person.find(role_actor_id)
parent = klass.find(roleable_id)
return true if parent.current_membership?
actor.current_membership?
end
end
I would like to return all instances of Role for who this method is true, however can't iterate through them with all.each as this takes around 20 seconds. I'm trying to use where statements, however they rely on an attribute of the model rather than a method:
Role.where(currently_active: true)
This obviously throws an error as there is no attribute called currently_active. How can I perform this query the most efficient way possible, and if possible using Active Records rather than arrays?
Thanks in advance
It seems impossible, in your case you have to do iterations. I think the best solution is to add a Boolean column in your table, so you can filter by query and this will be much faster.
After seeing your method after edit, it seems that it's not slow because of the loop, it is slow because Person.find and klass.find , you are doing alot of queries and database read here. (You better use associations and do some kind of eager loading, it will be much faster)
Another work-around is you can use ActiveModelSerializers , in the serializer you can get the attributes on the object based on condition. and after that you can work your logic to neglect the objects that have some kind of flag or attribute.
See here the documentation of active model serializer
Conditional attributes in Active Model Serializers
Wherever possible you better delegate your methods to SQL through activerecord when you're seeking better efficiency and speed and avoid iterating through objects in ruby to apply the method. I understand this is an old question but still many might get the wrong idea.
There is not enough information on current_membership? methods on associations but here's an example based on some guess-work from me:
roleables = roleable_type.pluralize
roleable_type_sym = roleable_type.to_sym
Role.joins(roleables_sym).where(" ? BETWEEN #{roleables}.membership_start_date AND #{roleables}.membership_end_date", DateTime.current).or(Role.joins(:person).where(" ? BETWEEN persons.membership_start_date AND persons.membership_end_date", DateTime.current))
so you might have to re-implement the method you have written in the model in SQL to improve efficiency and speed.
Try the select method: https://www.rubyguides.com/2019/04/ruby-select-method/
Role.all.select { |r| r.currently_active? }
The above can be shortened to Role.select(&:currently_active?)
Problem:
We're querying our database with a lot of entries. The overall application performance is okay. But I think it could be better if our bottleneck, querying a special table, could be done parallel.
We are using for that dynamic finders like:
Object.findAllByObjectCollectionAndParentIsNullAndDeletedByUserIsNull(objectCollection, [sort: 'attr1.number', fetch: [objectType: 'eager']])
I think about this solution: Dividing the query into 2 steps.
The first step loads only the Ids in a sorted way.
The second step (could be done in parallel threads) loads the objects itself by the id and inserts it into the resultset.
I already googled for some hints about that, but find nothing.
Is it possible, that it makes no sense?
Or is there already a adequate solution in grails standard/extensions?
If it makes sense and there is no solution: Can someone give me a hint implementing it? But, we need the ids in that sorted manner, explained in the example.
We're using grails 2.3.11, hibernate:3.6.10.13 with jdk 1.7 under it.
I found a solution!
ArrayList<Long> objectIds = Attribute.executeQuery("select o.id from Object o, ObjectType ot where o.objectCollection = :oc and o.parent is null and o.deletedByUser is null and oc.typeUsage = ot order by ot.nr asc", [oc: objectCollection])
Object[] tmp = new Object[objectIds.size()]
withPool(10, {
objectIds.eachWithIndexParallel {
Long objectId, i ->
Object o = Object.findById(objectId, [cache: true])
tmp[i] = a
}
})
for (Object a : tmp)
a.merge()
Its round about 3 times slower!
I think the reason is, that
we have to create the threads and databaseconnections too.
After loading the objects, they need to be merged into our local thread.
But in the end - we cant use it, beause the lazy loaded fields couldn't be loaded on demand later. So this could be only a solution for read-only Objects
I'm making my first steps in using Redis under ZF2.
I was wondering if there is a method to retrieve keys by pattern.
e.g.:
after setting multiple values with keys like: 'stackOverflow_'.time(), i would like to retrieve later all keys matching the 'stackOverflow_' pattern.
tried using getItems(array $keys) with wildcard in: \vendor\zendframework\zendframework\library\Zend\Cache\Storage\Adapter\AbstractAdapter.php
$redisKeyPattern = 'stackOverflow_';
$redis = $this->getServiceLocator()->get('Redis');
$values = $redis->getItems(array($redisKeyPattern.'*'));
with no succces.
any ideas?
UDPATE:
thanks guys. i ended up with duplicating the Redis adapter and adding my own functionality that utilizes the 'keys' function in the Redis extension:
public function getItemsByKeyPattern($pattern) {
$keys = $this->getRedisResource()->keys('*'.$pattern.'*');
if(empty($keys)) return null;
foreach($keys as &$key){
$key = explode(':', $key)[1];
}
$items = parent::getItems($keys);
return $items;
}
and it works for me :)
sadly to say there is no method present to return items with a wildcard, also redis don't support namespaces for stored items.
you need to define each item you want to receive, maybe you should look at a implementation like this
$receiveRedisKeys = [];
foreach($resultSet as $result)
{
$receiveRedisKeys[] = 'predefined_prefix_' . $result->getId();
}
$redisCacheResultSet = $redis->getItems($receiveRedisKeys);
i know that someone on github made a new repository where he modified redis to allow namespaces but this requires that you build the redis binarys by yourself from source. this leeds to a redis version you can't update anymore over apt-get
It's not possible, but there are some alternatives.
One idea is to keep a set with the keys you are interested in. That is the most common approach to this problem: each time you create one of the keys you will want to retrieve later, you add its name to a set. Then when you need to operate on one of those keys, you can grab it from the set. Read this article to get a general idea about this approach.
Another idea is to use the SCAN command to walk the keyspace with the pattern you are using, and as a second step retrieve the values with MGET followed by the keys you collected. This approach is good for administrative processes, but not as something that should be included in an application because the performance will be worse than that of the first idea. More about SCAN.
Finally, an option that is not recommended but I'm listing it just for completeness is to use the KEYS command to collect the keys you want, then proceed to get the values with MGET, as in the SCAN approach. This is not recommended as KEYS shouldn't be used in production environments. More about KEYS.
I'm trying to accelerate the performance of my app and wonder if there is a difference between accessing domain property value with instance.name and instance.getName()
If it is, which one is the best in terms of performance ?
Example
class User {
String name
}
User user = User.get(100);
//is it better this way
user.name
//or this way
user.getName()
Thank you
It doesn't matter for the usage you've provided, because user.name uses user.getName() behind scenes. So it's the same. If you want to access property directly you have to use # like this user.#name. See more here
But I don't think this is the way you can speed up your app.
It is very likely you will find a lot easier ways for improving performance of your code. Here are some ideas where to start if you like to improve performance.
A) Number of queries. Try to avoid the the N+1 problem. For example if one user hasMany [events: Event], code like user.events.each { access event.anyPropertyExceptId } will dispatch new queries for each event.
B) Efficiency of queries. Grails per default creates indexes for all gorm associations / other nested domains. However anything you use to search, filter etc. you need to do "manually" for example.
static mapping = {
anyDomainProperty index: 'customIndexName'
}
C) Only query for the data you are interested in, replace for example:
User.all.each { user ->
println user.events.size()
}
with
Event.withCriteria {
projections {
property('user')
countDistinct('id')
groupProperty('user')
}
}
D) If you really need to speed up your groovy code and your problem is rather a single request than general cpu usage, take a look at http://gpars.codehaus.org and http://grails.org/doc/2.3.8/guide/async.html and try to parallize work.
I doubt any performance issues in your application are related to how you are accessing your properties of your domain classes. In fact, if you profile/measure your application I'm sure you will see that is the case.
I have two object, a room type and a reservation. Simplified they are:
class Room {
String description
int quantity
}
class Reservation {
String who
Room room
}
I want to query for all rooms along with the number of rooms available for each type. In SQL this does what I want:
select id, quantity, occupied, quantity-coalesce(occupied, 0) as available
from room left join(select room_id, count(room_id) as occupied from reservation)
on id = room_id;
I'm not getting anywhere trying to work out how to do this with HQL.
I'd appreciate any pointers since it seems like I'm missing something fairly fundamental in either HQL or GORM.
The problem here is your trying to represent fields that are not your domain classes like available and occupied. Trying to get HQL\GORM to do this can be a bit a little frustrating, but not impossible. I think you have a couple options here...
1.) Build your domain classes so that there easier to use. Maybe your Room needs to know about it's Reservations via a mapping table or, perhaps write what you want the code to look like and then adjust the design.
For example. Maybe you want your code to look like this...
RoomReservation.queryAllByRoomAndDateBetween(room, arrivalDate, departureDate);
Then you would implement it like this...
class RoomReservation{
...
def queryAllByRoomAndDateBetween(def room, Date arrivalDate, Date departureDate){
return RoomReservation.withCriteria {
eq('room', room)
and {
between('departureDate', arrivalDate, departureDate)
}
}
}
2.) My second thought is... It's okay to use the database for what it's good for. Sometimes using sql in you code is simply the most effective way to do something. Just do it in moderation and keep it centralized and unit tested. I don't suggest you use this approach because you query isn't that complex, but it is an option. I use stored procedures for things like 'dashboard view's' that query millions of objects for summary data.
class Room{
...
def queryReservations(){
def sql = new Sql(dataSoruce);
return sql.call("{call GetReservations(?)}", [this.id]) //<-- stored procedure.
}
}
I'm not sure how you can describe a left join with a subquery in HQL. INn any case you can easily execute raw SQL in grails too, if HQL is not expressive enough:
in your service, inject the dataSource and create a groovy.sql.Sql instance
def dataSource
[...]
def sql= new Sql(dataSource)
sql.eachRow("...."){row->
[...]
}
I know it's very annoying when people try to patronize you into their way of thinking when you ask a question, instead of answering your question or just shut up, but in my opinion, this query is sufficiently complex that I would create a concept for this number in my data structure, perhaps an Availability table associated to the Room, which would keep count not only of the quantity but also of the occupied value.
This is instead of computing it every time you need it.
Just my $.02 just ignore it if it annoys you.