I'll try to explain this well, as it's quite strange.
I have a .NET MVC 4 app with Linq2SQL where I'm developing a search where users are able to seek elements by one or more different values.
I have a method that receive a string like: "value1&value2|value3&value4"
I pretend to return results where a determinated element contains value1 AND value2 OR that determinated element contains value3 AND value4.
I'm using an algorithm like this:
if (!String.IsNullOrEmpty(figuresSelected)) {
string[] groupsOR = figuresSelected.Split('|');
IQueryable<CENTROSOPERACIONES> sqlAuxOr = null;
foreach (string goupOR in groupsOR) {
string[] groupsAND = groupOR.Split('&');
IQueryable<CENTROSOPERACIONES>[] sqlAuxAnd = sql; //sql contains current query so far
foreach (string group in groupsAND) {
sqlAuxAnd = sqlAuxAnd.Where(o => o.CENTROS.FIGURASCENTROS.Any(f => f.FIGURAS.nombre == group));
}
if (sqlAuxAnd != null)
if (sqlAuxOr == null)
sqlAuxOr = sqlAuxAnd;
else
sqlAuxOr = sqlAuxOr.Union(sqlAuxAnd);
}
}
Well, the problem here is that, on every iteration inside the inner loop, sqlAuxAnd queries from the original data container on sql, not from the subresult obtained on the previous loop. I've followed the loop iteration by iteration and I've checked that sqlAuxAnd contains the expected subresult on the first iteration, but when reaching a second one that value is lost and a full query from the whole data is stored on it.
I've tried to store sqlAuxAnd partial results on different elements (I used an array) and checked it's evolution. On the first iteration sqlAuxAnd[0] contains the expected result and sqlAuxAnd[1] is empty, on the second iteration both elements, sqlAuxAnd[0] and sqlAuxAnd[1] contains the expected result from querying the second value from the whole dataset.
It seems I'm losing something here... probably it's something trivial, but I supose I'm too dizzy to see it. Anyone have any idea to share?
Feel free to ask for explanations on the matter, english is not my mother language and I supose that probably the question is not as well redacted as it should be.
You need to capture the group variable from the foreach:
foreach (string group in groupsAND) {
var captured = group;
sqlAuxAnd = sqlAuxAnd.Where(o => o.CENTROS.FIGURASCENTROS.Any(f => f.FIGURAS.nombre == captured));
}
See Linq query built in foreach loop always takes parameter value from last iteration.
Related
I'm modding a game. I'd like to optimize my code if possible for a frequently called function. The function will look into a dictionary table (consisting of estimated 10-100 entries). I'm considering 2 patterns a) direct reference and b) lookup with ipairs:
PATTERN A
tableA = { ["moduleName.propertyName"] = { some stuff } } -- the key is a string with dot inside, hence the quotation marks
result = tableA["moduleName.propertyName"]
PATTERN B
function lookup(type)
local result
for i, obj in ipairs(tableB) do
if obj.type == "moduleName.propertyName" then
result = obj
break
end
end
return result
end
***
tableB = {
[1] = {
type = "moduleName.propertyName",
... some stuff ...
}
}
result = lookup("moduleName.propertyName")
Which pattern should be faster on average? I'd expect the 'native' referencing to be faster (it is certainly much neater), but maybe this is a silly assumption? I'm able to sort (to some extent) tableB in a order of frequency of the lookups whereas (as I understand it) tableA will have in Lua random internal order by default even if I declare the keys in proper order.
A lookup table will always be faster than searching a table every time.
For 100 elements that's one indexing operation compared to up to 100 loop cycles, iterator calls, conditional statements...
It is questionable though if you would experience a difference in your application with so little elements.
So if you build that data structure for this purpose only, go with a look-up table right away.
If you already have this data structure for other purposes and you just want to look something up once, traverse the table with a loop.
If you have this structure already and you need to look values up more than once, build a look up table for that purpose.
Following the question I asked: Build a dynamic query using neo4j client
I got an answer about how can I return value dynamically using string only.
When I'm trying to use the syntax to return multi values from the query it failed,
I tried the following query:
var resQuery2 = WebApiConfig.GraphClient.Cypher
.Match("(movie:Movie {title:{title}})")
.OptionalMatch("(movie)<-[r]-(person:Person)")
.WithParam("title", title)
.Return(() => Return.As<string>("movie, collect([person.name, head(split(lower(type(r)), '_')), r.roles])"));
I'm getting the following error:
The deserializer is running in single column mode, but the response
included multiple columns which indicates a projection instead. If
using the fluent Cypher interface, use the overload of Return that
takes a lambda or object instead of single string. (The overload with
a single string is for an identity, not raw query text: we can't map
the columns back out if you just supply raw query text.)
Is it possible to return multiple nodes using only strings?
We can't get an output like in the question you asked previously - this is due to the fact that you are asking for a Node (the movie) and a Collection of strings (the collect) and they have no common properties, or even styles of property.
Firstly, let's look at the painful way to do this:
var q = gc.Cypher
.Match("(movie:Movie)")
.OptionalMatch("(movie)<-[r]-(person:Person)")
.Return(() => Return.As<string>("{movie:movie, roles:collect([person.name, head(split(lower(type(r)), '_')), r.roles])}"));
var results = q.Results;
Here we take the query items (movie, r, person) and create a type with them the {} around the results, and cast that to a string.
This will give you a horrible string with the Node data around the movie and then a collection of the roles:
foreach (var m in results)
{
//This is going to be painful to navigate/use
dynamic d = JsonConvert.DeserializeObject<dynamic>(m);
Console.WriteLine(d.movie);
Console.WriteLine(d.roles);
}
You'd be a lot better off doing something like:
var q = gc.Cypher
.Match("(movie:Movie)")
.OptionalMatch("(movie)<-[r]-(person:Person)")
.Return(() => new
{
Movie = Return.As<Node<string>>("movie"),
Roles = Return.As<IEnumerable<string>>("collect([person.name, head(split(lower(type(r)), '_')), r.roles])")
});
var res = q.Results;
You could either JsonConvert.DeserializeObject<dynamic>() the Movie node, at your leisure, or write a strongly typed class.
In terms of a 'dynamic' object, I don't know how you were wanting to interact with the collect part of the return statement, if this doesn't help, you might need to update the question to show a usage expectation.
my Linq query times out and the results are never retreived..
using (var repository = PersistenceService.GetRepository<Organisation>(UserId))
{
var organisation = repository.GetAll().FirstOrDefault(x => x.Id == organisationId);
if (organisation != null)
{
isCourseApproverRole = organisation.Contacts.FirstOrDefault(con => con.RoleName == "CourseApprover" &&
con.Individual.Id == individualId) != null;
}
}
When I try doing all this in one query it works fine..
Can some one explain why above query will time out??
Note: organisation.Contacts contain about 18,000 rows for the selected organisation.
It's because of massive lazy loading.
The first command...
var organisation = repository.GetAll().FirstOrDefault(x => x.Id == organisationId);
... pulls an Organisation object into memory. That shouldn't be any problem.
But then you access organisation.Contacts. It doesn't matter which LINQ method you apply to this collection, the whole collection is pulled into memory by lazy loading. The LINQ filter is applied afterwards.
However, though highly inefficient, this still shouldn't cause a timeout. Fetching 18000 records by an indexed search (I assume) shouldn't take more than 30s (I assume) unless something else is terribly wrong (like low resources, bad network).
The part con.Individual.Id == individualId is the culprit. If you would have monitored the executed SQL commands you'd have seen that this causes one query for each Individual until the predicate Id == individualId is matched. These queries run while the query organisation.Contacts is read. No doubt, this causes the timeout.
You could fix this by replacing the predicate by con.IndividualId == individualId (i.e. using the foreign key property). But I really don't understand why you don't do this in one query, which works fine, as you say. By the current approach you fetch large amounts of data, while in the end you only need one boolean value!
I have an Item model which has an attribute category. I want the items count grouped by category. I wrote a map reduce for this functionality. It was working fine. I recently wrote a script to create 5000 items. Now I realize my map reduce only gives the result for the last 80 records. The following is the code for the mapreduce function.
map = %Q{
function(){
emit({},{category: this.category});
}
}
reduce = %Q{
function(key, values){
var category_count = {};
values.forEach(function(value){
if(category_count.hasOwnProperty(value.category))
category_count[value.category]++;
else
category_count[value.category] = 1
})
return category_count;
}
}
Item.map_reduce(map,reduce).out(inline: true).first.try(:[],"value")
After researching a bit and I discovered mongodb invokes reduce function multiple times. How can achieve the functionality I intended for?
There is a rule you must follow when writing map-reduce code in MongoDB (a few rules, actually). One is that the emit (which emits key/value pairs) must have the same format for the value that your reduce function will return.
If you emit(this.key, this.value) then reduce must return the exact same type that this.value has. If you emit({},1) then reduce must return a number. If you emit({},{category: this.category}) then reduce must return the document of format {category:"string"} (assuming category is a string).
So that clearly can't be what you want, since you want totals, so let's look at what reduce is returning and work out from that what you should be emitting.
It looks like at the end you want to accumulate a document where there is a keyname for each category and its value is a number representing the number of its occurrences. Something like:
{category_name1:total, category_name2:total}
If that's the case then the correct map function would emit({},{"this.category":1}) in which case your reduce will need to add up the numbers for each key corresponding to a category.
Here is what the map should look like:
map=function (){
category = { };
category[this.category]=1;
emit({},category);
}
And here is the correct corresponding reduce:
reduce=function (key,values) {
var category_count = {};
values.forEach(function(value){
for (cat in value) {
if( !category_count.hasOwnProperty(cat) ) category_count[cat]=0;
category_count[cat] += value[cat];
}
});
return category_count;
}
Note that it satisfies two other requirements for MapReduce - it works correctly if the reduce function is never called (which will be the case if there is only one document in your collection) and it will work correctly if the reduce function gets called multiple times (which is what's happening when you have more than 100 documents).
A more conventional way to do that would be to emit category name as key and the number as value. This simplifies map and reduce:
map=function() {
emit(this.category, 1);
}
reduce=function(key,values) {
var count=0;
values.forEach(function(val) {
count+=val;
}
return count;
}
This will sum the number of times each category appears. This also satisfies requirements for MapReduce - it works correctly if the reduce function is never called (which will be the case for any category that only appears once) and it will work correctly if the reduce function gets called multiple times (which will happen if any category appears more than 100 times).
As others pointed out, aggregation framework makes the same exercise much simpler with:
db.collection.aggregate({$group:{_id:"$category",count:{$sum:1}}})
although that matches the format of the second mapReduce I showed, and not the original format that you had which is outputting category names as keys. However aggregation framework will always be significantly faster than MapReduce.
I agree with Neil Lunn's comment.
What I can see from the info that is provided is that if you are on a version of MongoDB greater or equal than 2.2 you can use the aggregation framework instead of map-reduce.
db.items.aggregate([
{ $group: { _id: '$category', category_count: { $sum: 1 } }
])
Which is a lot simpler and performant (see Map/Reduce vs. Aggregation Framework )
I do not really understand what is the difference of collections from other type of output in Cypher. Can somebody explain this to me, please?
For instance the query
match (c:Context) where c.name="health" or c.name="opinion" return collect(c);
returns 1 row, while the query
match (c:Context) where c.name="health" or c.name="opinion" return c;
returns 6 rows (I have 6 nodes in my database that match the criteria).
This seems to be the only difference.
So then, is it just about the way the data is represented? Or is there some sort of advantage to use collections?
Thank you for your help!
Collections return the entities in an array, instead of an individual "row" for each result.
The benefit of this is, for example: I want to get all addresses associated to a contact.
Match (c:Contact)-[:AddressRelation]->(a:Address)
return c,collect(a)
This would return a group of addresses for each contact, whereas without collect, it would return duplicate contact items (one for each address they have)
Collect returns something like this:
row = { name:"fred" } , [{address1},{address2},...]
Without collect:
row = { name:"fred"} , {address1}
row = { name:"fred"} , {address2}
...etc.
There are a lot of other things you can do, like return a property in an array, loop through each node in a foreach loop, etc.