I need to get all the buildings with "church" function that are far 100km from a specified point (lat, lng). I made in this way:
[{
"id": null,
"name": null,
"type": "/architecture/building",
"building_function" : [{"name" : 'church'}],
"/location/location/geolocation" : {"latitude" : 45.1603653, "longitude" : 10.7976976}
"/location/location/area" : 100
}]
but I alway get an empty response
code: "/api/status/ok"
result: []
status: "200 OK"
transaction_id: "cache;cache03.p01.sjc1:8101;2011-04-16T12:32:45Z;0035"
What am I missing?
Thanks
An area isn't a distance and you probably don't want an exact match to the value "100" anyway. You've asked for things which are precisely at that long/lat and have exactly that area.
Are you looking for churches which are less than a certain distance, more than a certain distance, or exactly the given distance? You probably want to look at the Geosearch API http://api.freebase.com/api/service/geosearch?help (although it's not a long term solution since it's been deprecated)
The /location/location/area property is used to query locations which cover a certain amount of area. So your query looks for buildings centered at (45.1603653, 10.7976976) which cover an area of 100km. Naturally there are no results that match.
Searching for topics within 100km of a those coordinates takes a little more work. You'll need to use the Geosearch service which is still in alpha. The following query should give you the results that you're looking for:
http://www.freebase.com/api/service/geosearch?location={%22type%22:%22Point%22,%22coordinates%22:[10.7976976,45.1603653]}&type=/architecture/building&within=100&indent=1
Once you have that list of buildings, you can query the MQL Read API to find out which ones are churches like this:
[{
"id": null,
"name": null,
"type": "/architecture/building",
"building_function" : [{"name" : 'church'}],
"filter:id|=":[
"/en/verona_arena",
"/en/basilica_palladiana",
"/en/teatro_olimpico",
"/en/palazzo_del_te",
"/en/villa_capra_la_rotonda",
"/en/villa_badoer",
"/en/san_petronio_basilica",
"/en/palazzo_schifanoia",
"/en/palazzo_chiericati",
"/en/basilica_di_santandrea_di_mantova",
"/en/basilica_of_san_domenico",
"/en/castello_estense",
"/en/palazzo_dei_diamanti",
"/en/villa_verdi",
"/en/cathedral_of_cremona",
"/en/monte_berico",
"/en/villa_pojana",
"/en/san_sebastiano",
"/en/cremona_baptistery",
"/en/palazzo_della_pilotta"
]
}]
Right now its only matching 2 results so you'll probably need to edit some of those topics to mark them as churches.
Related
I have articles data indexed to elastic as follows.
{
"id": 1011,
"title": "abcd",
"author": "author1"
"status": "published"
}
Now I wanted to get all the article id grouped by status.
Result should someway look like this
{
"published": [1011, 1012, ....],
"draft": [2011],
"deleted": [3011]
}
NB: I tried normal aggs (Article.search('*',aggs: [:status], load: false).aggs) , it just giving me the count of each items in, I want ids in each item instead
#Crazy Cat
You can transform you query in this way:
sort(Inc/Dec order) your response from ES over field "status".
Only Ask ES query to return only ID Field and status.
Now the usage of sorting would be it would sort your response to like this: [1st N results of "deleted" status, then N+1 to M results to "draft" and then M+1 to K results to "published"].
Now the advantages of this technique:
You will get flagged ids field of every document over which you can apply operations in you application.
Your query would be light weight as compared to Aggs query.
This way you would also get the metdata of every document ike docId of that document.
Now the Disadvantages:
You would always have to give a high upper bound of your page size, but You can also play around with count coming in the metadata.
Might take a bit more of network size as it returns redundant status in every document.
I Hope this redesign of your query might be helpful to you.
In a question on CouchDB I asked previously (Can you implement document joins using CouchDB 2.0 'Mango'?), the answer mentioned creating domain objects instead of storing relational data in Couch.
My use case, however, is not necessarily to store relational data in Couch but to flatten relational data. For example, I have the entity of Invoice that I collect from several suppliers. So I have two different schemas for that entity.
So I might end up with 2 docs in Couch that look like this:
{
"type": "Invoice",
"subType": "supplier B",
"total": 22.5,
"date": "10 Jan 2017",
"customerName": "me"
}
{
"type": "Invoice",
"subType": "supplier A",
"InvoiceTotal": 10.2,
"OrderDate": <some other date format>,
"customerName": "me"
}
I also have a doc like this:
{
"type": "Customer",
"name": "me",
"details": "etc..."
}
My intention then is to 'flatten' the Invoice entities, and then join on the reduce function. So, the map function looks like this:
function(doc) {
switch(doc.type) {
case 'Customer':
emit(doc.customerName, { doc information ..., type: "Customer" });
break;
case 'Invoice':
switch (doc.subType) {
case 'supplier B':
emit (doc.customerName, { total: doc.total, date: doc.date, type: "Invoice"});
break;
case 'supplier A':
emit (doc.customerName, { total: doc.InvoiceTotal, date: doc.OrderDate, type: "Invoice"});
break;
}
break;
}
}
Then I would use the reduce function to compare docs with the same customerName (i.e. a join).
Is this advisable using CouchDB? If not, why?
First of all apologizes for getting back to you late, I thought I'd look at it directly but I haven't been on SO since we exchanged the other day.
Reduce functions should only be used to reduce scalar values, not to aggregate data. So you wouldn't use them to achieve things such as doing joins, or removing duplicates, but you would for example use them to compute the number of invoices per customer - you see the idea. The reason is you can only make weak assumptions with regards to the calls made to your reduce functions (order in which records are passed, rereduce parameter, etc...) so you can easily end up with serious performance problems.
But this is by design since the intended usage of reduce functions is to reduce scalar values. An easy way to think about it is to say that no filtering should ever happen in a reduce function, filtering and things such as checking keys should be done in map.
If you just want to compare docs with the same customer name you do not need a reduce function at all, you can query your view the following parameters:
startkey=["customerName"]
endkey=["customerName", {}]
Otherwise you may want to create a separate view to filter on customers first, and return their names and then use these names to query your view in a bulk manner using the keys view parameter. Startkey/endkey is good if you only want to filter one customer at a time, and/or need to match complex keys in a partial way.
If what you are after are the numbers, you may want to do :
if(doc.type == "Invoice") {
emit([doc.customerName, doc.supplierName, doc.date], doc.amount)
}
And then use the _stats built-in reduce function to get statistics on the amount (sum, min, max,)
So that to get the amount spent with a supplier, you'd just need to make a reduce query to your view, and use the parameter group_level=2 to aggregate by the first 2 elements of the key. You can combine this with startkey and endkey to filter specific values of this key :
startkey=["name1", "supplierA"]
endkey=["name1", "supplierA", {}]
You can then build from this example to do things such as :
if(doc.type == "Invoice") {
emit(["BY_DATE", doc.customerName, doc.date], doc.amount);
emit(["BY_SUPPLIER", doc.customerName, doc.supplierName], doc.amount);
emit(["BY_SUPPLIER_AND_DATE", doc.customerName, doc.supplierName, doc.date], doc.amount);
}
Hope this helps
It is totally ok to "normalize" your different schemas (or subTypes) via a view. You cannot create views based on those normalized schemas, though, and on the long run, it might be hard to manage different schemas.
The better solution might be to normalize the documents before writing them to CouchDB. If you still need the documents in their original schema, you can add a sub-property original where you store your documents in their original form. This would make working on data much easier:
{
"type": "Invoice",
"total": 22.5,
"date": "2017-01-10T00:00:00.000Z",
"customerName": "me",
"original": {
"supplier": "supplier B",
"total": 22.5,
"date": "10 Jan 2017",
"customerName": "me"
}
},
{
"type": "Invoice",
"total": 10.2,
"date": "2017-01-12T00:00:00:00.000Z,
"customerName": "me",
"original": {
"subType": "supplier A",
"InvoiceTotal": 10.2,
"OrderDate": <some other date format>,
"customerName": "me"
}
}
I d' also convert the date to ISO format because it parses well with new Date(), sorts correctly and is human-readable. You can easily emit invoices grouped by year, month, day and whatever with that.
Use reduce preferably only with built-in functions, because reduces have to be re-executed on queries, and executing JavaScript on many documents is a complex and time-intensive operation, even if the database has not changed at all. You find more information about the reduce process in the CouchDB process. It makes more sense to preprocess the documents as much as you can before storing them in CouchDB.
I am queuing and getting a report through the API and javascript, but now I want to start filtering the report. I want the results that come back to apply only to the user (other filters are needed too) who is requesting the report. What is the best way to put a filter on the initial report queue?
The way I am doing it now is adding a selected element to the report description:
...
"elements": [
{ "id": "page" },{ "id": "evar23" , "selected": ["295424","306313"]}
...
But this only seems to apply to the breakdown section of the results, not the top level count that is returned. I would expect the top level count in the below example be 66, not 68:
...
"counts":[
"68"
],
"breakdown":[
{
"name":"306313",
"url":"",
"counts":[
"43"
]
},
{
"name":"295424",
"url":"",
"counts":[
"23"
]
}
]
}
,...
I know I can just crawl through the breakdown array and total up what I need, but the more filters I apply the messier it becomes. All of a sudden I am three levels deep in a nested array, making sure that all 3 breakdown names match my conditions. There must be a better way to do this, any ideas? Many thanks.
Although there are some possible limitations to them that I am still working through, it seems that segments is what I need, not elements.
"segments": [
{
"element": "evar23","selected": ["295424","306313"]
}]
https://marketing.adobe.com/developer/forum/reporting/report-filtering-with-api
how to get the distance or radian between two point on the earth with lng and lat?
You probably don't want mapReduce in this case but actually the aggregation framework. Apart from the general first stage query you can run via $geoNear which is more efficient in your purpose.
db.places.aggregate([
{ "$geoNear": {
"near": {
"type": "Point",
"coordinates": [ -88 , 30 ]
},
"distanceField": "dist"
}},
{ "$match": {
"loc": {
"$centerSphere": [ [ -88 , 30 ] , 0.1 ]
}
}}
])
Or frankly, because the initial $geoNear stage will "project" an additional field into the document containing the "distance" from the queried "point of origin", then you can just "filter" on that element in a subsequent stage:
db.places.aggregate([
{ "$geoNear": {
"near": {
"type": "Point",
"coordinates": [ -88 , 30 ]
},
"distanceField": "dist"
}},
{ "$match": {
"dist": { "$lte": 0.1 }
}}
])
Since this is one option that can "produce/project" a value representing the distance in the result then that satisfies your first criteria. The "chaining" nature of the "aggregation framework" allows "additional filtering" or any other operation you need to perform after the filtering of the initial query.
So $geoWithin works just as well in the aggregation framework under a $match stage as it would in any standard query since it is not "dependant" on an "index" of geospatial origin to be present. It performs better in an initial query with one, but it does not need it.
Since your requirement is the "distance" from the point of origin, then the most logical thing to do is to perform an operation that will return such information. Such as this does.
Would love to include all of the relevant links in this response, but as a new responder then two links is all I am allowed for now.
One more relevant note:
The measurement of "distance" or "radius" in any operation is dependant on how your data is stored. If it is in a "legacy" or "key/pair or plain array" format then the value will be expressed in "radians", otherwise where the data is expressed in GeoJSON format on the "location" then the "distance data" is expressed in "meters" instead.
That is an important consideration given the libraries implemented by the MongoDB service and how this interacts with the data as you have it stored. There is of course documentation on this in the official resources should you care to look at that properly. And again, I cannot add those links at this time, unless this response sees some much needed love.
https://github.com/mongodb/mongo/blob/master/src/third_party/s2/s2latlng.cc#L37
GetDistance() return a S1Angle, S1Angle::radians() will return the radians.
This belong to s2-geometry-library(google code will close,i export it to my github. Java).
I'm trying to do an exact location search, meaning that each term in the location should exactly match at least one location field. For example, if I search for "Sudbury, Middlesex, Massachusetts" then I want to only get results that have an exact match for each of those three terms. A result with location.city.name = Sudbury, location.county.name = Middlesex, and location.region.name = Massachusetts would match.
{
"multi_match": {
"fields": [
"location.city.name",
"location.region.name",
"location.county.name",
"location.country.name"
],
"query": "Sudbury, Middlesex, Massachusetts",
"type": "cross_fields",
"operator": "and"
}
This is very close, however I also get results for "East Sudbury." I don't want East Sudbury, I only want results that match the field exactly. How can I do this? I know that "type":"phrase" is wrong because then it would be searching for the entire phrase "Sudbury, Middlesex, Massachusetts" in each field and would get no results.
Sounds like the field location.city.name is being analysed and splitting 'East Sudbury' into 'East' and 'Sudbury' and getting returned for a search for 'Sudbury'
Try setting the field to not_analyzed if you are always searching for specific terms?