I'm trying to boost up the query using elastic search. Headlines that are published recently must be boosted up and should come on top. However, I have return the following code:
{
"query": {
"function_score": {
"query": {
"bool": {
"must": {
"match": {
"keywords": {
"query":"trump"
}
}
},
"should": [
{ "match": {
"type": {
"query": "headline"
}
}}
]}},
"functions": [
{ "boost": 5 },
{
"gauss": {
"versioncreated": {
"origin": "now/d",
"scale": "50w",
"offset": "4w",
"decay": "0.5"
}}}],
"score_mode": "sum"
}
}}
I'm getting this error :-
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "no [score_function] registered for [params]",
"line": 24,
"col": 20
}
],
"type": "parsing_exception",
"reason": "no [score_function] registered for [params]",
"line": 24,
"col": 20
},
"status": 400
}
You query is a bit wrong. What do you want do by this?
"functions": [
{ "boost": 5 },
If you want to boost it try to use this query
{
"query": {
"function_score": {
"query": {
"bool": {
"must": {
"match": {
"keywords": {
"query": "trump"
}
}
},
"should": [
{
"match": {
"type": {
"query": "headline"
}
}
}
]
}
},
"functions": [
{
"gauss": {
"versioncreated": {
"origin": "now/d",
"scale": "50w",
"offset": "4w",
"decay": "0.5"
}
}
}
],
"boost": 5,
"score_mode": "sum"
}
}
}
Related
I am learning ES and I am having problems with this query:
Given 2 products:
products/_source/1
{
"product_id": "58410-2",
"name": [
{
"locale": "en",
"translation": "CBC panel"
},
{
"locale": "vn",
"translation": "CBC panel VN"
}
],
"status": "active",
"category": {
"id": 8,
"name": [
{
"locale": "en",
"translation": "Hematology"
},
{
"locale": "vn",
"translation": "huyết học"
}
]
},
"children": [
{
"product_id": "6690-2",
"name": [
{
"locale": "en",
"translation": "Leukocytes"
},
{
"locale": "vn",
"translation": "Leukocytes vn"
}
],
"status": "active",
"category": {
"id": 8,
"name": [
{
"locale": "en",
"translation": "Hematology"
},
{
"locale": "vn",
"translation": "huyết học"
}
]
},
"children": []
}]}
and
products/_source/2
{
"product_id": "6690-2",
"name": [
{
"locale": "en",
"translation": "Leukocytes"
},
{
"locale": "vn",
"translation": "Leukocytes vn"
}
],
"status": "active",
"category": {
"id": 8,
"name": [
{
"locale": "en",
"translation": "Hematology"
},
{
"locale": "vn",
"translation": "huyết học"
}
]
},
"children": []
}
where a product is a single document but also can be nested in a children array of other products. Both products are different documents in the index.
and this index:
{
"products": {
"aliases": {},
"mappings": {
"dynamic": "false",
"properties": {
"category": {
"properties": {
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
}
}
},
"children": {
"type": "nested"
},
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
},
"product_id": {
"type": "keyword"
},
"status": {
"type": "keyword"
}
}
},
"settings": {
"index": {
"routing": {
"allocation": {
"include": {
"_tier_preference": "data_content"
}
}
},
"number_of_shards": "3",
"provided_name": "products",
"number_of_replicas": "1"
}
}
}
}
I want to be able to query for "Leuko" (or the category or the product_id) and retrieve both products, the single product and the root product.
I have tried using object field, nested, flattened but I think the problem is I don't know how to properly write the query, I have tried things like this (I am using a ruby library but I think it is easy to follow):
#query = {
query: {
query_string: {
fields: ['name.translation', 'children.name.translation', 'category.name.translation', 'children.product_id'],
query: "*#{text}*"
}
},
size: 50
}
#query = {
query: {
nested: {
path: 'children',
query: {
bool: {
should: [
term: { 'children.name.translation' => "*#{text}*" },
term: { 'name.translation' => "*#{text}*" }
]
}
}
}
}
}
but I think at some point I dunno what I am doing anymore and I am just randomly trying different stuff from the documentation.
Follow my query suggestion. Note that I had to add the fields in the Nested object to the mapping.
Mapping:
{
"mappings": {
"dynamic": "false",
"properties": {
"category": {
"properties": {
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
}
}
},
"children": {
"type": "nested",
"properties": {
"product_id": {
"type": "keyword"
},
"category": {
"properties": {
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
}
}
},
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
},
"status": {
"type": "keyword"
}
}
},
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
},
"product_id": {
"type": "keyword"
},
"status": {
"type": "keyword"
}
}
}
}
Query:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"nested": {
"path": "children",
"query": {
"wildcard": {
"children.name.translation": "leuko*"
}
}
}
},
{
"wildcard": {
"name.translation": "leuko*"
}
}
]
}
}
}
hint
See that you use translation. Avoid using array to make your queries simpler.
What I would do in your case is to create a field for each language, this makes the use of analyzer more flexible for each type of language and you stop using an array and work with an object.
PUT test
{
"mappings": {
"properties": {
"name":{
"type": "text",
"fields": {
"es":{
"type": "text",
"analyzer":"english"
},
"vn":{
"type": "text"
}
}
}
}
}
}
POST test/_doc/
{
"name": "Leukocytes"
}
An example query using field languages.
GET test/_search
{
"query": {
"multi_match": {
"query": "Leukocytes",
"fields": ["name.es", "name.vn"]
}
}
}
I have an elasticsearch index and am using the following query:
"_source": [
"title",
"content"
],
"size": 15,
"from": 0,
"query": {
"bool": {
"must": {
"multi_match": {
"query": "{{query}}",
"fields": [
"title",
"content"
],
"operator": "or"
}
},
"should": [
{
"multi_match": {
"query": "{{query}}",
"fields": [
"title.standard^16",
"content.standard^2"
],
"operator": "and"
}
},
{
"match_phrase": {
"content.standard": {
"query": "{{query}}",
"_name": "Phrase on title",
"boost": 1000
}
}
}
]
}
},
"highlight": {
"fields": {
"content": {}
},
"fragment_size": 100
}
}
Here is the mapping I set:
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "standard",
"filter": [
"lowercase",
"my_metaphone"
]
}
},
"filter": {
"my_metaphone": {
"type": "phonetic",
"encoder": "metaphone",
"replace": true
}
}
}
}
},
"mappings": {
"properties": {
"title": {
"type": "text",
"term_vector": "with_positions_offsets",
"analyzer": "my_analyzer",
"fields": {
"standard": {
"type": "text"
},
"stemmer": {
"type": "text",
"analyzer": "english"
}
}
},
"content": {
"type": "text",
"term_vector": "with_positions_offsets",
"analyzer": "my_analyzer",
"fields": {
"standard": {
"type": "text"
},
"stemmer": {
"type": "text",
"analyzer": "english"
}
}
}
}
}
}
Here is my logic with the query:
1) It will give the highest precedence to a phrase if it appears.
2) If not it will use the standard analyzer (that is the text, as is) and give it the highest precedence.
3) If all else doesn't match up, it will use the phonetic analyzer to get the results, that is the least precedence.
But obviously there is some fault to this as it seems to give higher precedence to the phonetic analyzer than the standard or phrase. For example, if I search for "Person of Indian Origin" it returns results on the top highlighting "Pursuant" "pursuing" and very, very less number of results with person of Indian origin although I know a large number of them exists. How do I solve this?
I looked over other answers that was having the same problem, but those answers don't seem to resolve this problem. How can I get the request to include the Author name? Although, this request returns Author in the response, it doesn't have the actual author details.
Am I missing something on the request or does it need some configuration adjustments?
Setup
We have ADFS configured with Sharepoint 2013 on Premise.
Endpoint I'm hitting
https://SHAREPOINTURL/_api/web/GetFolderByServerRelativeUrl('Documents')?$select=Author/Id,Author/Name,Author/Title,Editor/Id,Editor/Name,Editor/Title,*&$expand=Files/Author,Editor
Response
{
"d": {
"__metadata": {
"id": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')",
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')",
"type": "SP.Folder"
},
"Files": {
"results": [
{
"__metadata": {
"id": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')",
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')",
"type": "SP.File"
},
"Author": {
"__metadata": {
"id": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/Author",
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/Author",
"type": "SP.User"
},
"Groups": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/Author/Groups"
}
},
"Id": 1073741823,
"IsHiddenInUI": false,
"LoginName": "SHAREPOINT\\system",
"Title": "System Account",
"PrincipalType": 1,
"Email": "",
"IsSiteAdmin": false,
"UserId": {
"__metadata": {
"type": "SP.UserIdInfo"
},
"NameId": "S-1-0-0",
"NameIdIssuer": "urn:office:idp:activedirectory"
}
},
"CheckedOutByUser": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/CheckedOutByUser"
}
},
"ListItemAllFields": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/ListItemAllFields"
}
},
"LockedByUser": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/LockedByUser"
}
},
"ModifiedBy": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/ModifiedBy"
}
},
"Versions": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/Versions"
}
},
"CheckInComment": "",
"CheckOutType": 2,
"ContentTag": "{71501108-7ACC-46F6-82D7-33E5C5F0124C},3,4",
"CustomizedPageStatus": 0,
"ETag": "\"{71501108-7ACC-46F6-82D7-33E5C5F0124C},3\"",
"Exists": true,
"Length": "390144",
"Level": 1,
"MajorVersion": 1,
"MinorVersion": 0,
"Name": "FILENAME.xls",
"ServerRelativeUrl": "Documents/FILENAME.xls",
"TimeCreated": "2013-07-10T13:55:39Z",
"TimeLastModified": "2013-07-10T13:55:39Z",
"Title": "",
"UIVersion": 512,
"UIVersionLabel": "1.0"
}
]
},
"ListItemAllFields": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')/ListItemAllFields"
}
},
"ParentFolder": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')/ParentFolder"
}
},
"Properties": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')/Properties"
}
},
"Folders": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')/Folders"
}
},
"ItemCount": 18,
"Name": "Documents",
"ServerRelativeUrl": "Documents",
"WelcomePage": ""
}
}
Try this. I found this on the MSDN forum.
https://social.msdn.microsoft.com/Forums/office/en-US/04fc252c-adb9-4f50-b9b0-c326f88ad69e/retriving-author-name-using-rest-api?forum=appsforsharepoint
----- update
https://{SITE_URL}/_api/web/Lists/getbytitle('{DOCUMENT_PATH}')/items?$select=Title,Author/ID,Author/FirstName,Author/LastName,Author/Title,Author/Department,Author/EMail&$expand=Author/ID
i'm using elasticsearch-rails for a project, there is a combine search feature, all columns are in one table. I just write a custom search function, and the search dsl didn't work, can't have any results.
def self.combine_search_filter(remark=nil, sim_card_supplier_id=nil, work_mode=nil, operator_status=nil, platform_management_status=nil, online_status=nil, sim_card_set_id=nil, iccid_from=nil, iccid_to=nil, actived_at_from=nil, actived_at_to=nil, check_in_at_from=nil, check_in_at_to=nil, device_mac_from=nil, device_mac_to=nil)
response = __elasticsearch__.search(
"size": 1000,
"query": {
"filtered": {
"filter": {
"bool": {
"filter": [
{ "term": { "sim_card_supplier_id": sim_card_supplier_id } },
{ "term": { "work_mode": work_mode } },
{ "term": { "operator_status": operator_status } },
{ "term": { "platform_management_status": platform_management_status } },
{ "term": { "online_status": online_status } },
{ "term": { "sim_card_set_id": sim_card_supplier_id } }
{ "range": { "iccid": { "from": iccid_from, "to": iccid_to }}},
{ "range": { "check_in_at": { "from": check_in_at_from, "to": check_in_at_to }}},
{ "range": { "actived_at": { "from": actived_at_from, "to": actived_at_to }}},
{ "range": { "device_mac": { "from": device_mac_from, "to": device_mac_to }}}
]
}
}
}
}
)
end
and the params maybe pass nil, how can i do that make the search dsl valid?
I would rather suggest you to use searchkick gem for rails which makes elastic search as simple as that and makes your search more intelligent and queries simple.
My bad, i just figure this out, ruby keyword function should be like this:
def self.combine_search_filter(options = {})
response = __elasticsearch__.search(
"size": 1000,
"query": {
"filtered": {
"filter": {
"bool": {
"filter": [
{ "term": { "sim_card_supplier_id": 104 } },
{ "term": { "work_mode": options[:work_mode] } },
{ "term": { "operator_status": options[:operator_status] } },
{ "term": { "platform_management_status": options[:platform_management_status] } },
{ "term": { "online_status": options[:online_status] } },
{ "term": { "sim_card_set_id": 76 } },
{ "range": { "iccid": { "from": options[:iccid_from], "to": options[:iccid_to] }}},
{ "range": { "check_in_at": { "from": options[:check_in_at_from], "to": options[:check_in_at_to] }}},
{ "range": { "actived_at": { "from": options[:actived_at_from], "to": options[:actived_at_to] }}},
{ "range": { "device_mac": { "from": options[:device_mac_from], "to": options[:device_mac_to] }}}
]
}
}
}
}
)
end
this search will have the correct result. The options[:key] may be nil will cause the wrong result, so i'm gonna delete the nil key of the dsl hash, then pass it to search()
I have this scenario wherein there are two multi_match searches within the same query. The trouble is, when I create the JSON for it in ruby, a json with non-unique keys doesn't seem possible so only one of them appear.
Here is my query:
{
"fields": ["id", "title",
"address.city", "address.state", "address.country", "address.state_code", "address.country_code", "proxy_titles", "location"],
"size":2,
"query":{
"filtered":{
"filter": {
"range": {
"custom_score": {
"gte": 100
}
}
},
"query":{
"bool": {
"must": {
"multi_match":{
"query": "term 1",
"type": "cross_fields",
"fields": ["title^2", "proxy_titles^2","description"]
}
},
"must": {
"multi_match": {
"query": "us",
"fields": ["address.city", "address.country", "address.state",
"address.zone", "address.country_code", "address.state_code", "address.zone_code"]
}
}
}
}
}
},
"sort": {
"_score": { "order": "desc" },
"variation": {"order": "asc"},
"updated_at": { "order": "desc" }
}
}
I have also only recently started using elasticsearch so it be very helpful if you could suggest me a better query to accomplish the same as well.
You have the syntax wrong. For multiple "must" values in a "bool", they need to be in an array. The documentation is not always terribly helpful, unfortunately (the bool query page shows this for "should" but not "must").
Try this:
{
"fields": ["id","title","address.city","address.state","address.country","address.state_code","address.country_code","proxy_titles","location"],
"size": 2,
"query": {
"filtered": {
"filter": {
"range": {
"custom_score": {
"gte": 100
}
}
},
"query": {
"bool": [
{
"must": {
"multi_match": {
"query": "term 1",
"type": "cross_fields",
"fields": ["title^2","proxy_titles^2","description"]
}
}
},
{
"must": {
"multi_match": {
"query": "us",
"fields": ["address.city","address.country","address.state","address.zone","address.country_code","address.state_code","address.zone_code"]
}
}
}
]
}
}
},
"sort": {
"_score": {
"order": "desc"
},
"variation": {
"order": "asc"
},
"updated_at": {
"order": "desc"
}
}
}