Grails - ElasticSearch - QueryParsingException[[index] No query registered for [query]]; with elasticSearchHelper; JSON via curl works fine though - grails

I have been working on a Grails project, clubbed with ElasticSearch ( v 20.6 ), with a custom build of elasticsearch-grails-plugin(to support geo_point indexing : v.20.6)
have been trying to do a filtered Search, while using script_fields (to calculate distance). Following is Closure & the generated JSON from the GXContentBuilder :
Closure
records = Domain.search(searchType:'dfs_query_and_fetch'){
query {
filtered = {
query = {
if(queryTxt){
query_string(query: queryTxt)
}else{
match_all {}
}
}
filter = {
geo_distance = {
distance = "${userDistance}km"
"location"{
lat = latlon[0]?:0.00
lon = latlon[1]?:0.00
}
}
}
}
}
script_fields = {
distance = {
script = "doc['location'].arcDistanceInKm($latlon)"
}
}
fields = ["_source"]
}
GXContentBuilder generated query JSON :
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "5km",
"location": {
"lat": "37.752258",
"lon": "-121.949886"
}
}
}
}
},
"script_fields": {
"distance": {
"script": "doc['location'].arcDistanceInKm(37.752258, -121.949886)"
}
},
"fields": ["_source"]
}
The JSON query, using curl-way, works perfectly fine. But when I try to execute it from Groovy Code, I mean with this (taken from ElasticSearchService.groovy) where request is SearchRequest instance :
elasticSearchHelper.withElasticSearch { Client client ->
def response = client.search(request).actionGet()
}
It throws following error :
Failed to execute phase [dfs], total failure; shardFailures {[1][index][3]: SearchParseException[[index][3]: from[0],size[60]: Parse Failure [Failed to parse source [{"from":0,"size":60,"query_binary":"eyJxdWVyeSI6eyJmaWx0ZXJlZCI6eyJxdWVyeSI6eyJtYXRjaF9hbGwiOnt9fSwiZmlsdGVyIjp7Imdlb19kaXN0YW5jZSI6eyJkaXN0YW5jZSI6IjVrbSIsImNvbXBhbnkuYWRkcmVzcy5sb2NhdGlvbiI6eyJsYXQiOiIzNy43NTIyNTgiLCJsb24iOiItMTIxLjk0OTg4NiJ9fX19fSwic2NyaXB0X2ZpZWxkcyI6eyJkaXN0YW5jZSI6eyJzY3JpcHQiOiJkb2NbJ2NvbXBhbnkuYWRkcmVzcy5sb2NhdGlvbiddLmFyY0Rpc3RhbmNlSW5LbSgzNy43NTIyNTgsIC0xMjEuOTQ5ODg2KSJ9fSwiZmllbGRzIjpbIl9zb3VyY2UiXX0=","explain":true}]]]; nested: QueryParsingException[[index] No query registered for [query]]; }
The above Closure works if I only use filtered = { ... } script_fields = { ... } but it doesn't return the calculated distance.
Anyone had any similar problem ?
Thanks in advance :)
It's possible that I might have been dim to point out the obvious here :P

Related

keep getting this error what is the proper syntax fornConditionType 'NUMBER_BETWEEN' requires exactly two ConditionValues, but 1 value was supplied

I am trying to use the conditional filter to check a range between two numbers using google sheets api I keep getting this syntax error how do I fix it please here is my code:
const addFilterViewRequest = [
{
'addFilterView' : {
filter : {
title : 'PO_Log Precentage',
range : {
sheetId : 1701531392,
'startRowIndex': 1,
'startColumnIndex':1
},
'criteria': {
19:{
'condition': {
'type': 'NUMBER_BETWEEN',
'values':[
{
"userEnteredValue" : "80"
}
],
}
}
}
}
}
}
]
Sheets.Spreadsheets.batchUpdate({ requests: addFilterViewRequest },spreadsheet_id);
}
ConditionType
NUMBER_BETWEEN
The cell's value must be between the two condition values. Supported by data validation, conditional formatting and filters. Requires exactly two ConditionValues .
'values':[
{
"userEnteredValue" : "0"
},
{
"userEnteredValue" : "80"
}
],

Not able to retrieve the spreadsheet id from workspace add-on

I'm developing a workspace add-on with alternate runtime; I configured the add-on to work with spreadsheets and I need to retrieve the spreadsheet id when the user opens the add-on. For test purposes I created a cloud function that contains the business logic.
My deployment.json file is the following:
{
"oauthScopes": ["https://www.googleapis.com/auth/spreadsheets.currentonly", "https://www.googleapis.com/auth/drive.file"],
"addOns": {
"common": {
"name": "My Spreadsheet Add-on",
"logoUrl": "https://cdn.icon-icons.com/icons2/2070/PNG/512/penguin_icon_126624.png"
},
"sheets": {
"homepageTrigger": {
"runFunction": "cloudFunctionUrl"
}
}
}
}
However, the request I receive seems to be empty and without the id of the spreadsheet in which I am, while I was expecting to have the spreadsheet id as per documentation
Is there anything else I need to configure?
The relevant code is quite easy, I'm just printing the request:
exports.getSpreadsheetId = function addonsHomePage (req, res) { console.log('called', req.method); console.log('body', req.body); res.send(createAction()); };
the information showed in the log is:
sheets: {}
Thank you
UPDATE It's a known issue of the engineering team, here you can find the ticket
The information around Workspace Add-ons is pretty new and the documentation is pretty sparse.
In case anyone else comes across this issue ... I solved it in python using CloudRun by creating a button that checks for for the object then if there is no object it requests access to the sheet in question.
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route('/', methods=['POST'])
def test_addon_homepage():
req_body = request.get_json()
sheet_info = req_body.get('sheets')
card = {
"action": {
"navigations": [
{
"pushCard": {
"sections": [
{
"widgets": [
{
"textParagraph": {
"text": f"Hello {sheet_info.get('title','Auth Needed')}!"
}
}
]
}
]
}
}
]
}
}
if not sheet_info:
card = create_file_auth_button(card)
return card
def create_file_auth_button(self, card):
card['action']['navigations'][0]['pushCard']['fixedFooter'] = {
'primaryButton': {
'text': 'Authorize file access',
'onClick': {
'action': {
'function': 'https://example-cloudrun.a.run.app/authorize_sheet'
}
}
}
}
return card
#app.route('/authorize_sheet', methods=['POST'])
def authorize_sheet():
payload = {
'renderActions': {
'hostAppAction': {
'editorAction': {
'requestFileScopeForActiveDocument': {}
}
}
}
}
return payload

Accessing a specific item in a JSON return in Rails

I have this JSON from the NHL API, and I tried to access the value gamesPlayed:
"teams"=>[
{
"id"=>5,
"name"=>"Pittsburgh Penguins",
"link"=>"/api/v1/teams/5",
"venue"=>{
"id"=>5034,
"name"=>"PPG Paints Arena",
"link"=>"/api/v1/venues/5034",
"city"=>"Pittsburgh",
"timeZone"=>{
"id"=>"America/New_York",
"offset"=>-5,
"tz"=>"EST"
}
},
"abbreviation"=>"PIT",
"teamName"=>"Penguins",
"locationName"=>"Pittsburgh",
"division"=>{
"id"=>18,
"name"=>"Metropolitan",
"nameShort"=>"Metro",
"link"=>"/api/v1/divisions/18",
"abbreviation"=>"M"
},
"conference"=>{
"id"=>6,
"name"=>"Eastern",
"link"=>"/api/v1/conferences/6"
},
"franchise"=>{
"franchiseId"=>17,
"teamName"=>"Penguins",
"link"=>"/api/v1/franchises/17"
},
"teamStats"=>[
{
"type"=>{
"displayName"=>"statsSingleSeason"
},
"splits"=>[
{
"stat"=>{
"gamesPlayed"=>16,
"wins"=>7,
"losses"=>6,
"ot"=>3,
"pts"=>17,
"ptPctg"=>"53.1",
"goalsPerGame"=>3.313,
"goalsAgainstPerGame"=>3.063,
"evGGARatio"=>1.0833,
"powerPlayPercentage"=>"23.4",
"powerPlayGoals"=>11.0,
"powerPlayGoalsAgainst"=>8.0,
"powerPlayOpportunities"=>47.0,
"penaltyKillPercentage"=>"84.0",
"shotsPerGame"=>32.625,
"shotsAllowed"=>33.6875,
"winScoreFirst"=>0.6,
"winOppScoreFirst"=>0.167,
"winLeadFirstPer"=>0.5,
"winLeadSecondPer"=>1.0,
"winOutshootOpp"=>0.333,
"winOutshotByOpp"=>0.444,
"faceOffsTaken"=>1035.0,
"faceOffsWon"=>534.0,
"faceOffsLost"=>501.0,
"faceOffWinPercentage"=>"51.6",
"shootingPctg"=>10.2,
"savePctg"=>0.909
},
"team"=>{
"id"=>5,
"name"=>"Pittsburgh Penguins",
"link"=>"/api/v1/teams/5"
}
},
{
"stat"=>{
"wins"=>"24th",
"losses"=>"15th",
"ot"=>"9th",
"pts"=>"24th",
"ptPctg"=>"19th",
"goalsPerGame"=>"8th",
"goalsAgainstPerGame"=>"19th",
"evGGARatio"=>"11th",
"powerPlayPercentage"=>"10th",
"powerPlayGoals"=>"22nd",
"powerPlayGoalsAgainst"=>"4th",
"powerPlayOpportunities"=>"31st",
"penaltyKillOpportunities"=>"1st",
"penaltyKillPercentage"=>"6th",
"shotsPerGame"=>"12th",
"shotsAllowed"=>"27th",
"winScoreFirst"=>"15th",
"winOppScoreFirst"=>"27th",
"winLeadFirstPer"=>"27th",
"winLeadSecondPer"=>"7th",
"winOutshootOpp"=>"25th",
"winOutshotByOpp"=>"25th",
"faceOffsTaken"=>"25th",
"faceOffsWon"=>"19th",
"faceOffsLost"=>"6th",
"faceOffWinPercentage"=>"8th",
"savePctRank"=>"13th",
"shootingPctRank"=>"12th"
},
"team"=>{
"id"=>5,
"name"=>"Pittsburgh Penguins",
"link"=>"/api/v1/teams/5"
}
}
]
}
],
"shortName"=>"Pittsburgh",
"officialSiteUrl"=>"http://pittsburghpenguins.com/",
"franchiseId"=>17,
"active"=>true
}
}
I am working in ruby on rails and would like to access the gamesPlayed value.
So far I have:
url = 'https://statsapi.web.nhl.com/api/v1/teams/5?expand=team.stats'
uri = URI(url)
response = Net::HTTP.get(uri)
response = JSON.parse(response)
#awayteamgamesplayed = response["teams"][0]["teamStats"]["stat"]["gamesPlayed"]
I can get to the team name using: response["teams"][away_team]["name"] but cant't work out gamesPlayed.
But it doesn't seem to work for gamesPlayed.
Value of teamStats is an Array. You need to access it via an index.
Same for splits
response["teams"][0]["teamStats"][0]["splits"][0]["stat"]["gamesPlayed"]
# => 16
teamStats is an array try this
response["teams"][0]["teamStats"][0]["stat"]["gamesPlayed"]

get distinct values of string field using elasticsearch query Client java API

I'm using elasticsearch 5.4
I need to get distinct values of fields using elasticsearch query
I'm trying this query but is not wokring
GET /index/index_type/_search
{
"size": 0,
"aggs": {
"distinct": {
"terms": {
"field": "status"
}
}
}
thank you
solution with client java api elasticsearch 5.4
import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.aggregations.bucket.terms.Terms;
TransportClient client;
SearchResponse response = client.prepareSearch(indexName).setTypes(typeName)
.addAggregation(AggregationBuilders.terms("distinct_line_status").field("line_status.keyword"))
.setSize(0).get();
//AggregationProfileShardResult searchHits = response.getAggregations().getAsMap();
System.out.println(response);
Terms genders = response.getAggregations().get("distinct_line_status");
// For each entry
for (Terms.Bucket entry : genders.getBuckets()) {
System.out.println(entry.getKey()); // Term
System.out.println(entry.getDocCount()); // Doc count
}
You can use cardinality to achieve this.
GET /index/index_type/_search
{
"size": 0,
"aggs": {
"distinct": {
"cardinality": {
"field": "status"
}
}
}

Elasticsearch validate API explain query terms from more like this against single field getting highlighted terms

I have an index, with effectively the converted word or pdf document plain text "document_texts", built on a Rails stack the ActiveModel is DocumentText using the elasticsearch rails gems, for model, and API. I want to be able to match similar word documents or pdf's based on the document text
I have been able to match documents against each other by using
response = DocumentText.search \
query: {
filtered: {
query: {
more_like_this: {
ids: ["12345"]
}
}
}
}
But I want to see HOW did the result set get queried, what were the query terms used to match the documents
Using the elasticsearch API gem I can do the following
client=Elasticsearch::Client.new log:true
client.indices.validate_query index: 'document_texts',
explain: true,
body: {
query: {
filtered: {
query: {
more_like_this: {
ids: ['12345']
}
}
}
}
}
But I get this in response
{"valid":true,"_shards":{"total":1,"successful":1,"failed":0},"explanations":[{"index":"document_texts","valid":true,"explanation":"+(like:null -_uid:document_text#12345)"}]}
I would like to find out how did the query get built, it uses upto 25 terms for the matching, what were those 25 terms and how can I get them from the query?
I'm not sure if its possible but I would like to see if I can get the 25 terms used by elasticsearches analyzer and then reapply the query with boosted values on the terms depending on my choice.
I also want to highlight this in the document text but tried this
response = DocumentText.search \
from: 0, size: 25,
query: {
filtered: {
query: {
more_like_this: {
ids: ["12345"]
}
},
filter: {
bool: {
must: [
{match: { documentable_type: model}}
]
}
}
}
},
highlight: {
pre_tags: ["<tag1>"],
post_tags: ["</tag1>"],
fields: {
doc_text: {
type_name: {
content: {term_vector: "with_positions_offsets"}
}
}
}
}
But this fails to produce anything, I think I was being rather hopeful. I know that this should be possible but would be keen to know if anyone has done this or the best approach. Any ideas?
Including some stop words just for anyone else out there this will give an easy way for it to show the terms used for the query. It doesnt solve the highlight issue but can give the terms used for the mlt matching process. Some other settings are used just to show
curl -XGET 'http://localhost:9200/document_texts/document_text/_validate/query?rewrite=true' -d '
{
"query": {
"filtered": {
"query": {
"more_like_this": {
"ids": ["12345"],
"min_term_freq": 1,
"max_query_terms": 50,
"stop_words": ["this","of"]
}
}
}
}
}'
https://github.com/elastic/elasticsearch-ruby/pull/359
Once this is merged this should be easier
client.indices.validate_query index: 'document_texts',
rewrite: true,
explain: true,
body: {
query: {
filtered: {
query: {
more_like_this: {
ids: ['10538']
}
}
}
}
}

Resources