I am having several experiments a day storing the error of the experiment and a boolean value (if the result is ok) in elasticsearch.
Now, I would like to display the results in a graph (using highchart js).
I use an aggregation query like this to receive the aggregated errors for each day including the standard deviation:
query: {
filtered: {
filter: {
range : {
date: {
"gte":"2015-1-1",
"lte": "2016-1-1,
"time_zone": "+1:00"
}
}
}
}
},
// Aggregate on the results
aggs: {
group_by_date: {
terms:{
field:"date",
order: {_term:"asc"}
},
aggs:{
error_stats:{
extended_stats:{
field:"error"
}
}
}
}
}
The problem I face is that I cannot retrieve the boolean values the same way as I get the double errors from the DB.
When I just change the field name to "ok" in
aggs:{
error_stats:{
extended_stats:{
field:"ok"
}
}
}
I receive this error message:
ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData
However, it would be OK to aggreate all the boolean values usign true as 1 and false as zero and then to receive a mean value for each day.
Can anyone help me with this?
Thanks alot!
First 0/1 representation is not exactly ES Boolean representation. There is a Boolean type for as true/false.
Second stats aggregation can be only done on numeric field and not on string field.
That is why it worked for 0/1 representation.
You can transform this value using scripts in extended stats
{
"aggs" : {
...
"aggs" : {
"grades_stats" : {
"extended_stats" : {
"field" : "grade",
"script" : "_value == 'T' ? 1 : 0",
}
}
}
}
}
To see some example usage of scripting in aggregation , you can look here.
Related
I am trying to use the conditional filter to check a range between two numbers using google sheets api I keep getting this syntax error how do I fix it please here is my code:
const addFilterViewRequest = [
{
'addFilterView' : {
filter : {
title : 'PO_Log Precentage',
range : {
sheetId : 1701531392,
'startRowIndex': 1,
'startColumnIndex':1
},
'criteria': {
19:{
'condition': {
'type': 'NUMBER_BETWEEN',
'values':[
{
"userEnteredValue" : "80"
}
],
}
}
}
}
}
}
]
Sheets.Spreadsheets.batchUpdate({ requests: addFilterViewRequest },spreadsheet_id);
}
ConditionType
NUMBER_BETWEEN
The cell's value must be between the two condition values. Supported by data validation, conditional formatting and filters. Requires exactly two ConditionValues .
'values':[
{
"userEnteredValue" : "0"
},
{
"userEnteredValue" : "80"
}
],
I have tried applying a filter in Sensu, from referring Sensu document, for reducing our monitoring alerts. But problem is, this filter is sending email only when issue is in Resolved state, no critical or warning alerts. Filter looks like :
{
"filters": {
"state_change_only": {
"negate": false,
"attributes": {
"occurrences": "eval: value == 1 || ':::action:::' == 'resolve'"
}
}
}
}
Please help me in understanding this behaviour and what this eval: value == 1 actually means here.
Does it have to do anything with occurrences value? I have values like 2,3 etc. depending on the severity of the checks.
I'm using the following function score for outfits purchased:
{
"query": {
"function_score": {
"field_value_factor": {
"field": "purchased",
"factor": 1.2,
"modifier": "sqrt",
"missing": 1
}
}
}
}
However, when I create a search - I get the following error:
"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [purchased] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
The syntax is correct for the search as I've run it locally and it works perfectly. I'm now running it on my server and it's not workings. Do I need to define purchased as an integer somewhere or is this due to something else?
The purchased field is an analyzed string field, hence the error you see.
When indexing your documents, make sure that the numbers are not within double quotes, i.e.
Wrong:
{
"purchased": "324"
}
Right:
{
"purchased": 324
}
...or if you can't change the source documents (because you're not responsible for producing them), make sure that you create a mapping that defines the purchased field as being an integer field.
{
"your_type": {
"properties": {
"purchased": {
"type": "integer"
}
}
}
}
I'm attempting to learn Relay by implementing TodoMVC from scratch.
I can query my data like this which is working well:
query {
allTodos(first: 100) {
totalCount
completedCount
edges {
node {
id
text
completed
}
}
}
}
I got the idea to add the totalCount and completedCount metadata to the connection from here: http://graphql.org/learn/pagination/#end-of-list-counts-and-connections
It's similar in this example: https://github.com/graphql/swapi-graphql/blob/master/src/schema/index.js#L78
Now I am writing a mutation to change the completed field of a Todo given its id.
I gather I will need to return the new completedCount in the mutation payload, but I'm not sure how to implement getConfigs() to update this in the client-side store. I don't have an id for the connection, right? Is there is a flaw in my schema design? Thanks!
Assuming your mutation returns a viewer, you'll need to add the viewer to your fatQuery and getConfigs. I think this tutorial might be helpful. Here's the excerpt relevant to your task:
Adding a Todo is more complex. The reason for this is that we need to
update not only the state of a Todo object that we will create, but
also a connection where it is stored - the count of Todos will change,
as well as the listing of Todo nodes in edges.
import Relay from 'react-relay';
export default class AddTodoMutation extends Relay.Mutation {
static fragments = {
viewer: () => Relay.QL`fragment on ReindexViewer {
id
allTodos {
count,
}
}`
};
getMutation() {
return Relay.QL`mutation{ createTodo }`;
}
getVariables() {
return {
text: this.props.text,
complete: false,
};
}
getFatQuery() {
return Relay.QL`
fragment on _TodoPayload {
changedTodoEdge,
viewer {
id,
allTodos {
count
}
}
}
`;
}
getConfigs() {
return [{
type: 'RANGE_ADD',
parentID: this.props.viewer.id,
connectionName: 'allTodos',
edgeName: 'changedTodoEdge',
rangeBehaviors: {
'': 'prepend',
},
}, {
type: 'FIELDS_CHANGE',
fieldIDs: {
viewer: this.props.viewer.id,
},
}];
}
getOptimisticResponse() {
return {
changedTodoEdge: {
node: {
text: this.props.text,
complete: false,
},
},
viewer: {
id: this.props.viewer.id,
allTodos: {
count: this.props.viewer.allTodos.count + 1,
},
},
};
}
}
In order to perform this mutation, we need some data that might not be
available to the component - the id of viewer object and count of
allTodos connection. Therefore we need to specify fragments for the
mutation same way as we specify them for containers.
Our configs are more complex this time too - we need to add our new
Todo to a connection, so we use RANGE_ADD mutation config. Relay
expects an edge to be passed in payload, not just a Todo, Reindex
provides changedTodoEdge for this. Lastly we need to fetch updated
connection count from the server and for this viewer field is
available for every payload.
In our optimistic update we increment the count of allTodos, so that
we change our “total” display without any delay.
I have an index, with effectively the converted word or pdf document plain text "document_texts", built on a Rails stack the ActiveModel is DocumentText using the elasticsearch rails gems, for model, and API. I want to be able to match similar word documents or pdf's based on the document text
I have been able to match documents against each other by using
response = DocumentText.search \
query: {
filtered: {
query: {
more_like_this: {
ids: ["12345"]
}
}
}
}
But I want to see HOW did the result set get queried, what were the query terms used to match the documents
Using the elasticsearch API gem I can do the following
client=Elasticsearch::Client.new log:true
client.indices.validate_query index: 'document_texts',
explain: true,
body: {
query: {
filtered: {
query: {
more_like_this: {
ids: ['12345']
}
}
}
}
}
But I get this in response
{"valid":true,"_shards":{"total":1,"successful":1,"failed":0},"explanations":[{"index":"document_texts","valid":true,"explanation":"+(like:null -_uid:document_text#12345)"}]}
I would like to find out how did the query get built, it uses upto 25 terms for the matching, what were those 25 terms and how can I get them from the query?
I'm not sure if its possible but I would like to see if I can get the 25 terms used by elasticsearches analyzer and then reapply the query with boosted values on the terms depending on my choice.
I also want to highlight this in the document text but tried this
response = DocumentText.search \
from: 0, size: 25,
query: {
filtered: {
query: {
more_like_this: {
ids: ["12345"]
}
},
filter: {
bool: {
must: [
{match: { documentable_type: model}}
]
}
}
}
},
highlight: {
pre_tags: ["<tag1>"],
post_tags: ["</tag1>"],
fields: {
doc_text: {
type_name: {
content: {term_vector: "with_positions_offsets"}
}
}
}
}
But this fails to produce anything, I think I was being rather hopeful. I know that this should be possible but would be keen to know if anyone has done this or the best approach. Any ideas?
Including some stop words just for anyone else out there this will give an easy way for it to show the terms used for the query. It doesnt solve the highlight issue but can give the terms used for the mlt matching process. Some other settings are used just to show
curl -XGET 'http://localhost:9200/document_texts/document_text/_validate/query?rewrite=true' -d '
{
"query": {
"filtered": {
"query": {
"more_like_this": {
"ids": ["12345"],
"min_term_freq": 1,
"max_query_terms": 50,
"stop_words": ["this","of"]
}
}
}
}
}'
https://github.com/elastic/elasticsearch-ruby/pull/359
Once this is merged this should be easier
client.indices.validate_query index: 'document_texts',
rewrite: true,
explain: true,
body: {
query: {
filtered: {
query: {
more_like_this: {
ids: ['10538']
}
}
}
}
}