After upgrading to the spring-boot-starter:2.2.5.RELEASE, spring-cloud-dependencies:Hoxton.SR3, spring-cloud-stream-dependencies:Horsham.SR3 & spring-data-elasticsearch:3.2.5.RELEASE. The boolean fields are not stored in index/document. It was working earlier with spring boot 2.1.11.
I tried to create document manually using the ElasticSearch REST API. When tried directly with REST API, the boolean fields are stored in index.
Is there any changes done how the mappings are declared for boolean fields?
I'm using the ElasticsearchTemplate.index(IndexQuery) API to create an index document, where the IndexQuery is built with an document object having some boolean fields.
The following are the boolean fields in the CatalogIndex.java file.
#Document(indexName = "catalogentity")
public class CatalogIndex {
private boolean isType;
private boolean isAbstract;
private boolean isFinal;
private String stateId;
private String stageId;
//some other fields
public boolean isType() {
return isType;
}
public void setType(final boolean type) {
isType = type;
}
public boolean isAbstract() {
return isAbstract;
}
public void setAbstract(final boolean anAbstract) {
isAbstract = anAbstract;
}
public boolean isFinal() {
return isFinal;
}
public void setFinal(final boolean aFinal) {
isFinal = aFinal;
}
//some other setter and getters
The mappings are as follows
{
"properties": {
"type": {
"type": "boolean"
},
"abstract": {
"type": "boolean"
},
"final": {
"type": "boolean"
},
"stateId": {
"type": "text"
},
"stageId": {
"type": "keyword"
}
}
}
Thanks in advance,
Santhosh
The following is the working configuration for boolean fields. I was not sure why it was working fine before upgrade.
The following are the boolean fields in the CatalogIndex.java file.
#Document(indexName = "catalogentity")
public class CatalogIndex {
private boolean isType;
private boolean abstract1;
private boolean final1;
private String stateId;
private String stageId;
//some other fields
public boolean isType() {
return isType;
}
public void setType(final boolean type) {
isType = type;
}
public boolean isAbstract1() {
return abstract1;
}
public void setAbstract1(final boolean abstract1) {
this.abstract1 = abstract1;
}
public boolean isFinal1() {
return final1;
}
public void setFinal1(final boolean final1) {
this.final1 = final1;
}
//some other setter and getters
The mappings are as follows
{
"properties": {
"type": {
"type": "boolean"
},
"abstract1": {
"type": "boolean"
},
"final1": {
"type": "boolean"
},
"stateId": {
"type": "text"
},
"stageId": {
"type": "keyword"
}
}
}
Related
Here is my Dto for percolator query class.
#Data
#Document(indexName = "#{#es.indexName}")
#Builder(builderClassName = "RuleBuilder")
public class Rule {
#Id
private String id = UUID.randomUUID().toString();
private QueryBuilder query;
private RuleDataDto data;
public static class RuleBuilder {
private String id = UUID.randomUUID().toString();
}
}
Index Mapping
{
"mappings": {
"properties": {
"query": {
"type": "percolator"
},
"data": {
"properties": {
"subType": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"content": {
"type": "text"
}
}
}
}
based on criteria I am generating queries and trying to index those to percolator. but getting below exception
query malformed, no start_object after query name
what should be the query field Type? can someone help me on this
You are trying to store an Elasticsearch object (QueryBuilder) in Elasticsearch without specifying the mapping type.
You will need to annotate your query property as percolator type and might change the type of your property to a String:
#Document(indexName = "#{#es.indexName}")
public class Rule {
#Id
private String _id;
#Field(type = FieldType.Percolator)
private String query;
// ...
}
Or, if you want to have some other class for the query property you'll need a custom converter that will convert your object into a valid JSON query - see the documentation for the percolator mapping.
I have json like following
{"data": [
{
"instance": { ...
"inner"" {....
.............}
}
}]
"isvalid":true
"nextVal" : <some num>
}
and POJO like
class A{
private String data;
private boolean isvalid;
private String nextVal;
//with getter setters and proper jackson annotations
}
These can have variable structure inside data, so with object mapper.read I want to take entire data object in string!
have tried direct serialization to my simple object which obviously gives error and also tried JSONNode
mapper.readValue(jsonString, JsonNode.class);
String content = node.get("data").textValue();
This returns blank
anyway I can achieve that to take entire data object value in string with objectmapper?
I tried toString and returned just fine what I wanted - entire data object as String
JsonNode node = (ObjectNode) mapper.readValue(jsonString, JsonNode.class);
node.get("data").toString();
The reason it returns blank is because, data is an array. You need to deseralise it in to JsonArray. Assuming your JSON structure as below,
{
"data": [
{"instance": {
"inner": {
"id": "1"
}
}
}],
"isvalid": true,
"nextVal": 1
}
This will be deserialised using below code (in JSONNode),
List<JsonNode> list = node.findValues("data");
for(JsonNode n: list){
JsonNode in1 = n.findValue("instance");
JsonNode in2 = in1.findValue("inner");
String abc = in2.findValue("id").textValue();
System.out.println(abc);
}
You need to have the POJO structure as shown above. The data will be list of instance object. instance object will have to have inner object.
Update:
Outer node = mapper.readValue(jsonstr, Outer.class);
The classes which needs to be created would be as shown below.
public class Outer {
private List<Data> data;
Boolean valid;
Integer nextval;
public List<Data> getData() {
return data;
}
public void setData(List<Data> data) {
this.data = data;
}
public Boolean isValid() {
return valid;
}
public void setValid(Boolean valid) {
this.valid = valid;
}
public Integer getNextval() {
return nextval;
}
public void setNextval(Integer nextval) {
this.nextval = nextval;
}
}
public class Data {
Instance instance;
public Instance getInstance() {
return instance;
}
public void setInstance(Instance instance) {
this.instance = instance;
}
}
public class Instance {
private Inner inner;
public Inner getInner() {
return inner;
}
public void setInner(Inner inner) {
this.inner = inner;
}
}
public class Inner {
private String id;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
}
I'm trying to serialize the class below:
public class Person : GLib.Object {
public string name { get; set; }
public int age { get; set; }
public bool alive { get; set; }
public Person (string name, int age, bool alive = true) {
Object (
name: name,
age: age,
alive: alive
);
}
}
public int main () {
var person = new Person ("mike", 33, false);
var node = Json.gobject_serialize (person);
string obj = Json.to_string (node, true);
print (obj+"\n");
return 0;
}
While i expect the output be...
{
"name": "mike",
"age": 32,
"alive": false
}
I'm getting...
{
"name": "mike",
"age": 32
}
How do i get the Boolean serialized even if the value is false?
The default serialization function of json-glib doesn't serialize a property if it contains its default value. For boolean properties, this is false, unless explicitly specified otherwise.
To make sure that serialization does happen in this case, you should explicitly implement the Serializable.serialize_property() method yourself.
I have the following nodes and relationships defined:
CarMaker and Models
A CarModel is made CarMaker in multiple years, and that is represented as a property of the MADE_IN relationship.
A CarModel is made by one CarMaker only.
A CarMaker can make multiple CarModels in multiple years.
I have defined the following Classes to represent the nodes: CarModel, CarMaker and the relationship MADE_IN
CarModel
#NodeEntity
public class CarModel {
private Long id;
private String name;
#Relationship (type="MADE_IN", direction = Relationship.UNDIRECTED)
private Set<MadeIn> madeIns = new HashSet<MadeIn>();
private Set<String> years = new HashSet<String>();
public CarModel() {
}
public Long getId() {
return id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public void addMadeIn(MadeIn madeIn) {
System.out.println ("Found CarMaker: " + madeIn.getCarMaker());
this.madeIns.add(madeIn);
}
private Set<MadeIn> getMadeIn() {
return madeIns;
}
public Set<String> getYears() {
Iterator<MadeIn> itr = madeIns.iterator();
while (itr.hasNext()) {
years.add(((MadeIn) itr.next()).getYear());
}
Set<String> sortedYears = years.stream().collect(Collectors.toCollection(TreeSet::new));
return sortedYears;
}
}
CarMaker
public class CarMaker {
#GraphId private Long id;
private String name;
#Relationship (type="MADE_IN", direction = Relationship.UNDIRECTED)
private Set<CarModel> carModels = new HashSet<>();
public CarMaker() {
}
public Long getId() {
return id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Set<CarModel> getCarModels() {
return carModels;
}
public void setCarModels(CarModel carModel) {
carModels.add(carModel);
}
}
MADE_IN
#RelationshipEntity(type="MADE_IN")
public class MadeIn {
#GraphId private Long relationshipId;
#Property private String year;
#StartNode private CarMaker carMaker;
#EndNode private CarModel carModel;
public MadeIn() {
}
public MadeIn(CarMaker carMaker, CarModel carModel, String year) {
this.carMaker = carMaker;
this.carModel = carModel;
this.year = year;
}
public Long getRelationshipId() {
return relationshipId;
}
public void setCarMaker(CarMaker carMaker) {
this.carMaker = carMaker;
}
public CarMaker getCarMaker() {
return this.getCarMaker();
}
public void setCarModel(CarModel carModel) {
this.carModel = carModel;
}
public CarModel getCarModel() {
return this.getCarModel();
}
public String getYear() {
return year;
}
public void setYear(String year) {
this.year = year;
}
}
When I make a request to retrieve a CarModel, I receive a response with the details of that model and all years when it was manufactured:
{
"id": 260248,
"name": "Ulysse",
"years": [
"1994",
"1995",
"1996",
"1997",
"1998",
"1999",
"2000",
"2001",
"2004",
"2005",
"2006",
"2007",
"2008",
"2009",
"2010",
"2011",
"2012"
]
}
The problem is when I try to request the CarModels made by a CarMaker:
{
"id": 4152072,
"name": "BMW",
"carModels": []
}
I noticed that if I reverse the annotations #StartNode and #EndNode on the MadeIn class I get the information about the CarModels made by a CarMaker, however I will not longer get the information about the years when those models were made.
{
"id": 4152072,
"name": "BMW",
"carModels": [
{
"id": 260852,
"name": "120",
"years": []
},
{
"id": 261430,
"name": "Z18",
"years": []
},
{
"id": 262044,
"name": "L7",
"years": []
},
Any idea on what am I missing, or what I am doing wrong ?
Thanks in advance for any help.
--MD
I am trying to use spring-data-elasticsearch to set up mappings for a type that is equivalent to the json configuration below and I am running into issues.
{
"_id" : {
"type" : "string",
"path" : "id"
},
"properties":{
"addressName":{
"type":"multi_field",
"fields":{
"addressName":{
"type":"string"
},
"edge":{
"analyzer":"edge_search_analyzer",
"type":"string"
}
}
},
"companyName":{
"type":"multi_field",
"fields":{
"companyName":{
"type":"string"
},
"edge":{
"analyzer":"edge_search_analyzer",
"type":"string"
}
}
}
}
}
Here is the entity class as it stands now:
#Document(indexName = "test", type="address")
#Setting(settingPath="/config/elasticsearch/test.json") // Used to configure custom analyzer/filter
public class Address {
#Id
private Integer id;
#MultiField(
mainField = #Field(type = FieldType.String),
otherFields = {
#NestedField(dotSuffix = "edge", type = FieldType.String, indexAnalyzer = "edge_search_analyzer")
}
)
private String addressName;
#MultiField(
mainField = #Field(type = FieldType.String),
otherFields = {
#NestedField(dotSuffix = "edge", type = FieldType.String, indexAnalyzer = "edge_search_analyzer")
}
)
private String companyName;
public Address() {}
public Address(AddressDTO dto) {
id = dto.getId();
addressName = dto.getAddressName();
companyName = dto.getCompanyName();
}
public Integer getId() {
return id;
}
public void setId(Integer id) {
Preconditions.checkNotNull(id);
this.id = id;
}
public String getAddressName() {
return addressName;
}
public void setAddressName(String addressName) {
this.addressName = addressName;
}
public String getCompanyName() {
return companyName;
}
public void setCompanyName(String companyName) {
this.companyName = companyName;
}
}
What I am finding is that this results in a mapping like:
"address" : {
"properties" : {
"addressName" : {
"type" : "string",
"fields" : {
"addressName.edge" : {
"type" : "string",
"index_analyzer" : "edge_search_analyzer"
}
}
},
"companyName" : {
"type" : "string",
"fields" : {
"companyName.edge" : {
"type" : "string",
"index_analyzer" : "edge_search_analyzer"
}
}
}
}
}
Which results in two issues:
Id mapping is not done so Elasticsearch generates the ids
Searches on the properties with the custom analyzer do not work properly
If I add the following to override the use of the annotations everything works fine:
#Mapping(mappingPath="/config/elasticsearch/address.json")
Where "address.json" contains the mapping configuration from the top of this post.
So, can anyone point out where I might be going wrong with what I'm trying to achieve?
Is is possible using available annotations or am I going to have to stick with json configuration?
Also, is there a "correct" way to setup index-level configuration via spring-data-elasticsearch, or is that not a supported approach? Currently, I am using the #Setting annotation for this purpose