I am trying to use spring-data-elasticsearch to set up mappings for a type that is equivalent to the json configuration below and I am running into issues.
{
"_id" : {
"type" : "string",
"path" : "id"
},
"properties":{
"addressName":{
"type":"multi_field",
"fields":{
"addressName":{
"type":"string"
},
"edge":{
"analyzer":"edge_search_analyzer",
"type":"string"
}
}
},
"companyName":{
"type":"multi_field",
"fields":{
"companyName":{
"type":"string"
},
"edge":{
"analyzer":"edge_search_analyzer",
"type":"string"
}
}
}
}
}
Here is the entity class as it stands now:
#Document(indexName = "test", type="address")
#Setting(settingPath="/config/elasticsearch/test.json") // Used to configure custom analyzer/filter
public class Address {
#Id
private Integer id;
#MultiField(
mainField = #Field(type = FieldType.String),
otherFields = {
#NestedField(dotSuffix = "edge", type = FieldType.String, indexAnalyzer = "edge_search_analyzer")
}
)
private String addressName;
#MultiField(
mainField = #Field(type = FieldType.String),
otherFields = {
#NestedField(dotSuffix = "edge", type = FieldType.String, indexAnalyzer = "edge_search_analyzer")
}
)
private String companyName;
public Address() {}
public Address(AddressDTO dto) {
id = dto.getId();
addressName = dto.getAddressName();
companyName = dto.getCompanyName();
}
public Integer getId() {
return id;
}
public void setId(Integer id) {
Preconditions.checkNotNull(id);
this.id = id;
}
public String getAddressName() {
return addressName;
}
public void setAddressName(String addressName) {
this.addressName = addressName;
}
public String getCompanyName() {
return companyName;
}
public void setCompanyName(String companyName) {
this.companyName = companyName;
}
}
What I am finding is that this results in a mapping like:
"address" : {
"properties" : {
"addressName" : {
"type" : "string",
"fields" : {
"addressName.edge" : {
"type" : "string",
"index_analyzer" : "edge_search_analyzer"
}
}
},
"companyName" : {
"type" : "string",
"fields" : {
"companyName.edge" : {
"type" : "string",
"index_analyzer" : "edge_search_analyzer"
}
}
}
}
}
Which results in two issues:
Id mapping is not done so Elasticsearch generates the ids
Searches on the properties with the custom analyzer do not work properly
If I add the following to override the use of the annotations everything works fine:
#Mapping(mappingPath="/config/elasticsearch/address.json")
Where "address.json" contains the mapping configuration from the top of this post.
So, can anyone point out where I might be going wrong with what I'm trying to achieve?
Is is possible using available annotations or am I going to have to stick with json configuration?
Also, is there a "correct" way to setup index-level configuration via spring-data-elasticsearch, or is that not a supported approach? Currently, I am using the #Setting annotation for this purpose
Related
Here is my Dto for percolator query class.
#Data
#Document(indexName = "#{#es.indexName}")
#Builder(builderClassName = "RuleBuilder")
public class Rule {
#Id
private String id = UUID.randomUUID().toString();
private QueryBuilder query;
private RuleDataDto data;
public static class RuleBuilder {
private String id = UUID.randomUUID().toString();
}
}
Index Mapping
{
"mappings": {
"properties": {
"query": {
"type": "percolator"
},
"data": {
"properties": {
"subType": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"content": {
"type": "text"
}
}
}
}
based on criteria I am generating queries and trying to index those to percolator. but getting below exception
query malformed, no start_object after query name
what should be the query field Type? can someone help me on this
You are trying to store an Elasticsearch object (QueryBuilder) in Elasticsearch without specifying the mapping type.
You will need to annotate your query property as percolator type and might change the type of your property to a String:
#Document(indexName = "#{#es.indexName}")
public class Rule {
#Id
private String _id;
#Field(type = FieldType.Percolator)
private String query;
// ...
}
Or, if you want to have some other class for the query property you'll need a custom converter that will convert your object into a valid JSON query - see the documentation for the percolator mapping.
After upgrading to the spring-boot-starter:2.2.5.RELEASE, spring-cloud-dependencies:Hoxton.SR3, spring-cloud-stream-dependencies:Horsham.SR3 & spring-data-elasticsearch:3.2.5.RELEASE. The boolean fields are not stored in index/document. It was working earlier with spring boot 2.1.11.
I tried to create document manually using the ElasticSearch REST API. When tried directly with REST API, the boolean fields are stored in index.
Is there any changes done how the mappings are declared for boolean fields?
I'm using the ElasticsearchTemplate.index(IndexQuery) API to create an index document, where the IndexQuery is built with an document object having some boolean fields.
The following are the boolean fields in the CatalogIndex.java file.
#Document(indexName = "catalogentity")
public class CatalogIndex {
private boolean isType;
private boolean isAbstract;
private boolean isFinal;
private String stateId;
private String stageId;
//some other fields
public boolean isType() {
return isType;
}
public void setType(final boolean type) {
isType = type;
}
public boolean isAbstract() {
return isAbstract;
}
public void setAbstract(final boolean anAbstract) {
isAbstract = anAbstract;
}
public boolean isFinal() {
return isFinal;
}
public void setFinal(final boolean aFinal) {
isFinal = aFinal;
}
//some other setter and getters
The mappings are as follows
{
"properties": {
"type": {
"type": "boolean"
},
"abstract": {
"type": "boolean"
},
"final": {
"type": "boolean"
},
"stateId": {
"type": "text"
},
"stageId": {
"type": "keyword"
}
}
}
Thanks in advance,
Santhosh
The following is the working configuration for boolean fields. I was not sure why it was working fine before upgrade.
The following are the boolean fields in the CatalogIndex.java file.
#Document(indexName = "catalogentity")
public class CatalogIndex {
private boolean isType;
private boolean abstract1;
private boolean final1;
private String stateId;
private String stageId;
//some other fields
public boolean isType() {
return isType;
}
public void setType(final boolean type) {
isType = type;
}
public boolean isAbstract1() {
return abstract1;
}
public void setAbstract1(final boolean abstract1) {
this.abstract1 = abstract1;
}
public boolean isFinal1() {
return final1;
}
public void setFinal1(final boolean final1) {
this.final1 = final1;
}
//some other setter and getters
The mappings are as follows
{
"properties": {
"type": {
"type": "boolean"
},
"abstract1": {
"type": "boolean"
},
"final1": {
"type": "boolean"
},
"stateId": {
"type": "text"
},
"stageId": {
"type": "keyword"
}
}
}
I have two controller FooController and BooController (the last is for a backward compatibility), I want only the FooController will return its model with upper camel case notation ("UpperCamelCase").
For example:
public class MyData
{
public string Key {get;set;}
public string Value {get;set;}
}
public class BooController : ControllerBase
{
public ActionResult<MyData> GetData() { ... }
}
public class FooController : ControllerBase
{
public ActionResult<MyData> GetData() { ... }
}
The desired GET output:
GET {{domain}}/api/Boo/getData
[
{
"key": 1,
"value": "val"
}
]
GET {{domain}}/api/Foo/getData
[
{
"Key": 1,
"Value": "val"
}
]
If I'm using the AddJsonOptions extension with option.JsonSerializerOptions.PropertyNamingPolicy = null like:
services.AddMvc()
.AddJsonOptions(option =>
{
option.JsonSerializerOptions.PropertyNamingPolicy = null;
});
both BooController and FooController returns the data with upper camel case notation.
How to make only the FooController to return the data in upper camel case notation format?
Add [JsonProperty("name")] in the response class property.
stackoverflow.com/a/34071205/5952008
Solved (although kind of bad performance and a hacky solution - hoping for someone to suggest a better solution):
Inside the legacy BooController I'm serializing the response just before returning it using a custom formater of JsonSerializerSettings:
public class BooController : ControllerBase
{
public ActionResult<MyData> GetData()
{
var formatter = JsonConvert.DefaultSettings = () => new JsonSerializerSettings
{
Formatting = Formatting.Indented,
ContractResolver = new CamelCasePropertyNamesContractResolver()
};
return Ok(JsonConvert.SerializeObject(response, Formatting.Indented, formatter()));
}
}
Result in:
GET {{domain}}/api/Boo/getData
[
{
"key": 1,
"value": "val"
}
]
I have the following java REST service
#GET
#Path("/{customerId}")
#Operation(
summary = "Test summary",
description = "Test desciption",
responses = {
#ApiResponse(
responseCode = "200",
content = #Content(
schema = #Schema(implementation = CustomerDto.class),
mediaType = "application/json"),
description = "CustomerDto description"),
#ApiResponse(
responseCode = "400",
description = "Description for 400 status")
//Here you can add all response statuses and their's descriptions
})
public CustomerDto findCustomerById(#Parameter(description = "Id of the customer", required = true) #PathParam("customerId") Long customerId) {
return apiFacade.getCustomerDtoByCustomerId(customerId);
}
Simple dto-class
public class CustomerDto {
private TestInfo testInfo;
}
Simple subclass
public class TestInfo {
private Long testId;
private Long checkId;
public TestInfo(Long testId, Long checkId) {
this.testId = testId;
this.checkId = checkId;
}
public Long getTestId() {
return testId;
}
public Long getCheckId() {
return checkId;
}
public boolean isDecisionMade() {
return testId != null || checkId != null;
}
}
I'd like to see TestInfo-json without 'decisionMade' section. Can you help me It is possible?
"TestInfo" : {
"type" : "object",
"properties" : {
"testId" : {
"type" : "integer",
"format" : "int64"
},
"checkId" : {
"type" : "integer",
"format" : "int64"
},
"decisionMade" : {
"type" : "boolean"
}
}
}
Is it possible to tell swagger use class-fields instead of getter+setter?
I use swagger version 2.2.2. Generate json via BaseOpenApiResource. All configs are default.
I have found the decision:
Json.mapper().setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.NONE);
Json.mapper().setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
I have the following nodes and relationships defined:
CarMaker and Models
A CarModel is made CarMaker in multiple years, and that is represented as a property of the MADE_IN relationship.
A CarModel is made by one CarMaker only.
A CarMaker can make multiple CarModels in multiple years.
I have defined the following Classes to represent the nodes: CarModel, CarMaker and the relationship MADE_IN
CarModel
#NodeEntity
public class CarModel {
private Long id;
private String name;
#Relationship (type="MADE_IN", direction = Relationship.UNDIRECTED)
private Set<MadeIn> madeIns = new HashSet<MadeIn>();
private Set<String> years = new HashSet<String>();
public CarModel() {
}
public Long getId() {
return id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public void addMadeIn(MadeIn madeIn) {
System.out.println ("Found CarMaker: " + madeIn.getCarMaker());
this.madeIns.add(madeIn);
}
private Set<MadeIn> getMadeIn() {
return madeIns;
}
public Set<String> getYears() {
Iterator<MadeIn> itr = madeIns.iterator();
while (itr.hasNext()) {
years.add(((MadeIn) itr.next()).getYear());
}
Set<String> sortedYears = years.stream().collect(Collectors.toCollection(TreeSet::new));
return sortedYears;
}
}
CarMaker
public class CarMaker {
#GraphId private Long id;
private String name;
#Relationship (type="MADE_IN", direction = Relationship.UNDIRECTED)
private Set<CarModel> carModels = new HashSet<>();
public CarMaker() {
}
public Long getId() {
return id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Set<CarModel> getCarModels() {
return carModels;
}
public void setCarModels(CarModel carModel) {
carModels.add(carModel);
}
}
MADE_IN
#RelationshipEntity(type="MADE_IN")
public class MadeIn {
#GraphId private Long relationshipId;
#Property private String year;
#StartNode private CarMaker carMaker;
#EndNode private CarModel carModel;
public MadeIn() {
}
public MadeIn(CarMaker carMaker, CarModel carModel, String year) {
this.carMaker = carMaker;
this.carModel = carModel;
this.year = year;
}
public Long getRelationshipId() {
return relationshipId;
}
public void setCarMaker(CarMaker carMaker) {
this.carMaker = carMaker;
}
public CarMaker getCarMaker() {
return this.getCarMaker();
}
public void setCarModel(CarModel carModel) {
this.carModel = carModel;
}
public CarModel getCarModel() {
return this.getCarModel();
}
public String getYear() {
return year;
}
public void setYear(String year) {
this.year = year;
}
}
When I make a request to retrieve a CarModel, I receive a response with the details of that model and all years when it was manufactured:
{
"id": 260248,
"name": "Ulysse",
"years": [
"1994",
"1995",
"1996",
"1997",
"1998",
"1999",
"2000",
"2001",
"2004",
"2005",
"2006",
"2007",
"2008",
"2009",
"2010",
"2011",
"2012"
]
}
The problem is when I try to request the CarModels made by a CarMaker:
{
"id": 4152072,
"name": "BMW",
"carModels": []
}
I noticed that if I reverse the annotations #StartNode and #EndNode on the MadeIn class I get the information about the CarModels made by a CarMaker, however I will not longer get the information about the years when those models were made.
{
"id": 4152072,
"name": "BMW",
"carModels": [
{
"id": 260852,
"name": "120",
"years": []
},
{
"id": 261430,
"name": "Z18",
"years": []
},
{
"id": 262044,
"name": "L7",
"years": []
},
Any idea on what am I missing, or what I am doing wrong ?
Thanks in advance for any help.
--MD