I am learning ES and I am having problems with this query:
Given 2 products:
products/_source/1
{
"product_id": "58410-2",
"name": [
{
"locale": "en",
"translation": "CBC panel"
},
{
"locale": "vn",
"translation": "CBC panel VN"
}
],
"status": "active",
"category": {
"id": 8,
"name": [
{
"locale": "en",
"translation": "Hematology"
},
{
"locale": "vn",
"translation": "huyết học"
}
]
},
"children": [
{
"product_id": "6690-2",
"name": [
{
"locale": "en",
"translation": "Leukocytes"
},
{
"locale": "vn",
"translation": "Leukocytes vn"
}
],
"status": "active",
"category": {
"id": 8,
"name": [
{
"locale": "en",
"translation": "Hematology"
},
{
"locale": "vn",
"translation": "huyết học"
}
]
},
"children": []
}]}
and
products/_source/2
{
"product_id": "6690-2",
"name": [
{
"locale": "en",
"translation": "Leukocytes"
},
{
"locale": "vn",
"translation": "Leukocytes vn"
}
],
"status": "active",
"category": {
"id": 8,
"name": [
{
"locale": "en",
"translation": "Hematology"
},
{
"locale": "vn",
"translation": "huyết học"
}
]
},
"children": []
}
where a product is a single document but also can be nested in a children array of other products. Both products are different documents in the index.
and this index:
{
"products": {
"aliases": {},
"mappings": {
"dynamic": "false",
"properties": {
"category": {
"properties": {
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
}
}
},
"children": {
"type": "nested"
},
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
},
"product_id": {
"type": "keyword"
},
"status": {
"type": "keyword"
}
}
},
"settings": {
"index": {
"routing": {
"allocation": {
"include": {
"_tier_preference": "data_content"
}
}
},
"number_of_shards": "3",
"provided_name": "products",
"number_of_replicas": "1"
}
}
}
}
I want to be able to query for "Leuko" (or the category or the product_id) and retrieve both products, the single product and the root product.
I have tried using object field, nested, flattened but I think the problem is I don't know how to properly write the query, I have tried things like this (I am using a ruby library but I think it is easy to follow):
#query = {
query: {
query_string: {
fields: ['name.translation', 'children.name.translation', 'category.name.translation', 'children.product_id'],
query: "*#{text}*"
}
},
size: 50
}
#query = {
query: {
nested: {
path: 'children',
query: {
bool: {
should: [
term: { 'children.name.translation' => "*#{text}*" },
term: { 'name.translation' => "*#{text}*" }
]
}
}
}
}
}
but I think at some point I dunno what I am doing anymore and I am just randomly trying different stuff from the documentation.
Follow my query suggestion. Note that I had to add the fields in the Nested object to the mapping.
Mapping:
{
"mappings": {
"dynamic": "false",
"properties": {
"category": {
"properties": {
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
}
}
},
"children": {
"type": "nested",
"properties": {
"product_id": {
"type": "keyword"
},
"category": {
"properties": {
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
}
}
},
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
},
"status": {
"type": "keyword"
}
}
},
"name": {
"properties": {
"locale": {
"type": "keyword"
},
"translation": {
"type": "text"
}
}
},
"product_id": {
"type": "keyword"
},
"status": {
"type": "keyword"
}
}
}
}
Query:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"nested": {
"path": "children",
"query": {
"wildcard": {
"children.name.translation": "leuko*"
}
}
}
},
{
"wildcard": {
"name.translation": "leuko*"
}
}
]
}
}
}
hint
See that you use translation. Avoid using array to make your queries simpler.
What I would do in your case is to create a field for each language, this makes the use of analyzer more flexible for each type of language and you stop using an array and work with an object.
PUT test
{
"mappings": {
"properties": {
"name":{
"type": "text",
"fields": {
"es":{
"type": "text",
"analyzer":"english"
},
"vn":{
"type": "text"
}
}
}
}
}
}
POST test/_doc/
{
"name": "Leukocytes"
}
An example query using field languages.
GET test/_search
{
"query": {
"multi_match": {
"query": "Leukocytes",
"fields": ["name.es", "name.vn"]
}
}
}
Related
I have an elasticsearch index and am using the following query:
"_source": [
"title",
"content"
],
"size": 15,
"from": 0,
"query": {
"bool": {
"must": {
"multi_match": {
"query": "{{query}}",
"fields": [
"title",
"content"
],
"operator": "or"
}
},
"should": [
{
"multi_match": {
"query": "{{query}}",
"fields": [
"title.standard^16",
"content.standard^2"
],
"operator": "and"
}
},
{
"match_phrase": {
"content.standard": {
"query": "{{query}}",
"_name": "Phrase on title",
"boost": 1000
}
}
}
]
}
},
"highlight": {
"fields": {
"content": {}
},
"fragment_size": 100
}
}
Here is the mapping I set:
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "standard",
"filter": [
"lowercase",
"my_metaphone"
]
}
},
"filter": {
"my_metaphone": {
"type": "phonetic",
"encoder": "metaphone",
"replace": true
}
}
}
}
},
"mappings": {
"properties": {
"title": {
"type": "text",
"term_vector": "with_positions_offsets",
"analyzer": "my_analyzer",
"fields": {
"standard": {
"type": "text"
},
"stemmer": {
"type": "text",
"analyzer": "english"
}
}
},
"content": {
"type": "text",
"term_vector": "with_positions_offsets",
"analyzer": "my_analyzer",
"fields": {
"standard": {
"type": "text"
},
"stemmer": {
"type": "text",
"analyzer": "english"
}
}
}
}
}
}
Here is my logic with the query:
1) It will give the highest precedence to a phrase if it appears.
2) If not it will use the standard analyzer (that is the text, as is) and give it the highest precedence.
3) If all else doesn't match up, it will use the phonetic analyzer to get the results, that is the least precedence.
But obviously there is some fault to this as it seems to give higher precedence to the phonetic analyzer than the standard or phrase. For example, if I search for "Person of Indian Origin" it returns results on the top highlighting "Pursuant" "pursuing" and very, very less number of results with person of Indian origin although I know a large number of them exists. How do I solve this?
I'm currently writing swagger 3.0 documentation and using reDoc to render as nice UI for it. I have a few scenarios in my documentation where based on a previous properties enum I would want to display different schema object properties. Sadly I cant seam to figure out how to wire this together properly in my documentation. So far I have the following test endpoint:
{
"post": {
"operationId" : "test",
"summary": "test",
"description": "test",
"tags": [ "test" ],
"consumes": "application/json",
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"oneOf": [
{
"$ref": "./schemas/test1.json"
},
{
"$ref": "./schemas/test2.json"
}
],
"discriminator": {
"propertyName": "pet_type",
"mapping": {
"click": "./schemas/test1.json",
"open": "./schemas/test2.json"
}
}
}
}
}
},
"responses": {
"200": {
"description": "Success"
}
}
}
}
The test1.json looks like this:
{
"Cat": {
"type": "object",
"properties": {
"pet_type": {
"type": "string"
},
"hunts": {
"type": "boolean"
},
"age": {
"type": "integer"
}
},
"discriminator": {
"propertyName": "pet_type"
}
}
}
And the test2.json like this:
{
"Dog": {
"type": "object",
"properties": {
"pet_type": {
"type": "string"
},
"bark": {
"type": "boolean"
},
"breed": {
"type": "string",
"enum": [
"Dingo",
"Husky",
"Retriever",
"Shepherd"
]
}
},
"discriminator": {
"propertyName": "pet_type"
}
}
}
The desired out come would be to toggle between the two "test" jsons based on an enum (the drop down seen in the reDoc sample). What am I missing to get this result?
You can see an example of the discriminator result here under the feature section (the first gif)
After more digging I was able to figure out the issue... my structure for the most part.
On my index.json file I updated my components section to point at my components folder containing the schema as such:
"components": {
"$ref": "./components/test.json"
},
The test.json looks like the following:
{
"schemas": {
"Refinance": {
"description": "A representation of a cat",
"allOf": [
{
"$ref": "#/schemas/Pet"
},
{
"type": "object",
"properties": {
"huntingSkill": {
"type": "string",
"description": "The measured skill for hunting",
"default": "lazy",
"enum": [
"clueless",
"lazy",
"adventurous",
"aggressive"
]
}
},
"required": [
"huntingSkill"
]
}
]
},
"Purchase": {
"description": "A representation of a dog",
"allOf": [
{
"$ref": "#/schemas/Pet"
},
{
"type": "object",
"properties": {
"packSize": {
"type": "integer",
"format": "int32",
"description": "The size of the pack the dog is from",
"default": 1,
"minimum": 1
},
"foobar": {
"type": "string",
"description": "some ol bullshit"
}
},
"required": [
"packSize"
]
}
]
},
"Pet": {
"type": "object",
"discriminator": {
"propertyName": "petType"
},
"properties": {
"petType": {
"description": "Type of a pet",
"type": "string"
}
},
"xml": {
"name": "Pet"
}
}
}
}
And finally the schema for the endpoint gets referenced as follows:
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "../../index.json#/components/schemas/Pet"
}
}
}
},
I looked over other answers that was having the same problem, but those answers don't seem to resolve this problem. How can I get the request to include the Author name? Although, this request returns Author in the response, it doesn't have the actual author details.
Am I missing something on the request or does it need some configuration adjustments?
Setup
We have ADFS configured with Sharepoint 2013 on Premise.
Endpoint I'm hitting
https://SHAREPOINTURL/_api/web/GetFolderByServerRelativeUrl('Documents')?$select=Author/Id,Author/Name,Author/Title,Editor/Id,Editor/Name,Editor/Title,*&$expand=Files/Author,Editor
Response
{
"d": {
"__metadata": {
"id": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')",
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')",
"type": "SP.Folder"
},
"Files": {
"results": [
{
"__metadata": {
"id": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')",
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')",
"type": "SP.File"
},
"Author": {
"__metadata": {
"id": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/Author",
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/Author",
"type": "SP.User"
},
"Groups": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/Author/Groups"
}
},
"Id": 1073741823,
"IsHiddenInUI": false,
"LoginName": "SHAREPOINT\\system",
"Title": "System Account",
"PrincipalType": 1,
"Email": "",
"IsSiteAdmin": false,
"UserId": {
"__metadata": {
"type": "SP.UserIdInfo"
},
"NameId": "S-1-0-0",
"NameIdIssuer": "urn:office:idp:activedirectory"
}
},
"CheckedOutByUser": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/CheckedOutByUser"
}
},
"ListItemAllFields": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/ListItemAllFields"
}
},
"LockedByUser": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/LockedByUser"
}
},
"ModifiedBy": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/ModifiedBy"
}
},
"Versions": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFileByServerRelativeUrl('Documents/FILENAME.xls')/Versions"
}
},
"CheckInComment": "",
"CheckOutType": 2,
"ContentTag": "{71501108-7ACC-46F6-82D7-33E5C5F0124C},3,4",
"CustomizedPageStatus": 0,
"ETag": "\"{71501108-7ACC-46F6-82D7-33E5C5F0124C},3\"",
"Exists": true,
"Length": "390144",
"Level": 1,
"MajorVersion": 1,
"MinorVersion": 0,
"Name": "FILENAME.xls",
"ServerRelativeUrl": "Documents/FILENAME.xls",
"TimeCreated": "2013-07-10T13:55:39Z",
"TimeLastModified": "2013-07-10T13:55:39Z",
"Title": "",
"UIVersion": 512,
"UIVersionLabel": "1.0"
}
]
},
"ListItemAllFields": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')/ListItemAllFields"
}
},
"ParentFolder": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')/ParentFolder"
}
},
"Properties": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')/Properties"
}
},
"Folders": {
"__deferred": {
"uri": "https://SHAREPOINT_URL/_api/Web/GetFolderByServerRelativeUrl('Documents')/Folders"
}
},
"ItemCount": 18,
"Name": "Documents",
"ServerRelativeUrl": "Documents",
"WelcomePage": ""
}
}
Try this. I found this on the MSDN forum.
https://social.msdn.microsoft.com/Forums/office/en-US/04fc252c-adb9-4f50-b9b0-c326f88ad69e/retriving-author-name-using-rest-api?forum=appsforsharepoint
----- update
https://{SITE_URL}/_api/web/Lists/getbytitle('{DOCUMENT_PATH}')/items?$select=Title,Author/ID,Author/FirstName,Author/LastName,Author/Title,Author/Department,Author/EMail&$expand=Author/ID
I am new to swagger. Following is the error generated:
✖ Swagger Error
A deterministic version of a JSON Schema object.
Jump to line 7
Details
Object
{
"swagger": "2.0",
"info": {"description": "Fast Healthcare Interoperability Resources (FHIR, pronounced \"Fire\") defines a set of \"Resources\" that represent granular clinical concepts. The resources can be managed in isolation, or aggregated into complex documents. Technically, FHIR is designed for the web; the resources are based on simple XML or JSON structures, with an http-based RESTful protocol where each resource has predictable URL. Where possible, open internet standards are used for data representation. \n",
"version":"1.1.1",
"title": "FhirServer"
},
"definitions": {
"Account": {
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "",
"type": "object",
"properties": {
"resourceType": {
"type": "string",
"minLength": 1
},
"id": {
"type": "string",
"minLength": 1
},
"text": {
"type": "object",
"properties": {
"status": {
"type": "string",
"minLength": 1
},
"div": {
"type": "string",
"minLength": 1
}
},
"required": [
"status",
"div"
]
}
},
"required": [
"resourceType",
"id",
"text"
]
}
},
"paths": {
},
"schemes": [
"http"
],
"basePath": "/open",
"tags": [
{
"name": "Account"
},
{
"name": "AllergyIntolerance"
},
{
"name": "Appointment"
},
{
"name": "AppointmentResponse"
},
{
"name": "AuditEvent"
},
{
"name": "Basic"
},
{
"name": "Binary"
},
{
"name": "BodySite"
},
{
"name": "Bundle"
},
{
"name": "CarePlan"
},
{
"name": "Claim"
},
{
"name": "ClaimResponse"
},
{
"name": "ClinicalImpression"
},
{
"name": "Communication"
},
{
"name": "CommunicationRequest"
},
{
"name": "Composition"
},
{
"name": "ConceptMap"
},
{
"name": "Condition"
},
{
"name": "Conformance"
},
{
"name": "Contract"
},
{
"name": "Coverage"
},
{
"name": "DataElement"
},
{
"name": "DetectedIssue"
},
{
"name": "Device"
},
{
"name": "DeviceComponent"
},
{
"name": "DeviceMetric"
},
{
"name": "DeviceUseRequest"
},
{
"name": "DeviceUseStatement"
},
{
"name": "DiagnosticOrder"
},
{
"name": "DiagnosticReport"
},
{
"name": "DocumentManifest"
},
{
"name": "DocumentReference"
},
{
"name": "EligibilityRequest"
},
{
"name": "EligibilityResponse"
},
{
"name": "Encounter"
},
{
"name": "EnrollmentRequest"
},
{
"name": "EnrollmentResponse"
},
{
"name": "EpisodeOfCare"
},
{
"name": "ExplanationOfBenefit"
},
{
"name": "FamilyMemberHistory"
},
{
"name": "Flag"
},
{
"name": "Goal"
},
{
"name": "Group"
},
{
"name": "HealthcareService"
},
{
"name": "ImagingObjectSelection"
},
{
"name": "ImagingStudy"
},
{
"name": "Immunization"
},
{
"name": "ImmunizationRecommendation"
},
{
"name": "ImplementationGuide"
},
{
"name": "List"
},
{
"name": "Location"
},
{
"name": "Media"
},
{
"name": "Medication"
},
{
"name": "MedicationAdministration"
},
{
"name": "MedicationDispense"
},
{
"name": "MedicationOrder"
},
{
"name": "MedicationStatement"
},
{
"name": "NamingSystem"
},
{
"name": "NutritionOrder"
},
{
"name": "Observation"
},
{
"name": "OperationDefinition"
},
{
"name": "OperationOutcome"
},
{
"name": "Order"
},
{
"name": "OrderResponse"
},
{
"name": "Organization"
},
{
"name": "Parameters"
},
{
"name": "Patient"
},
{
"name": "PaymentNotice"
},
{
"name": "PaymentReconciliation"
},
{
"name": "Person"
},
{
"name": "Practitioner"
},
{
"name": "Procedure"
},
{
"name": "ProcedureRequest"
},
{
"name": "ProcessRequest"
},
{
"name": "ProcessResponse"
},
{
"name": "Provenance"
},
{
"name": "Questionnaire"
},
{
"name": "QuestionnaireResponse"
},
{
"name": "ReferralRequest"
},
{
"name": "RelatedPerson"
},
{
"name": "RiskAssessment"
},
{
"name": "Schedule"
},
{
"name": "SearchParameter"
},
{
"name": "Slot"
},
{
"name": "Specimen"
},
{
"name": "StructureDefinition"
},
{
"name": "Subscription"
},
{
"name": "Substance"
},
{
"name": "SupplyDelivery"
},
{
"name": "SupplyRequest"
},
{
"name": "TestScript"
},
{
"name": "ValueSet"
},
{
"name": "VisionPrescription"
}
]
}
Remove the $schema attribute in your definition. It's not allowed by the OAI Specification
I'm using Elasticsearch and create an index with the following information for mapping and settings. The problem I have is that my field geography.locality which should use the 'name_analyser' doesn't seem to use it.
{
"index": "programs",
"body": {
"settings": {
"number_of_shards": 5,
"analysis": {
"filter": {
"elision": {
"type": "elision",
"articles": [
"l",
"m",
"t",
"qu",
"n",
"s",
"j",
"d"
]
},
"multi_words": {
"type": "shingle",
"min_shingle_size": 2,
"max_shingle_size": 10
},
"name_filter": {
"type": "edgeNGram",
"max_gram": 100,
"min_gram": 2
}
},
"tokenizer": {
"name_tokenizer": {
"type": "edgeNGram",
"max_gram": 100,
"min_gram": 2
}
},
"analyser": {
"name_analyser": { // <-- analyser I want to use on geography.locality
"tokenizer": "whitespace",
"type": "custom",
"filter": [
"lowercase",
"multi_words",
"name_filter",
"asciifolding"
]
},
"french": {
"tokenizer": "letter",
"filter": [
"asciifolding",
"lowercase",
"elision",
"stop"
]
},
"city_name": {
"type": "custom",
"tokenizer": "letter",
"filter": [
"lowercase",
"asciifolding"
]
}
}
}
},
"mappings": {
"program": {
"properties": {
"nid": {
"type": "integer",
"index": "not_analyzed"
},
"title": {
"type": "string"
},
"language": {
"type": "string",
"index": "not_analyzed"
},
"regulation": {
"type": "integer"
},
"sales_state": {
"type": "integer"
},
"enabled_dwell": {
"type": "boolean"
},
"enabled_invest": {
"type": "boolean"
},
"delivery_date": {
"type": "date"
},
"address": {
"properties": {
"country": {
"type": "string",
"index": "not_analyzed"
},
"locality": {
"type": "string",
"analyser": "name_analyser"
},
"postal_code": {
"type": "integer"
},
"thoroughfare": {
"type": "string",
"index": "not_analyzed"
},
"premise": {
"type": "string",
"index": "not_analyzed"
}
}
},
"location": {
"type": "geo_point"
},
"geography": {
"properties": {
"locality": {
"type": "string",
"analyser": "name_analyser" // ... here :-/
},
"department": {
"type": "string",
"index": "not_analyzed"
},
"region": {
"type": "string",
"index": "not_analyzed"
}
}
},
"lots": {
"type": "nested",
"include_in_all": false,
"properties": {
"lot_type": {
"type": "integer"
},
"rooms": {
"type": "integer"
},
"price_vat_inc": {
"type": "integer"
},
"price_reduced_vat_inc": {
"type": "integer"
},
"price_vat_ex": {
"type": "integer"
}
}
}
}
}
}
}
}
Here's the output given by ES for the mapping registered for this index.
{
"program": {
"properties": {
"address": {
"properties": {
"country": {
"index": "not_analyzed",
"type": "string"
},
"premise": {
"index": "not_analyzed",
"type": "string"
},
"locality": {
"type": "string"
},
"postal_code": {
"type": "integer"
},
"thoroughfare": {
"index": "not_analyzed",
"type": "string"
}
}
},
"sales_state": {
"type": "integer"
},
"nid": {
"type": "integer"
},
"language": {
"index": "not_analyzed",
"type": "string"
},
"title": {
"type": "string"
},
"enabled_invest": {
"type": "boolean"
},
"geo_point": {
"type": "string"
},
"lots": {
"include_in_all": false,
"type": "nested",
"properties": {
"rooms": {
"include_in_all": false,
"type": "integer"
},
"price_vat_inc": {
"include_in_all": false,
"type": "integer"
},
"price_vat_ex": {
"include_in_all": false,
"type": "integer"
},
"lot_type": {
"include_in_all": false,
"type": "integer"
},
"price_reduced_vat_inc": {
"include_in_all": false,
"type": "integer"
}
}
},
"enabled_dwell": {
"type": "boolean"
},
"delivery_date": {
"format": "dateOptionalTime",
"type": "date"
},
"regulation": {
"type": "integer"
},
"geography": {
"properties": {
"locality": {
"type": "string" // name_analyser should show up here right?????
},
"department": {
"index": "not_analyzed",
"type": "string"
},
"region": {
"index": "not_analyzed",
"type": "string"
}
}
},
"location": {
"type": "geo_point"
}
}
}
}
Does anybody knows what I am doing wrong? I'm kind of lost about this.
You have a typo :-), actually two:
"locality": {
"type": "string",
"analyser": "name_analyser"
},
in both address and geography. It should be analyzer not analyser (with an s).
Also, the same here:
"analyser": {
"name_analyser": {
"tokenizer": "whitespace",
...
I am guessing that the index exists and you are trying to update the settings with a new analyser. This is not permitted on a live index.
Do you have any errors when you submit the updated settings?
Have a look at this thread - Change settings and mappings on existing index in Elasticsearch
and here http://www.elastic.co/guide/en/elasticsearch/reference/1.x/indices-update-settings.html#update-settings-analysis