I have the following JSON:
{
"votecategory": [
{
"id": "nlvfl2",
"title": "Best Song",
"pollQuestion": {
"id": "nbprqp",
"title": "best-song",
"displayText": "Best Song",
"answer": [
{
"id": "qylaw4",
"title": "Bruno Mars – Locked Out Of Heaven",
"relatedItems": [
{
"Name": "Bruno Mars",
"id": "sljkur",
"Bio": "Bio info here"
},
{} //Sometimes there's an empty object
],
"winner": "true"
},
{
"id": "q05sb3",
"title": "Daft Punk – Get Lucky (ft. Pharrell Williams)",
"displayText": "Daft Punk – Get Lucky (ft. Pharrell Williams)",
"relatedItems": [
{
"Name": "Daft Punk",
"id": "d9sd84",
"Bio": "Bio info here"
}
]
},
...
]
}
},
...
]
}
Which maps to the following entities:
Category (votecategory values)
Nomination (answer values)
Artist (relatiedItems values)
Ive managed to setup object and relationship mappings for votecategory (category) -> answer (nomination), however I'm having a problem mapping nomination to artist.
What I need to do is have a 1:1 core data relationship setup between nomination and artist, and 1:N relationship setup between artists and nomination (one artist can have multiple nominations).
The problem is that "relatedItems" is an array, but in reality only contains 1 usable value, the related artist. This "should" be a 1:1 relationship from a data perspective, however the JSON maps it as a 1:N relationship, this confuses restkit (rightfully so).
How can I store the single item in the JSON relatedItems response as a single 1:1 relationship?
Thanks
Oli
You could look at using a custom value transformer on that mapping which converts the array into a single object. Check out this question for some additional details.
Related
My goal is to use an input array of strings (fake emails) as a search query for documents in my MongoDB database, which I am powering using TypeORM. This way if I want to search for documents using more than one email at a time, I can do that. Meaning I want to be able to feed in:
query = ["kim#gmail.com", "jim#gmail.com", "sarah#gmail.com"] and get 3 different documents where document one has kim#gmail.com as the attendee, jim#gmail.com is another document's attendee field, and sarah#gmail.com is the third document's attendee (or is among them).
I want to use an email as a query to search for and return all documents where the array field has the email in the array.
So as an example here is the results for the "get all documents" endpoint right now:
[
{
"_id": "6283d7ad706445dc33319bcb",
"hostUsername": "jack",
"hostEmail": "jack#outlook.com",
"meetingName": "nervous-fish-hautily-vetting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"kate#gmail.com",
"sawyer#gmail.com"
]
},
{
"_id": "6284235e662f7dfb073e2cbc",
"hostUsername": "jacob",
"hostEmail": "jacob#gmail.com",
"meetingName": "eager-fish-hautily-vetting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"kate#gmail.com",
"benjaminlinus#gmail.com"
]
},
{
"_id": "6283d7c3706445dc33319bcc",
"hostUsername": "richard",
"hostEmail": "richard#outlook.com",
"meetingName": "eager-cat-hautily-subtracting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"johnlocke#gmail.com",
"hurley#gmail.com"
]
},
{
"_id": "6283d82b706445dc33319bcd",
"hostUsername": null,
"hostEmail": "richard#outlook.com",
"meetingName": "nervous-cat-hautily-jumping",
"startTime": "1970-01-01T00:00:00.000Z",
"attendees": null
},
{
"_id": "6283d8af706445dc33319bce",
"hostUsername": null,
"hostEmail": "richard#outlook.com",
"meetingName": "eager-plant-ignorantly-jumping",
"startTime": "1970-01-01T00:00:00.000Z",
"attendees": null
}
]
I want to query the database with ["kate#gmail.com"] and get back the two results that have "kate#gmail.com" in the attendees field.
The closest solution (that doesn't work) is the one I found in this GitHub issue and also another close solution (that doesn't work) in this StackOverflow question
Here is me implementing those two suggestions:
import { In } from "typeorm";
async searchMeetingsByDetails(
attendees?: string[]
): Promise<IMeeting[]> {
console.log(attendees, 39);
const meetingsByAttendees = attendees
? await this.meetingRepository.find({
where: {
attendees: In([...attendees]),
},
})
: [];
return [
meetingsByAttendees,
].flat();
}
This gives me an empty array [] when the input is ["kate#gmail.com"] so if the In() thing worked, it would give results.
const meetings = await this.meetingRepository
.createQueryBuilder("meeting")
.where("meeting.attendees IN (:attendees)", {
attendees: [...attendees],
});
This one gives ERROR [ExceptionsHandler] Query Builder is not supported by MongoDB. TypeORMError: Query Builder is not supported by MongoDB.
The API I'm trying to describe has a structure where the root object can contain an arbitrary number of child objects (properties that are themselves objects). The "key", or property in the root object, is the unique identifier of the child object, and the value is the rest of the child object's data.
{
"child1": { ... bunch of stuff ... },
"child2": { ... bunch of stuff ... },
...
}
This could similarly be modeled as an array, e.g.:
[
{ "id": "child1", ... bunch of stuff ... },
{ "id": "child2", ... bunch of stuff ... },
...
]
but this both makes it structurally less clear what the identifying property is and makes uniqueness among the children's ID implicit rather than explicit, so we want to use an object, or a map.
I've seen the Swagger documentation for Model with Map/Dictionary Properties, but that doesn't adequately suit my use case. Writing something like:
"Parent": {
"additionalProperties": {
"$ref": "#/components/schemas/Child",
}
Yields something like this:
This adequately communicates the descriptiveness of the value in the property, but how do I document what the restrictions are for the "key" in the object? Ideally I'd like to say something like "it's not just any arbitrary string, it's the ID that corresponds to the child". Is this supported in any way?
Your example is correct.
how do I document what the restrictions are for the "key" in the object? Ideally I'd like to say something like "it's not just any arbitrary string, it's the ID that corresponds to the child". Is this supported in any way?
OpenAPI 3.1
OAS 3.1 fully supports JSON Schema 2020-12, including patternProperties. This keyword lets you define the format of dictionary keys by using a regular expression:
"Parent": {
"type": "object",
"patternProperties": {
"^child\d+$": {
"$ref": "#/components/schemas/Child"
}
},
"description": "A map of `Child` schemas, where the keys are IDs that correspond to the child"
}
Or, if the property names are defined by an enum, you can use propertyNames to define that enum:
"Parent": {
"type": "object",
"propertyNames": {
"enum": ["foo", "bar"]
},
"additionalProperties": {
"$ref": "#/components/schemas/Child"
}
}
OpenAPI 3.0 and 2.0
Dictionary keys are assumed to be strings, but there's no way to limit the contents/format of keys. You can document any restrictions and specifics verbally in the schema description. Adding schema examples could help illustrate what your dictionary/map might look like.
"Parent": {
"type": "object",
"additionalProperties": {
"$ref": "#/components/schemas/Child"
},
"description": "A map of `Child` schemas, where the keys are IDs that correspond to the child",
"example": {
"child1": { ... bunch of stuff ... },
"child2": { ... bunch of stuff ... },
}
If the possible key names are known (for example, they are part of an enum), you can define your dictionary as a regular object and the keys as individual object properties:
// Keys can be: key1, key2, key3
"Parent": {
"type": "object",
"properties": {
"key1": { "$ref": "#/components/schemas/Child" },
"key2": { "$ref": "#/components/schemas/Child" },
"key3": { "$ref": "#/components/schemas/Child" }
}
}
Then you can add "additionalProperties": false to really ensure that only those keys are used.
I was able to create author nodes directly from the json file . But the challenge is on what basis or how we have to link the data. Linking "Author" to "organization". since the data is dynamic we cannot generalize it. I have tried with using csv file but, it fails the conditions when dynamic data is coming. For example one json record contain 2 organization and 3 authors, next record will be different. Different json record have different author and organization to link. organization/1 represent organization1 and organization/2 represents organization 2. Any help or hint will be great. Thank you. Please find the json file below.
"Author": [
{
"seq": "3",
"type": "abc",
"identifier": [
{
"idtype:auid": "10000000"
}
],
"familyName": "xyz",
"indexedName": "MI",
"givenName": "T",
"preferredName": {
"familyName": "xyz1",
"givenName": "a",
"initials": "T.",
"indexedName": "bT."
},
"emailAddressList": [],
"degrees": [],
"#id": "https:abc/2009127993/author/person/3",
"hasAffiliation": [
"https:abc/author/organization/1"
],
"organization": [
[
{
"identifier": [
{
"#type": "idtype:uuid",
"#subtype": "idsubtype:affiliationInstanceId",
"#value": "aff2"
},
{
"#type": "idtype:OrgDB",
"#subtype": "idsubtype:afid",
"#value": "12345"
},
{
"#type": "idtype:OrgDB",
"#subtype": "idsubtype:dptid"
}
],
"organizations": [],
"addressParts": [],
"sourceText": "",
"text": " Medical University School of Medicine",
"#id": "https:abc/author/organization/1"
}
],
[
{
"identifier": [
{
"#type": "idtype:uuid",
"#subtype": "idsubtype:affiliationInstanceId",
"#value": "aff1"
},
{
"#type": "idtype:OrgDB",
"#subtype": "idsubtype:afid",
"#value": "7890"
},
{
"#type": "idtype:OrgDB",
"#subtype": "idsubtype:dptid"
}
],
"organizations": [],
"addressParts": [],
"sourceText": "",
"text": "K University",
"#id": "https:efg/author/organization/2"
}
]
Hi I see that Organisation is part of the Author data, so you have to model it like wise. So for instance (Author)-[:AFFILIATED_WITH]->(Organisation)
When you use apoc.load.json which supports a stream of author objects you can load the data.
I did some checks on your JSON structure with this cypher query:
call apoc.load.json("file:///Users/keesv/work/check.json") yield value
unwind value as record
WITH record.Author as author
WITH author.identifier[0].`idtype:auid` as authorId,author, author.organization[0] as organizations
return authorId, author, organizations
To get this working you will need to create include apoc in the plugins directory, and add the following two lines in the apoc.conf file (create one if it is not there) in the 'conf' directory.
apoc.import.file.enabled=true
apoc.import.file.use_neo4j_config=false
I also see a nested array for the organisations in the output why is that and what is the meaning of that?
And finally I see also in the JSON that an organisation can have a reference to other organisations.
explanation
In my query I use UNWIND to unwind the base Author array. This means you get for every author a 'record' to work with.
With a MERGE or CREATE statement you can now create an Author Node with the correct properties. With the FOREACH construct you can walk over all the Organization entry and create/merge an Organization node and create the relation between the Author and the Organization.
here an 'psuedo' example
call apoc.load.json("file:///Users/keesv/work/check.json") yield value
unwind value as record
WITH record.Author as author
WITH author.identifier[0].`idtype:auid` as authorId,author, author.organization[0] as organizations
// creating the Author node
MERGE (a:Author { id: authorId })
SET a.familyName = author.familyName
...
// walk over the organizations
// determine
FOREACH (org in organizations |
MERGE (o:Organization { id: ... })
SET o.name = org.text
...
MERGE (a)-[:AFFILIATED_WITH]->(o)
// if needed you can also do a nested FOREACH here to process the Org Org relationship
)
Here is the JSON file I used I had to change something at the start and the end
[
{
"Author":{
"seq":"3",
"type":"abc",
"identifier":[
{
"idtype:auid":"10000000"
}
],
"familyName":"xyz",
"indexedName":"MI",
"givenName":"T",
"preferredName":{
"familyName":"xyz1",
"givenName":"a",
"initials":"T.",
"indexedName":"bT."
},
"emailAddressList":[
],
"degrees":[
],
"#id":"https:abc/2009127993/author/person/3",
"hasAffiliation":[
"https:abc/author/organization/1"
],
"organization":[
[
{
"identifier":[
{
"#type":"idtype:uuid",
"#subtype":"idsubtype:affiliationInstanceId",
"#value":"aff2"
},
{
"#type":"idtype:OrgDB",
"#subtype":"idsubtype:afid",
"#value":"12345"
},
{
"#type":"idtype:OrgDB",
"#subtype":"idsubtype:dptid"
}
],
"organizations":[
],
"addressParts":[
],
"sourceText":"",
"text":" Medical University School of Medicine",
"#id":"https:abc/author/organization/1"
}
],
[
{
"identifier":[
{
"#type":"idtype:uuid",
"#subtype":"idsubtype:affiliationInstanceId",
"#value":"aff1"
},
{
"#type":"idtype:OrgDB",
"#subtype":"idsubtype:afid",
"#value":"7890"
},
{
"#type":"idtype:OrgDB",
"#subtype":"idsubtype:dptid"
}
],
"organizations":[
],
"addressParts":[
],
"sourceText":"",
"text":"K University",
"#id":"https:efg/author/organization/2"
}
]
]
}
}
]
IMPORTANT create unique constraints for Author.id and Organization.id!!
In this way you can process any json file with an unknown number of author elements and an unknown number of affiliated organisations
I have a ruby (2.4.0p0) rails (5.0.2) controller from which I wish to return a json result containing a list of Thing objects as well as some high level info (such as next and previous from Kaminari paging).
Consider a Thing with an association to Owner. Thing has a owner_id attribute.
For #things = Thing.page(1).per(2) I will be able to use
render json: #things
and get;
[
{ "id": 1, "owner_id": 1, "name": "thing1" },
{ "id": 2, "owner_id": 1, "name": "thing2" }
]
Good. If I then create a serializer called ThingSerializer.rb and define owner such that it adds "owner":"CatInHat" instead of "owner_id":1
This works as well;
[
{ "id": 1, "owner": "CatInHat", "name": "thing1" },
{ "id": 2, "owner": "CatInHat", "name": "thing2" }
]
This is good, but, my problem comes when I want to add higher level data and label the list as "results" such as when I try;
render json: { next:"some_url_link",previous:"some_other_url_link", results: #bags}
I'd like to get;
{ "next":some_url_link,
"prev":some_other_url_link,
"results":[ { "id": 1, "owner": "CatInHat", "name": "thing1" }, { "id": 2, "owner": "CatInHat", "name": "thing2" } ]
}
What I get is nearly the above but with "owner_id":1 instead of "owner":"CatInHat" - my serializer does not seem to be used when I label and nest my list of things. What is the appropriate way to use my serializer and get this output?
If I create config/initializers/active_model_serializers.rb and add
ActiveModel::Serializer.config.adapter = :json_api
It gives me an api which is similar but I don't know if it can be customized to fit the spec I need above.
thank you for any help
It looks like the serialization logic in render json: ... only kicks in if the attribute is an ActiveRecord object or an array of ActiveRecord objects. Since you are giving it a hash, it will not inspect the individual attributes and recursively apply the serializers.
You can try manually applying ThingSerializer:
render json: {
next: ...,
prev: ...,
results: #things.map { |thing|
ThingSerializer.new(thing).attributes
},
}
I am using the elasticsearch-rails and elasticsearch-model gems for my Ruby on Rails app, which is like a question-and-answer site.
My main question is: how do you tell Elasticsearch which documents to update when there was a change to a nested object that's nested in multiple documents?
I have one index my_index and mappings for question, and answer. In particular, question has a nested object with a user:
"question": {
"properties": {
"user": {
"type": "nested",
"properties": {
"created_at": {
"type": "date",
"format": "dateOptionalTime"
},
"name": {
"type": "string"
},
"id": {
"type": "long"
},
"email": {
"type": "string"
}
}
}
...
}
}
It's possible for a user to change his name, and I have hooks to update the user in Elasticsearch:
after_commit lambda { __elasticsearch__.index_document}, on: :update
But this isn't updating the appropriate question objects correctly, and I don't know what to pass to the index_document call to make sure it updates all the corresponding questions with the new user name. Does anyone know? It might even help me to see what a RESTful/curl request should look like?
Any help would be appreciated!
There are a couple of different ways you can go about this. They are all probably going to require some code changes, though. I don't think there is a way to do what you are asking directly, with your current setup.
You can read about the various options here. If you can set things up as a one-to-many relationship, then the parent/child relationship is probably the way to go. Then you could set up something like this:
PUT my_index
{
"mappings": {
"user": {
"properties": {...}
},
"question": {
"_parent": {
"type": "user"
},
"properties": {...}
}
}
}
And in that case you would be able to update users independently of questions. But it makes querying more complicated, which may or may not be a problem in your application code.
Given that you already have nested documents set up, you could simply query for all the documents that have that particular user as a nested document, with something like:
POST /test_index/question/_search
{
"filter": {
"nested": {
"path": "user",
"filter": {
"term": {
"user.id": 2
}
}
}
}
}
and once you have all the affected question documents you can modify the user name in each one and update all the documents with a bulk index request.
Here is some code I used to play around with that last bit:
http://sense.qbox.io/gist/d2a319c6b4e7da0d5ff910b4118549228d90cba0