Storing a List in a Vertex using Gremlin and Azure Cosmos Graph - neo4j

I am trying to store a list of changes made to a Vertex in the Vertex itself. Ideally I would want something like this:
{
"id": "95fcfa87-1c03-436d-b3ca-340cea926ee9",
"label": "person",
"type": "vertex",
"log": [{
"user": "user#user.dk",
"action": "update",
"timestamp": "22-03-2017",
"field": "firstName",
"oldValue": "Marco"
}
]
}
Using this method chain I am able to a achieve the following structure
graph.addV('person')
.property('firstName', 'Thomas')
.property(list, 'log', '22-03-2017')
.properties('log')
.hasValue('22-03-2017', '21-03-2017')
.property('user','user#user.dk')
.property('action', 'update')
.property('field', 'firstName')
.property('oldValue', 'Marco')
{
"id": "95fcfa87-1c03-436d-b3ca-340cea926ee9",
"label": "person",
"type": "vertex",
"properties": {
"firstName": [{
"id": "f23482a9-48bc-44e0-b783-3b74a2439a11",
"value": "Thomas"
}
],
"log": [{
"id": "5cfa35e1-e453-42e2-99b1-eb64cd853f22",
"value": "22-03-2017",
"properties": {
"user": "user#user.dk",
"action": "update",
"field": "firstName",
"oldValue": "Marco"
}
}
]
}
}
However this seems overly complex, as I will have to store a value and add properties to it.
Is it possible to add anonymous objects (i.e. without id and value) with the above mentioned data?

Not an actual solution to storing proper objects in a history log, but if you just use it as a log and don't have to access or query it by its properties, you could just put the serialised JSON in the value?
Something like along these lines should approximate the structure you're requesting:
dynamic entry = new JObject();
entry.user = "user#user.dk";
entry.action = "update";
entry.timestamp = "22-03-2017 12:34:56";
entry.field = "firstName";
entry.oldValue = "Marco";
graph.addV('person')
.property('firstName', 'Thomas')
.property(list, 'log', entry.ToString());
{
"id": "95fcfa87-1c03-436d-b3ca-340cea926ee9",
"label": "person",
"type": "vertex",
"properties": {
"firstName": [{
"id": "f23482a9-48bc-44e0-b783-3b74a2439a11",
"value": "Thomas"
}
],
"log": [{
"id": "5cfa35e1-e453-42e2-99b1-eb64cd853f22",
"value": "{\"user\":\"user#user.dk\",\"action\":\"update\",\"timestamp\":\"22-03-2017\",\"field\":\"firstName\",\"oldValue\":\"Marco\"}"
}
]
}
}
These log entries can easily be read, deserialised, used, and presented, but will not do much for queriability.

Related

Validate multiple duplicate objects in JSON-SCHEMA

I'd like to validate a json object with the json-schema, but that json object can duplicate its values ​​as many times as the user wants.
The keys of that object can be repeated as many times as the user wishes at the times the user create his json.
example 1: (collection with object)
{
"info":
[
{
"name": "aaron",
"email": "aaron.com"
}
]
}
JSON-SCHEMA of Example 1
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"email": {
"type": "string"
}
},
"required": [
"name",
"email"
]
}
example 2: (collection with 2 object)
{
"info":
[
{
"name": "aaron",
"email": "aaron.com"
},
{
"name": "misa",
"email": "misa.com"
}
]
}
JSON SCHEMA of example 2
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"info": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"email": {
"type": "string"
}
},
"required": [
"name",
"email"
]
},
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"email": {
"type": "string"
}
},
"required": [
"name",
"email"
]
}
]
}
},
"required": [
"info"
]
}
In short, what I am looking for is a dynamic json schema that no matter how many times the collection grows, it can use only 1 and not generate several.
As you're using draft-04, I'll quote from the draft-04 specification.
This means you want items to have an object value as opposed to an array of objects.
The value of "items" MUST be either an object or an array. If it is
an object, this object MUST be a valid JSON Schema. If it is an
array, items of this array MUST be objects, and each of these objects
MUST be a valid JSON Schema.
Draft-04 specificiation https://datatracker.ietf.org/doc/html/draft-fge-json-schema-validation-00#section-5.3.1
In JSON Schema 2020-12, items may ONLY be an object value, and you must use a different keyword for tuple like validation.
This worked for me-
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"info": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"email": {
"type": "string"
}
},
"required": [
"name",
"email"
]
}
]
"additionalItems":{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"email": {
"type": "string"
}
},
"required": [
"name",
"email"
]
}
}
},
"required": [
"info"
]
}

Avro Schema array without name

Question 1
I'm wondering whether below schema is valid or not for an Avro schema. Note that it is missing name in the first object of fields array.
{
"name": "AgentRecommendationList",
"type": "record",
"fields": [
{
"type": {
"type": "array",
"items": {
"name": "friend",
"type": "record",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "phoneNumber",
"type": "string"
},
{
"name": "email",
"type": "string"
}
]
}
}
}
]
}
Which actually designed to target below kind of data
[
{
"Name": "1",
"phoneNumber": "2",
"email": "3"
},
{
"Name": "1",
"phoneNumber": "2",
"email": "3"
},
{
"Name": "1",
"phoneNumber": "2",
"email": "3"
}
]
Based on reading below, seems like array without name like this are not permitted
Avro Schema failure
There is no way to define and avro schema with an array without a field name.
https://avro.apache.org/docs/current/spec.html#schema_complex
name: a JSON string providing the name of the field (required), and
I'm suspecting that below is the correct ones
{
"name": "AgentRecommendationList",
"type": "record",
"fields": [
{
"name": "friends",
"type": {
"type": "array",
"items": {
"name": "friend",
"type": "record",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "phoneNumber",
"type": "string"
},
{
"name": "email",
"type": "string"
}
]
}
}
}
]
}
And it should have a data like below, in order to do the avro conversion successfully
{
"friends": [
{
"Name": "1",
"phoneNumber": "2",
"email": "3"
},
{
"Name": "1",
"phoneNumber": "2",
"email": "3"
},
{
"Name": "1",
"phoneNumber": "2",
"email": "3"
}
]
}
Question 2
Does below schema is a valid schema? This target the array without name in first example...
{
"name": "AgentRecommendationList",
"type": "array",
"items": {
"name": "friend",
"type": "record",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "phoneNumber",
"type": "string"
},
{
"name": "email",
"type": "string"
}
]
}
}
I will appreciate if anyone can confirm my understanding... thanks!
For question 1...
Everything you have written is right. The first schema, as you mentioned, is not valid because each field within a record needs to have a name. The corrected schema is valid and the corrected data is right for the updated schema.
For question 2...
The schema in question two is valid, but the AgentRecommendationList name will get ignored. Arrays don't have names. This might sound strange after looking at the examples in question one, but in those the name is part of the field specification, not the array.

How to extract a nested nullable Avro Schema

The complete schema is the following:
{
"type": "record",
"name": "envelope",
"fields": [
{
"name": "before",
"type": [
"null",
{
"type": "record",
"name": "row",
"fields": [
{
"name": "username",
"type": "string"
},
{
"name": "timestamp",
"type": "long"
}
]
}
]
},
{
"name": "after",
"type": [
"null",
"row"
]
}
]
}
I wanted to programmatically extract the following sub-schema:
{
"type": "record",
"name": "row",
"fields": [
{
"name": "username",
"type": "string"
},
{
"name": "timestamp",
"type": "long"
}
]
}
As you see, field "before" is nullable. I can extract it's schema by doing:
schema.getField("before").schema()
But the schema is not a record as it contains NULL at the beginning(UNION type) and I can't go inside to fetch schema of "row".
["null",{"type":"record","name":"row","fields":[{"name":"username","type":"string"},{"name":"tweet","type":"string"},{"name":"timestamp","type":"long"}]}]
I want to fetch the sub-schema because I want to create GenericRecord out of it. Basically I want to create two GenericRecords "before" and "after" and add them to the main GenericRecord created from full schema.
Any help will be highly appreciated.
Good news, if you have a union schema, you can go inside to fetch the list of possible options:
Schema unionSchema = schema.getField("before").schema();
List<Schema> unionSchemaContains = unionSchema.getTypes();
At that point, you can look inside the list to find the one that corresponds to the Type.RECORD.

Webhook input not available in scripting step?

I have an internal app that uses a webhook listener and some scripting to manipulate the input data. I'm posting this to it:
curl -X POST -d '{
"assignment_id": 12345,
"updated_custom_fields": [{
"name": "RNVIDAYEBB",
"value": "updated!"
},
{
"name": "QUFTXSIBYA",
"value": "and me too"
}
],
"custom_fields": [{
"id": 981,
"name": "RDEXDPVKRD",
"fields": [
{
"id": 4096,
"name": "RNVIDAYEBB",
"default": "EDJEAJICYW",
"required": true,
"value": "Blah"
},
{
"id": 4097,
"name": "QUFTXSIBYA",
"default": "",
"required": true,
"value": ""
}]
}]
}' "https://hooks.zapier.com/hooks/catch/......"
My script is as follows:
update_custom_fields_by_name_pre_write: function(bundle) {
var updatedFields = _.map(bundle.request.data.custom_fields, function(group) {
return _.map(group.fields, function(field) {
return _.extend(field, _.findWhere(bundle.request.data.updated_custom_fields, { name: field.name} ));
});
});
bundle.request.data = updatedFields;
return bundle.request;
}
I know that the merging logic is good, but it appears that the custom_fields and updated_custom_fields arrays are not present in the bundle.request.data object. Anyone know how to get access to them in the script?
It seems like you should be using update_custom_fields_by_name_catch_hook to capture the incoming static webhook data (instead of _pre_write). If you use that, you can capture the data within bundle.cleaned_request.custom_fields and bundle.cleaned_request.updated_custom_fields.

How can I describe complex json model in swagger

I'm trying to use Swagger to describe web-api I'm building.
The problem is that I can't understand how to describe complex json object?
For example, how to describe this objects:
{
name: "Jhon",
address: [
{
type: "home",
line1: "1st street"
},
{
type: "office",
line1: "2nd street"
}
]
}
Okay, so based on the comments above, you want the following schema:
{
"definitions": {
"user": {
"type": "object",
"required": [ "name" ],
"properties": {
"name": {
"type": "string"
},
"address": {
"type": "array",
"items": {
"$ref": "#/definitions/address"
}
}
}
},
"address": {
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": [ "home", "office" ]
},
"line1": {
"type": "string"
}
}
}
}
}
I've made a few assumptions to make the sample a bit more complicated, to help in the future.
For the "user" object, I've declared that the "name" field is mandatory. If, for example, you also need the address to be mandatory, you can change the definition to "required": [ "name", "address" ].
We basically use a subset of json-schema to describe the models. Of course not everyone knows it, but it's fairly simple to learn and use.
For the address type you can see I also set the limit to two options - either home or office. You can add anything to that list, or remove the "enum" entirely to remove that constraint.
When the "type" of a property is "array", you need to accompany it with "items" which declares the internal type of the array. In this case, I referenced another definition, but that definition could have been inline as well. It's normally easier to maintain that way, especially if you need the "address" definition alone or within other models.
As requested, the inline version:
{
"definitions": {
"user": {
"type": "object",
"required": [
"name"
],
"properties": {
"name": {
"type": "string"
},
"address": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": [
"home",
"office"
]
},
"line1": {
"type": "string"
}
}
}
}
}
}
}
}

Resources