I'm parsing some JSON from a mixed content source, and with it trying to store it with ActiveRecord.
At the moment I'm using a ton of variables:
json['settings']['newsletters']['weekly']
json['info']['address']['city']
Or trying to make things a little easier:
newsletters = json['settings']['newsletters']
newsletters['weekly']
address = json['info']['address']
address['city']
But this is all getting very messy, and not DRY.
I think the better way to do this would be to iterate over each element that is a hash (and therefore 'complex'), and assign it it's own object. This way, I don't have to declare a trillion variables, they can instead be assigned from the context of the JSON input.
So, I can do something like this:
user = json['input']
user.settings.newsletters.weekly
user.info.address.city
This is inspired by what ActiveResource documents:
# Any complex element (one that contains other elements) becomes its own object:
#
# {"id":1,"first":"Tyler","address":{"street":"Paper St.","state":"CA"}}
tyler = Person.find(1)
tyler.address # => <Person::Address::xxxxx>
tyler.address.street # => 'Paper St.'
Here is the JSON, reduced for brevity's sake:
{
"username": "robert_fitzsimmonds",
"emails": [{
"id_number": 1,
"address": "robert_fitzsimmonds#yahoo.com",
"confirmed": false
}, {
"id_number": 2,
"address": "robert_fitzsimmonds#gmail.com",
"confirmed": true
}],
"settings": {
"marketing": {
"main": true,
"weekly": false,
"daily": false
},
"language": "English"
},
"info": {
"address": {
"line_1": "31 Mole Road",
"line_2": "",
"city": "London",
"post_code": "NE4 5RJ"
},
"shared_account": false
}
}
Would such an iteration be the most efficient solution, or is it best to stick to long, messy variables?
Use the hash_dot gem if you can https://github.com/adsteel/hash_dot
Related
The API I'm trying to describe has a structure where the root object can contain an arbitrary number of child objects (properties that are themselves objects). The "key", or property in the root object, is the unique identifier of the child object, and the value is the rest of the child object's data.
{
"child1": { ... bunch of stuff ... },
"child2": { ... bunch of stuff ... },
...
}
This could similarly be modeled as an array, e.g.:
[
{ "id": "child1", ... bunch of stuff ... },
{ "id": "child2", ... bunch of stuff ... },
...
]
but this both makes it structurally less clear what the identifying property is and makes uniqueness among the children's ID implicit rather than explicit, so we want to use an object, or a map.
I've seen the Swagger documentation for Model with Map/Dictionary Properties, but that doesn't adequately suit my use case. Writing something like:
"Parent": {
"additionalProperties": {
"$ref": "#/components/schemas/Child",
}
Yields something like this:
This adequately communicates the descriptiveness of the value in the property, but how do I document what the restrictions are for the "key" in the object? Ideally I'd like to say something like "it's not just any arbitrary string, it's the ID that corresponds to the child". Is this supported in any way?
Your example is correct.
how do I document what the restrictions are for the "key" in the object? Ideally I'd like to say something like "it's not just any arbitrary string, it's the ID that corresponds to the child". Is this supported in any way?
OpenAPI 3.1
OAS 3.1 fully supports JSON Schema 2020-12, including patternProperties. This keyword lets you define the format of dictionary keys by using a regular expression:
"Parent": {
"type": "object",
"patternProperties": {
"^child\d+$": {
"$ref": "#/components/schemas/Child"
}
},
"description": "A map of `Child` schemas, where the keys are IDs that correspond to the child"
}
Or, if the property names are defined by an enum, you can use propertyNames to define that enum:
"Parent": {
"type": "object",
"propertyNames": {
"enum": ["foo", "bar"]
},
"additionalProperties": {
"$ref": "#/components/schemas/Child"
}
}
OpenAPI 3.0 and 2.0
Dictionary keys are assumed to be strings, but there's no way to limit the contents/format of keys. You can document any restrictions and specifics verbally in the schema description. Adding schema examples could help illustrate what your dictionary/map might look like.
"Parent": {
"type": "object",
"additionalProperties": {
"$ref": "#/components/schemas/Child"
},
"description": "A map of `Child` schemas, where the keys are IDs that correspond to the child",
"example": {
"child1": { ... bunch of stuff ... },
"child2": { ... bunch of stuff ... },
}
If the possible key names are known (for example, they are part of an enum), you can define your dictionary as a regular object and the keys as individual object properties:
// Keys can be: key1, key2, key3
"Parent": {
"type": "object",
"properties": {
"key1": { "$ref": "#/components/schemas/Child" },
"key2": { "$ref": "#/components/schemas/Child" },
"key3": { "$ref": "#/components/schemas/Child" }
}
}
Then you can add "additionalProperties": false to really ensure that only those keys are used.
My project is a micro-services that connect two major services, my project fetches data from one server, format the data, and then use the data to generate an XML file, and then upload the XML to another service. I'm just wondering if there is any design pattern for this kind of micro-services.
this is the JSON received from the backend server:
{
"employee_id": 100464,
"organization_id": 93,
"start_date": "2018-09-05",
"first_name": "Tom",
"departments": [
{
"id": 2761,
"name": "Sale",
"organization_id": 93
},
{
"id": 2762,
"name": "Product",
"organization_id": 93
}
],
"primary_department": {
"id": 2761,
"name": "Product",
"organization_id": 93
}
}
This is the data format I want for, so I need to do some data formatting:
{
"employee_id": 100464,
"organization_id": 93,
"first_name": "Tom",
"target_department": {
"department_id": 2761,
"name": "Product",
"organization_id": 93,
"is_primary_department": true
}
}
the logic to determine the target_department is
departments = hsh.delete :departments
primary_department = hsh.delete :primary_department
hsh[:target_department] = departments.select do |department|
department[:id] ==another_obj[:group_id]
end.first
hsh[:target_department][:is_home_department] = (hsh[:target_department][:id] == primary_department[:id])
hsh[:target_department][:department_id] = hsh[:target_department].delete :id
As you can see, I need to remove, rename, and reformat attributes and the structure of the data.
During the process, there are many potential issues: attributes not existed?
My question is what's the best practice for dealing with this issue from programming designing perspective?
I am using Rails, so any good gem or project is dealing with a similar issue?
I don't know about best practices, but this ought to do what you want.
{
"employee_id": 100464,
"organization_id": 93,
"start_date": "2018-09-05",
"first_name": "Tom",
"departments": [
{
"id": 2761,
"name": "Sale",
"organization_id": 93
},
{
"id": 2762,
"name": "Product",
"organization_id": 93
}
],
"primary_department": {
"id": 2761,
"name": "Product",
"organization_id": 93
}
}.with_indifferent_access.except(:start_date, :departments).transform_keys do |k|
k == 'primary_department' ? 'target_department' : k
end.tap do |hsh|
if hsh['target_department']
hsh['target_department']['is_primary_department'] = true
hsh['target_department']['department_id'] = hsh['target_department']['id']
hsh['target_department'].delete('id')
end
puts hsh
end
In console, this will return:
{
"employee_id"=>100464,
"organization_id"=>93,
"first_name"=>"Tom",
"target_department"=>{
"name"=>"Product",
"organization_id"=>93,
"is_primary_department"=>true,
"department_id"=>2761
}
}
BTW, in your example output, you show the target_department name as "aProduct". Is that a typo?
Working directly with json/xml type data programmatically is often tedious. I once worked at a company that did this EVERYWHERE and it was painful. I would suggest deserializing your data into an object graph representation. Also have a result class that can be constructed by querying/processing your input object graph. Then just serialize your result object back to json at the end.
You can likely find a built-in or open source solution for handling the serialization and deserialization for you.
I have a ruby (2.4.0p0) rails (5.0.2) controller from which I wish to return a json result containing a list of Thing objects as well as some high level info (such as next and previous from Kaminari paging).
Consider a Thing with an association to Owner. Thing has a owner_id attribute.
For #things = Thing.page(1).per(2) I will be able to use
render json: #things
and get;
[
{ "id": 1, "owner_id": 1, "name": "thing1" },
{ "id": 2, "owner_id": 1, "name": "thing2" }
]
Good. If I then create a serializer called ThingSerializer.rb and define owner such that it adds "owner":"CatInHat" instead of "owner_id":1
This works as well;
[
{ "id": 1, "owner": "CatInHat", "name": "thing1" },
{ "id": 2, "owner": "CatInHat", "name": "thing2" }
]
This is good, but, my problem comes when I want to add higher level data and label the list as "results" such as when I try;
render json: { next:"some_url_link",previous:"some_other_url_link", results: #bags}
I'd like to get;
{ "next":some_url_link,
"prev":some_other_url_link,
"results":[ { "id": 1, "owner": "CatInHat", "name": "thing1" }, { "id": 2, "owner": "CatInHat", "name": "thing2" } ]
}
What I get is nearly the above but with "owner_id":1 instead of "owner":"CatInHat" - my serializer does not seem to be used when I label and nest my list of things. What is the appropriate way to use my serializer and get this output?
If I create config/initializers/active_model_serializers.rb and add
ActiveModel::Serializer.config.adapter = :json_api
It gives me an api which is similar but I don't know if it can be customized to fit the spec I need above.
thank you for any help
It looks like the serialization logic in render json: ... only kicks in if the attribute is an ActiveRecord object or an array of ActiveRecord objects. Since you are giving it a hash, it will not inspect the individual attributes and recursively apply the serializers.
You can try manually applying ThingSerializer:
render json: {
next: ...,
prev: ...,
results: #things.map { |thing|
ThingSerializer.new(thing).attributes
},
}
I currently have a nested JSON object which resembles
{
"People": [
{
"Name": "James",
"Age": "18",
"Gender": "Male",
"Sports": []
},
{
"Name": "Sarah",
"Age": "19",
"Gender": "Female",
"Sports": [
"Soccer",
"Basketball",
"Football"
]
}
]
}
Being new to Ruby, I aim to filter throught the entire json and return only the json object/objects in which the "Sports" array has content. So in the above scenario I expect to obtain the object below as a final outcome:
{
"Name": "Sarah",
"Age": "19",
"Gender": "Female",
"Sports": [
"Soccer",
"Basketball",
"Football"
]
}
Will I have to initiate a new method to perform such an act? Or would using regular ruby calls work in this case?
Although #philipyoo answer is right, it miss an explanation on how to "filter" the parsed JSON. If you are new to ruby, take a look at Array#keep_if : http://ruby-doc.org/core-2.2.0/Array.html#method-i-keep_if
require 'json'
people = JSON.parse("{long JSON data ... }")
people_with_sports = people.fetch('People', []).keep_if do |person|
!person.fetch('Sports', []).empty?
end
If you're getting a JSON object from a request, you want to parse it and then you can traverse the hash and arrays to find the information you need. See http://ruby-doc.org/stdlib-2.0.0/libdoc/json/rdoc/JSON.html
In your case, something like this:
require 'json'
parsed_json = JSON.parse('{"People": [ ... ]}')
parsed_json["People"].each do |person|
puts person if person["name"] == "Sarah"
end
I am thinking of writing a web application that crawls an API and returns this information in JSON form.
However, I am only after one number, then current price (in this sample, "227"). How can I access that in Ruby? I have no clue where to begin. I've never dealt with text like this.
For discussion's sake, suppose I save this output into instance variable #information
{
"item": {
"icon": "http://services.runescape.com/m=itemdb_rs/4332_obj_sprite.gif?id=4798",
"icon_large": "http://services.runescape.com/m=itemdb_rs/4332_obj_big.gif?id=4798",
"id": 4798,
"type": "Ammo",
"typeIcon": "http://www.runescape.com/img/categories/Ammo",
"name": "Adamant brutal",
"description": "Blunt adamantite arrow...ouch",
"current": {
"trend": "neutral",
"price": 227
},
"today": {
"trend": "neutral",
"price": 0
},
"day30": {
"trend": "positive",
"change": "+1.0%"
},
"day90": {
"trend": "positive",
"change": "+1.0%"
},
"day180": {
"trend": "positive",
"change": "+2.0%"
},
"members": "true"
}
}
First follow this post to parse this JSON in to Hash
Parsing a JSON string in Ruby
say the parsed hash name is my_hash then the following should give you price
my_hash['item']['current']['price']
Edit:
As you said you want to save it in #information
#information = my_hash['item']['current']['price']
Even you can use hashie it gives your json into readable structural code
Install Hashie
gem install hashie
then in your code all that json take in a variable my_json
myhash = Hashie::Mash.new(my_json)
#information = my_hash.item.current.price
Tips:-
if your json is dynamic and it may respond some other structural element so you can maintain exceptional code
#information = my_hash.item.try(:current).try(:price)