I have a JSON File which looks like:
{"business_id": "vcNAWiLM4dR7D2nwwJ7nCA", "full_address": "4840 E Indian School Rd\nSte 101\nPhoenix, AZ 85018", "hours": {"Tuesday": {"close": "17:00", "open": "08:00"}, "Friday": {"close": "17:00", "open": "08:00"}, "Monday": {"close": "17:00", "open": "08:00"}, "Wednesday": {"close": "17:00", "open": "08:00"}, "Thursday": {"close": "17:00", "open": "08:00"}}, "open": true, "categories": ["Doctors", "Health & Medical"], "city": "Phoenix", "review_count": 7, "name": "Eric Goldberg, MD", "neighborhoods": [], "longitude": -111.98375799999999, "state": "AZ", "stars": 3.5, "latitude": 33.499313000000001, "attributes": {"By Appointment Only": true}, "type": "business"}
{"business_id": "JwUE5GmEO-sH1FuwJgKBlQ", "full_address": "6162 US Highway 51\nDe Forest, WI 53532", "hours": {}, "open": true, "categories": ["Restaurants"], "city": "De Forest", "review_count": 26, "name": "Pine Cone Restaurant", "neighborhoods": [], "longitude": -89.335843999999994, "state": "WI", "stars": 4.0, "latitude": 43.238892999999997, "attributes": {"Take-out": true, "Good For": {"dessert": false, "latenight": false, "lunch": true, "dinner": false, "breakfast": false, "brunch": false}, "Caters": false, "Noise Level": "average", "Takes Reservations": false, "Delivery": false, "Ambience": {"romantic": false, "intimate": false, "touristy": false, "hipster": false, "divey": false, "classy": false, "trendy": false, "upscale": false, "casual": false}, "Parking": {"garage": false, "street": false, "validated": false, "lot": true, "valet": false}, "Has TV": true, "Outdoor Seating": false, "Attire": "casual", "Alcohol": "none", "Waiter Service": true, "Accepts Credit Cards": true, "Good for Kids": true, "Good For Groups": true, "Price Range": 1}, "type": "business"}
I want to to traverse this file and convert individual structures into a Ruby hash so that I can access the different hashes to perform actions on them.
This is a hash of the first structure of the JSON file:
{"business_id"=>"uUsfpN81JCMKyH6c0D0bTg", "full_address"=>"1910 Village Center Cir\nSte 6\nSummerlin\nLas Vegas, NV 89134", "hours"=>{}, "open"=>true, "categories"=>["Food", "Desserts", "Italian", "Pizza", "Restaurants"], "city"=>"Las Vegas", "review_count"=>6, "name"=>"Rocco's NY Pizza & Pasta", "neighborhoods"=>["Summerlin"], "longitude"=>-115.3041999, }
Similarly I want to have hashes of all other JSON structures. How can I do this in Ruby on Rails?
The file you got it's not a JSON file, but simply a text file with a bunch of JSON-encoded objects. Simply loop over the lines.
require 'json'
File.read("file.txt").split("\n").each do |line|
JSON.parse(line)
end
or use .map instead of .each if you want return an Array of Hash.
File.foreach("file.txt").map do |line|
JSON.parse(line)
end
File.read("file.txt").split("\n").map do |line|
JSON.parse(line)
end
Also note this has nothing to do with Rails, it's pure Ruby code.
Something like this will return an array of hashes retrieved from the file:
require 'json'
array_of_hashes = File.foreach('file.txt').map{ |l| JSON[l] }
Be warned though, this isn't scalable because you'll be pulling the entire file into memory, and if that resulting array is larger than the available space you'll see your program go to a crawl.
A more scalable approach would be to read each line, convert it to a Ruby object using JSON's parser, then immediately stuff the resulting object into a database record, and finally iterate over the database to do your massaging. SQLite3, PostgreSQL, MySQL would all be reasonable choices for this.
Have you tried JSON.parse
require 'json'
JSON.parse(your_json) #=> your expected result
Related
According to https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-project-versions/#api-group-project-versions the Jira v2 REST API supports a query param for project versions.
Filter the results using a literal string. Versions with matching name or description are returned (case insensitive).
I can't seem to make this work as advertised.
https://my-jira-host/rest/api/2/project/project-key/version?query=july&orderBy=name returns versions that have nothing to do with "july" at all.
{
"self": "https://my-jira-host/rest/api/2/project/project-key/version?maxResults=50&orderBy=name&startAt=0",
"nextPage": "https://my-jira-host/rest/api/2/project/project-key/version?maxResults=50&orderBy=name&startAt=50",
"maxResults": 50,
"startAt": 0,
"total": 56,
"isLast": false,
"values": [
{
"self": "https://my-jira-host/rest/api/2/version/10302",
"id": "10302",
"name": "2017.07 Phoenix",
"archived": false,
"released": true,
"startDate": "2017-07-03",
"releaseDate": "2017-07-31",
"userStartDate": "03/Jul/17",
"userReleaseDate": "31/Jul/17",
"projectId": 10607
},
{
"self": "https://my-jira-host/rest/api/2/version/10303",
"id": "10303",
"name": "2017.08 Zelos",
"archived": false,
"released": true,
"startDate": "2017-08-01",
"releaseDate": "2017-08-31",
"userStartDate": "01/Aug/17",
"userReleaseDate": "31/Aug/17",
"projectId": 10607
},
{
"self": "https://my-jira-host/rest/api/2/version/10304",
"id": "10304",
"name": "2017.09 Horen",
"archived": false,
"released": true,
"startDate": "2017-09-01",
"releaseDate": "2017-09-30",
"userStartDate": "01/Sep/17",
"userReleaseDate": "30/Sep/17",
"projectId": 10607
},
Turns out I was looking at the wrong documentation - Jira Cloud instead of Jira Server. The later doesn't support the query parameter.
Source: https://docs.atlassian.com/software/jira/docs/api/REST/9.0.0/#project-getProjectVersionsPaginated
I have a simple avro schema, from which I generated a java class using the avro-maven-plugin.
The avro schema is as follows:
{
"type": "record",
"name": "addressGeo",
"namespace": "com.mycompany",
"doc": "Best record address and list of geos",
"fields": [
{
"name": "version",
"type": "int",
"default": 1,
"doc": "version the class"
},
{
"name": "eventType",
"type": "string",
"default": "addressGeo",
"doc": "event type"
},
{
"name": "parcelId",
"type": "long",
"doc": "ParcelID of the parcel. Join parcelid and sequence with ParcelInfo"
},
{
"name": "geoCodes",
"type": {"type": "array", "items": "com.mycompany.geoCode"},
"doc": "Multiple Geocodes, with restrictions information"
},
{
"name": "brfAddress",
"type": ["null", "com.mycompany.address"],
"doc": "Address cleansed version of BRF"
}
]
}
If I construct a simple object using the builder, and serialize it using json, I get the following output:
{
"version": 1,
"eventType": {
"bytes": [
97,
100,
100,
114,
101,
115,
115,
71,
101,
111
],
"length": 10,
"string": null
},
"parcelId": 1,
"geoCodes": [
{
"version": 1,
"latitude": 1,
"longitude": 1,
"geoQualityCode": "g",
"geoSourceTypeID": 1,
"restrictions": "NONE"
}
],
"brfAddress": {
"version": 1,
"houseNumber": "1",
"houseNumberFraction": null,
"streetDirectionPrefix": null,
"streetName": "main",
"streetSuffix": "street",
"streetDirectionSuffix": null,
"fullStreetAddress": "1 main street, seattle, wa, 98101",
"unitPrefix": null,
"unitNumber": null,
"city": "seattle",
"state": "wa",
"zipCode": "98101",
"zipPlusFour": null,
"addressDPV": "Y",
"addressQualityCode": "good",
"buildingNumber": "1",
"carrierRoute": "t",
"censusTract": "c",
"censusTractAndBlock": "b",
"dataCleanerTypeID": 1,
"restrictions": "NONE"
}
}
Note the output of the eventType field. It is coming through as an array of bytes whereas the type of the field is a CharSequence.
Any idea why serialization is doing this? It works fine for other types that are strings.
I am using google-gson to serialize the object to json.
You might be working with a older version of avro, that uses CharSequence. Ideally string type should be java String type. I would suggest to update the avro version or have a look at this one - Apache Avro: map uses CharSequence as key
I want to be able to retrieve the tier and division from this code, however when using the response object from HTTParty and doing res[0]["#{id}"]["tier"] it comes up with "cannot implicitly convert string to integer", which means it expects an integer, but I don't know where
This is the response I get (I'm doing it in a loop which is why I'm putting in the ID with "#{id}")
{"37714607": [
{
"queue": "RANKED_SOLO_5x5",
"name": "Diana's Patriots",
"entries": [{
"leaguePoints": 32,
"isFreshBlood": false,
"isHotStreak": false,
"division": "IV",
"isInactive": false,
"isVeteran": false,
"losses": 65,
"playerOrTeamName": "Wicked7000",
"playerOrTeamId": "37714607",
"wins": 59
}],
"tier": "GOLD"
},
{
"queue": "RANKED_TEAM_5x5",
"name": "Nasus's Justicars",
"entries": [{
"leaguePoints": 81,
"isFreshBlood": false,
"isHotStreak": false,
"division": "V",
"isInactive": false,
"isVeteran": false,
"losses": 73,
"playerOrTeamName": "Pink Fedoras",
"playerOrTeamId": "TEAM-5ffedf90-45ba-11e4-9e4b-c81f66db8bc5",
"wins": 73
}],
"tier": "SILVER"
},
{
"queue": "RANKED_TEAM_3x3",
"name": "Cassiopeia's Marksmen",
"entries": [{
"leaguePoints": 0,
"isFreshBlood": false,
"isHotStreak": true,
"division": "I",
"isInactive": false,
"isVeteran": false,
"losses": 3,
"playerOrTeamName": "The Booty Brothers",
"playerOrTeamId": "TEAM-53a65b60-ff2d-11e4-9e51-c81f66dba0e7",
"wins": 7
}],
"tier": "BRONZE"
}
]}
As your json something like below
{"37714607": [
{
"queue": "RANKED_SOLO_5x5",
"name": "Diana's Patriots",
"entries": [{
"leaguePoints": 32,
"isFreshBlood": false,
"isHotStreak": false,
"division": "IV",
"isInactive": false,
"isVeteran": false,
"losses": 65,
"playerOrTeamName": "Wicked7000",
"playerOrTeamId": "37714607",
"wins": 59
}],
"tier": "GOLD"
},
so it will first id = "37714607" then an array start([) the array contains hashes so first hash has "tier" key
so it should be
tiers = []
res["#{id}"].each do |result| #id = 37714607
tiers << result["tier"]
end
Seems like you need to do res[id.to_s][0]["tier"] instead – first take the root key, then first element (you did it vice versa).
I'm new to the SurveyMonkey API and it hasn't been too difficult to get payloads back from API calls, but right now I'm trying to get back what responses a specific respondent gave.
I have a survey which has two respondents, the first question on the survey asks the user to enter three pieces of information: Their Name, an ID and today's date.
So, if I do a call to get_survey_details, I can see the questions just fine. For example
obj.pages[0].questions[0].answers[0].answerid: "xxxxxxxx" //some long ID
obj.pages[0].questions[0].answers[0].text: "Enter Your Name"
obj.pages[0].questions[0].answers[0].type: "row"
There's a couple more pieces of information in that object, like whether the question is visible, etc., but these seem to be the pertinent pieces to the question I have.
So! I make another call to get_responses using the same survey_id and respondent_id (there's only two so actually I get them both).
In the resulting payload I get an array of 2 objects (one to hold each respondents responses). So I look in the first (obj[0]) and I see an array of questions and the respondent id. Fine. I look in the questions array and I see one object for each question and in each of those an answers object.
so that's:
obj[0].questions[0].answers[0].col: "yyyyyy" //some long ID
obj[0].questions[0].answers[0].row: "nnnnnn" //some other long ID
No response text. just this row/col business.
At this point, I'm super-confused (which is like regular confused, but with a cape). Where the heck are the respondents actual responses?
What the heck does "row" and "column" reference? Do I have to do some other API call with the row and/or column in order to get the text of the respondent's response?
I've looked through the documentation (and will continue to do so after posting this) and through stackoverflow to see if anyone else has asked this before. There was one question that came close, but really they were just forgetting to pair 'get_responses' with 'get_survey_details'. I'm doing that, but am still lost as ever. And I don't see any documentation really explaining in detail how this row/column concept works for mapping responses to the text of the response. :/
I know this is a really long-winded question, but I'm just so confused as to how to actually get responses out of this API. :(
Thanks for reading.
The text for a given response should come through under the "text" key. e.g. for a survey that only consists of an essay style question:
{
"status": 0,
"data": [
{
"respondent_id": "123456",
"questions": [
{
"answers": [
{
"text": "This is an essay style answer.",
"row": "0"
}
],
"question_id": "78910"
}
]
}
]
}
"row" and "col" literally reference the row and column of an answer - e.g. in a matrix question, there will be a list of rows for different questions ("what did you think of the hotel?") and ratings ("bad, okay, great") - and each answer is a combination of these. For a regular multiple choice question there will be multiple rows and only one column.
Calling "get_responses" with the correct respondent_id should provide you with the text response that you want. It's only the fixed details of the answer stored in the survey itself you should have to look up (provided in get_survey_details).
Using GET : /surveys/{survey_id}/details, we can get the corresponding question Ids along with the answer Ids.
{
"pages": [
{
"href": "https://api.surveymonkey.net/v3/surveys/87263608/pages/260492760",
"description": "",
"questions": [
{
"sorting": null,
"family": "matrix",
"subtype": "rating",
"required": {
"text": "This question requires an answer.",
"amount": "0",
"type": "all"
},
"answers": {
"rows": [
{
"visible": true,
"text": "",
"position": 1,
"id": "10788526669"
}
],
"choices": [
{
"description": "Not at all likely",
"weight": -100,
"id": "10788526670",
"visible": true,
"is_na": false,
"text": "Not at all likely - 0",
"position": 1
},
{
"description": "",
"weight": -100,
"id": "10788526671",
"visible": true,
"is_na": false,
"text": "1",
"position": 2
},
{
"description": "",
"weight": -100,
"id": "10788526672",
"visible": true,
"is_na": false,
"text": "2",
"position": 3
},
{
"description": "",
"weight": -100,
"id": "10788526673",
"visible": true,
"is_na": false,
"text": "3",
"position": 4
},
{
"description": "",
"weight": -100,
"id": "10788526674",
"visible": true,
"is_na": false,
"text": "4",
"position": 5
},
{
"description": "",
"weight": -100,
"id": "10788526675",
"visible": true,
"is_na": false,
"text": "5",
"position": 6
},
{
"description": "",
"weight": -100,
"id": "10788526676",
"visible": true,
"is_na": false,
"text": "6",
"position": 7
},
{
"description": "",
"weight": 0,
"id": "10788526677",
"visible": true,
"is_na": false,
"text": "7",
"position": 8
},
{
"description": "",
"weight": 0,
"id": "10788526678",
"visible": true,
"is_na": false,
"text": "8",
"position": 9
},
{
"description": "",
"weight": 100,
"id": "10788526679",
"visible": true,
"is_na": false,
"text": "9",
"position": 10
},
{
"description": "Extremely likely",
"weight": 100,
"id": "10788526680",
"visible": true,
"is_na": false,
"text": "Extremely likely - 10",
"position": 11
}
]
},
"visible": true,
"href": "https://api.surveymonkey.net/v3/surveys/87263608/pages/260492760/questions/1044924866",
"headings": [
{
"heading": "How likely is it that you would recommend XYZ to a friend or colleague?"
}
],
"position": 1,
"validation": null,
"id": "1044924866",
"forced_ranking": false
},
{
"sorting": null,
"family": "single_choice",
"subtype": "vertical",
"required": null,
"answers": {
"choices": [
{
"visible": true,
"text": "High Interest",
"position": 1,
"id": "10788529403"
},
{
"visible": true,
"text": "Long process",
"position": 2,
"id": "10788529404"
},
{
"visible": true,
"text": "Low XYZ Amount",
"position": 3,
"id": "10788529405"
},
{
"visible": true,
"text": "Lot of Documents",
"position": 4,
"id": "10788529406"
},
{
"visible": true,
"text": "Bad customer service",
"position": 5,
"id": "10788529407"
}
]
},
"visible": true,
"href": "https://api.surveymonkey.net/v3/surveys/87263608/pages/260492760/questions/1044925207",
"headings": [
{
"heading": "What is the most important issue which we need to address for overall a better service?"
}
],
"position": 2,
"validation": null,
"id": "1044925207",
"forced_ranking": false
}
],
"title": "",
"position": 1,
"id": "260492760",
"question_count": 2
}
],
}
We can use these ids to decipher the answer we get after fetching responses using get response API(Bulk or each respondent).
For eg:,
If my survey has two questions, like
Then after fetching the responses we get a json like this:
{
"total_time": 34,
"href": "https://api.surveymonkey.net/v3/collectors/94630092/responses/5120000552",
"custom_variables": {},
"ip_address": "182.76.20.30",
"id": "5120000552",
"logic_path": {},
"date_modified": "2016-12-01T11:01:11+00:00",
"response_status": "completed",
"custom_value": "LAI100023",
"analyze_url": "http://www.surveymonkey.com/analyze/browse/EvaBWWcU9K1XTH_2FFFBTfFul4ge94MwVWvBk0eAFDJ3c_3D?respondent_id=5120000552",
"pages": [
{
"id": "260492760",
"questions": [
{
"id": "1044924866",
"answers": [
{
"choice_id": "10788526677",
"row_id": "10788526669"
}
]
},
{
"id": "1044925207",
"answers": [
{
"choice_id": "10788529404"
}
]
}
]
}
],
"page_path": [],
"recipient_id": "2743199128",
"collector_id": "94630092",
"date_created": "2016-12-01T11:00:37+00:00",
"survey_id": "87263608",
"collection_mode": "default",
"edit_url": "http://www.surveymonkey.com/r/?sm=SfTljxZSoBFvaRUeGSI6L813qctjfG_2FDCVcqCks7CDc4TcJC_2BNHqmPYD7NNTcvST",
"metadata": {
"contact": {
"first_name": {
"type": "string",
"value": "John"
},
"last_name": {
"type": "string",
"value": "Doe"
},
"email": {
"type": "string",
"value": "neeta#xyz.com"
}
}
}
}
We can map the questions and answers using their IDs in this response with the ids we got from survey details. For open ended text questions, we get direct typed responses.
I am trying to download tweets from the London area using
curl --user xx:xx -X POST -d 'locations=-0.651,51.208,0.425,51.781' https://stream.twitter.com/1.1/statuses/filter.json > twitter.london.json
I am getting tweets that are way off. For example in (-6.71658144,50.48812681). Why is that?
The tweet:
{
"created_at": "Mon Apr 08 12:26:40 +0000 2013",
"id": 321237656321286144,
"id_str": "321237656321286144",
"text": "4dc9c54f70000000",
"source": "\u003ca href=\"http:\/\/www.google.com\/\" rel=\"nofollow\"\u003eGoogle\u003c\/a\u003e",
"truncated": false,
"in_reply_to_status_id": null,
"in_reply_to_status_id_str": null,
"in_reply_to_user_id": null,
"in_reply_to_user_id_str": null,
"in_reply_to_screen_name": null,
"user": {
"id": 61043461,
"id_str": "61043461",
"name": "GooGuns",
"screen_name": "googuns_prod",
"location": "",
"url": null,
"description": null,
"protected": false,
"followers_count": 86,
"friends_count": 0,
"listed_count": 2,
"created_at": "Tue Jul 28 22:49:22 +0000 2009",
"favourites_count": 0,
"utc_offset": -18000,
"time_zone": "Eastern Time (US & Canada)",
"geo_enabled": true,
"verified": false,
"statuses_count": 417708,
"lang": "en",
"contributors_enabled": false,
"is_translator": false,
"profile_background_color": "C0DEED",
"profile_background_image_url": "http:\/\/a0.twimg.com\/images\/themes\/theme1\/bg.png",
"profile_background_image_url_https": "https:\/\/si0.twimg.com\/images\/themes\/theme1\/bg.png",
"profile_background_tile": false,
"profile_image_url": "http:\/\/a0.twimg.com\/sticky\/default_profile_images\/default_profile_3_normal.png",
"profile_image_url_https": "https:\/\/si0.twimg.com\/sticky\/default_profile_images\/default_profile_3_normal.png",
"profile_link_color": "0084B4",
"profile_sidebar_border_color": "C0DEED",
"profile_sidebar_fill_color": "DDEEF6",
"profile_text_color": "333333",
"profile_use_background_image": true,
"default_profile": true,
"default_profile_image": true,
"following": null,
"follow_request_sent": null,
"notifications": null
},
"geo": {
"type": "Point",
"coordinates": [50.48812681, -6.71658144]
},
* * "coordinates": {
"type": "Point",
"coordinates": [-6.71658144, 50.48812681]
},
* * "place": {
"id": "6416b8512febefc9",
"url": "http:\/\/api.twitter.com\/1\/geo\/id\/6416b8512febefc9.json",
"place_type": "country",
"name": "United Kingdom",
"full_name": "United Kingdom",
"country_code": "GB",
"country": "United Kingdom",
"contained_within": [],
"geometry": null,
"polylines": [],
"bounding_box": {
"type": "Polygon",
"coordinates": [
[
[-8.662663, 49.1626564],
[-8.662663, 60.86165],
[1.768926, 60.86165],
[1.768926, 49.1626564]
]
]
},
"attributes": {}
},
"contributors": null,
"retweet_count": 0,
"favorite_count": 0,
"entities": {
"hashtags": [],
"urls": [],
"user_mentions": []
},
"favorited": false,
"retweeted": false,
"filter_level": "medium",
"lang": "en"
}
According to the Twitter documentation, the location parameter accepts a comma separated list of longitude,latitude pairs specifying a set of bounding boxes.
If you just pass in a single latitude and longitude, then it will return the nearest Tweets to that location. Have you also specified a filter query, if so this will be massively cutting down the number of Tweets returned, meaning that the closest Tweet to London, is actually miles away. I had a similar problem.
Having said that, I just tried this out, if you use the lat and lon below, with NO other filter queries it works well.
{locations: [51.50,0.12,51.75,1.25]}
Another point to note, is that not a lot of Tweets actually have locations attached anymore, and often the ones which do are very inaccurate as they can actually be coming from the authors home location specified on their profile.