I'm trying to obtain response of solr for a given query. I've checked the solr query interface on localhost:8983 and its working fine, giving the required response to the queries. I want to select the productId field from each response and print it. Following is the code I'm using:
#!/usr/bin/ruby
# encoding: utf-8
require 'rubygems'
require 'solr'
solr = Solr::Connection.new('http://localhost:8983/solr')
response = solr.query('necklace')
puts "the response is:"
#puts response
puts "\n"
response.each do |hit|
puts hit['productId']
end
But it does not print anything, my response seems to be empty. How can I fix this
Try changing line 16 to:
puts hit['productId']
Related
I have a rake task which loops over pages of card game database and checks for the cards in each deck. Until recently this was working fine (it's checked 34000 pages of 25 decks each no problem) but recently this has stopped working when I run the rake task and I get the error:
JSON::ParserError: 765: unexpected token at ''
In order to debug this I have tried running each line of the get request and json parse manually in the rails console and it works fine every time. Weirder still I have installed pry and it works every time I go through the json parse manually with pry (takes ages though).
Here is the rake task:
desc "Create Cards"
require 'net/http'
require 'json'
task :create_cards => :environment do
# Get the total number of pages of decks
uri = URI("https://www.keyforgegame.com/api/decks/")
response = Net::HTTP.get(URI(uri))
json = JSON.parse(response)
deck_count = json["count"]
# Set variables
page_number = 1
page_size = 25 # 25 is the max page size
page_limit = deck_count / 25
card_list = Card.where(is_maverick: false)
# Updates Card List (non-mavericks) - there are 740 cards so we stop when we have that many
# example uri: https://www.keyforgegame.com/api/decks/?page=1&page_size=30&search=&links=cards
puts "Updating Card List..."
until page_number > page_limit || Card.where(is_maverick: false).length == 740
uri = URI("https://www.keyforgegame.com/api/decks/?page=#{page_number}&page_size=#{page_size}&search=&links=cards")
response = Net::HTTP.get(URI(uri))
json = JSON.parse(response) # task errors here!
cards = json["_linked"]["cards"]
cards.each do |card|
unless Card.exists?(:card_id => card["id"])
Card.create({
card_id: card["id"],
amber: card["amber"],
card_number: card["card_number"],
card_text: card["card_text"],
card_title: card["card_title"],
card_type: card["card_type"],
expansion: card["expansion"],
flavor_text: card["flavor_text"],
front_image: card["front_image"],
house: card["house"],
is_maverick: card["is_maverick"],
power: card["power"],
rarity: card["rarity"],
traits: card["traits"],
})
end
end
puts "#{page_number}/#{page_limit} - Cards: #{Card.where(is_maverick: false).length}"
page_number = (page_number + 1)
end
end
The first json parse where it gets the total number of pages of decks works okay. It's the json parse in the until block that is failing (I've marked the line with a comment to that effect).
As I say, if I try this in the console it works fine and I can parse the json without error, literally copying and pasting the lines from the file into the rails console.
Since you're looping over an api, it's possible there are rate limits. Public APIs normally have per second rate limits. You could try adding a sleep to slow down your requests, not sure how many your making per second. I tested with a simple loop and looks like response returns an empty string if you hit the api too fast.
url='https://www.keyforgegame.com/api/decks/?page=1&page_size=30&search=&links=cards'
uri = URI(url)
i = 1
1000.times do
puts i.to_s
i += 1
response = Net::HTTP.get(URI(uri))
begin
j = JSON.parse(response)
rescue
puts response
#= ""
end
end
I played with this until the loop stopped returning empty string after the 3rd request and got it to work with sleep 5 inside each loop, so you can probably add as the first line inside your loop. But you should probably add error handling to your rake task in case you encounter any other API errors.
So for now you can probably just do this
until page_number > page_limit || Card.where(is_maverick: false).length == 740
sleep 5
# rest of your loop code, maybe add a rescue like I've shown
end
I'm getting a no implicit conversion of String into Integer error that has me stumped, and unable to import user records and seed my database with them.
So far I have no problem accessing the data, but receive an error referencing the '[]' on the line with User.find... on it
The code I'm using is as follows:
require 'net/http'
require 'uri'
require 'json'
require 'faker'
#this script imports APR user data from the zendesk api and populates
the database with it.
uri = URI.parse("https://blahsupport.zendesk.com/api/v2/users.json")
request = Net::HTTP::Get.new(uri)
request.content_type = "application/json"
request.basic_auth("blah#blah.com", "blahpass")
req_options = {
use_ssl: uri.scheme == "https",
}
#response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
http.request(request)
end
puts #response.body
puts #response.message
puts #response.code
info = #response.body
info.force_encoding("utf-8")
File.write('blahusers1.json', info)
puts "File Created Successfully!"
file = File.read('blahusers1.json')
users = JSON.load(file)
users.each do |a|
User.find_or_create_by_zendesk_id(:zendesk_id => a['id'], :url => a['url'], :name => a['name'], :email => a['email'])
end
Any ideas on how I've gotten this error? Thank you for any help!
**Edit
Below is an example of the data being returned.
{"users":[{"id":333653859,"url":"https://blahblah.zendesk.com/api/v2/users/333653859.json","name":"Randy Blah","email":"randy#blah.com","created_at":"2014-08-06T14:31:24Z","updated_at":"2018-04-04T14:22:06Z","time_zone":"Pacific Time (US & Canada)","phone":null,"shared_phone_number":null,"photo":{"url":"https://aprtechsupport.zendesk.com/api/v2/attachments/68955389.json","id":68955389,"file_name":"Work.jpg","content_url":"https://aprtechsupport.zendesk.com/system/photos/6895/5389/Work.jpg","mapped_content_url":"https://blahblah.zendesk.com/system/photos/6895/5389/Work.jpg","content_type":"image/jpeg","size":2528,"width":80,"height":80,"inline":false,"thumbnails":[{"url":"https://blahblah.zendesk.com/api/v2/attachments/68955399.json","id":68955399,"file_name":"Work_thumb.jpg","content_url":"https://blahblah.zendesk.com/system/photos/6895/5389/Work_thumb.jpg","mapped_content_url":"https://blahblah.zendesk.com/system/photos/6895/5389/Work_thumb.jpg","content_type":"image/jpeg","size":2522,"width":32,"height":32,"inline":false}]},"locale_id":1,"locale":"en-US","organization_id":null,"role":"admin","verified":true,"external_id":null,"tags":[],"alias":"","active":true,"shared":false,"shared_agent":false,"last_login_at":"2018-04-04T14:21:44Z","two_factor_auth_enabled":null,"signature":"Thanks for contacting the helpdesk!\n-Randy","details":"","notes":"","role_type":null,"custom_role_id":null,"moderator":true,"ticket_restriction":null,"only_private_comments":false,"restricted_agent":false,"suspended":false,"chat_only":false,"default_group_id":21692179,"user_fields":{}}
The example data you posted has a root object users that contains the array of user objects. So when you loop users using users.each, a is actually an Array and not a user Hash like you expected.
When you try to access an element of an Array using a 'String' index, it gives you the exception – no implicit conversion of String into Integer
So, try changing
users = JSON.load(file)
to
users = JSON.load(file)['users']
to get it working like how you'd expect.
I am using RestClient to make a post request and i made it so i an error response back so i can print those error messages in console
i tried the following per the restclient gem documentation
begin
response = RestClient.post base_uri, params.to_json, content_type: 'application/json', accept: 'application/json'
rescue RestClient::ExceptionWithResponse => err
error = err.response
p "this is the error response #{error}"
end
when i print err.response i get the following
"this is the error response {\"error\":{\"message\":\"An active access token must be used to query information about the current us
er.\",\"type\":\"OAuthException\",\"code\":2500,\"fbtrace_id\":\"HTzmJ0CcIfd\"}}"
how do i access the message in the above hash to display it in console?
tried
p "this is the error response #{error.message}"
and it gives me "Bad request" - have no idea where it gets that
If you're just looking to output it:
error = JSON.load(err.response)
puts error['error']['message']
You can always format it a bit better:
puts '[Code %d %s] %s' % [
error['error']['code'],
error['error']['type'],
error['error']['message']
]
Note that using puts inside of a Rails process is not going to work very well. You might want to use Rails.logger.debug instead.
The response you received is in JSON. You'll need to decode the JSON first and then interact with the data. Personally, I like MultiJson for this:
begin
response = RestClient.post base_uri, params.to_json, content_type: 'application/json', accept: 'application/json'
rescue RestClient::ExceptionWithResponse => err
error = MultiJson.load(err.response)
p "this is the error response #{error[:message]}"
end
I´m using Rails and Savon 2 to get data from a SOAP Webservice.
This is the code:
client = Savon.client(wsdl: "http://www.webservicex.net/periodictable.asmx?WSDL",
log_level: :debug,
pretty_print_xml: true)
message = {'ElementName' => 'Zinc'}
response = client.call(:get_element_symbol, message: message)
logger.debug "Body=" + response.body.to_s
symbol = response.to_hash[:get_element_symbol_response][:get_element_symbol_result][:NewDataDet][:Table][:Symbol]
The request is ok and I´m getting the data in the response:
Body={:get_element_symbol_response=>{:get_element_symbol_result=>"<NewDataSet>\n <Table>\n <Symbol>Zn</Symbol>\n </Table>\n</NewDataSet>", :#xmlns=>"http://www.webserviceX.NET"}}
But now, I don´t know how to parse this response correctly to get the "Symbol".
I´m getting this error:
`TypeError (no implicit conversion of Symbol into Integer):`
UPDATE:
If I do:
symbol = response.to_hash[:get_element_symbol_response][:get_element_symbol_result]
logger.debug "Symbol=" + symbol.inspect
I get this: Symbol="<NewDataSet>\n <Table>\n <Symbol>Zn</Symbol>\n </Table>\n</NewDataSet>"
I think the error is that I´m trying to get Symbol in hash mode, and it is not. But how can I get the symbol? I can´t believe I have to parse the string manually...
You can use nokogiri to parse XML text:
require 'nokogiri'
text = response.body[:get_element_symbol_response][:get_element_symbol_result]
Nokogiri::XML(text).css('Symbol').text # => Zn
# or
Nokogiri::XML(text).xpath('//NewDataSet/Table/Symbol').text # => Zn
My rb file reads:
require "csv"
puts "Program1 initialized."
contents = CSV.open "data.csv", headers: true
contents.each do |row|
name = row[4]
puts name
end
...but when i run it in ruby it wont load the program. it gives me the error message about the headers:
syntax error, unexpected ':', expecting $end
contents = CSV.open "data.csv", headers: true
so I'm trying to figure out, why won't ruby let me parse this file? I've tried using other csv files I have and it won't load, and gives me an error message. I'm trying just to get the beginning of the program going! I feel like it has to do with the headers. I've updated as much as I can, mind you I'm using ruby 1.8.7. I read somewhere else that I could try to run the program in irb but it didn't seem like it needed it. so yeah... thank you in advance!!!!
Since you are using this with Ruby 1.8.7, :headers => true won't work in this way.
The simplest way to ignore the headers and get your data is to shift the first row in the data, which would be the headers:
require 'csv'
contents = CSV.open("data.csv", 'r')
contents.shift
contents.each do |row|
name = row[4]
puts name
end
If you do want to use the syntax with headers in ruby 1.8, you would need to use FasterCSV, something similar to this:
require 'fastercsv'
FasterCSV.foreach("data.csv", :headers => true) do |fcsv_obj|
puts fcsv_obj['name']
end
(Refer this question for further read: Parse CSV file with header fields as attributes for each row)