If I load a section of my website with Net::HTTP in Rails, will this get loaded every time or will it get cached along with the rest of the footer?
EDIT: I mean the rest of the footer is currently cached. Would the Net:HTTP results, which get rendered inside the footer, also become cached? I would like it to reload the results every time.
No, Net::HTTP will not cache anything for you. You will have to implement caching, or use a gem that does it for you. But depending on what you do with Rails, Rails can do it - look into fragment caching.
Doesn't look like it does, at least not by default as of 2011. There's also a segment in the net/http.rb file in the ruby source that has the following code commented out:
# The following example performs a conditional GET using the
# If-Modified-Since header. If the files has not been modified since the
# time in the header a Not Modified response will be returned. See RFC 2616
# section 9.3 for further details.
#
uri = URI('http://example.com/cached_response')
file = File.stat 'cached_response'
req = Net::HTTP::Get.new(uri)
req['If-Modified-Since'] = file.mtime.rfc2822
res = Net::HTTP.start(uri.hostname, uri.port) {|http|
http.request(req)
}
open 'cached_response', 'w' do |io|
io.write res.body
end if res.is_a?(Net::HTTPSuccess)
The source file is dated July 9. Hope that helps.
Related
I am using trello api to attach an image to a card. the documentation says
require 'uri'
require 'net/http'
url = URI("https://api.trello.com/1/cards/id/attachments")
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(url)
response = http.request(request)
puts response.read_body
After putting my key and my token, I tried to upload a file and the binary data goes in the url itself, not only it seems too ugly but it also doesn't work because the request is really too long. I've tried using multipart and rest client gems from in my code to upload and attach a file to a trello card but everytime I get errors like bad request or SSL errors, can anyone please give me a piece of code that really works? thanks
actually I am sending the image data via AJAX (I'm generating it from a charjs view), so the data sent is binary, it would be better if the solution upload an image from binary data.
Their documentation does indeed encourage you to add the whole encoded file object into the URL, which I also find ugly. I wonder if it will work to add it into the POST body instead? Try this:
request = Net::HTTP::Post.new(url)
request.set_form_data({file: put_encoded_file_contents_here})
I'm using Net::HTTP with Ruby to crawl an URL.
I don't want to crawl streaming audio such as: http://listen2.openstream.co/334
in fact i only want to crawl Html content, so no pdfs, video, txt..
Right now, I have both open_timeout and read_timeout set to 10, so even if I do crawl these streaming audio pages they will timeout.
url = 'http://listen2.openstream.co/334'
path = uri.path
req= Net::HTTP::Get.new(path, {'Accept' => '*/*', 'Content-Type' => 'text/plain; charset=utf-8', 'Connection' => 'keep-alive','Accept-Encoding' => 'Identity'})
uri = Addressable::URI.parse(url)
resp = Net::HTTP.start(uri.host, uri.inferred_port) do |httpRequest|
httpRequest.open_timeout = 10
httpRequest.read_timeout = 10
#how can I read the headers here before it's streaming the body and then exit b/c the content type is audio?
httpRequest.request(req)
end
However, is there a way to check the header BEFORE I read the body of a http response to see if it's an audio? I want to do so without sending a separate HEAD request.
net/http supports streaming, you can use this to read the header before the body.
Code example,
url = URI('http://stackoverflow.com/questions/41306082/ruby-nethttp-read-the-header-before-the-body-without-head-request')
Net::HTTP.start(url.host, url.port) do |http|
request = Net::HTTP::Get.new(url)
http.request(request) do |response|
# check headers here, body has not yet been read
# then call read_body or just body to read the body
if true
response.read_body do |chunk|
# process body chunks here
end
end
end
end
I will add a ruby example later tonight. However, for a quick response. There is a simple trick to do this.
You can use HTTP Range header to indicate if which range of bytes you want to receive from the server. Here is an example:
curl -XGET http://www.sample-videos.com/audio/mp3/crowd-cheering.mp3 -v -H "Range: bytes=0-1"
The above example means the server will return data from 0 to 1 byte range.
See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests
Since I did not find a way to properly do this in Net::HTTP, and I saw that you're using the addressable gem as an external dependency already, here's a solution using the wonderful http gem:
require 'http'
response = HTTP.get('http://listen2.openstream.co/334')
# Here are the headers
puts response.headers
# Everything ok? Start streaming the response
body = response.body
body.stream!
# now just call `readpartial` on the body until it returns `nil`
# or some other break condition is met
Sorry if you're required to use Net::HTTP, hopefully someone else will find an answer. A separate HEAD request might indeed be the way to go in that case.
You can do a whole host of net related things without using a gem. Just use the net/http module.
require 'net/http'
url = URI 'http://listen2.openstream.co/334'
Net::HTTP.start(url.host, url.port){|conn|
conn.request_get(url){|resp|
resp.each{|k_header, v_header|
# process headers
puts "#{k_header}: #{v_header}"
}
#
# resp.read_body{|body_chunk|
# # process body
# }
}
}
Note: while processing headers, just make sure to check the content-type header. For audio related content it would normally contain audio/mpeg value.
Hope, it helped.
I will replace my command line
`curl -XPUT 'host:port/url' -d '{"val": "some_json"}'̀
by a Rails command, and get the result...
Somewhere like this :
response = call('put', 'host:port/url', '{"val" : "some_json"}')
Is there any predefined method to do this in Rails, or some gem ?
I know the command get of HTTP, but I will do a 'PUT' method.
Net::HTTP.get(URI.parse('host:port/url'))
Thanks for your replies
You can use Net::HTTP to send any standard http requests.
Here is a way, you can connect to any url ( http / https ), with any valid http methods with or without parameters.
def universal_connector(api_url, api_parameters={}, method="Get")
# Do raise Error, if url is invalid and Method is invalid
uri = URI(api_url)
req = eval("Net::HTTP::#{method.capitalize}.new('#{uri}')")
req.set_form_data(api_parameters)
Net::HTTP.start(uri.host, uri.port,:use_ssl => uri.scheme == 'https') do |http|
response = http.request(req)
return response.body
end
end
There are many alternatives available as well. Specifically, Faraday. Also, read this before making a choice.
#get is just a simple shortcut for the whole code (Net::HTTP Ruby library tends to be very verbose). However, Net::HTTP perfectly supports PUT requests.
Another alternative is to use an HTTP client as a wrapper. The most common alternatives are HTTParty and Faraday.
HTTParty.put('host:port/url', { body: {"val" : "some_json"} })
As a side note, please keep in mind that Rails is a framework, not a programming language. Your question is about how to perform an HTTP PUT request in Ruby, not Rails. It's important to understand the difference.
I am getting a LoadError - "Too many open files" when using Feedzirra. I am running it on my development server using the default WEBrick server.
I am parsing only 2 feeds. What is the problem?
I had the same issue with Feedzirra. You can notice that it leaves TCP connections in CLOSE_WAIT state forever, hence causing the problem.
It appears to be curb gem specific that is used to fetch feeds. Another project depending on libcurl had the same issue. They have fixed it by setting 'CURLOPT_FORBID_REUSE' option.
I've tried to do the same for Feedzirra but didn't succeed. Even with this option I had a growing number of CLOSE_WAIT sessions and Too many open files error eventually.
So I did the most straightforward thing, I download feeds using Net::HTTP:
def get_contents(furl)
url = URI.parse(furl)
req = Net::HTTP::Get.new(url.to_s)
res = Net::HTTP.start(url.host, url.port) { |http|
http.request(req)
}
unless res.kind_of? Net::HTTPSuccess
puts "can't get feed #{url.to_s}: #{res.code}"
return nil
end
res.body
end
Then I parse the XML with Feedzirra:
xml = get_contents(furl)
feedin = Feedzirra::Feed.parse xml
No more stuck connections and no more errors. You may also want to add better error handling to this sample code.
In Rail my final goal is to write a Net::HTTP client to connect to my REST API that is returning JSON and parse it, pass it to View , etc....
But first things first!
What is the simplest thing I can start with?
I am looking at this page:http://ruby-doc.org/stdlib-1.9.3/libdoc/net/http/rdoc/Net/HTTP.html
and I get the impression that if I have one .rb file with these two lines of code in it, it should show me something?
require 'net/http'
Net::HTTP.get('example.com', '/index.html')
url = URI.parse("http://example.com")
req = Net::HTTP::Get.new(url.path)
#resp = Net::HTTP.new(url.host, url.port).start {|http| http.request(req)}
in a view
<%= "The call to example.com returned this: #{#resp}" %>
You could start testing with something like this:
require 'net/http'
response = Net::HTTP.get_response("www.google.com","/")
puts response.body
I'll recommend you take a look at the docs: Net::HTTPSession