The following code gives me a Timeout::Error
open("#{local_file}", 'wb') do |file|
file << open(remote_file_url).read
end
How can I increase the timeout? The file to download is big and usually takes more than 2 minutes in the browser.
I notice you are opening a remote file "remote_file_url". I imagine this is where the timeout is coming from.
For starters, I wrote a net/http connection class that includes timeout code you can use or model. https://github.com/bf4/Notes/blob/master/code/connection.rb (I believe the open method is using net/http)
You can also get the response in a block
e.g. something like
local_file, url, timeout = 'foo', 'http://www.example.com/index.html', 60
File.open(local_file, 'wb') do |file|
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.open_timeout = timeout
http.read_timeout = timeout
http.use_ssl = (uri.scheme == 'https')
http.request_get(uri.request_uri) do |response|
response.read_body do |segment|
file.write(segment)
end
end
end
see http://www.ruby-doc.org/stdlib-1.8.7/libdoc/net/http/rdoc/Net/HTTPResponse.html and http://www.ruby-doc.org/stdlib-1.8.7/libdoc/net/http/rdoc/Net/HTTP.html
Related
Let's say that I am calling the following code from inside a loop with a 1-second sleep/delay between each iteration and the URL is an API. How do I make sure that Net::HTTP is using the same API session for all the calls? I know the documentation says Net::HTTP.new will try to reuse the same connection. But how do I verify that? Is there a session ID that I can pull out of Net::HTTP?
request = Net::HTTP::Put.new(url)
url = URI(url)
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
request["Accept"] = 'application/json'
request["Content-Type"] = 'application/json'
request["Authorization"] = #auth_key
request["cache-control"] = 'no-cache'
request.body = request_body.to_json if request_body
response = http.request(request)
double check the following against the ruby version you are running on
For one, I don't think there is any session ID from what I can see which would be quite a useful feature. Next, looking at the source code, we see the variable setting in lib/net/http.rb in such methods as:
def do_finish
#started = false
#socket.close if #socket
#socket = nil
end
# Returns true if the HTTP session has been started.
def started?
#started
end
# Finishes the HTTP session and closes the TCP connection.
# Raises IOError if the session has not been started.
def finish
raise IOError, 'HTTP session not yet started' unless started?
do_finish
end
Where do_finish sets the instance variable #socket to nil and #socket is used as a BufferedIO instance to run HTTP requests through
So I would write an override method for the finish method and raise an alert when it calls on do_finish.
Looking through the comments start is the safest bet to use the same session, so you could use a start block and compare the id of the instance variable does not change
Net::HTTP.start(url) do |http|
before = http.instance_variable_get(:#socket)
loop do
instance_var = http.instance_variable_get(:#socket)
break unless before == instance_var
end
end
I have making a NET:HTTP call to a third party payment API using form_post method from NET:HTTP using rails.
res = Net::HTTP.post_form(uri, payment_details)
logger.info "RESPONSE: #{res}"
descrypted_res = JSON.parse(res.body)
logger.info "RESPONSE: #{descrypted_res}"
sometimes, I got the response back saying HTTPOK, sometimes i dont get anything at all. I am suspecting the payment server did not respond to my request in time and the connection is terminated. My question is, is there any way to increase the post_form timeout duration?
I want to increase it to 60 seconds.
Thanks.
You can adapt the ruby method:
https://docs.ruby-lang.org/en/2.0.0/Net/HTTP.html#method-c-post_form
like this:
def _post_form(url, params)
req = Net::HTTP::Post.new(url)
req.form_data = params
req.basic_auth url.user, url.password if url.user
Net::HTTP.start(
url.hostname,
url.port,
use_ssl: url.scheme == 'https',
read_timeout: 600
) do |http|
http.request(req)
end
end
I'm trying to call a service with Digest Auth from a rails application and it always returns a 400 bad request error.
I've used net-http-digest_auth gem to create the headers but I think I've missed something.
def get_digest(url)
uri = URI.parse(url)
http = Net::HTTP.new uri.host, uri.port
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
req = Net::HTTP::Get.new(uri.request_uri)
# Fist call with the 401 and auth headers
digest_response = http.request(req)
digest_auth_request = Net::HTTP::DigestAuth.new
uri.user = digest_auth[:user]
uri.password = digest_auth[:password]
auth = digest_auth_request.auth_header uri, digest_response['www-authenticate'], 'GET', true
req.add_field 'Authorization', auth
response = http.request(req)
# Response is always #<Net::HTTPBadRequest 400 Bad Request readbody=true>
if response.code.to_i == 200
response_body = response.body
else
error
end
response_body
end
The request's headers look like this:
Digest username=\"myuser#mydomain.com\", realm=\"Digest\", algorithm=MD5-sess, qop=\"auth\", uri=\"/path/WS/my%20user/path/path/path/path/service.svc\", nonce=\"+Upgraded+v1e3f88bce1c32bd15avn421e440ca6622ebadd4522f7ed201fab1421c39d8fd15b771b972c9eb59894f8879307b9e6a5544476bc05cc7885a\", nc=00000000, cnonce=\"d42e6ea8a37aadsasdbea1231232456709\", response=\"7fbfc75cc3aasdasd342230ebf57ac37df\""
I can't figure out what's happening, is there any other gem to make this easier?
Finally found the problem by comparing browser header vs ruby header.
I wasn't calculating "nc" (calls counter) correctly. After adding +1 it started to return a 401 error (now I have a different problem ;)).
I am trying to send a http.get request to different websites. Here is the code I am using:
def makePing
begin
url = URI.parse(#URI)
req = Net::HTTP::Get.new(url.to_s)
res = Net::HTTP.start(url.host, url.port) {|http|
http.read_timeout = #request_timeout_limit
http.request(req)
}
# debugger
rescue Exception => echo
puts "Error is: Failed to open TCP connection to #{#URI}"
end
end
It returns the result of 200 for 'http://www.example.com'
but
for http://www.google.com or http://www.facebook.com
it returns
<Net::HTTPNotFound 404 Not Found readbody=true>
1-I am wondering why it happens like this?
2-How can I get the body of the response?
3- I expect that, the request get expired exactly after #request_timeout_limit, and it stop trying, but it is not working in this way?
I have a post happening to a rails application from a ruby script. The script creates a variable request as
request = Net::HTTP::Post.new(url.path)
which is then used as follows
request.content_type = "application/json"
request.body = JSON.generate( params )
response = Net::HTTP.start(url.host, url.port) {|http| http.request(request)}
There is quite a lot of processing happening on the server side, and I'm getting a Net::ReadTimeout error
I tried to specify a timeout period
request.read_timeout = 500
as per this stackoverflow answer but I got a
undefined method `read_timeout=' for #<Net::HTTP::Post POST> (NoMethodError)
error. I assume that I'm missing something simple somewhere. All clues gratefully received
Technical info:
Ruby 2.0.0p247
Rails 4.0.0
Windows 7 32 bit ruby
Solved via this stackoverflow answer
I've changed my
response = Net::HTTP.start(url.host, url.port) {|http| http.request(request)}
line to be
response = Net::HTTP.start(url.host, url.port, :read_timeout => 500) {|http| http.request(request)}
and this seems to have got around this problem.
The read_timeout is available with a plain Net::HTTP object:
url = URI.parse('http://google.com')
http = Net::HTTP.new(url.host, url.port)
http.read_timeout = 5 # seconds
http.request_post(url.path, JSON.generate(params)) do |response|
# do something with response
p response
end
One thing to keep in mind is that if read_timeout is set to a small value such that a timeout does occur...Net::HTTP will "helpfully" retry the request. For a slow HTTP server, a timeout error may not be raised to the code calling Net::HTTP until 2x the read_timeout value.
This certainly was not the behavior I expected.
More info on this topic and how possible solutions differ for Ruby < 2.5 and >= 2.5 may be found here:
https://stackoverflow.com/a/59186209/5299483
I catch both OpenTimeout and ReadTimeout and it's work. test in Ruby:2.6.5
def ping(host, port)
begin
url = URI.parse("http://#{host}:#{port}/ping")
req = Net::HTTP::Get.new(url.to_s)
# setting both OpenTimeout and ReadTimeout
res = Net::HTTP.start(url.host, url.port, :open_timeout => 3, :read_timeout => 3) {|http|
http.request(req)
}
if JSON.parse(res.body)["ok"]
# return true
STDERR.puts "#{host}:#{port} is reachable"
else
STDERR.puts "#{host}:#{port} is NOT reachable"
end
rescue Net::ReadTimeout => exception
STDERR.puts "#{host}:#{port} is NOT reachable (ReadTimeout)"
rescue Net::OpenTimeout => exception
STDERR.puts "#{host}:#{port} is NOT reachable (OpenTimeout)"
end
end
ping("#{ENV['FIRST_HOST']}", 2345)
ping("#{ENV['SECOND_HOST']}", 2345)
If anyone is still facing timeout setting issue and Net::HTTP timeout not working as expected, then you may follow below approach as well:
begin
Timeout::timeout(10) {
####
## YOUR REQUEST CODE WILL BE HERE
####
}
rescue
408
end