I'm using ruby 1.9.3 and trying to use open-uri to get a url and try posting using Net:HTTP
Im trying to use proxy authentication for both:
Trying to do a POST request with net/http:
require 'net/http'
require 'open-uri'
http = Net::HTTP.new("google.com", 80)
headers = { 'User-Agent' => 'Ruby 193'}
resp, data = http.post("/", "name1=value1&name2=value2", headers)
puts data
And for open-uri which I can't get to do POST I use:
data = open("http://google.com/","User-Agent"=> "Ruby 193").read
How would I modify these to use a proxy with HTTP Authentication
I've tried (for open-uri)
data = open("http://google.com/","User-Agent"=> "Ruby 193", :proxy_http_basic_authentication => ["http://proxy.com:8000/", "proxy-user", "proxy-password"]).read
However all I will get is a OpenURI::HTTPError: 407 Proxy Authentication Required. I've verified all and it works in the browser with the same authentication and proxy details but I can't get ruby to do it.
How would I modify the code above to add http authentication properly? Has anyone gone through this atrocity?
Try:
require "open-uri"
proxy_uri = URI.parse("http://proxy.com:8000")
data = open("http://www.whatismyipaddress.com/", :proxy_http_basic_authentication => [proxy_uri, "username", "password"]).read
puts data
As for Net::HTTP, I recently implemented support for proxies with http authentication into a Net::HTTP wrapper library called http. If you look at my last pull-request, you'll see the basic implementation.
EDIT: Hopefully this will get you moving in the right direction.
Net::HTTP::Proxy(proxy_uri.host, proxy_uri.port,"username","password").start('whatismyipaddress.com') do |http|
puts http.get('/').body
end
EDIT 11/24/2020: Net::HTTP::Proxy is now considered obsolete. You can now configure proxies when creating a new instance of Net::HTTP. See the documentation for Net::HTTP.new for more details.
Related
I have a very simple API, that I would like to make a POST to using ruby and NOT using a GEM just the built in libraries net/http, uri, and openssl if needed.
Anyway, I am using the code below to make a very simple POST request but am getting some VERY strange results and was hoping someone else has seen this.
I have also tested the same request below in POSTMAN and NodeJS and BOTH work as expected, the only one I can not get to work is Ruby.
require 'uri'
require 'net/http'
require 'openssl'
url = URI("https://somesite.dev/devices")
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(url)
request["key"] = '1234567'
response = http.request(request)
puts response.read_body
The result is something I have not seen before: I am getting the header key twice... So the log to the API shows a query like this:
SELECT * FROM device where key = '1234567, 1234567' LIMIT ...
As stated above I can make the same request via POSTMAN or NodeJS and get the correct result.
NOTE: Because I have a local copy of the API I can test locally BUT it's not SSL it's all over http. When making the request locally it works just fine. So from what I can tell this issue only presents it self when SSL is introduced.
Any help would be amazing! Thanks!!
Ruby Version 2.2.1
The issue was with something I did not list in my initial question. The API was using AWS API Gateway, and HTTP_PROXY was enabled on the method causing this strange issue. After I removed HTTP_PROXY the issue cleared up and the above code worked!
Versions:
Ruby 2.2.4
Rails 4.2
Omniauth-oauth2 1.3.1
Omniauth-bnet 1.1.0
Issue:
Trying to complete the authorization and token request process to Blizzard's Battle Net Community API. While I can get the authorization_code returned, when I attempt to construct a POST back to the token endpoint it keeps telling me that its an invalid request/internal server error or just returns back the following object: <Net::HTTPFound 302 Found readbody=true> which has a blank string for a response body. Details for how Blizzard recommends handling the OAuth 2 process are located here: Battle.net OAuth 2 Guide. The omniauth-bnet gem is the one Blizzard suggested but doesn't seem to handle the entire OAuth authorization and token process but I'll freely admit I'm brand new when it comes to anything OAuth related so I could be wrong.
Any help you all can provide would be very welcome!
Controller Code:
def index
client_id = ENV[BNET_CLIENT_ID]
client_secret = ENV[BNECT_CLIENT_SECRET]
uri = URI('https://us.battle.net/auth/token?
redirect_uri=https%3A%2F%2f127%2f0%2f0%2f1%3A3001%2Fauth%2Fbnet%2Fcallback
&grant_type=authorization_code
&code=' + params["code"])
req = Net::HTTP::Post.new(uri)
req.basic_auth(client_id, client_secret)
res = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https') {|http| http.request(req)}
#bnet_response = res.body
end
Process:
Click link on index.html page which triggers the authorization process to start by sending a GET request to https://us.battle.net/oauth/authorize?client_id=&client_secret=&redirect_uri=&response_type=code
BNet API accepts the request and redirects user to Battle.Net Login screen
User authorizes app and is redirected (via redirect_uri) back to the index page with query parameters for the authorization_code
My app SHOULD make a POST request using the controller code above and the response should contain the actual access token.
This is just some hastily cobbled together code while I'm trying to get this working. I'll clean it up once I get over this little speed bump. I wouldn't hardcode the URI and such normally, just getting frustrated with things.
Things I've tried:
Using Rest-Client : There's a callback redirect that it can't handle apparently
Testing Postman : When using their OAuth 2 Authorization it works just fine, also works fine if I use Postman to get the authorization code (GET) and token (POST), so I'm assuming things work on the Blizz side and my code just sucks.
config/routes.rb
Rails.application.routes.draw do
root to: 'pages#index'
get '/auth/:provider/callback', to: 'pages#index'
end
I'm not familiar with the Battle.net API, however reading your process it seems you are getting an authorization code back from the redirect in the form of a query parameter (accessed by params[:code]) correct?
You also mention that this works using Postman indicating the flaw must be somewhere in the code.
I would suggest using the following snippet and let me know if it works. I would encourage using puts url after url = ... code to ensure the URI looks exactly as you want it (and has the correct code, client id, secret, redirect_uri).
require 'uri'
require 'net/http'
# Remember to change the redirect uri to match yours
redirect_uri = 'https://my-domain-name.com/auth/bnet/callback'
# Double check the environment variable names for BNET ID / Secret
url = URI("https://us.battle.net/oauth/token?redirect_uri=#{redirect_uri}&code=#{params[:code]}" \
"&grant_type=authorization_code&scope=wow.profile&client_id=#{ENV['BNET_CLIENT_ID']}" \
"&client_secret=#{ENV['BNET_CLIENT_SECRET']}")
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(url)
response = http.request(request)
puts response.read_body
Ensure your domain is accessible over https as this seems to be required.
I will replace my command line
`curl -XPUT 'host:port/url' -d '{"val": "some_json"}'̀
by a Rails command, and get the result...
Somewhere like this :
response = call('put', 'host:port/url', '{"val" : "some_json"}')
Is there any predefined method to do this in Rails, or some gem ?
I know the command get of HTTP, but I will do a 'PUT' method.
Net::HTTP.get(URI.parse('host:port/url'))
Thanks for your replies
You can use Net::HTTP to send any standard http requests.
Here is a way, you can connect to any url ( http / https ), with any valid http methods with or without parameters.
def universal_connector(api_url, api_parameters={}, method="Get")
# Do raise Error, if url is invalid and Method is invalid
uri = URI(api_url)
req = eval("Net::HTTP::#{method.capitalize}.new('#{uri}')")
req.set_form_data(api_parameters)
Net::HTTP.start(uri.host, uri.port,:use_ssl => uri.scheme == 'https') do |http|
response = http.request(req)
return response.body
end
end
There are many alternatives available as well. Specifically, Faraday. Also, read this before making a choice.
#get is just a simple shortcut for the whole code (Net::HTTP Ruby library tends to be very verbose). However, Net::HTTP perfectly supports PUT requests.
Another alternative is to use an HTTP client as a wrapper. The most common alternatives are HTTParty and Faraday.
HTTParty.put('host:port/url', { body: {"val" : "some_json"} })
As a side note, please keep in mind that Rails is a framework, not a programming language. Your question is about how to perform an HTTP PUT request in Ruby, not Rails. It's important to understand the difference.
I'm new to open-uri and trying to set an outgoing IP address using open-uri in ruby on rails. I used this post as a reference to get started. I'm porting an app from PHP where I could use CURLOPT_INTERFACE in curl_setopt. What's the best way to do this using open-uri in rails? (Doing this from the controller - not command line.)
If there's not a way to do this - any suggestions on an alternative to open-uri? My goal is to take in and parse JSON data.
What I understand from your questions is you want to hit another server from a specific IP which suggests you have a server with couple of addresses.
What I can suggest you is try to execute curl directly and do what you want to do or use a wrapper for it.
Doesn't look like open-uri can do this. But with net/https it's fairly easy.
require 'net/https'
require 'json'
uri = URI('https://jsonvat.com/')
http = Net::HTTP.new(uri.host, uri.port)
http.local_host = '1.2.3.4'
http.use_ssl = true
request = Net::HTTP::Get.new('/')
request.content_type = 'application/json'
request.initialize_http_header('Content-Type' => 'application/json')
response = http.request(request)
json = JSON.parse(response.body)
Probably you don't need the "require" lines inside Rails Controllers.
You can specify the outgoing IP address with the http.local_host line.
https://stackoverflow.com/a/24896074/1371731
https://yukimotopress.github.io/http
A client of mine has asked me to integrate a 3rd party API into their Rails app. The only problem is that the API uses SOAP. Ruby has basically dropped SOAP in favor of REST. They provide a Java adapter that apparently works with the Java-Ruby bridge, but we'd like to keep it all in Ruby, if possible. I looked into soap4r, but it seems to have a slightly bad reputation.
So what's the best way to integrate SOAP calls into a Rails app?
I built Savon to make interacting with SOAP webservices via Ruby as easy as possible.
I'd recommend you check it out.
We used the built in soap/wsdlDriver class, which is actually SOAP4R.
It's dog slow, but really simple. The SOAP4R that you get from gems/etc is just an updated version of the same thing.
Example code:
require 'soap/wsdlDriver'
client = SOAP::WSDLDriverFactory.new( 'http://example.com/service.wsdl' ).create_rpc_driver
result = client.doStuff();
That's about it
We switched from Handsoap to Savon.
Here is a series of blog posts comparing the two client libraries.
I also recommend Savon. I spent too many hours trying to deal with Soap4R, without results. Big lack of functionality, no doc.
Savon is the answer for me.
Try SOAP4R
SOAP4R
Getting Started with SOAP4R
And I just heard about this on the Rails Envy Podcast (ep 31):
WS-Deathstar SOAP walkthrough
Just got my stuff working within 3 hours using Savon.
The Getting Started documentation on Savon's homepage was really easy to follow - and actually matched what I was seeing (not always the case)
Kent Sibilev from Datanoise had also ported the Rails ActionWebService library to Rails 2.1 (and above).
This allows you to expose your own Ruby-based SOAP services.
He even has a scaffold/test mode which allows you to test your services using a browser.
I have used HTTP call like below to call a SOAP method,
require 'net/http'
class MyHelper
def initialize(server, port, username, password)
#server = server
#port = port
#username = username
#password = password
puts "Initialised My Helper using #{#server}:#{#port} username=#{#username}"
end
def post_job(job_name)
puts "Posting job #{job_name} to update order service"
job_xml ="<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:ns=\"http://test.com/Test/CreateUpdateOrders/1.0\">
<soapenv:Header/>
<soapenv:Body>
<ns:CreateTestUpdateOrdersReq>
<ContractGroup>ITE2</ContractGroup>
<ProductID>topo</ProductID>
<PublicationReference>#{job_name}</PublicationReference>
</ns:CreateTestUpdateOrdersReq>
</soapenv:Body>
</soapenv:Envelope>"
#http = Net::HTTP.new(#server, #port)
puts "server: " + #server + "port : " + #port
request = Net::HTTP::Post.new(('/XISOAPAdapter/MessageServlet?/Test/CreateUpdateOrders/1.0'), initheader = {'Content-Type' => 'text/xml'})
request.basic_auth(#username, #password)
request.body = job_xml
response = #http.request(request)
puts "request was made to server " + #server
validate_response(response, "post_job_to_pega_updateorder job", '200')
end
private
def validate_response(response, operation, required_code)
if response.code != required_code
raise "#{operation} operation failed. Response was [#{response.inspect} #{response.to_hash.inspect} #{response.body}]"
end
end
end
/*
test = MyHelper.new("mysvr.test.test.com","8102","myusername","mypassword")
test.post_job("test_201601281419")
*/
Hope it helps. Cheers.
I have used SOAP in Ruby when i've had to make a fake SOAP server for my acceptance tests. I don't know if this was the best way to approach the problem, but it worked for me.
I have used Sinatra gem (I wrote about creating mocking endpoints with Sinatra here) for server and also Nokogiri for XML stuff (SOAP is working with XML).
So, for the beginning I have create two files (e.g. config.rb and responses.rb) in which I have put the predefined answers that SOAP server will return.
In config.rb I have put the WSDL file, but as a string.
##wsdl = '<wsdl:definitions name="StockQuote"
targetNamespace="http://example.com/stockquote.wsdl"
xmlns:tns="http://example.com/stockquote.wsdl"
xmlns:xsd1="http://example.com/stockquote.xsd"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns="http://schemas.xmlsoap.org/wsdl/">
.......
</wsdl:definitions>'
In responses.rb I have put samples for responses that SOAP server will return for different scenarios.
##login_failure = "<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<LoginResponse xmlns="http://tempuri.org/">
<LoginResult xmlns:a="http://schemas.datacontract.org/2004/07/WEBMethodsObjects" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<a:Error>Invalid username and password</a:Error>
<a:ObjectInformation i:nil="true"/>
<a:Response>false</a:Response>
</LoginResult>
</LoginResponse>
</s:Body>
</s:Envelope>"
So now let me show you how I have actually created the server.
require 'sinatra'
require 'json'
require 'nokogiri'
require_relative 'config/config.rb'
require_relative 'config/responses.rb'
after do
# cors
headers({
"Access-Control-Allow-Origin" => "*",
"Access-Control-Allow-Methods" => "POST",
"Access-Control-Allow-Headers" => "content-type",
})
# json
content_type :json
end
#when accessing the /HaWebMethods route the server will return either the WSDL file, either and XSD (I don't know exactly how to explain this but it is a WSDL dependency)
get "/HAWebMethods/" do
case request.query_string
when 'xsd=xsd0'
status 200
body = ##xsd0
when 'wsdl'
status 200
body = ##wsdl
end
end
post '/HAWebMethods/soap' do
request_payload = request.body.read
request_payload = Nokogiri::XML request_payload
request_payload.remove_namespaces!
if request_payload.css('Body').text != ''
if request_payload.css('Login').text != ''
if request_payload.css('email').text == some username && request_payload.css('password').text == some password
status 200
body = ##login_success
else
status 200
body = ##login_failure
end
end
end
end
I hope you'll find this helpful!
I was having the same issue, switched to Savon and then just tested it on an open WSDL (I used http://www.webservicex.net/geoipservice.asmx?WSDL) and so far so good!
https://github.com/savonrb/savon