How can I parse urls like
http://www.1800contacts.com/productlist.aspx?dl=P&source=cj&ac=8.2.0007
and only get
http://www.1800contacts.com
?
PS. Some urls have subdomains etc so I can't use regexps here.
Try to use 'uri' library:
require 'uri'
address = 'http://www.1800contacts.com/productlist.aspx?dl=P&source=cj&ac=8.2.0007'
uri = URI.parse(address)
puts "#{uri.scheme}://#{uri.host}" # => http://www.1800contacts.com
I like #skojin's answer number 1 (sorry I opened another answer, it's only a long comment) because it makes more general code for both cases:
require 'uri'
uri = URI "http://www.1800contacts.com/productlist.aspx?dl=P&source=cj&ac=8.2.0007"
uri.query = uri.fragment = nil
uri.path = ""
uri.to_s
# => "http://www.1800contacts.com"
uri = URI "http://example.com:8080/you-found-me.php"
uri.query = uri.fragment = nil
uri.path = ""
uri.to_s
# => "http://example.com:8080"
2 alternative ways
uri = URI.parse(url); uri.path = ''; uri.query = nil; uri.to_s
url.split('/')[0,3].join('/')
Related
Hoping for some help as this one has me baffled...
I created a user account and API credentials at FTX.com.
They have an interesting Auth setup which is detailed here: https://docs.ftx.com/?python#authentication
They only provide code examples for python, javascript and c#, but I need to implement the integration on a RoR app.
Here's a link which also provides an example for both GET and POST calls: https://blog.ftx.com/blog/api-authentication/
I'm using:
ruby '3.0.1'
gem 'rails', '~> 6.1.4', '>= 6.1.4.1'
also,
require 'uri'
require 'net/https'
require 'net/http'
require 'json'
I got the authentication working for GET calls as follows:
def get_market
get_market_url = 'https://ftx.com/api/markets/BTC-PERP/orderbook?depth=20'
api_get_call(get_market_url)
end
def api_get_call(url)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}GET/api/markets"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s
}
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
rsp = http.get(uri, headers)
JSON.parse(rsp.body)
end
This works great and I get the correct response:
=>
{"success"=>true,
"result"=>
{"bids"=>
[[64326.0, 2.0309],
...
[64303.0, 3.1067]],
"asks"=>
[[64327.0, 4.647],
...
[64352.0, 0.01]]}}
However, I can't seem to authenticate correctly for POST calls (even though as far as I can tell I am following the instructions correctly). I use the following:
def create_subaccount
create_subaccount_url = 'https://ftx.com/api/subaccounts'
call_body =
{
"nickname": "sub2",
}.to_json
api_post_call(create_subaccount_url, call_body)
end
def api_post_call(url, body)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}POST/api/subaccounts#{body}"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s
}
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
request = Net::HTTP::Post.new(uri, headers)
request.body = body
response = http.request(request)
JSON.parse(response.body)
end
Also tried passing headers via request[] directly:
def api_post_call(url, body)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}POST/api/subaccounts#{body}"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
request = Net::HTTP::Post.new(uri)
request['FTX-KEY'] = ENV['FTX_API_KEY']
request['FTX-SIGN'] = signature
request['FTX-TS'] = ts.to_s
request.body = body
response = http.request(request)
JSON.parse(response.body)
end
This is the error response:
=> {"success"=>false, "error"=>"Not logged in: Invalid signature"}
My feeling is the issue is somewhere in adding the body to signature_payload before generating the signature via HMAC here..?:
signature_payload = "#{ts}POST/api/subaccounts#{body}"
Thinking this because, if I leave out #{body} here, like so:
signature_payload = "#{ts}POST/api/subaccounts"
the response is:
=> {"success"=>false, "error"=>"Missing parameter nickname"}
I have tried several iterations of setting up the POST call method using various different net/https examples but have had no luck...
I have also contacted FTX support but have had no response.
Would truly appreciate if anyone has some insight on what I am doing wrong here?
try this headers
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s,
'Content-Type' => 'application/json',
'Accepts' => 'application/json',
}
Here's a working example of a class to retrieve FTX subaccounts. Modify for your own purposes. I use HTTParty.
class Balancer
require 'uri'
require "openssl"
include HTTParty
def get_ftx_subaccounts
method = 'GET'
path = '/subaccounts'
url = "#{ENV['FTX_BASE_URL']}#{path}"
return HTTParty.get(url, headers: headers(method, path, ''))
end
def headers(*args)
{
'FTX-KEY' => ENV['FTX_API_KEY'],
'FTX-SIGN' => signature(*args),
'FTX-TS' => ts.to_s,
'Content-Type' => 'application/json',
'Accepts' => 'application/json',
}
end
def signature(*args)
OpenSSL::HMAC.hexdigest(digest, ENV['FTX_API_SECRET'], signature_payload(*args))
end
def signature_payload(method, path, query)
payload = [ts, method.to_s.upcase, "/api", path].compact
if method==:post
payload << query.to_json
elsif method==:get
payload << ("?" + URI.encode_www_form(query))
end unless query.empty?
payload.join.encode("UTF-8")
end
def ts
#ts ||= (Time.now.to_f * 1000).to_i
end
def digest
#digest ||= OpenSSL::Digest.new('sha256')
end
end
Let's say I have a string like:
url = "https://example.com/user/tr_auth.php?key=34432&cmp_id=344&tr_id={user_id}"
I want to update the cmp_id=344 to be say cmp_id=44553. What's the best way to accomplish this? I can't gsub per say because I don't know what cmp_id might be equal, only that it will be a URL parameter in the string.
It seems like I want to do something like
uri = URI.parse(url)
params = CGI.parse(uri.query)
But then, how do I re-build the string swapping out the cmp_id parameter to be 44553?
Thanks!
If you are dealing with a web application (and/or Rails as the tag seems to indicate), then you certainly have Rack available. Rack::Utils has methods to parse and build a query.
url = "https://example.com/user/tr_auth.php?key=34432&cmp_id=344&tr_id={user_id}"
uri = URI.parse(url)
query = Rack::Utils.parse_query(uri.query)
# => {"key"=>"34432", "cmp_id"=>"344", "tr_id"=>"{user_id}"}
# Replace the value
query["cmp_id"] = 44553
uri.query = Rack::Utils.build_query(query)
uri.to_s
# => "https://example.com/user/tr_auth.php?key=34432&cmp_id=44553&tr_id=%7Buser_id%7D"
Also note that Rack, by default, escapes the query.
url = "https://example.com/user/tr_auth.php?key=34432&cmp_id=344&tr_id={user_id}"
uri = URI.parse(url)
params = CGI.parse(uri.query)
params['cmp_id'] = 44553
new_str = uri.host + uri.path + '?' + params.to_query
First, you can parse the url for params:
require 'cgi'
url = 'https://example.com/user/tr_auth.php?key=34432&cmp_id=344&tr_id={user_id}'
string_params = url.split('?')[1]
hash = CGI::parse(string_params)
Then you can iterate the hash by keys and change values:
hash.keys.each {|key| hash[key]='new value'}
url_params = hash.to_param
I'd like to open my stackoverflow.com page via ruby.
And I'd like to see it as if I am authenticated.
I took usr cookie from Google Chrome and created the following snippet:
require 'net/http'
require 'cgi'
url = "http://stackoverflow.com/users/1650525/alex-smolov"
uri = URI(url)
http = Net::HTTP.new(uri.host, 80)
request = Net::HTTP::Get.new(uri.request_uri)
cookie = CGI::Cookie.new("usr", "[my cookie is here]")
request['Cookie'] = cookie
r = http.request(request)
puts r.body
It does output a page, but I'm not authenticated there.
Is it possible to make a Net::HTTP::Get request in Ruby with cookie?
You need to call CGI::Cookie.to_s method.
request['Cookie'] = cookie.to_s
Try following code with / without .to_s.
require 'net/http'
require 'cgi'
uri = URI("http://httpbin.org/cookies")
http = Net::HTTP.new(uri.host, 80)
request = Net::HTTP::Get.new(uri.request_uri)
cookie1 = CGI::Cookie.new('usr', 'blah')
request['Cookie'] = cookie1.to_s # <---
r = http.request(request)
puts r.body
UPDATE
As the other answer mentioned, the resulted string is for server output. You need to strip out ; path= part.
CGI::Cookie.new('usr', 'value').to_s.sub(/; path=$/, '')
The accepted answer is imho incorrect. CGI::Cookie#to_s generates
string which should SERVER send to client, not something Net::HTTP should
use. It can be easily demonstrated:
[1] pry(main)> require 'cgi'
=> true
[2] pry(main)> CGI::Cookie.new('usr', 'value').to_s
=> "usr=value; path="
Code like this should work better.
require 'net/http'
require 'cgi'
uri = URI("http://httpbin.org/cookies")
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Get.new(uri.request_uri)
request['Cookie'] = "usr=#{CGI.encode cookie_value}"
r = http.request(request)
puts r.body
Or in case you have multiple cookies in a hash:
h = {'cookie1' => 'val1', 'cookie2' => 'val2'}
req['Cookie'] = h.map { |k,v| "#{k}=#{CGI.encode v}" } .join('; ')
We have a Rails 3.2.12 app that fetches data from a partner API by making POST requests. However, we're finding that our code seems to be a bottleneck, as it takes longer to process requests than expected. Can we do anything to speed up the code?
Should we use another XML parser? Is there a faster way to post SSL requests in Ruby?
Thanks!
def ced( ad )
# Set vars
u = 'e'
pw = 'p'
ad = ad.join ','
url = 'https://r.e.com/interface.asp?ResponseType=XML&Command=BC' + '&UID=' + u + '&PW=' + pw + '&DL=' + ad
results = []
# Invoke API command
uri = URI.parse url
http = Net::HTTP.new uri.host, uri.port
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
req = Net::HTTP::Get.new uri.request_uri
resp = http.request req
# Parse API response
resp_xml = Nokogiri.XML resp.body
resp_xml.remove_namespaces!
num_errors = resp_xml.at_xpath('//ErrCount').content
# Any errors?
if num_errors != '0'
error = true
# No errors, process domains
else
resp_xml.xpath('//CheckResult').each do |elem|
results << {
:domain => elem.xpath('D').text.downcase,
:status => case elem.xpath('RRPCode').text
when '210' then 'a'
when '211' then 't'
else
error = true
'error'
end
}
end
end
<truncated>
end
I'm trying to do a simple GET with ruby to the Facebook fql.query method without success.
The url is basically structured like this:
https://api.facebook.com/method/fql.query?query=SELECT total_count FROM link_stat WHERE url = "http://twitter.com/"&format=json
I've read in a few posts here on StackOverflow about how to make those requests, but even tho I keep getting:
/usr/lib/ruby/1.8/net/http.rb:560:in `initialize': getaddrinfo: Name or service not known (SocketError)
On the first line of http_get function.
def http_get(domain,path,params)
return Net::HTTP.get(domain, "#{path}?".concat(params.collect { |k,v| "#{k}=#{CGI::escape(v.to_s)}" }.join('&'))) if not params.nil?
return Net::HTTP.get(domain, path)
end
def getFacebookStats(url)
params = {
:query => 'SELECT total_count FROM link_stat WHERE url = "' + url + '"',
:format => 'json'
}
http = http_get('https://api.facebook.com', '/method/fql.query', params)
puts http
end
The http call accepts a host, not a URL:
def http_get(domain,path,params)
path = unless params.blank
path + "?" + params.collect { |k,v| "#{k}=#{CGI::escape(v.to_s)}" }.join('&')
else
path
end
request = Net::HTTP.get(domain, path)
end
def get_facebook_stats(url)
params = {
:query => 'SELECT total_count FROM link_stat WHERE url = "' + url + '"',
:format => 'json'
}
http = http_get('api.facebook.com', '/method/fql.query', params)
puts http
end
Please do not use camel case on method names on Ruby.
If you want to make HTTPS calls, you will have to use a different call:
require 'net/http'
require 'net/https'
http = Net::HTTP.new('somehost.com', 443)
http.use_ssl = true
path = '/login.html'
resp, data = http.get(path, nil)