Given any valid HTTP/HTTPS string, I would like to parse/transform it such that the end result is exactly the root of the string.
So given URLs:
http://foo.example.com:8080/whatsit/foo.bar?x=y
https://example.net/
I would like the results:
http://foo.example.com:8080/
https://example.net/
I found the documentation for URI::Parser not super approachable.
My initial, naïve solution would be a simple regex like:
/\A(https?:\/\/[^\/]+\/)/
(That is: Match up to the first slash after the protocol.)
Thoughts & solutions welcome. And apologies if this is a duplicate, but my search results weren't relevant.
With URI::join:
require 'uri'
url = "http://foo.example.com:8080/whatsit/foo.bar?x=y"
baseurl = URI.join(url, "/").to_s
#=> "http://foo.example.com:8080/"
Use URI.parse and then set the path to an empty string and the query to nil:
require 'uri'
uri = URI.parse('http://foo.example.com:8080/whatsit/foo.bar?x=y')
uri.path = ''
uri.query = nil
cleaned = uri.to_s # http://foo.example.com:8080
Now you have your cleaned up version in cleaned. Taking out what you don't want is sometimes easier than only grabbing what you need.
If you only do uri.query = '' you'll end up with http://foo.example.com:8080? which probably isn't what you want.
You could use uri.split() and then put the parts back together...
WARNING: It's a little sloppy.
url = "http://example.com:9001/over-nine-thousand"
parts = uri.split(url)
puts "%s://%s:%s" % [parts[0], parts[2], parts[3]]
=> "http://example.com:9001"
Related
I'd like to use the URI or CGI libraries to get the path from the query part of this url. In other words, just: '/scouting/amateur'. Is this possible or do I need to use regexp?
http://10.241.180.63:3149/login?redirect_path=http%3A%2F%2F10.241.180.63%3A3149%2Fscouting%2Famateur
Suggestion with Ruby built-ins (if designing a method, you might want to implement some error handling).
require 'uri'
query = URI("http://10.241.180.63:3149/login?redirect_path=http%3A%2F%2F10.241.180.63%3A3149%2Fscouting%2Famateur").query
path = URI(URI.decode(query).split('=')[1]).path
You may find the gem uri-query_params helpful / more elegant (it will decode query params automatically). E.g.
require 'uri/query_params'
uri = URI("http://10.241.180.63:3149/login?redirect_path=http%3A%2F%2F10.241.180.63%3A3149%2Fscouting%2Famateur")
URI(uri.query_params["redirect_path"]).path
Try this -
url = URI.parse('http:://10.241.180.63:3149/login?redirect_path=http%3A%2F%2F10.241.180.63%3A3149%2Fscouting%2Famateur')
redirect_path = u.opaque.split("redirect_path=").last
# redirect_path is now {"redirect_path"=>["http://10.241.180.63:3149/scouting/amateur"]}
result = redirect_path.split("/").last(2).join("/")
# result = 'scouting/amateur'
How to remove random excess of slashes from url or just validate it?
For example,
valid statements:
http://domain.com/url/url2
https://domain.com/url/url2
www.domain.com/url/url2
invalid statements:
http://domain.com//url/url2
https://domain.com/////url/url2
www.domain.com/url/////////url2
Thanks for help!
Use regular expressions:
require 'uri'
url = URI.parse('https://domain.com/////url/url2')
url.path.gsub! %r{/+}, '/'
p url.to_s
this pattern do the job (with http(s) or not) :
"https://domain.com/////url/url2".gsub! %r{(?<!:)/+(?=/)}, ''
The other answers do not remove a trailing slash from the URL - which can be important for SEO purposes. There are many ways to do this, but for example:
require 'uri'
url = URI.parse('https://example.com/////url/url2/')
url.path.gsub! %r{/+}, '/'
url.path.sub! %r{/$}, ''
Or:
require 'uri'
url = URI.parse('https://example.com/////url/url2/')
url.path.squeeze!('/')
url.path.chomp!('/')
See: String#squeeze! and String#chomp!.
I'm having trouble with a regex in Ruby (on Rails). I'm relatively new to this.
The test string is:
http://www.xyz.com/017010830343?$ProdLarge$
I am trying to remove "$ProdLarge$". In other words, the $ signs and anything between.
My regular expression is:
\$\w+\$
Rubular says my expression is ok. http://rubular.com/r/NDDQxKVraK
But when I run my code, the app says it isn't finding a match. Code below:
some_array.each do |x|
logger.debug "scan #{x.scan('\$\w+\$')}"
logger.debug "String? #{x.instance_of?(String)}"
x.gsub!('\$\w+\$','scl=1')
...
My logger debug line shows a result of "[]". String is confirmed as being true. And the gsub line has no effect.
What do I need to correct?
Use /regex/ instead of 'regex':
> "http://www.xyz.com/017010830343?$ProdLarge$".gsub(/\$\w+\$/, 'scl=1')
=> "http://www.xyz.com/017010830343?scl=1"
Don't use a regex for this task, use a tool designed for it, URI. To remove the query:
require 'uri'
url = URI.parse('http://www.xyz.com/017010830343?$ProdLarge$')
url.query = nil
puts url.to_s
=> http://www.xyz.com/017010830343
To change to a different query use this instead of url.query = nil:
url.query = 'scl=1'
puts url.to_s
=> http://www.xyz.com/017010830343?scl=1
URI will automatically encode values if necessary, saving you the trouble. If you need even more URL management power, look at Addressable::URI.
If I have two URLs in Rails, (whether they be in string form or URI objects) what's the best way to determine if they are equal? It seems like a fairly simple problem, but I need the solution to work even if one of the URLs is relative and the other is absolute, or if one of the URLs has different parameters than the other.
I already looked at What is the best way in Rails to determine if two (or more) given URLs (as strings or hash options) are equal? (and several other questions), but the question was pretty old and the suggested solution doesn't work the way I need it to.
Provided you have url1 and url2 being some string containing a URL :
def is_same_controller_and_action?(url1, url2)
hash_url1 = Rails.application.routes.recognize_path(url1)
hash_url2 = Rails.application.routes.recognize_path(url2)
[:controller, :action].each do |key|
return false if hash_url1[key] != hash_url2[key]
end
return true
end
1) convert URL to canonical form
In my current project I am using addressable gem in order to do that:
def to_canonical(url)
uri = Addressable::URI.parse(url)
uri.scheme = "http" if uri.scheme.blank?
host = uri.host.sub(/\www\./, '') if uri.host.present?
path = (uri.path.present? && uri.host.blank?) ? uri.path.sub(/\www\./, '') : uri.path
uri.scheme.to_s + "://" + host.to_s + path.to_s
rescue Addressable::URI::InvalidURIError
nil
rescue URI::Error
nil
end
Example:
> to_canonical('www.example.com') => 'http://example.com'
> to_canonical('http://example.com') => 'http://example.com'
2) compare your URLs: canonical_url1 == canonical_url2
UPD:
Does it work with sub-domains? - No. I mean, we cannot say that translate.google.com and google.com are equal. Of course, you can modify it depending on your needs.
Checkout the addressable gem and specifically the normalize method (and its documentation), and the heuristic_parse method (and its documentation). I've used it in the past and found it to be very robust.
Addressable even handles URLs with unicode characters in them:
uri = Addressable::URI.parse("http://www.詹姆斯.com/")
uri.normalize
#=> #<Addressable::URI:0xc9a4c8 URI:http://www.xn--8ws00zhy3a.com/>
Removing the query string from a URL in Ruby could be done like this:
url.split('?')[0]
Where url is the complete URL including the query string (e.g. url = http://www.domain.extension/folder?schnoo=schnok&foo=bar).
Is there a faster way to do this, i.e. without using split, but rather using Rails?
edit: The goal is to redirect from http://www.domain.extension/folder?schnoo=schnok&foo=bar to http://www.domain.extension/folder.
EDIT: I used:
url = 'http://www.domain.extension/folder?schnoo=schnok&foo=bar'
parsed_url = URI.parse(url)
new_url = parsed_url.scheme+"://"+parsed_url.host+parsed_url.path
Easier to read and harder to screw up if you parse and set fragment & query to nil instead of rebuilding the URL.
parsed = URI::parse("http://www.domain.extension/folder?schnoo=schnok&foo=bar#frag")
parsed.fragment = parsed.query = nil
parsed.to_s
# => "http://www.domain.extension/folder"
url = 'http://www.domain.extension/folder?schnoo=schnok&foo=bar'
u = URI.parse(url)
p = CGI.parse(u.query)
# p is now {"schnoo"=>["schnok"], "foo"=>["bar"]}
Take a look on the : how to get query string from passed url in ruby on rails
You can gain performance using Regex
'http://www.domain.extension/folder?schnoo=schnok&foo=bar'[/[^\?]+/]
#=> "http://www.domain.extension/folder"
Probably no need to split the url. When you visit this link, you are pass two parameters to back-end:
http://www.domain.extension/folder?schnoo=schnok&foo=bar
params[:schnoo]=schnok
params[:foo]=bar
Try to monitor your log and you will see them, then you can use them in controller directly.