I want to know the route(all the intermediate servers/routers/etc. and final destination) taken by a query/url to fetch data fetched from a server.
For example: http://coreapi.imagin8ors.org:8080/v1/child/140df552-eba9-42c3-8c9e-6d478637925f/learningpods_new/
Any Online tools or software's to do the same?
Any help is appreciated?
You can use the command traceroute to trace down all the nodes in the path while querying a domain name.
But, please note that the command may fail for URLs containing parameters like the one you have provided in your question. The command takes only the domain name as argument, not complete URL with extra parameters. Example:
traceroute coreapi.imagin8ors.org
and not:
traceroute http://coreapi.imagin8ors.org:8080/v1/child/140df552-eba9-42c3-8c9e-6d478637925f/learningpods_new/
The point to understand here is that the domain name is mapped to a specific source of a resource and the extra parameters after the domain name in a URL is to specify a particular resource from that source. Hence no matter what the full URL is, only the domain name matters for the path to be taken by the packets.
So, given any URL you could safely extract out the domain name and trace the path using traceroute command with the URL to get the path you are looking for.
Hope this helps. Thanks.
Is there a Unique Resource Identifier (URI) schema which allows to uniquely refer to IP addresses and domain names?
For example "http://8o.232.240.50" or "http://supersite.com" are not refering to IP address or a domain name, rather they are refering to some resource which can be accessed via HTTP protocol and it's HTTP path happens to include this given IP address or domain name. Therefore they are more of Unique Resource Locators (URL) than Unique Resource Identifiers (URI) or Unique Resource Names (URN).
What I am looking for are schemas like "mailto:someemai#something.com" for IP addresses and/or domain names.
Something like "domain:supersite.com" and "ip:80.232.240.50".
I could make up schemes like these for my own use, but I am wondering if there isn't something like this already out there. And if there isn't, why?
RFC 4501: Domain Name System Uniform Resource Identifiers
The Domain Name System (DNS) […] is a widely deployed system
used, among other things, to translate host names into IP addresses.
Several protocols use Uniform Resource Identifiers (URIs) to refer to
data. By defining a URI scheme for DNS data, the gap between these
two worlds is bridged. The DNS URI scheme defined here can be used
to reference any data stored in the DNS.
From the examples:
dns:www.example.org.?clAsS=IN;tYpE=A
dns:www.example.org
dns:simon.example.org?type=CERT
dns://192.168.1.1/ftp.example.org?type=A
dns:world%20wide%20web.example%5c.domain.org?TYPE=TXT
dns://fw.example.org/*.%20%00.example?type=TXT
According to the syntax, it should be possible to use the dnsauthority with an empty dnsname. dnsauthority takes a host as defined in RFC 3986, which says:
host = IP-literal / IPv4address / reg-name
So it should be possible to use domain names as well as IP addresses (an empty dnsname "is to be interpreted as the root itself").
http://example.com/something/somewhere//somehow/script.js
Does the double slash break anything on the server side? I have a script that parses URLs and i was wondering if it would break anything (or change the path) if i replaced multiple slashes with a single slash. Especially on the server side, some frameworks like CodeIgniter and Joomla use segmented url schemes and routing. I would just want to know if it breaks anything.
HTTP RFC 2396 defines path separator to be single slash.
However, unless you're using some kind of URL rewriting (in which case the rewriting rules may be affected by the number of slashes), the uri maps to a path on disk, but in (most?) modern operating systems (Linux/Unix, Windows), multiple path separators in a row do not have any special meaning, so /path/to/foo and /path//to////foo would eventually map to the same file.
An additional thing that might be affected is caching. Since both your browser and the server cache individual pages (according to their caching settings), requesting same file multiple times via slightly different URIs might affect the caching (depending on server and client implementation).
The correct answer to this question is it depends upon the implementation of the server!
Preface: Double-slash is syntactically valid according to RFC 2396, which defines URL path syntax. As amn explains, it therefore implies an empty URI segment. Note however that RFC 2396 only defines the syntax, not semantics of paths, including empty path segments, so it is up to your server to decide the semantics of the empty path.
You didn't mention the server software stack you're using, perhaps you're even rolling your own? So please use your imagination as to what the semantics could be!
Practically, I would like to point out some everyday semantic-related reasons which mean you should avoid double slashes even though they are syntactically valid:
Since empty being valid is somehow not expected by everyone, it can cause bugs. And even though your server technology of today might be compatible with it, either your server technology of tomorrow or the next version of your server technology of today might decide not to support it any more. Example: ASP.NET MVC Web API library throws an error when you try to specify a route template with a double slash.
Some servers might interpret // as indicating the root path. This can either be on-purpose, or a bug - and then likely it is a security bug, i.e. a directory traversal vulnerability.
Because it is sometimes a bug, and a security bug, some clever server stacks and firewalls will see the substring '//', deduce you are possibly making an attempt at exploiting such a bug, and therefore they will return 403 Forbidden or 400 Bad Request etc, and refuse to actually do any further processing of the URI.
URLs don't have to map to filesystem paths. So even if // in a filesystem path is equivalent to /, you can't guarantee the same is true for all URLs.
Consider the declaration of the relevant path-absolute non-terminal in "RFC3986: Uniform Resource Identifier (URI): Generic Syntax" (specified, as is typical, in ABNF syntax):
path-absolute = "/" [ segment-nz *( "/" segment ) ]
Then consider the segment declaration a few lines further down in the same document:
segment = *pchar
If you can read ABNF, the asterisk (*) specifies that the following element pchar may be repeated multiple times to make up a segment, including zero times. Learning this and re-reading the path-absolute declaration above, you can see that a potentially empty segment imples that the second "/" may repeat indefinitely, hence allowing valid combinations like ////// (arbitrary length of at least one /) as part of path-absolute (which itself is used in specifying the rule describing a URI).
As all URLs are URIs we can conclude that yes, URLs are allowed multiple consecutive forward slashes, per quoted RFC.
But it's not like everyone follows or implements URI parsers per specification, so I am fairly sure there are non-compliant URI/URL parsers and all kinds of software that stacks on top of these where such corner cases break larger systems.
One thing you may want to consider is that it might affect your page indexing in a search engine. According to this web page,
A URL with the same path repeated 3 times will not be indexed in Google
The example they use is:
example.com/path/path/path/
I haven't confirmed this would also be true if you used example.com///, but I would certainly want to find out if SEO optimization was critical for my website.
They mention that "This is because Google thinks it has hit a URL trap." If anyone else knows the answer for sure, please add a comment to this answer; otherwise, I thought it relevant to include this case for consideration.
Yes, it can most definitely break things.
The spec considers http://host/pages/foo.html and http://host/pages//foo.html to be different URIs, and servers are free to assign different meanings to them. However, most servers will treat paths /pages/foo.html and /pages//foo.html identically (because the underlying file system does too). But even when dealing with such servers, it's easily possible for extra slash to break things. Consider the situation where a relative URI is returned by the server.
http://host/pages/foo.html + ../images/foo.png = http://host/images/foo.png
http://host/pages//foo.html + ../images/foo.png = http://host/pages/images/foo.png
Let me explain what that means. Say your server returns an HTML document that contains the following:
<img src="../images/foo.png">
If your browser obtained that page using
http://host/pages/foo.html # Path has 2 segments: "pages" and "foo.html"
your browser will attempt to load
http://host/images/foo.png # ok
However, if your browser obtained that page using
http://host/pages//foo.html # Path has 3 segments: "pages", "" and "foo.html"
you'll probably get the same page (because the server probably doesn't distinguish /pages//foo.html from /pages/foo.html), but your browser will erroneously try to load
http://host/pages/images/foo.png # XXX
You may be surprised for example when building links for resources in your app.
<script src="mysite.com/resources/jquery//../angular/script.js"></script>
will not resolve to mysite.com/resources/angular/script.js but to mysite.com/resources/jquery/angular/script.js what you probably didn't want
Double slashes are evil, try to avoid them.
Your question is "does it break anything". In terms of the URL specification, extra slashes are allowed. Don't read the RFC, here is a quick experiment you can try to see if your browser silently mangles the URL:
echo '<?= $_SERVER['REQUEST_URI'];' > tmp.php
php -S localhost:4000 tmp.php
I tested macOS 10.14 (18A391) with Safari 12.0 (14606.1.36.1.9) and Chrome 69.0.3497.100 and both get the result:
/hello//world
This indicated that using an extra slash is visible to the web application.
Certain use cases will be broken when using a double slash. This includes URL redirects/routing that are expecting a single-slashed URL or other CGI applications that are analyzing the URI directly.
But for normal cases of serving static content, such as your example, this will still get the correct content. But the client will get a cache miss against the same content accessed with different slashes.
I currently do automatic detection of hyperlinks within text in my program. I made it very simple and only look for http:// or www.
However, a user suggested to me that I extend it to other forms, e.g.: https:// or .com
Then I realized it might not stop there because there's ftp and mailto and file, all the other top level domains, and even email addresses and file paths.
What I think is best is to limit it to what is practical by following some often-used standard set of hyperlink detection rules that are currently in use. Maybe how Microsoft Word does it, or maybe how RichEdit does it or maybe you know of a better standard.
So my question is:
Is there a built in function that I can call from Delphi to do the detection, and if so, what would the call look like? (I plan in the future to go to FireMonkey, so I would prefer something that will work beyond Windows.)
If there isn't a function available, is there some place I can find a documented set of rules of what is detected in Word, in RichEdit, or any other set of rules of what should be detected? That would then allow me to write the detection code myself.
Try the PathIsURL function which is declarated in the ShLwApi unit.
Following regex taken from RegexBuddy's library might get you started (I can't make any claims about performance).
Regex
Match; JGsoft; case insensitive:
\b(https?|ftp|file)://[-A-Z0-9+&##/%?=~_|$!:,.;]*[A-Z0-9+&##/%=~_|$]
Explanation
URL: Find in full text
The final character class makes sure that if an URL is part of some text,
punctuation such as a comma or full stop after the URL is not interpreted as part
of the URL.
Matches (whole or partial)
http://regexbuddy.com
http://www.regexbuddy.com
http://www.regexbuddy.com/
http://www.regexbuddy.com/index.html
http://www.regexbuddy.com/index.html?source=library
You can download RegexBuddy at http://www.regexbuddy.com/download.html.
Does not match
regexbuddy.com
www.regexbuddy.com
"www.domain.com/quoted URL with spaces"
support#regexbuddy.com
For a set of rules you might look into RFC 3986
A Uniform Resource Identifier (URI) is a compact sequence of
characters that identifies an abstract or physical resource. This
specification defines the generic URI syntax and a process for
resolving URI references that might be in relative form, along with
guidelines and security considerations for the use of URIs on the
Internet
A regex that validates a URL as specified in RFC 3986 would be
^
(# Scheme
[a-z][a-z0-9+\-.]*:
(# Authority & path
//
([a-z0-9\-._~%!$&'()*+,;=]+#)? # User
([a-z0-9\-._~%]+ # Named host
|\[[a-f0-9:.]+\] # IPv6 host
|\[v[a-f0-9][a-z0-9\-._~%!$&'()*+,;=:]+\]) # IPvFuture host
(:[0-9]+)? # Port
(/[a-z0-9\-._~%!$&'()*+,;=:#]+)*/? # Path
|# Path without authority
(/?[a-z0-9\-._~%!$&'()*+,;=:#]+(/[a-z0-9\-._~%!$&'()*+,;=:#]+)*/?)?
)
|# Relative URL (no scheme or authority)
([a-z0-9\-._~%!$&'()*+,;=#]+(/[a-z0-9\-._~%!$&'()*+,;=:#]+)*/? # Relative path
|(/[a-z0-9\-._~%!$&'()*+,;=:#]+)+/?) # Absolute path
)
# Query
(\?[a-z0-9\-._~%!$&'()*+,;=:#/?]*)?
# Fragment
(\#[a-z0-9\-._~%!$&'()*+,;=:#/?]*)?
$
Regular Expressions may be the way to go here, to define the various patterns which you deem to be appropriate hyperlinks.
I have the following two lines of codes in my model, however, both virtual and path have values "\". Where have I gone wrong?
var virtual = VirtualPathUtility.ToAbsolute(HttpContext.Current.Request.ApplicationPath);
var path =HttpContext.Current.Request.ApplicationPath;
From MSDN:
Gets the ASP.NET application's virtual application root path on the server.
So this is the part of the URL's path that is the root of the IIS Web Application the code is running in. The root URL ("http://domain/") is always an IIS Application, so will give "/" as its ApplicationPath.
You perhaps need to convert some child (virtual) folder into an IIS Application to see a longer path result.
Updated from comment:
I want to have the part ":/..."
This information is all available within the properties of Request.Uri. In particular "http" is Uri.Scheme, severname is Uri.Host and the port is Uri.Port (but check Uri.IsDefaultPort to check if you need to specify it).
http://weblogs.asp.net/srkirkland/archive/2009/09/17/a-urlhelper-extension-for-creating-absolute-action-paths-in-asp-net-mvc.aspx