I want to run imposm to import OSM data to a PostGIS DB.
My problem is that the password for the DB contains special characters like ":" (a colon). I cannot change the password.
The documentation states to use the connection parameter like this:
The -connection option takes a URL in the format
postgis://username:password#host:port/databasename or a list of
parameters like postgis: host=/tmp dbname=osm
However, this produces an error because of the special characters in the password.
parsing database connection URL: parse "postgres://user:pss?wo:r?d!#host/osm": invalid port ":r?d" after host
The documentation also says to use a list of parameters but it is not clear to me how these parameters have to be named.
Does anyone know how to pass the connection parameters with a list instead of the URL encryption? I can only find host and dbname and I dont't know how to write the connection excatly. Or is there a possibility to escape special characters in the URL?
You must encode the special characters.
If your password is pss?wo:r?d!, the connection URL would be
postgresql://user:pss%3Fwo%3Ar%3Fd%21#host/osm
Related
I have an application that stores information about a person onto a database, but when I try to use the URL to GET a user based on their email address users with a + in their email cannot be found.
Example URL that returns person:
https://www.someURL.com/api/people/johnsmith#someemail.com
Example URL that does not return person (returns null):
https://www.someURL.com/api/people/jane+doe#someemail.com
Both emails are in the database as written in the URL so it does not appear to be a typo issue, and I am using postman to test the GET method. Why am I not able to find them, and how can I make it so that they can be found even with the + character?
Working postman request
NOT working postman request
When I search with id I am able to find the person so I know the person exists.
Verification that person exists
My suggestion would be: change your server implementaion from GET to POST and provide an email as a String parameter within the body of request. It'll prevent this and any similar issue with escaping special characters in URI.
If it's not possible, try to frame email address with a single ' or double " quotes, depending on how your web server treats incoming request it may help as well.
Nice to know that "+" is not really a 'valid' character for a lot of email providers for a reason. For instance, Gmail will not let you to create an email address with anything but [A-z0-9] (alphanumeric) and dot (.) characters. I'm pretty sure they were tired of validating input emails with complex regular expression and just limited it to basic ones.
'+' is a reserved character in URIs, so in order to prevent it being interpreted as a space character you would need to percent-encode it. In your example, replace '+' with '%2B'.
https://www.someURL.com/api/people/jane%2Bdoe#someemail.com
There are other characters that are allowed in email addresses but are reserved characters in URIs, so it would be best to percent-encode the whole email address, just in case.
Is there a way to make a GET request in Siesta, while providing parameter, like http://example.com/api/list.json?myparam=1?
I tried with
myAPI.resource("list.json?myparam=1")
but the question mark gets escaped.
Then I tried with
myAPI.resource("list.json").request(.GET, urlEncoded:["myparam": "1"])
but it always fails with "The network connection was lost.", but all other requests succeed, so the message is wrong.
You are looking for withParam:
myAPI.resource("list.json").withParam("myparam", "1")
The Service.resource(_:) method you are trying to use in your first example specifically avoids interpreting special characters as params (or anything except a path). From the docs:
The path parameter is simply appended to baseURL’s path, and is never interpreted as a URL. Strings such as .., //, ?, and https: have no special meaning; they go directly into the resulting resource’s path, with escaping if necessary.
This is a security feature, meant to prevent user-submitted strings from bleeding into other parts of the URL.
The Resource.request(_:urlEncoded:) method in your second example is for passing parameters in a request body (i.e. with a POST or PUT), not for parameters in the query string.
Note that you can always use Service.resource(absoluteURL:) to construct a URL yourself if you want to bypass Siesta’s URL component isolation and escaping features.
I need users to be able to pass a file path as a parameter of a get url (the file would not be uploaded and only the local file path is used for some security reasons). Now it's difficult for them to go and change all the backslashes to "%5". I was wondering if there is a way to force encoding of a part of the url. For example something as simple as putting it in double quotes, which doesn't work...
http://example.com/"c:\user\somone\somefile.txt"/dosomething
I ended up using pattern matching of rest routes at the server level. Something like this:
/example.com/*path/dosomething
So it would match any path even with slashes/backslashes. At last I do a decoding of the url to get rid of the escaped characters passed by browser for chars like space.
java.net.URLDecoder.decode(path, "UTF-8")
Just say I have the following url that has a query string parameter that's an url:
http://www.someSite.com?next=http://www.anotherSite.com?test=1&test=2
Should I url encode the next parameter? If I do, who's responsible for decoding it - the web browser, or my web app?
The reason I ask is I see lots of big sites that do things like the following
http://www.someSite.com?next=http://www.anotherSite.com/another/url
In the above, they don't bother encoding the next parameter because I'm guessing, they know it doesn't have any query string parameters itself. Is this ok to do if my next url doesn't include any query string parameters as well?
RFC 2396 sec. 2.2 says that you should URL-encode those symbols anywhere where they're not used for their explicit meanings; i.e. you should always form targetUrl + '?next=' + urlencode(nextURL).
The web browser does not 'decode' those parameters at all; the browser doesn't know anything about the parameters but just passes along the string. A query string of the form http://www.example.com/path/to/query?param1=value¶m2=value2 is GET-requested by the browser as:
GET /path/to/query?param1=value¶m2=value2 HTTP/1.1
Host: www.example.com
(other headers follow)
On the backend, you'll need to parse the results. I think PHP's $_REQUEST array will have already done this for you; in other languages you'll want to split over the first ? character, then split over the & characters, then split over the first = character, then urldecode both the name and the value.
According to RFC 3986:
The query component is indicated by the first question mark ("?")
character and terminated by a number sign ("#") character or by the
end of the URI.
So the following URI is valid:
http://www.example.com?next=http://www.example.com
The following excerpt from the RFC makes this clear:
... as query components are often used to carry identifying
information in the form of "key=value" pairs and one frequently used
value is a reference to another URI, it is sometimes better for
usability to avoid percent-encoding those characters.
It is worth noting that RFC 3986 makes RFC 2396 obsolete.
Using Selenium I am accessing protected pages. I need to put the credentials into the URL to prevent the .htaccess popup from appearing. This is the method suggested in Selenium documentation.
One of the locations I need to access has a hash character in the password, and this causes the browser (both Chrome and Firefox) to not understand the URL and treat it as a search term.
e.g. http://user:pass#example.com/ gets through, but http://user:pa#ss#example.com/ is not recognised as a URL.
How can I "encode" the hash?
You should use Percent-encoding to encode the hash with %23.
See also:
How to escape hash character in URL