I've discovered when using a custom primary key in django admin, sometimes the assigned url is inconsistent with the id used.
When the primary key is 829_011
class Run(models.Model):
id = models.CharField(max_length=50, primary_key=True)
This url works:
https://localhost/admin/myapp/run/829_5F011/
but this does not - 404 error:
https://localhost/admin/myapp/run/829_011/
I don't know why django assigns an inconsistent url.
Does anyone know whether there is a simple fix for this?
Changing the model is going to be a major pain.
It's interesting that I consistently get the "5F" automatically put in the routing so if my primary key is 44_66 the url will end with 44_5F66.
I had to change the schema which was a major work to avoid this inconsistency.
In short, do not use a string primary key with django.
This is simply a misunderstanding of how custom primary keys in Django admin work. Since you can put any characters, valid or invalid for urls, into a string primary key, Django has to encode these primary keys so that they do not confuse the admin url parser. This is described here: https://github.com/django/django/blob/master/django/contrib/admin/utils.py#L63
The special characters : / and _ are special characters in django urls, so for a primary key with the value 44_66 Django has to escape the _ and this is why you see 44_5F66 in the url. The primary key in the database is still 44_66.
Related
I've created a Rest API for a existing system utilizing Rails and am attempting to consume it in an external system via ActiveResource. Unfortunately the primary key of one of the core tables is an arbitrary string defined by the user so many non-URL friendly characters have been used over the years. We've ended up with keys such as "CR 1400/2400 A-C", which ActiveResource is not encoding correctly into a restful URL. It is deals with the spaces correctly, but does not encode the forward slashes amongst other characters.
I would like to be able to call the find method with the primary key containing these forbidden characters such as:
p = Project.find('CR 1400/2400 A-C')
which would result in a url such as:
http://localhost:3000/projects/CR%201400%2F2400%20A-C.json
instead of:
http://localhost:3000/projects/CR%201400/2400%20A-C.json
I cannot change the database schema even though currently very little would bring me greater joy.
Is there a way to tell ActiveResource to encode additional characters, or intercept the call to encode them prior to constructing the URL?
Thanks in advance
Neil
I'm building a little token-based authentication library for my (rails based) api server which uses redis to store generated auth tokens. The line I'm worried about is: user_id = $redis.get("auth:#{token}"), where token is what's passed in to authenticate_or_request_with_http_token.
If this were SQL, that'd be a huge red flag - string interpolated SQL queries are pretty insecure. As far as I can tell, however, doing string interpolation on a redis key query isn't insecure.
My source for the above claim is the redis documentation here: http://redis.io/topics/security (under the string escaping and nosql injection header), but I wanted to make sure that this is the case before I get a Bobby Tables attack.
The documentation you are pointing to is quite explicit:
The Redis protocol has no concept of string escaping, so injection is impossible under normal circumstances using a normal client library. The protocol uses prefixed-length strings and is completely binary safe.
There is a small attack vector for these kinds of string injections. While the redis documentation is clear about the difficulty of executing multiple commands on the database, it does not mention that the key separator (':' in your example) usually needs to be escaped when used as the part of a key.
I have seen a redis database using these keys:
oauth_token:123456 (which contained a hash of OAuth token parameters) and
oauth_token:123456:is_temp (which contained a boolean property to indicate whether the OAuth token is a temporary token)
Trusting the user input without escaping might result in GET oauth_token:#{token} accidentally ending up as GET oauth_token:123456:is_temp (when token has been set to 123456:is_temp by the user).
So I highly recommend to properly escape colons from potential user input to make sure your key paths cannot be tricked like this.
NOTE: Someone recommended to fix the example above by using oauth_token:123456 and oauth_token:is_temp:123456, but that is flawed (for the user-provided token is_temp:123456). The correct solution to that problem (without escaping) would be to use keys oauth_token:info:123456 and oauth_token:is_temp:123456 to make sure these keys cannot overlap whatever the user-provided input was (or simply escape colons).
Basically Redis is immune from escaping issues when the input string is used verbatim. For example:
SET mykey <some-attacker-chosen-data>
However Redis is not immune from issues arising by using non validate input in the context of string interpolation, as showed by Sven Herzberg. In order to turn the Sven example into a safe one, it is possible to just use an Hash, and avoid reverting to interpolation. Otherwise either use not common prefixes to use in conjunction with keys interpolation, or use some basic form of sanity check on the input, which is, filtering away the separator used, or better, validate that the input is actually a number (in the specific example).
So while Redis does not suffer from the typical injection attacks of SQL, when used untrusted input in the context of a string interpolation used to create key names, or even worse, Lua scripts, some care should be taken.
snatching my hair to fix this problem but I can't.
I am parsing id in url to pull data on next page according to that id. So rather than parsing id=123 I encrypted it something like process.php?token=TG4n6iv_aoO7sU3AngFY4WLSppLvueEoh-MnYE6k7NA, and decrypted it on process.php page by collecting it with $_GET, before using it in sql query. This is not proper URL, I need url like process.php?token=9878799889 and I need to decrypt this 9878799889 on process.php which would give me my original user id.
So here I can not use md5 or base64_encode which give me ugly string.
What would be best thing to do here?
id is unique so generated long digits should be unique as well and not easy to guess.
Right now I am using encrypt logarithm with salt. Actually want to parse like www.sitename.com/process/token/9878799889..this can be achieve with .htaccess so not worried about it..
Any help will be much appreciated..
What you could do is add an association table in your databse, which would contains a UUID as primary key (a randomly generated number) and your true ID reference (and other information you may want to store there, like a "valid until" date or other things...)
You'd have to generate the entry in that table as you parse the UUID
let's say INSERT INTO uuid_table (uuid, real_id) VALUES (9878799889, 123);
now when you process the url process.php?token=9878799889
you would only have to SELECT real_id FROM uuid_table WHERE uuid=9878799889;
it would retern the read id 123
You should also DELETE FROM uuid_table WHERE uuid=9878799889 when you're done.
Note that it would also work with md5 or base_64, but indeed it makes the url uglier.
I have read the term "composite URI", but I'm not familiar with what it means. Is there such a thing?
I have a CompositeUri class in an application I'm building and I would like to know if I should choose a different name.
Composite generally means something made up on a few things. For example, composite keys in databases are made up from multiple fields, which together guarentee uniqueness.
A composite URL could be anything that fits that description, but would maybe be like the SO urls which include what your viewing (question), it's ID and some friendlier string.
In ShrinkWrap, a composite URI is one that refers to an aggregate of multiple CSS or javascript files e.g.
http://example.com/css/hello.css+world.css
That composite URI denotes the contents of http://example.com/css/world.css appended to the contents of http://example.com/css/hello.css.
I see often (rewritten) URLs without ID in it, like on some wordpress installations. What is the best way of achieve this?
Example: site.com/product/some-product-name/
Maybe to keep an array of page names and IDs in cache, to avoid DB query on every page request?
How to avoid conflicts, and what are other issues on using urls without IDs?
Using an ID presents the same conundrum, really--you're just checking for a different value in your database. The "some-product-name" part of your URL above is also something unique. Some people call them slugs (Wordpress, also permalinks). So instead of querying the database for a row that has the particular ID, you're querying the database for a row that has a particular slug. You don't need to know the ID to retrieve the record.
As long as product names are unique it shouldn't be an issue. It won't take any longer (at least not significant) to look up a product by unique name than numeric ID as long as the column is indexed.
Wordpress has a field in the wp_posts table for the slug. When you create the post, it creates a slug from the post title (if that's how you have it configured), replacing spaces with dashes (or I think you can set it to underscores). It also takes out the apostrophes, commas, or whatnot. I believe it also limits the overall length of the slug, too.
So, in short, it isn't dynamically decoding the URL into the post's title--there's a field in the table that matches the URL version of the post name directly.
As you may or may not know, the URLs are being re-written with Apache's mod_rewrite module. As mentioned here, Wordpress is, in the background, assigning a slug after sanitizing the title or post name.
But, to answer your question, what you're describing is Wordpress' "Pretty Permalinks" feature and you can learn more about it in the Wordpress codex. Newer versions of Wordpress do the re-writing internally (no .htaccess editin, wp_rewrite instead). Which is why you'll see the same ruleset for any permalink structure.
Though, if you do some digging you can find the old rewrite rules. For example:
RewriteRule ^([0-9]{4})/([0-9]{1,2})/([0-9]{1,2})/?$ /index.php?year=$1&monthnum=$2&day=$3 [QSA,L]
Will take a URL like /2008/01/01/ and direct it to /index.php?year=2008&monthnum=01&day=01 (and load a date category).
But, as mentioned, a page like product-name exists only because Wordpress already sanitized the post title and stored it as a field in the database.