While trying to understand how to create a better URL for ease of user and not compromising on SEO, I was looking at the structure for different sites. I have a few queries related to the below ones:
Amazon -> /typical-product-slug/dp/{id}
stackoverflow -> /questions/{id}/question-slug
yardsellr -> for_sale#!/product-slug--{id}
[1] Why does amazon place the slug right after the domain name when doing it like stackoverflow's makes it more easy to understand and also provides a directory structure?
Also, keeping the {id} at the end makes it more prone to cut-n-paste errors.
[2] Is the difference in amazon and stackoverflow's positioning of the {id} has something to do with the {id} being alphanumeric for amazon and only digits for stackoverflow? Maybe alphanumeric is more cryptic to users and hence amazon putting it on the right most position, from SEO perspective?
[3] For amazon, isn't using the keyword 'products' or even the high level category like 'books' or 'music' in place of 'dp' makes more SEO effective?
[4] yardsellr for it's trailing {id} doesn't use a '/' but '--' as the marker. Google says depth of URL dir structure doesn't really matter much. What is yardsellr benefiting from this structure, is it a dash just because the slug and the id both unique and same? Is there something to do with the depth? If they had placed the id before the slug, does that impact the SEO?
I know there are no straight forward answers to most of these. Looking for your experience and thoughts so that I can learn.
Thanks.
SEO. Have you tried the URLs without slug?! SO can put slugs at the end, because titles can change frequently and then the URL changes too. So its better to have permalinks to SO only with the question/ID. Same with Amazon but in the other direction, they have the bigger competition and normally titles for a product don't change that often and maybe have shorter lifetime, but can also become permalinked without slugs.
ID is ID. Regardless. Cryptic like YT have a bigger range with less positions than auto-increasing numbers which look better. There are a lot of pros and cons.
Nobody searches for 'product', 'book' or 'cd'. Only titles and authors are important.
Always keep slugs in front for SEO, like in 1. Depth is always important.
Related
For an upcoming project we need to have unique real world identifiers that are exposed to users for things like Account Numbers or Case Numbers (like a bug tracking ID). These will always be system generated and unchangeable. Right now we plan to run strictly on Heroku.
While (as my name would suggest) I am new to the wonderfulness that is Ruby on Rails, I have a long background in enterprise application development. I'm trying to bridge between what I have done in the past while doing in the "RoR way"
Obviously RoR has wonderful primary key support. I have read dozens of posts here recommending to adapt business requirements to just use the out of the box id/key methodology.
So let me describe what I am trying to accomplish and please let me know if you have faced similar objectives and what approach you took.
1) Would like to have a human readable key with a consistent length. There is value in always having an Account ID or Transaction ID that is the same length (for form validation, training sales staff, etc.) Using Ruby's innate key generation one could just add buffer characters (e.g. 100000 instead of 1).
2) Compactness: My initial plan was to go with a base 36 unique key (e.g. 36 values [0..9],[a..z]). As part of our API/interface we plan on exposing certain non-confidential objects based on a shortform URL (e.g. xx.co/000001). I like the idea of being able to have a five character identifier in base 36 vs. 7+ in decimal.
So I can think of two possible approaches:
a) add my own field and develop my own unique key generator (or maybe someone will point me to one).
b) Pad leading digits (and I assume I can force the unique key generation to start at 1xxxxxxx rather than 0000001). Then use the to_s(36) method to convert it to and from base 36 for all interactions with humans. Maybe even store the actual ID value in the database in the base 36 format to avoid ongoing conversions, but always do the conversion before a query to avoid the need to have another index.
I'm leaning towards approach B, as it seems like it would be optimal from a DB performance standpoint and that it would require the least investment in non-value added overhead. Once again, any real world experience with these topics and thoughts on the best approach would be greatly appreciated.
Thanks in advance!
I would never use the primary key in a Rails table for anything of business importance. There will come a day when someone on the business end will want to change it, and it'll end up being an enormous pain in the butt and will invalidate a bunch of URLs you and your users thought were canonical and will mess up all your foreign keys and blah blah blah. It's just a really bad idea and I would encourage you not to do it.
The Rails way to do this is have a new column, called something like number or bug_tracking_number or whatever strikes your fancy, and before_validation implement a callback that gives it a value. This is where you can let your creativity shine; something like this sounds like what you want:
before_validation( :on => :create ) do
self.number = CaseNumber.count + 1
end
You can pad the number there, ensure its uniqueness, or do whatever else you want.
We all love REST, especially when it comes to the development of APIs. Doing so for the last years I always stumble upon the same problem: nested resources. It seems we're living at the two edges of a scale. Let me introduce an example.
/galaxies/8/solarsystems/5/planets/1/continents/4/countries.json
Neato. Cases like that seem to happen everywhere, no matter in what shape they materialize. Now I'd like to being able to fetch all the countries in a solar system while being able to fetch countries deeply scoped as shown above.
It seems I have two choices here. The first one, I flatten my nested structure and introduce a lot of GET parameters (that need to be well documented and understood by my API user) like so:
/countries.json?galaxy=8&solarsystem=5&planet=1&continent=4
I could flatten all my resources like so and won a unique endpoint base URL for each one. Good point … unique endpoints per resource!
But what's the price? Something that does not feel natural, is not discoverable and does not behave like the tree structure below my resources. Conclusion: Bad idea, but well practiced.
On the other hand I could try to get rid of as many additional GET parameters as possible, creating endpoints like that:
/galaxies/8/solarsystems/5/countries.json
But I also needed:
/galaxies/8/solarsystems/5/planets/1/continents/4/countries.json
This seems to be the other side of the scale. Least number of additional GET parameters, more natural behave but still not what I expected as an API user.
The most APIs I worked with in the last year follow the one or the other paradigm. It seems there is at least one bullet to bite. So why not doing the following:
If there are resources that nest naturally, lets nest them exactly in the way we'd expect them to be nested. What we achive is at first a unique endpoint for every resource when we stay like that:
/galaxies.json
/galaxies/8/solarsystems.json
/galaxies/8/solarsystems/5/planets.json
/galaxies/8/solarsystems/5/planets/1/continents.json
/galaxies/8/solarsystems/5/planets/1/continents/4/countries.json
Ok, but how to solve the initial problem, I wanted to fetch all the countries in a solar system while still being able to fetch countries fully scoped under galaxies, solar systems, planets and continents? Here's what feels natural for me:
/galaxies/8/solarsystems/5/planets/0/continents/0/countries.json # give me all countries in the solarsystem 5
/galaxies/8/solarsystems/0/planets/0/continents/0/countries.json # give me all countries in the galaxy 8
… and so on, and so on. Now you may argue "ok, but the zero there ….." and you are right. Does not look really nice. So why not change the two upper calls to something like that:
/galaxies/8/solarsystems/5/planets/all/continents/all/countries.json # give me all countries in the solarsystem 5
/galaxies/8/solarsystems/all/planets/all/continents/all/countries.json # give me all countries in the galaxy 8
Neat eh? So what do we achive? No additional GET parameters and still stable base URLs for each resources endpoint. What's the price? Yep, at least longer URLs especially during testing by hand using tools like curl.
I wonder wether this could be a way to improve not only the maintainability but also the ease of use of APIs. If so, why does not anyone take an approach like that. I can not imagine to be the first one having that idea. So there must be valid counter arguments against an approach like that. I don't see any. Do you?
I would really like to hear your opinion and arguments for or against an approach like that. Maybe there are ideas for improvement … would be great to hear from you. In my opinion this could lead to much better structured APIs, so hopefully someone will read that and reply.
Regards.
Jan
It would all depend on upon how the data is presented. Would the user really need to the know the galaxy # to find a specific country? If so them what you propose makes sense. However, it seems to me that what you are proposing, while structured and presented well, doesn't allow for clients to search for child element unless the parent is a known quantity.
In your example, if I had a specific id for a continent I would need to know the planet, solar system and galaxy as well. In order to find the specific continent I would need to get all for each possible parent until I found the continent.
Presenting structured data in this manner if fine. Using this structure when you only have a piece of the data may be a bit cumbersome. It all depends upon what you are trying to accomplish.
Nested resource URLs are usually bad. The approach I generally take is to use unique IDs.
Design your DB so that it is only going to have one continent with ID 4. Then, instead of the horrible /galaxies/8/solarsystems/5/planets/1/continents/4/countries.json, all you need is the simple /continents/4/countries.json. Clear, sufficient, and memorable.
The :shallow routing option in Rails does this automatically.
For "all countries in a solar system", I'd use /solar_systems/5/countries.json -- that is, don't try to shoehorn it into the generic URL scheme. (And note the underscore.)
I have two RESTful API design questions. Let's say I have a fruitstand web app. I want to return information about all the fruits I carry and the counts those fruits, and I do it with:
http://myfruitstand.com/fruits
Question 1:
If I have 10 oranges, then I'm thinking that I can get information about a particular orange with:
http://myfruitstand.com/fruits/oranges/3
But is the above url RESTful--don't I need an id between 'fruits' and 'oranges' to conform to REST standards? Or is this url with 'fruits' being immediately followed with the subclass 'oranges' okay?
Question 2:
Similarly, if I want to have a discussion forum about my oranges (not a particular orange), can I put it here:
http://myfruitstand.com/fruits/oranges/comments
Again, is the above url RESTful since there's no id between 'oranges' and 'comments' (an id here, of course, would imply a discussion about a particular orange and I don't want that)? Here, there's no subclass rational of 'oranges' being immediately followed by 'comments.'
thanks in advance, Chuck
It would be more RESTful to construct by resource-collection/id pairs, e.g. /fruits/:id1/items/:id2, where id1 = orange, id2 = 3
/fruits/orange/items/3
Then the answer to the second question would be also correct if you lose the 's' on oranges, as you are treating orange as an instance of a resource and not a resource collection.
/fruits/orange/comments
e.g. slide 48 of http://www.slideshare.net/Wombert/phpnw10-designing-http-services-and-restful-interfaces
Just been asking questions about RESTful style myself. But my take on this is:
If it's a fruit store, isn't 'fruit' implied? How about simply:
/oranges/{id}
Alternatively:
/fruits/{id}
Which is just a fruit of type Orange.
/fruits?type=orange could return you a list of oranges
Yes all urls are restful as long as they are deterministic. Restful is more about the http-methods you are using (DELETE for instance to delete, POST to edit, GET to read, ...).
I don't believe from your explanation of usage that you need an id for the fruit resource.
Similarly no id needed for the second one either.
As long as your urls model your object organization then I believe you're good to go.
Also, my personal feeling is that if you find a case where you need to depart ever so slightly from any specification to meet your application needs, that's a reasonable thing to do. Do not allow adherence to this sort of 'model' interfere with getting the job done. For example in the RESTful context, sometimes you perform actions related to some objects that just don't fall clearly under the DELETE/PUT/POST/GET verbs. Do you smoosh those things into there in order to maintain perfect RESTfulness?
I am now hiding behind my desk so as not to be knocked unconscious by the RESTful gurus/religionists that will inevitably beat up on me now :)
This is more of an open question. What is your opinion on query strings in a URL? While creating sites in ASP.NET MVC you spend a lot of time thinking about and crafting clean URLs only for them to be shattered the first time you have to use query strings, especially on a search form.
For example I recently did a fairly simple search form with half a dozen text field and two or three lists of checkboxes and selects. This produced the query string below when submitted
countrylcid=2057&State=England&StateId=46&Where=&DateFrom=&DateTo=&Tags=&Keywords=&Types
=1&Types=0&Types=2&Types=3&Types=4&Types=5&Costs=0.0-9.99&Costs=10.00-29.99&Costs=30.00-59.99&Costs=60.00-10000.00
Beautiful I think you'll agree. Half the fields had no information in them and the list inputs are very verbose indeed.
A while ago I implemented a simple solution to this for paging which produced a url such as
www.yourdomain.com/browse/filter-on/page-1/perpage-50/
This used a catchall route to grab what is essentially a replacement query string after the filter-on portion. Works quite well but breaks down when doing form submissions.
Id be keen to hear what other solutions people have come up with? There are lots of articles on clean urls but are aimed at asp.net developers creating basic restful urls which MVC has covered. I am half considering diving into model binding to produce a proper solution along those lines. With the above convention the large query string could be rewritten as:
filter-on/countrylcid-2057/state-England/stateId-46/types-{1,0,2,3,4,5}/costs-{0.0-9.99,10.00-29.99,30.00-59.99,60.00-10000.00}/
Is this worth the effort?
Thanks,
My personal view is that where users are likely to want to either bookmark or pass on URLs to other people then a nice, clean "friendly" URL is the way to go. Aesthetically they are much nicer. For simple pagination and ordering then a re-written URL is a good idea.
However, for pages that have a large number of temporary, dynamic fields (such as a search) then I think the humble query string is fine. Like wise for pages whose contents are likely to change significantly given the exact same URL in the future. In these cases, URLs with query strings are fine and, perhaps, even preferable as they at least indicate to the observant user that the page is dynamic. However, in these cases it may be better to use form POST variables, anyway, that way visitors are not tempted to "fiddle" with the values.
In addition to what others have said, a URL implies a hierarchy that is semantic. Whether true today or not, the ancestry is directories and people still think of it as such. That's why you have controller/action/id. Likewise, to me a querystring implies options or queries.
Personally, I think a rewritten URL is best when you can't tell if there's an interpreter behind it -- maybe it's just a generated HTML file?
So however you choose to do it (and it's a pain on the client in a search form -- I'd say more trouble than it's worth), I'd support you doing it for hierarchies.
E.g. /Search/Country/State/City
but once you start getting into prices and types, or otherwise having to preface a "directory" with the type of value (e.g. /prices=50.00/ or worse, with an array), then that's where you've lost me.
In fact, if all elements are filled in, then all you've really done is taken the querystring, replaced "&" with "/", and combined your arrays into a single field.
If you're going to be writing the javascript anyways, why don't you just loop through the form elements and:
Remove the empty ones, cleaning up the querystring from the "&price_low=&price_high=&" sorts of things.
Combine multiple values into an array structure
But then submit as a querystring.
James
Aren't the values of the different fields available in the FormsCollection anyway on post?
Imagine you had a group of product categories organized in a nice tree hierarchy and you wanted to provide hackable urls to browse these. You could do something like this
/catalog/categorya/categoryb/categoryc
You could then quite easily figure out which category you should list the products for (note that the full URL is needed since you could have categories with the same name but at different locations in the hierarchy)
Now what would be a good approach to add product information in that as well? To give you an example, you wanted to display the product Oblivion for this category
/catalog/games/consoles/playstation/adventure
It's tempting to just add the product at the end of the url
/catalog/games/consoles/playstation/adventure/oblivion
but the moment you do so you loose the ability to know if its category or a product which is called oblivion. I personally feel that not being forced to add a suffix such as .html
/catalog/games/consoles/playstation/adventure/oblivion.html
would be the nicest solution and using some sort of prefix, such as
/catalog/games/consoles/playstation/adventure/product:oblivion
You could also add some sort of trigger like
/catalog/games/consoles/playstation/adventure/PRODUCT/oblivion
not as nice either and you would (even though its very unlikely it would be a problem) restrict yourself from having a category called product
So far a suffix solution looks like the most user-friendly approach that I can think of from the top of my head but I'm not fond of having to use an extension
What are your thoughts on this?
Deep paths irk me. They're hideous to share.
/product/1234/oblivion --> direct page
/product/oblivion --> /product/1234/oblivion if oblivion is a unique product,
--> ~ Diambiguation page if oblivion is not a unqiue product.
/product/1234/notoblivion -> /product/1234/oblivion
/categories/79/adventure --> playstation adventure games
/categories/75/games --> console games page
/categories/76/games --> playstation games page
/categories/games --> Disambiguation Page.
Otherwise, the long urls, while seeming hackable, require you to get all node elements right to hack it.
Take php.net
php.net/str_replace --> goes to
http://nz2.php.net/manual/en/function.str-replace.php
And this model is so hackable people use it all the time blindly.
Note: The .html suffix is regarded by the W3C as functionally meaningless and redundant, and should be avoided in URLs.
http://www.w3.org/Provider/Style/URI
Lets disect your URL in order to be more DRY (non-repetitive). Here is what you are starting with:
/catalog/games/consoles/playstation/adventure/oblivion
Really, the category adventure is redundant as the game can belong to multiple genres.
/catalog/games/consoles/playstation/oblivion
The next thing that strikes me is that consoles is also not needed. It probably isn't a good idea to differentiate between PC's and Console machines as a subsection. They are all types of machines and by doing this you are just adding another level of complexity.
/catalog/games/playstation/oblivion
Now you are at the point of making some decisions about your site. I would recommend removing the playstation category on your page, as a game can exist across multiple platforms and also the games category. Your url should look like:
/catalog/oblivion
So how do you get a list of all the action games for the Playstation?
/catalog/tags/playstation+adventure
or perhaps
/catalog/tags/adventure/playstation
The order doesn't really matter. You have to also make sure that tags is a reserved name for a product.
Lastly, I am assuming that you cannot remove the root /catalog due to conflicts. However, if your site is tiny and doesn't have many other sections then reduce everything to the root level:
/oblivion
/tags/playstation/adventure
Oh and if oblivion isn't a unique product just construct a slug which includes it's ID:
/1234-oblivion
Those all look fine (except for the one with the colon).
The key is what to do when they guess wrong -- don't send them to a 404 -- instead, take the words you don't know and send them to your search page results for that word -- even better if you can spell check there.
If you see the different pieces as targets then the product itself is just another target.
All targets should be accessable by target.html or only target.
catalog/games/consoles/playstation.html
catalog/games/consoles/playstation
catalog/games/consoles/playstation/adventure.html
catalog/games/consoles/playstation/adventure
catalog/games/consoles/playstation/adventure/oblivion.html
catalog/games/consoles/playstation/adventure/oblivion
And so on to make it consistent.
My 5 cents...
One problem is that your user's notion of a "group of product categories organized in a nice tree hierarchy" may match yours.
Here's a google tech talk by David Weinberger's "Everything is Miscellaneous" with some interesting ideas on categorizing stuff:
http://www.youtube.com/watch?v=x3wOhXsjPYM
#Lou Franco yeah either method needs a sturdy fallback mechanism and sending it to some sort of suggestion page or seach engine would be good candidates
#Stefan the problem with treating both as targets are how to distinguish them (like I described). At worst case scenario is that you first hit your database to see if there is a category which satisfies the path and if it doesn't then you check if there is a product which does. The problem is that for each product path you will end up making a useless call to the database to make sure its not a category.
#some yeah a delimiter could be a possible solution but then a .html suffix is more userfriendly and commonly known of.
i like /videogames/consolename/genre/title" and use the amount of /'s to distinguish between category or product. The only thing i would be worried about multi (or hard to distinguish) genre. I highly recommend no extension on title. You could also do something like videogames(.php)?c=x360;t=oblivion; and just guess the missing information however i like the / method as it looks more neat. Why are you adding genre? it may be easier to use the first letter of the title or just to do videogame/console/title/
My humble experience, although not related to selling games, tells me:
editors often don't use the best names for these "slugs", they don't chose them wisely.
many items belong (logically) to several categories, so why restrict them (technically) to a single category?
Better design item urls by ids, (i.e. /item/435/ )
ids are stable (generated by the db, not editable by the editor), so the url stands a much bigger chance at not being changed over time
they don't expose (or depend on) the organization of the objects in the database like the category/item_name style of urls does. What if you change the underlying design (object structure) to allow an item to belong to multiple categories? the category/item urls suddenly won't make sense anymore; you'll change your url design and old urls might not work anymore.
Labels are better than categories. That is to say, allowing an item to belong to several categories is a better approach than assigning one category to each item.
the problem with treating both as
targets are how to distinguish them
(like I described). At worst case
scenario is that you first hit your
database to see if there is a category
which satisfies the path and if it
doesn't then you check if there is a
product which does. The problem is
that for each product path you will
end up making a useless call to the
database to make sure its not a
category.
So what? There's no real need to make a hard distinction between products and categories, least of all in the URI, except maybe a performance concern over an extra database call. If that's really such a big deal to you, consider these two suggestions:
Most page views will presumably be on products, not categories. So doing the check for a product first will minimize the frequency with which you need to double up on the database lookups.
Add code to your app to display the time taken to generate each page, then go out to the nearest internet cafe (not your internal LAN!) with a stopwatch. Bring up some pages from your site and time how long each takes to come up. Subtract the time taken to generate the page. Also compare the time taken to generate one-database-lookup pages vs. two-database-lookup pages. Then ask yourself, when it takes maybe 1-2 seconds total to establish a network connection, generate the content, and download the content, does it really matter whether you're spending an extra 0.05 second or less for an additional database lookup or not?
Optimize where it matters, like making URLs that will be human-friendly (as in Chris Lloyd's answer). Don't waste your time trying to shave off the last possible fraction of a percent.