I had an old asp.net website (aspx) and have redesigned it with asp.net MVC using SSL. Anyway, after I promoted the new site I saw a ton of errors which were being generated by bots looking for old pages; the errors were looking something like this: The controller for path '/blablabla/moreBlalbalba/page.aspx' was not found or does not implement IController. So, I updated my error handling to return a 301 response and redirect to the home page and added a sitemap. The google 404 console errors went away for around a month, but now have a ton of 404 errors and they are all pointing to the old site structure. As a side note, the new MVC/SSL site has no 404 errors in the webmaster console, all the errors are on the non SSL site. So, what is the best way to update the bots for the new site structure.
Thanks!
I would suggest you to redirect all traffic from HTTP to HTTPS, using this example. This may solve your problem already.
And secondly create a sitemap for Google to check the relevant pages of the website, example here
Related
I have developed an ASP.NET MVC 5 website that uses jQuery and ajax requests to pull and post data. Google's crawlers found my POST action urls in Javascript code and tried to index them.
In Webmaster Tools I see a lot of errors like that /Account/Login with a 500 error response because obviously a name and a password were not provided. How can I solve this problem? I don't want any crawl error, but I don't know how to say to Google not to follow these urls.
Thank you!
Use Google's instructions to create a robots.txt file, which is a request to a search engine to not index.
A sample robots.txt to put in the root of your domain may look like this:
User-agent: Googlebot
Disallow: /path/to/my/post/url
I have a web application that I am trying not to recompile since there is little documentation and the environment is a little sensitive.
With that in mind, all I am trying to do is hijack the authentication mechanism to redirect to one of a couple replacement websites. To that end, there is an authentication service and an authentication controller. The website redirects to /auth/login when the user comes unauthenticated.
In the views folder I have built an alternative /auth/login_new.gsp and from there can authenticate the user and get a redirection back to /auth/redirect.gsp at some frequency but not 100%. That redirect page takes a value from the DB and redirects the user to the correct follow on website. When I run authentication from /auth/login, the site ignores the redirect request to /auth/redirect.gsp.
I had set the show pages for all the different controllers to window.location.href="/auth/redirect.gsp" but I can't get it to go there 100%. I have also reset the layout/domain.gsp file to gut the other functionality of the site and script redirect as well. I was getting errors with duplicate redirect attempts, but now I just go to a dead/gutted homepage...
Any suggestions on how I can dodge the recompile?
Thanks
Leif
I am in the process of launching 2 sites that have been recently redesigned (one in RoR and one in WordPress) they both have a very large amount of inbound links coming in from search engines and outside sources. This has been something I have been curious for quite some time on an efficient way to implement redirects on all links.
My main purpose of this is so the site does not lose the work it has done SEO wise and in addition not leave any old backlinks forwarding to a 404.
What is the best practice when launching a new site for redirecting old URIs?
You'll find that most of your back-links are to your home-page anyway, so that will take care of the bulk of them. In terms of mitigating 404s from broken back-links, try to create a pattern-match (regex) redirect sending a 301 (Moved Permanently) header - using .htaccess (since you're using RoR/WP).
WordPress does have some plugins to handle migrations and redirections - simply search on the wordpress.org site.
Ensure you register your site with Google's Webmaster Tools and monitor your 404 pages (or log them server-side) to catch ones you've missed.
Lastly, to ensure that you get your new URLs indexed and canonicalization (beyond ensuring rel=canonical is used correctly), submit an XML sitemap of all your new pages.
In terms of redirecting old links to new links, it is general practice to do a 301 redirect (for SEO purposes). In the absolute worst case you cannot do this, redirect to the homepage at the very least to not lose visitors to 404 pages.
On the book Agile Web development with Rails, it is proposed that when someone tries to access some data in your web site and the record doesn't exist anymore, that the user should be redirected to a working page and display a message.
A user would go to /book/1, but a book with id 1 doesn't exist anymore, so it is redirected to /books and shown a message "That book doesn't exist". It seems to be a good user experience but to break the HTTP protocol. Should it be a temporary redirect? if so a web crawler will keep hitting that page. Should it be a permanent redirection? If so the previous content should be available there, and it isn't.
I think that a record-not-found page should issue a 404. Am I wrong? Hitting /book/1 where 1 doesn't exist anymore would return a 404 with the HTML showing exactly the same thing as /books, and maybe an error message.
Agile Web development with Rails is against that option because the user might keep hitting /book/1 generating 404s only to see what can be seen in /books.
What do you think?
If the resource does not exist, send the 404 status code. It’s really that simple. Redirecting means that only the URL is (temporarily) not valid but the resource does exist.
If there's no 404 , search engines have no way to discover that the object has been deleted. So I suppose it's a must.
I think there's a good compromise where you render a 404 template (complete with 404 status code) that prompts the user to continue to /books or /whatever.
if the record doesn't exist anymore, than you should probably use a 301 status code, "permanent redirect".
The difference between 301 and 404, is that a 404 error code should be used in cases when the resource never existed and 301 when the resource existed, but moved.
This one is causing me a few nightmares as I'm on the live box trying to work out what is going wrong!
If someone accesses our ASP.Net MVC website with the full URL http://www..net all is OK. If they go to: http://.net then our custom error page is shown. This used to work OK before we moved the site to MVC.
We do have an Application_OnError event in the Global.asax but I know that is not being hit in this situation, as I log to the event log and that is not happening.
If I switch custom errors off in the web.config, the site behaves correctly!
We're using the MVC Beta at the moment. Edit: We're running on IIS6 and using the MVC routing for friendly URLs.
This is impossible to test locally which is fustrating as it only happens on live without the www. I wonder if it is something to do with routing......
Thanks!
The problem is too vague at this stage for me to be able to give you a good answer but I would look firstly at your URL rewriting - what version of IIS are you using? If IIS5 or 6, are you using Isapi Rewrite? This could be interfering with your response.
As for why the error goes away when you turn customErrors off, well, I have no idea sorry.
On a side-note, if you're concerned with Google ranking, you may want to use a rewriting tool (like Isapi Rewrite and I think built-in to IIS7) to send an automatic redirect (HTTP 301 response) that will send users from the non-www version to the www version. Google sees both of these as individual sites with duplicate content and this will dilute your Page-Rank. This will also avoid the problem you're experimenting altogether as users will only ever see the www version.
Also, I'm not sure if Application_Error is really the best way to deal with errors in ASP.Net MVC. Do some research into the HandleError Action Filter as this to see if this might provide you with a better approach to error handling. Check out Scott Gu's post on this for more info.
I hope this helps.
Cheers,
Zac
i was having the same problem in my MVC .net site but it worked out for me when i enter both domain.com and www.domain.com in the host header in IIS.