We are hosted on Heroku, and have the NewRelic add on. Every day I check the errors, and almost every day this error comes up.
Action and Type
Middleware/Rack/Rack::MethodOverride#call
EOFError
Message
bad content body
This is a Rails Application, and so I figure it's not doing anything in particular other than returning a 440 response status because there is nothing at the url they are trying to access.
URL
/wp-admin/admin-ajax.php
Through some google-fu I found an article pertaining to this being a brute force attack on wordpress sites.
My specific question is:
Do I worry about this?
I inherited the site and am not sure if this is just something that happens, and if it is something that rails applications don't have to worry about? It seems fairly targeted towards wordpress, but I can't find any documentation on whether I should be doing more to stop this.
Other frequently pinged urls that don't exist on my application
/sites/all/libraries/elfinder/php/connector.minimal.php
/license.php
/tiny_mce/plugins/tinybrowser/upload_file.php
Any enlightenment on the subject would be great. Stack trace available if needed. Thanks in advance, overflowers.
As long as you don't have a route configured to handle those requests you then only have to worry about getting spammed these requests and losing network resources. They'll recieve a 404 Not Found error when they try to reach it and so there is nothing they can really do except slow your site if they spam requests. If they do it often you can ban their IP address.
Related
I have a Rails app running on Heroku that serves as the API for a front-end application.
I noticed that for a specific, dynamic URL, /bands/:band_id/members it consistently throws net::ERR_CONNECTION_CLOSED errors which breaks the app.
That specific URL doesn't throw an error when I run the Rails app locally and other URLs work fine on Heroku so I suspect this is a Heroku error but I'm not sure.
I couldn't get deeper in analyzing the problem as the request doesn't even appear in the Heroku logs.
Setup error monitoring on heorku. There are many addons listed under "Errors and Exceptions" category here - https://elements.heroku.com/addons
Eg. You can try Airbrake or Bugsnag. Most likely error is coming from you application. It's best practise to setup error monitoring but even before that you can check your server logs to debugs the issue - https://devcenter.heroku.com/articles/heroku-cli-commands#heroku-logs
Without more details I'm afraid I can only try to help you troubleshoot. Post as much code as you can. The route, the controller action, the view it's rendering, and any relevant logs from localhost and heroku would be a great start.
I've had Heroku requests timeout on my rails apps many times - in development there is often no time limit but if your request is taking too long that definitely could be the issue. How long does the request take on development? It could be as simple as shaving off a few seconds.
Otherwise I would say to check this out:
Heroku websocket connection
Also be sure to clear everything you can on your browser, try other browsers, incognito mode, all of that. Try to isolate the problem to one area - even though Heroku is throwing the error it is almost certainly not causing the error.
Check your routes. Look at everything that is happening with that request in your dev and prod logs and try and find something different about this request. Compare it to others.
It is also a good idea to understand your logs and increase their verbosity -
https://devcenter.heroku.com/articles/logging
What levels of logging are available for Heroku?
Good luck!
I am looking to implement First Click Free in my rails application. Google has this information on how to verify a if a googlebot is viewing your site here.
I have been searching to see if there is anything existing for Rails to do this but I have been unable to find anything. So firstly, does anyone know of anything? If not, could anyone point me in the right direction of how to go about implementing what they have suggested in that page about how to verify?
Also, in that solution, it has to do a lookup every time to try and detect google, that seems like its going to be a big performance hit if I have to do it every page load? I could cache the IP if it has been verified in the past but Google have stated that their IP's change so at some point it may no longer belong to them. Although it probably doesn't happen regularly so it may not be that big of an issue.
Many thanks!!
Check out the browser gem: https://github.com/fnando/browser
What I'd do is use the
browser.bot?
method to check if your site is being accessed by a bot or not. If you care about the Googlebot specifically, you could check if
browser.name
includes googlebot. Keep in mind that this gem just checks the user agent sent by the client's browser, which could of course be spoofed. Sounds like that isn't a huge concern for your purposes.
I've built a Ruby gem for that recently, it's called "legitbot".
You may learn if a Web request comes from a supported bot using
bot = Legitbot.bot(userAgent, ip)
"legitbot" does this looking into User-agent and searching for a bot signature, i.e. how bots identify themselves. This doesn't guarantee that the Web request IP really comes from e.g. Googlebot. To make sure it is, call
bot.detected_as # => "Google"
bot.valid? # => true
bot.fake? # => false
Supported bots are Googlebot, Yandex bots, Bing, Baidu, DuckDuckGo.
All the errors that occur in our web application is logged to a database, and I'm finding a 404 error occurring hundreds of times in the last month. The page users are attempting to access is "https://companysite.com/applicationsite/:/0"
The application is a classic ASP site with some ASP.NET MVC 3 included through i-frames, although this error appears to be occurring on the classical ASP side judging by the URL.
I've done a search through the entire code (classic and .NET) for the string ":/0" but I'm not seeing anything. I'm at a loss at how this error is occurring. It is happening too often and for too many users to be intentional.
Would anyone happen to know why users are getting this error? Unfortunately I only have the database logs so I'm not really user how to reproduce this error, nor do I know how users are coming across it.
I would suspect that someone (outside of your site) is hitting that URL, which does not exist.
It could simply be that a spider has that URL indexed and is trying to crawl it. Or maybe that is a path to some application that has a vulnerability and someone is testing to see if you are running that application.
Try logging the IP address of where the request is coming from and also the User-Agent. If it is a web crawler, you should be able to see it in the User-Agent.
You could also block the IP address from accessing your site.
I have a little problem with google bot, I have a server working on windows server 2009, the system called Workcube and it works on coldfusion, there is an error reporter built-in, thus i recieve every message of error, especially it concerned with google bot, that trying to go to a false link, which doesn't exist! the links looks like this:
http://www.bilgiteknolojileri.net/index.cfm?fuseaction=objects2.view_product_list&product_catid=282&HIERARCHY=215.005&brand_id=hoyrrolmwdgldah
http://www.bilgiteknolojileri.net/index.cfm?fuseaction=objects2.view_product_list&product_catid=145&HIERARCHY=200.003&brand_id=hoyrrolmwdgldah
http://www.bilgiteknolojileri.net/index.cfm?fuseaction=objects2.view_product_list&product_catid=123&HIERARCHY=110.006&brand_id=xxblpflyevlitojg
http://www.bilgiteknolojileri.net/index.cfm?fuseaction=objects2.view_product_list&product_catid=1&HIERARCHY=100&brand_id=xxblpflyevlitojg
of course with definition like brand_id=hoyrrolmwdgldah or brand_id=xxblpflyevlitojg is false, i don't have any idea what can be the problem?! need advice! thank you all for help! ;)
You might want to verify your site with Google Webmaster Tools which will provide URLs that it finds that error out.
Your logs are also valid, but you need to verify that it really is Googlebot hitting your site and not someone spoofing their User Agent.
Here are instructions to do just that: http://googlewebmastercentral.blogspot.com/2006/09/how-to-verify-googlebot.html
Essentially you need to do a reverse DNS lookup and then a forward DNS lookup after you receive the host from the reverse lookup.
Once you've verified it's the real Googlebot you can start troubleshooting. You see Googlebot won't request URLs that it hasn't naturally seen before, meaning Googlebot shouldn't be making direct object reference requests. I suspect it's a rogue bot with a User Agent of Googlebot, but if it's not you might want to look through your site to see if you're accidentally linking to those pages.
Unfortunately you posted the full URLs, so even if you clean up your site, Googelbot will see the links from Stack Overflow and continue to crawl them because it'll be in their crawl queue.
I'd suggest 301 redirecting these URLs to someplace that make sense to your users. Otherwise I would 404 or 410 these pages so Google know to remove these pages from their index.
In addition, if these are pages you don't want indexed, I would suggest adding the path to your robots.txt file so Googlebot can't continue to request more of these pages.
Unfortunately there's no real good way of telling Googlebot to never ever crawl these URLs again. You can always go into Google Webmaster Tools and request the URLs to be removed from their index which may stop Googlebot from crawling them again, but that doesn't guarantee it.
I am completely new to ruby and I inherited a ruby system for a product catalogue. Most of my users are able to view everything as they should but overseas users (specifically Mexico) cannot contact the server once logged in. They are an active user. I'm sorry I cannot be more specific, and the system is private so I cannot grant access.
Has anyone had any issues similar to this before? Is it a user-end issue or a system error?
Speaking as somebody who regularly ends up on your user's side of the fence, the number one culprit for this symptom is "Clueless administrator". There are many, many sites which generically block either large blocks of IP space or which geolocate and carve out big portions of the world.
For example, a surprising number of American blogs block Asian countries (including Japan) out of a misplaced effort to avoid DDOS attacks (which actually probably originated in Russia or China but, hey, this species of administrator isn't very good on fine tuning solutions). I have to hop over to my American proxy server to access those sites.
So the first thing I'd do to diagnose your problems is to see whether your Mexican users are making it to the server at all, or whether they're being blocked somewhere earlier (router? firewall? etc). Then, to determine whether the problem is on your end or their end, I'd try to replicate the issue with you proxying your connection through a Mexican proxy and repeating the actions they took to cause the issue.
The fact that they get blocked after logging in could indicate that you have https issues , for example with an HTTPS accelerator installed [1], or it could be that your frontend server is properly serving up the static content but doing the checking on dynamic requests only.
[1] We've seen some really weird bugs at work caused by a malfunctioning HTTPS accelerator.
If it's working for everyone else then it would appear that the problem is not with Ruby or Rails working, since they are...
My first thought would be to check for a network issue: are the Mexican users all behind the same proxy server and/or firewall?
Is login handled within the Rails application or via some other resource? Can you see any evidence that requests from Mexican users are reaching your web server at all?
Login is handled by the rails app. Am currently trying to hunt down the logs, taking some time as again I am new to this system.
Cheers guys
Maybe INS is cracking down on cyber-immigration.