We have a rails app(v5.2.6) running inside a unicorn.
we are bombarded with log msgs like the following:
24.33.33.243, 12.0.52.41 - - [26/Jun/2022:13:43:39 +0000] "GET /our_path HTTP/1.1" 200 - 0.0078
No idea where this log is coming from? its bombarding our log and I must find a way to filter it.
We have a custom Rails.logger, where we also filter some msg that applies to some regexs, the problem is that this msg gets to the log from another place
Related
I have a Flask app set up with a few endpoints. These endpoints work fine when accessed from postman or a browser.
I also have a React Native app set up (using an Expo template), in which I try to query this resource from iOS:
const testApiCall = async (): Promise<string> => {
try {
const url = `${baseUrl}/users`;
const response = await axios.get(url);
console.log(response.data);
return response.data.toString();
} catch (error) {
console.log(error.request)
console.log(" a " + error.request?._response);
console.log(error);
}
}
baseUrl is the full IP, not just localhost.
The /users endpoint returns json of the app's users. However, when calling this, I get the following output:
a Invalid imageTag: 192.168.1.191:80/users
[AxiosError: Network Error]
(excluding the error.request since it's long and probably doesn't provide any additional context - will edit this post if it would be useful!)
I can't find any reason for the "invalid imageTag" error. Intermittently, the flask app will get logs including a 404 request for the favicon:
[06/May/2022 21:55:05] "GET /users HTTP/1.1" 200 -
[06/May/2022 21:55:05] "GET /favicon.ico HTTP/1.1" 404 -
[06/May/2022 21:56:15] "GET /users HTTP/1.1" 200 -
[06/May/2022 21:56:16] "GET /favicon.ico HTTP/1.1" 404 -
I even tried to make an endpoint in the flask app to return a generic .ico file when requesting /favicon.ico but behavior didn't change. Now, I stopped getting any kind of request on my flask app when using my React Native app. I still get logs when accessing the endpoints from postman, though.
I also used flask_cors:
CORS(app, resources={r'/*': {'origins': '*'}})
This one has been a real head-scratcher for me and I'm having trouble finding any similar issues, so I'd really appreciate any help! Happy to provide additional context that could be useful, too.
A request to https://jsonplaceholder.typicode.com/todos/1 worked without an issue. Tried with http:// and that worked too.
Added this to app.json but it didn't help:
"ios": {
"supportsTablet": true,
"infoPlist": {
"NSAppTransportSecurity" : {
"NSAllowsArbitraryLoads" : true,
"NSAllowsLocalNetworking": true
}
}
},
Turns out my logic for getting the baseUrl excluded the http://. Prepending the baseUrl with 'http://' solved the issue. It's always something obvious that I miss!
I'm using Graylog to manage my server logs.
I would like to filter the apache logs access to keep the logs with a http response code 4** and 5**
So I would like to use a regex:
If I search /HTTP/ I have the corresponding logs like:
[...] "HEAD /register HTTP/1.1" 301 460 "-" [...]
But if I search /HTTP\//, I have no message. I also tried with /HTTP\\// but same result.
I tried with the regex /HTTP(?:.*?)"\s[4|5](?:\d{2})/ but no message found.
How to search a simple pattern like HTTP/ with a regex in Graylog ?
Thx
Why don't you use an extractor to map http-status-codes to fields.
Then you can easily filter and group your logs to find those with special codes.
Please see the following links
Extractors
How to use a JSON extractor
I want to implement access log in the ruby layer of my application. I want to log like this for each request.
192.168.2.20 - - [28/Jul/2006:10:27:10 -0300] "GET /cgi-bin/try/ HTTP/1.0" 200 3395
127.0.0.1 - - [28/Jul/2006:10:22:04 -0300] "GET / HTTP/1.0" 200 2216
I want to create a different file for access log only like apache access log. We can log the request in the rails logger that is easily achievable but how can we create separate log file only for access log. I am using trinidad.
Everything you need can be found in the response header and body. Add a global action to your ApplicationController with a before_ or around_ action like:
/application_controller.rb
around_filter :logging_traffic
def logging_traffic
logger.info "USERAGENT: #{request.headers['HTTP_USER_AGENT']}"
begin
yield
ensure
logger.info "response_status: #{response.status}"
end
end
I've been checking my production.log today and there's a number of requests hitting my site that appear to be malicious, but I'm confused as to how they're even getting to us.
For example:
Processing PublicController#unknown_request (for 217.23.4.13 at 2009-11-09 09:15:52) [GET]
Parameters: {"anything"=>["results.aspx"], "action"=>"unknown_request", "first"=>"200", "controller"=>"public", "q"=>"\"bbs/cbbs.cgi?\" intitle:\"Book\" intext:\"2008\" site:.uz ", "count"=>"200", "FORM"=>"PERE"}
Completed in 16ms (View: 12, DB: 0) | 200 OK [http : // search . live .com /results.aspx?q=%22bbs/cbbs.cgi%3F%22%20intitle%3A%22Book%22%20intext%3A%222008%22%20site%3A.uz%20&count=200&first=200&FORM=PERE]
These are happening every 30 seconds or so. Obviously, PublicController/Unknown_request is my controller/action 404 error.
The access log shows these requests as:
217.23.4.13 - - [09/Nov/2009:09:57:25 +1000] "GET http://search.live.com/results.aspx?q=%22en-gb.html%22%20intitle%3A%22Home%22%20intext%3A%222006%22%20site%3A.mn%20&count=200&first=400&FORM=PERE HTTP/1.1" 200 3626 "-" "Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.1$
How are these requests even hitting my site? Does anyone have any ideas?
I think this might be the same problem you're having: http://penguinpetes.com/b2evo/index.php?p=567&more=1&c=1&tb=1&pb=1
Basically, live/bing are doing some sort of testing that involves going to your site looking like someone searched something completely irrelevant to the content you have.
I have a rails app that is working fine except for one thing.
When I request something that doesn't exist (i.e. /not_a_controller_or_file.txt) and rails throws a "No Route matches..." exception, the response is this (blank line intentional):
HTTP/1.1 200 OK
Date: Thu, 02 Oct 2008 10:28:02 GMT
Content-Type: text/html
Content-Length: 122
Vary: Accept-Encoding
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Status: 500 Internal Server Error
Content-Type: text/html
<html><body><h1>500 Internal Server Error</h1></body></html>
I have the ExceptionLogger plugin in /vendor, though that doesn't seem to be the problem. I haven't added any error handling beyond the custom 500.html in public (though the response doesn't contain that HTML) and I have no idea where this bit of html is coming from.
So Something, somewhere is adding that HTTP/1.1 200 status code too early, or the Status: 500 too late. I suspect it's Apache because I get the appropriate HTTP/1.1 500 header (at the top) when I use Webrick.
My production stack is as follows:
Apache 2
Mongrel (5 instances)
RubyOnRails 2.1.1 (happens in both 1.2 and 2.1.1)
I forgot to mention, the error is caused by a "no route matches..." exception
This is a fairly old thread, but for what it's worth I found a great resource that includes a detailed description of the problem and the solution. Apparently this bug affects Rails < 2.3 when used with Mongrel.
The article that helped me understand the problem & write my own patch.
An official Rails bug ticket that includes a patch for Rails 2.2.2.
This html file is coming from Rails. It is encountering some sort of error (probably an exception of some kind, or some other unrecoverable error).
If the extra blank line between the Status: header and the actual headers is there, and not just a typo, then this would go a long way to explaining why Apache is reporting a 200 OK message.
The Status header is how Rails, PHP, or whatever tells Apache "There was an error, please return this code instead of 200 OK." The fact there is a blank line means something extra is going on and Ruby is outputting a blank line before the error output for whatever reason. Maybe it's previous output from your script. The long and short of it is though, the extra blank line means that Apache thinks "Oh, blank line, no extra headers, this is all content now.", which would be consistent with the Content-Length header you provided.
My guess for why there's a blank line would be previous script output, perhaps a line ending at the end of a fully script page. As to why the 500 error is happening, there isn't nearly enough info here to tell you that. Maybe a file I/O error.
Edit: Given the extra information provided by Dave about the internals, I'd say this is actually an issue with the proxying that goes on behind the scenes... I couldn't tell you exactly what though, beyond what's already been said.
This is coming from rails itself.
http://github.com/rails/rails/tree/master/actionpack/lib/action_controller/dispatcher.rb#L60
The dispatcher is return an error page with the status code of 200 (Success).