Preventing rapid-fire login attempts with Rack::Attack - ruby-on-rails

We have been reading the Definitive guide to form based website authentication with the intention of preventing rapid-fire login attempts.
One example of this could be:
1 failed attempt = no delay
2 failed attempts = 2 sec delay
3 failed attempts = 4 sec delay
etc
Other methods appear in the guide, but they all require a storage capable of recording previous failed attempts.
Blocklisting is discussed in one of the posts in this issue (appears under the old name blacklisting that was changed in the documentation to blocklisting) as a possible solution.
As per Rack::Attack specifically, one naive example of implementation could be:
Where the login fails:
StorageMechanism.increment("bad-login/#{req.ip")
In the rack-attack.rb:
Rack::Attack.blacklist('bad-logins') { |req|
StorageMechanism.get("bad-login/#{req.ip}")
}
There are two parts here, returning the response if it is blocklisted and check if a previous failed attempt happened (StorageMechanism).
The first part, returning the response, can be handled automatically by the gem. However, I don't see so clear the second part, at least with the de-facto choice for cache backend for the gem and Rails world, Redis.
As far as I know, the expired keys in Redis are automatically removed. That would make impossible to access the information (even if expired), set a new value for the counter and increment accordingly the timeout for the refractory period.
Is there any way to achieve this with Redis and Rack::Attack?
I was thinking that maybe the 'StorageMechanism' has to remain absolutely agnostic in this case and know nothing about Rack::Attack and its storage choices.

Sorry for the delay in getting back to you; it took me a while to dig out my old code relating to this.
As discussed in the comments above, here is a solution using a blacklist, with a findtime
# config/initilizers/rack-attack.rb
class Rack::Attack
(1..6).each do |level|
blocklist("allow2ban login scrapers - level #{level}") do |req|
Allow2Ban.filter(
req.ip,
maxretry: (20 * level),
findtime: (8**level).seconds,
bantime: (8**level).seconds
) do
req.path == '/users/sign_in' && req.post?
end
end
end
end
You may wish to tweak those numbers as desired for your particular application; the figures above are only what I decided as 'sensible' for my particular application - they do not come from any official standard.
One issue with using the above that when developing/testing (e.g. your rspec test suite) the application, you can easily hit the above limits and inadvertently throttle yourself. This can be avoided by adding the following config to the initializer:
safelist('allow from localhost') do |req|
'127.0.0.1' == req.ip || '::1' == req.ip
end
The most common brute-force login attack is a brute-force password attack where an attacker simply tries a large number of emails and passwords to see if any credentials match.
You should mitigate this in the application by use of an account LOCK after a few failed login attempts. (For example, if using devise then there is a built-in Lockable module that you can make use of.)
However, this account-locking approach opens a new attack vector: An attacker can spam the system with login attempts, using valid emails and incorrect passwords, to continuously re-lock all accounts!
This configuration helps mitigate that attack vector, by exponentially limiting the number of sign-in attempts from a given IP.
I also added the following "catch-all" request throttle:
throttle('req/ip', limit: 300, period: 5.minutes, &:ip)
This is primarily to throttle malicious/poorly configured scrapers; to prevent them from hogging all of the app server's CPU.
Note: If you're serving assets through rack, those requests may be counted by rack-attack and this throttle may be activated too quickly. If so, enable the condition to exclude them from tracking.
I also wrote an integration test to ensure that my Rack::Attack configuration was doing its job. There were a few challenges in making this test work, so I'll let the code+comments speak for itself:
class Rack::AttackTest < ActionDispatch::IntegrationTest
setup do
# Prevent subtle timing issues (==> intermittant test failures)
# when the HTTP requests span across multiple seconds
# by FREEZING TIME(!!) for the duration of the test
travel_to(Time.now)
#removed_safelist = Rack::Attack.safelists.delete('allow from localhost')
# Clear the Rack::Attack cache, to prevent test failure when
# running multiple times in quick succession.
#
# First, un-ban localhost, in case it is already banned after a previous test:
(1..6).each do |level|
Rack::Attack::Allow2Ban.reset('127.0.0.1', findtime: (8**level).seconds)
end
# Then, clear the 300-request rate limiter cache:
Rack::Attack.cache.delete("#{Time.now.to_i / 5.minutes}:req/ip:127.0.0.1")
end
teardown do
travel_back # Un-freeze time
Rack::Attack.safelists['allow from localhost'] = #removed_safelist
end
test 'should block access on 20th successive /users/sign_in attempt' do
19.times do |i|
post user_session_url
assert_response :success, "was not even allowed to TRY to login on attempt number #{i + 1}"
end
# For DOS protection: Don't even let the user TRY to login; they're going way too fast.
# Rack::Attack returns 403 for blocklists by default, but this can be reconfigured:
# https://github.com/kickstarter/rack-attack/blob/master/README.md#responses
post user_session_url
assert_response :forbidden, 'login access should be blocked upon 20 successive attempts'
end
end

Related

Allauth user_logged_in signal does not trigger in a TestCase

I'm using allauth for user management in my Django web app and I have functionality which finds session data stored when the user was not logged in and stores it when they next log in on the same session.
It is triggered on login via the standard allauth signal:
#receiver(user_logged_in)
def update_on_login(request, user, **kwargs):
print('Signal triggered')
# Do some stuff with session data here, e.g.:
for session_key in request.session.items():
...
This works perfectly when logging in via a web browser, but in TestCase functions the signal is not triggered at all when logging in with:
self.client = Client()
logged_in = self.client.login(username=self.username, password=self.password)
(I've also tried force_login and force_authenticate, but info on these indicates that they actually skip all real validation and just assume the user is logged in).
I think I've understood that client.login doesn't work because the test client doesn't really handle the request in the same way as a web browser would, but I can't say I really understand it.
I've also seen some indication that RequestFactory might be able to help in someway, but I haven't been able to trigger the signal with this either:
request = RequestFactory().post(reverse('account_login'), { 'username': self.username, 'password': self.password })
request.user = AnonymousUser()
response = login(request)
I've seen some comments about needing to call middleware as well, but pretty out of date and nothing clear enough for me to understand and try.
Any suggestions to point me in the right direction would be much appreciated.
Thanks in advance.

Accessing response status after it is sent

I'm working on a feature to store requests made to specific endpoints in my app.
after_action :record_user_activity
def record_user_activity
return unless current_user
return if request.url =~ /assets|packs/
#session.navigation.create!(
uri: request.url,
request_method: request.method,
response_status: response.code.to_i,
access_time: Time.now
)
end
The problem is that, even if we get an error response, when getting the response.code at this point (after_action), the response code is still a 2xx. I imagine it's probably because the server hasn't yet faced whatever problem it may face during the data access process.
How can I properly store the status code that was actually sent to the user?
The rails logs already store the requests and responses. If you just need to track all user requests, you can simply add the user id to your logs. Unless there's a specific reason to keep this data in the DB, which would grow exponentially with the amount of user activity, add this to either config/application.rb or config/environments/production.rb
MyAppName::Application.configure do
#...whatever you already have in configs
# then add the following line
config.log_tags = [
-> request { "user-#{request.cookie_jar.signed[:user_id]}" }
]
end
Then you can tail the logs and use grep, or write some other processes to parse over the logs with other analytics. There are many tools available for this type of work. But here's a basic example
tail -f logs/production.log | grep user-101
# this would show all log requests from user with id 101
If you still need this however, you may wanna try prepend_after_action instead, see
http://api.rubyonrails.org/classes/AbstractController/Callbacks/ClassMethods.html#method-i-prepend_after_action

How would I use rspec to test a method who's job is to post to a webhook?

I'm using rspec to test my application and I'm having a hard time figuring out how to test this. The Slack::Notifier's job is to send a post request to a webhook. Once I call this method in Rspec, I don't know how to see the response. Also, is it possible to match the format of this text to an expected text somewhere? My method is below. Thanks.
def notify
offset = 14400 #UTC to EST
notifier = Slack::Notifier.new Rails.application.secrets.slack_organization_name, Rails.application.secrets.slack_token, channel: "##{Rails.application.secrets.slack_channel}", username: Rails.application.secrets.slack_user_name
notifier.ping(":white_check_mark: *USAGE SUMMARY for #{(Time.now - offset).to_formatted_s(:long) }*")
count = 0
current_time = Time.now.to_i
live_response.each do |r|
if r["properties"]["time"] > ((current_time - offset) - 60) #&& r["properties"]["$initial_referring_domain"] == "capture.com"
notifier.ping("
*Name:* #{r["properties"]["$name"]}
*Event:* #{r["event"]}
*Keywords:* #{r["properties"]["keywords"]}
*Organization:* #{r["properties"]["organizationName"]}
*Email:* #{r["properties"]["$email"]}
*Time:* #{Time.at(r["properties"]["time"] + offset).utc.to_datetime.in_time_zone("Eastern Time (US & Canada)").to_formatted_s(:long_ordinal)}
*More Data:* #{ANALYTICS_URL}#{r["properties"]["distinct_id"]}
__________________________________________________
")
count +=1
end
end
notifier.ping("*There were #{count} events in this report.*")
end
Testing network communications (like API calls) is a tricky thing. Personally I would rely on programming by contract and testing in isolation - i.e. assume the external service is working fine and it responds positively for valid request.
Then you test your client code by checking that you are actually sending a valid request. For this stub the method where control exits your code into a library/system code. For example if you are making a HTTP GET request using a gem like HTTParty, then stub HTTParty.get i.e. HTTParty.stub(:get) and in that stub verify that correct parameters were sent.
On the other side of the spectrum you should also simulated both positive and negative responses from the web service and make sure your client code handles it in expected manner.
If you are making a real then you are introducing a lot of dependencies on your test : a test setup of external service, risk of network issues (timeout, n/w breakdown, etc) problems with external service and may be more.
If you yourself are writing that webservice too then test that one also in isolation, i.e by simulating valid and invalid inputs and making sure they are handled properly. This part is pretty much your controller specs or request specs.
Once again, this is my opinion. Suggestions to do this in a better way and constructive criticism on the shortcomings of this approach are definitely welcome.

Queuing API calls to fit rate limit

Using Full Contact API, but they have a rate limit of 300calls/minute. I currently have it to set that it does an API call when uploading the CSV file of emails. I want to queue it such that once it hits the rate limit or does 300 calls, it waits for 1 minute and proceeds. Then I will put delayed_job on it. How can I do that? A quick fix is to use
sleep 60
but how do I find it such that it made 300 calls already, make it sleep or queue it for next set?
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
hashy = row.to_hash
email = hashy["email"]
begin
Contact.create!(email: email, contact_hash: FullContact.person(email: email).to_json)
rescue FullContact::NotFound
Contact.create!(email: email, contact_hash: "Not Found")
end
end
end
There are several issues to think about here - is there going to be a single process using your API key at any one time, or is it possible that multiple processes would be running at once? If you have multiple delayed_job workers, I think the latter is likely. I haven't used delayed_jobs enough to give you a good solution to that, but my feeling is you would be restricted to a single worker.
I am currently working on a similar problem with an API with a restriction of 1 request every 0.5 seconds, with a maximum of 1000 per day. I haven't worked out how I want to track the per-day usage yet, but I've handled the per-second restriction using threads. If you can frame the restriction as "1 request every 0.2 seconds", that might free you up from having to track it on a minute-by-minute basis (though you still have the issue of how to keep track multiple workers).
The basic idea is that I have an request method that splits a single request into a queue of request parameters (based on the maximum number of objects allowed per request by the api), and then another method iterates over that queue and calls a block which sends the actual request to the remote server. Something like this:
def make_multiple_requests(queue, &block)
result = []
queue.each do |request|
timer = Thread.new { sleep REQUEST_INTERVAL }
execution = Thread.new { result << yield(request) }
[timer, execution].each(&:join)
end
result
end
To use it:
make_multiple_requests(queue) do |request|
your_request_method_goes_here(request)
end
The main benefit here is that if a request takes longer than the allowed interval, you don't have to wait around for the sleep to finish, and you can start your next request right away. It just guarantees that the next request won't start until at least the interval has passed. I've noticed that even though the interval is set correctly, I occasionally get an 'over-quota' response from the API. In those cases, the request is retried after the appropriate interval has passed.

Optimal way to structure polling external service (RoR)

I have a Rails application that has a Document with the flag available. The document is uploaded to an external server where it is not immediately available (takes time to propogate). What I'd like to do is poll the availability and update the model when available.
I'm looking for the most performant solution for this process (service does not offer callbacks):
Document is uploaded to app
app uploads to external server
app polls url (http://external.server.com/document.pdf) until available
app updates model Document.available = true
I'm stuck on 3. I'm already using sidekiq in my project. Is that an option, or should I use a completely different approach (cron job).
Documents will be uploaded all the time and so it seems relevant to first poll the database/redis to check for Documents which are not available.
See this answer: Making HTTP HEAD request with timeout in Ruby
Basically you set up a HEAD request for the known url and then asynchronously loop until you get a 200 back (with a 5 second delay between iterations, or whatever).
Do this from your controller after the document is uploaded:
Document.delay.poll_for_finished(#document.id)
And then in your document model:
def self.poll_for_finished(document_id)
document = Document.find(document_id)
# make sure the document exists and should be polled for
return unless document.continue_polling?
if document.remote_document_exists?
document.available = true
else
document.poll_attempts += 1 # assumes you care how many times you've checked, could be ignored.
Document.delay_for(5.seconds).poll_for_finished(document.id)
end
document.save
end
def continue_polling?
# this can be more or less sophisticated
return !document.available || document.poll_attempts < 5
end
def remote_document_exists?
Net::HTTP.start('http://external.server.com') do |http|
http.open_timeout = 2
http.read_timeout = 2
return "200" == http.head(document.path).code
end
end
This is still a blocking operation. Opening the Net::HTTP connection will block if the server you're trying to contact is slow or unresponsive. If you're worried about it use Typhoeus. See this answer for details: What is the preferred way of performing non blocking I/O in Ruby?

Resources