Best practice for deploying low traffic rails app without cost concerns - ruby-on-rails

Question

My questions
Cost related pit falls to avoid when deploying rails app?
Attacks are welcome as it would teach me what to expect and brace myself against.
I would rather avoid big bills at the end of month, however.
Easy cloud hosting services to use?
I picked AWS because it seems scalable and I thought I can avoid leaning another service later.
I have no regrets but AWS is overwhelming, if there was significantly simpler service, I should have used it.
My current concern
Dos attack or get request flooding on aws S3 could raise hosting cost significantly as I'm uploading some contents there.
Billing alarm is useful, but without automatic shutdown I feel a little uncomfortable taking a break and going into a jungle or an inhabited island where I get no INTERNET connection to be informed of or to shut down my service...
Obvious fix for my case
Stop using S3 and save user uploads to database where I can control scaling behavior. But then, most people seems to be using S3 with carrierwave, why?
What I'm doing
Making my first ever home page using:
elastic beanstalk
rails5
Carrierwave gem configured to save user uploads in S3
Edit
In the end, I could not find any real solution to the no cap for S3 issue.
The below is more or less my note.
I'm guessing S3 has some basic built in defense against attacks because I have not heard of sad stories about people using S3 to host static web sites and getting a bill over 10000 US, which can still happen though regardless of how good amazon's defense might be.
mitigation
A script that periodically checks for s3 log files and calls an action that disables s3 resource serving when the cumulative size of those files is too large.
S3 log sometimes takes more than an hour before they become available, so it's no solution but better than nothing.
class LogObserver
def initialize
aws_conf = Aws.config.update({
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
region: 'ap-northeast-1'})
#bucket_name="bucket name that holds s3 log"
#last_checked_log_timestamp = Time.now.utc
log "started at: #{Time.now}"
end
def run
bucket = Aws::S3::Resource.new.bucket(#bucket_name)
while true
prv_log_ts = #last_checked_log_timestamp
log_size = fetch_log_size(bucket)
log "The total size of S3 log accumulated since last time this script was executed: #{log_size}"
time_range = #last_checked_log_timestamp - prv_log_ts # float
log_size_per_second = log_size/time_range
if log_size_per_second > (500.kilobyte/60)
log "Disabling S3 access as S3 log size is greater than expected."
`curl localhost/static_pages/disable_s3`
end
sleep 60*60
end
end
def log text
puts text
File.open('./s3_observer_log.txt','a') do |f|
f << text
end
end
def fetch_log_size(bucket)
log_size=0
bucket.objects(prefix: 'files').each do |o|
time_object = o.last_modified
if time_object < #last_checked_log_timestamp
next
end
#last_checked_log_timestamp = time_object
log_size += o.size.to_i
end
return log_size
end
end
Rake task:
namespace :s3_log do
desc "Access S3 access log files and check their cumulative size. If the size is above the expected value, disables s3 access."
task :start_attack_detection_loop do
require './s3_observer.rb'
id=Process.fork do
o=LogObserver.new
o.run
end
puts "Forked a new process that watches s3 log. Process id: #{id}"
end
end
controller action:
before_action :ensure_permited_ip, only: [:enable_s3, :disable_s3]
def enable_s3
# allow disabling s3 only from localhost
CarrierWave.configure do |config|
config.fog_authenticated_url_expiration = 3
end
end
def disable_s3
# allow disabling s3 only from localhost
CarrierWave.configure do |config|
config.fog_authenticated_url_expiration = 0
end
end
private
def ensure_permited_ip
if request.remote_ip!= "127.0.0.1" # allow access only from localhost
redirect_to root_path
end
end
Gems:
gem 'aws-sdk-rails'
gem 'aws-sdk-s3', '~> 1'

Answer 1

My experiences are limited but my suggestions would be:
Cost related pit falls to avoid when deploying rails app?
if you're gonna be using a background-job, use rufus-scheduler instead of sidekiq or delayed_job, because it runs on top of your rails server and would not require additional memory / additional dedicated processes. This allows you to procure the smallest/cheapest possible instance: t2.nano, which I did once before.
Easy cloud hosting services to use?
Heroku would be a good choice, because it is a lot easy to set it up. However if you're doing this for the experience, I would suggest to procure unmanaged hosting like AWS EC2 or Linode. I recently migrated my server from AWS to Vpsdime 3 months ago because it's cheap and has big memory; so far so good.
My current concern
For carrierwave, you may restrict S3 access. See reference. This then prevents hotlinking and would then require a user to view your Rails pages first in order to download or view or show the S3 files. Now that Rails now have control over the S3 files, you can just simply use something like Rack::Attack to prevent DDOS or excessive requests. If your Rails app is configured with Apache or Nginx, you can instead set up DDOS rules there instead of using Rack::Attack. Or, if you are gonna be using AWS load balancer to manage / route the requests, then you can use AWS Shield ... haven't really used this yet though.