I'm having troubles with capistrano and sidekiq monit.
I setup a user for capistrano and everything was going smoothly until I installed Sidekiq.
My problem is when I try to execute cap staging sidekiq:monit:config (sidekiq:monit:start has the same permission problem).
Everytime I've tried, it "freezes" because it asks for the password.
Then I tryed to set sidekiq_monit_use_sudo to false. It's ok, it doesn't use sudo, but then it doesn't have permission to copy the /tmp/monit.conf into /etc/monit/conf.d/ folder.
It's the first time I'm setting up a server and I'm kinda lost here =|
Maybe try to config the sidekiq monit manually?
I'm using ruby 2.5 and these gems:
capistrano 3.10
capistrano-sidekiq 1.0
rails 5.1
Also I have the :pty config set to true as I don't feel comfortable not using a password.
Thank you!
You have a couple of options, of which I'll define two, the right one, and the easy/bad one.
Local User Monit
My personal usage of Monit is on a shared server on which I do not have root access. So I run Monit itself as a non-root user.
In order to do this, I compiled Monit with its prefix as $HOME/apps, so that the config files are in $HOME/apps/etc. This avoids the sudo issue. If you have access to the package manager and installed Monit that way, you can run monit as your user with the -c param to define where it should look for configuration files:
monit -c $HOME/config/monitrc
In order to get Capistrano to recognize the local monit, you will need some extra parameters in config/deploy.rb:
#set :monit_bin, '/usr/bin/monit' # Use this if you compile monit yourself.
set :sidekiq_monit_conf_dir, '/home/myuser/config/monit.d' # Feel free to customize.
set :sidekiq_monit_use_sudo, false
In the monitrc file you have defined with the -c option, you will need to make sure whatever folder you define in :sidekiq_monit_conf_dir is pulled in via includes:
include /home/myuser/config/monit.d/*.conf
Since I don't have an init system, I have Cron start Monit every 30 minutes, which is a noop if it is already running:
# Restart monit if it dies
*/30 * * * * $HOME/apps/bin/monit > /dev/null
If you have root access, you can improve upon this by having an init script (or systemd unit file) start Monit as your local user.
Bad option: give your user access to the conf dir
You can edit /etc/monit/monitrc to include your local user config directory as above. Similarly, you can allow your user to write to /etc/monit/conf.d. The major downside of these solutions is that you are now allowing your non-root user to create files which will be executes as root, opening a privilege escalation vulnerability. If your user ever got compromised, you certainly don't want an easy way for the attacker to get to root.
I include this option mostly because it's commonly considered, and should be avoided in the vast majority of cases (such as whenever you care about security). However, this might be useful in occasional rare cases (such as when you have a short term server for internal use only behind a firewall with only trusted users, and you need to set it up in a hurry).
Related
I have a Rails 6 application running on Debian buster. In one place I am using "low-level" caching. Here is the relevant code:
# Get the value.
def self.ae_enabled?()
Rails.cache.fetch("ae_enabled", expires_in: 1.hour)
end
# Change the value.
def self.ae_toggle()
ac = AdminConfiguration.find_by(name: "ae-enabled")
ac.value = ! ac.value
ac.save()
# Invalidate the cache.
Rails.cache.delete("ae_enabled")
return ac
end
This works fine ... for a while. At some point, and for reasons I cannot figure out, the cache directory tmp/cache/3F1/ where the above value is cached changes ownership from www-data:www-data (the user Apache runs under) to root:root. Once this happens Apache can no longer read this cached value and the application throws an error.
The odd thing is none of the other directories under tmp/cache/ have their permissions change, it is only the one associated with this low-level cache.
Why is that particular cache directory changing ownership?
Technical details: Rails version 6.0.3.3.
Apache usually does not relate to rails cache, unless you're using passenger, in which case it may be passenger's bug/misconfiguration, check if user sandboxing is enabled and configured correctly.
A typical rails deployment usually has multiple processes:
a web server handling static files and proxying requests to rails (usually nginx, you've mentioned apache)
rails web server (in case of passenger - "inside" the previous, but in fact there's still a child process)
some background workers or processes run from cron
File ownership confusion most probably originates from one of the above writing to disk while running under a different os user.
Look into how your processes are started. First suspect is some cron job that may be configured as system-wide, these run under root.
I have a Rails app which has been running for quite some time; however, occasionally it will decide to create files and directories under its "tmp" directory (particularly "tmp/cache") that are world-writable.
As this is an intranet, our system admins find these and complain.
I can certainly set up a cron job to remove the world-write permission, but I'd prefer to address the problem at the source.
For whatever it's worth, this is running under nginx using the Passenger 5.0.9 gem.
Thanks!
Does the umask of the app user and the nginix user affect this?
I have a Rails runner task that I want to run from cron, but of course cron runs as root and so the environment is set up improperly to get RVM to work properly. I've tried a number of things and none have worked thus far. The crontab entry is:
* 0 * * * root cd /home/deploy/rails_apps/supercharger/current/ && /usr/local/rvm/wrappers/ruby-1.9.3-p484/ruby bundle exec rails runner -e production "Charger.start"
Apologies for the super long command line. Anyhow, the error I'm getting from this is:
ruby: No such file or directory -- bundle (LoadError)
So ruby is being found in the RVM directory, but again, the environment is wrong.
I tried rvm alias delete [alias_name] and it seemed to do something, but darn if I know where the wrapper it generated went. I looked in /usr/local/rvm/wrappers and didn't see one with the name I had specified.
This seems like a common problem -- common enough that the whenever gem exists. The runner command I'm using is so simple, it seemed like a slam dunk to just put this entry in the crontab and go, but not so much...
Any help with this is appreciated.
It sounds like you could use a third-party tool to tether your Rails app to cron: Whenever. You already know about it, but it seems you never tried it. This gem includes a simple DSL that could be applied in your case like:
every :day # Or specify another period, or something else, see README
runner "Charger.start"
end
Once you've defined your schedule, you'll need to write it into crontab with whenever command line utility. See README file and whenever --help for details.
It should not cause any performance impact at runtime since all it does is conversion into crontab format upon deployment or explicit command. It's not needed, once the server is running, everything is done by cron after that.
If you don't want an extra gem anyway, you might as well check what command does it issue for executing your task. Still, an automated way of adding a cron task is easier to maintain and to deploy. Sure, just tossing a line into the crontab is easier — just for you and just this once. Then it starts to get repetitive and tiring, not to mention confusion for other potential developers who will have to set up something similar on their own machines.
You can run cron as different user than root. Even in your example the task begins with
* 0 * * * root cd
root is the user that runs the command. You can edit it with crontab -e -u username.
If you insist on running cron task as root or running as other user does not work for some reason, you can switch user with su. For example:
su - username -c "bundle exec rails runner -e production "Charger.start"
I don't really understand what model of unix accounts/permissions is intended with Capistrano.
Let's say I've got a rails app called Widget and I'll be deploying with passenger. In general, pre-capistrano, I want the entire ./widget directory to be owned by a user called 'widget'. And then by default default passenger will run the app process as user 'widget' too, because passenger runs as user that owns the file.
And the whole point of this is for that 'widget' account to have fairly limited permissions, right? Since a web app will be running under that account?
So since I want the files to be owned by 'widget', I tell cap
set :user, "widget"
But now when I run "cap deploy:setup", it wants to 'sudo' from that account. No way that 'widget' account gets sudo privileges, the whole point is keeping this account limited privs.
Okay, I can tell cap not to use sudo... but then it won't actually have privs to do what it needs, maybe.
I can find a workaround to this too. But I start thinking, why do I keep having to re-invent the wheel? I mistakenly thought the point of cap recipes was to give me some best practices here. Anyway... what do people actually do here?
Use one unix account for install, but then have cap somehow 'chown' it to something else? Use one unix account, but have something non-cap (puppet?) do enough setup so that account doesn't need to sudo to get things started? What? What am I missing?
You can avoid some of the headache by using Passenger most commonly with Nginx as your webserver.
Then to restart web services the unprivileged Widget user creates a file in his path and Passenger will automatically restart Nginx when it sees that file being present.
This is enabled via the following in your config/deploy.rb:
namespace :deploy do
task :start do ; end
task :stop do ; end
task :restart, :roles => :app, :except => { :no_release => true } do
run "touch #{File.join(current_path,'tmp','restart.txt')}"
end
end
As for other privileged tasks for MySQL/DB administration your database.yml provides the credentials necessary to handle rake migration tasks.
So really the only time you would need something more privileged would be for system wide installation of gems, ruby, or rails updates, but a lot of that depends on how your production environment was setup/installed.
Given Passenger+Nginx and separate credentials for DB you can disable sudo and see if you encounter any errors during your Capistrano deploy process and then pickup from there.
The GitHub guys recently released their background processing app which uses Redis:
http://github.com/defunkt/resque
http://github.com/blog/542-introducing-resque
I have it working locally, but I'm struggling to get it working in production. Has anyone got a:
Capistrano recipe to deploy workers (control number of workers, restarting them, etc)
Deployed workers to separate machine(s) from where the main app is running, what settings were needed here?
gotten redis to survive a reboot on the server (I tried putting it in cron but no luck)
how did you work resque-web (their excellent monitoring app) into your deploy?
Thanks!
P.S. I posted an issue on Github about this but no response yet. Hoping some SO gurus can help on this one as I'm not very experienced in deployments. Thank you!
I'm a little late to the party, but thought I'd post what worked for me. Essentially, I have god setup to monitor redis and resque. If they aren't running anymore, god starts them back up. Then, I have a rake task that gets run after a capistrano deploy that quits my resque workers. Once the workers are quit, god will start new workers up so that they're running the latest codebase.
Here is my full writeup of how I use resque in production:
http://thomasmango.com/2010/05/27/resque-in-production
I just figured this out last night, for Capistrano you should use san_juan, then I like the use of God to manage deployment of workers. As for surviving a reboot, I am not sure, but I reboot every 6 months so I am not too worried.
Although he suggest different ways of starting it, this is what worked easiest for me. (Within your deploy.rb)
require 'san_juan'
after "deploy:symlink", "god:app:reload"
after "deploy:symlink", "god:app:start"
To manage where it runs, on another server, etc, he covers that in the configuration section of the README.
I use Passenger on my slice, so it was relatively easy, I just needed to have a config.ru file like so:
require 'resque/server'
run Rack::URLMap.new \
"/" => Resque::Server.new
For my VirtualHost file I have:
<VirtualHost *:80>
ServerName resque.server.com
DocumentRoot /var/www/server.com/current/resque/public
<Location />
AuthType Basic
AuthName "Resque Workers"
AuthUserFile /var/www/server.com/current/resque/.htpasswd
Require valid-user
</Location>
</VirtualHost>
Also, a quick note. Make sure you overide the resque:setup rake task, it will save you lots of time for spawning new workers with God.
I ran into a lot of trouble, so if you need any more help, just post a comment.
Garrett's answer really helped, just wanted to post a few more details. It took a lot of tinkering to get it right...
I'm using passenger also, but nginx instead of apache.
First, don't forget you need to install sinatra, this threw me for a while.
sudo gem install sinatra
Then you need to make a directory for the thing to run, and it has to have a public and tmp folder. They can be empty but the problem is that git won't save an empty directory in the repo. The directory has to have at least one file in it, so I made some junk files as placeholders. This is a weird feature/bug in git.
I'm using the resque plugin, so I made the directory there (where the default config.ru is). It looks like Garrett made a new 'resque' directory in his rails_root. Either one should work. For me...
cd MY_RAILS_APP/vendor/plugins/resque/
mkdir public
mkdir tmp
touch public/placeholder.txt
touch tmp/placeholder.txt
Then I edited MY_RAILS_APP/vendor/plugins/resque/config.ru so it looks like this:
#!/usr/bin/env ruby
require 'logger'
$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/lib')
require 'resque/server'
use Rack::ShowExceptions
# Set the AUTH env variable to your basic auth password to protect Resque.
AUTH_PASSWORD = "ADD_SOME_PASSWORD_HERE"
if AUTH_PASSWORD
Resque::Server.use Rack::Auth::Basic do |username, password|
password == AUTH_PASSWORD
end
end
run Resque::Server.new
Don't forget to change ADD_SOME_PASSWORD_HERE to the password you want to use to protect the app.
Finally, I'm using Nginx so here is what I added to my nginx.conf
server {
listen 80;
server_name resque.seoaholic.com;
root /home/admin/public_html/seoaholic/current/vendor/plugins/resque/public;
passenger_enabled on;
}
And so it gets restarted on your deploys, probably something like this in your deploy.rb
run "touch #{current_path}/vendor/plugins/resque/tmp/restart.txt"
I'm not really sure if this is the best way, I've never setup rack/sinatra apps before. But it works.
This is just to get the monitoring app going. Next I need to figure out the god part.
Use these steps instead of making configuration with web server level and editing plugin:
#The steps need to be performed to use resque-web with in your application
#In routes.rb
ApplicationName::Application.routes.draw do
resources :some_controller_name
mount Resque::Server, :at=> "/resque"
end
#That's it now you can access it from within your application i.e
#http://localhost:3000/resque
#To be insured that that Resque::Server is loaded add its requirement condition in Gemfile
gem 'resque', :require=>"resque/server"
#To add basic http authentication add resque_auth.rb file in initializers folder and add these lines for the security
Resque::Server.use(Rack::Auth::Basic) do |user, password|
password == "secret"
end
#That's It !!!!! :)
#Thanks to Ryan from RailsCasts for this valuable information.
#http://railscasts.com/episodes/271-resque?autoplay=true
https://gist.github.com/1060167