In akk-http-microservice i have seen a procfile for the first time. I try to know about it. I came to know that procfile is meant for executing commands. But how it differs from shell script which also do same. When should i prefer to it?
A Procfile is a mechanism for declaring what commands are run by your application's dynos on the Heroku platform. It follows the process model. You can use a Procfile to declare various process types, such as multiple types of workers, a singleton process like a clock, or a consumer of the Twitter streaming API. Thus it's not related to Akka-Http.
Related
It's nice to see Rails 4.2 come with Active Job as a common interface for background jobs. But I can't find how to start a worker in the document. It seems that the document is still immature (e.g. the right version of Sneakers is only referred to in Rails' Gemfile), so I'm not sure if the "running workers" part is not in Active Job or just not mentioned in docs.
So with Active Job, do I still need to manually start the job watcher threads like sidekiq or in my case, rake sneakers:run? If so, where should I put these commands to let rails server run these parallel tasks automatically in a develop environment?
ActiveJob is just a common interface. You still need the backend gem, and you still need to launch it separately from your server (it is a separated process, which is the objective).
Sample using resque:
In the Gemfile:
gem 'resque'
In the terminal, launching a worker:
bin/resque work
The case is similary when using sidekick, delayed job or something else.
If you want to launch the server & worker in a single command, you can create a short bash script for it, but I would advise not doing so: having two separated console helps to watch what is happening on each side (web app & worker).
A better solution would be to use the foreman gem to manage starting & stopping your process.
You can create a simple Procfile with the processes to start:
web: bundle exec rails s
job: bundle exec resque work
And then just start both using foreman:
foreman start
By default, foreman will interleave the logs of the process in the console, but this can be configured.
You still have to run the job thread watcher.
I am running a server with multiple Rails apps. I want a different unicorn process name for each app, e.g:
unicorn_ecommerce for the ecommerce app.
unicorn_hotsite for the hotsite.
Instead of the traditional "unicorn_rails" for every app. So when I type in the terminal pgrep -lf unicorn I get the apps currently running. I also would like to know if there is a better way to organize your unicorn processes.
I found this question while trying to differentiate my unicorn processes in ps aux | grep unicorn, not sure if you have the same kind of issues as Renaming Unicorn processes or if you'd be better served by one of the options in https://askubuntu.com/questions/157075/why-does-ps-aux-grep-x-give-better-results-than-pgrep-x
Just in case someone is looking for better names in ps aux there is a pretty simple solution that I'm sure most people figure out by themselves. If you're like me and needed a little help just use the following tip from the manpage:
Using an absolute path for CONFIG_FILE is recommended as it makes multiple instances of Unicorn easily distinguishable when viewing ps(1) output.
http://manpages.ubuntu.com/manpages/saucy/man1/unicorn_rails.1.html (-c, --config-file CONFIG_FILE section)
rename your /etc/init.d/unicorn_rails file to anything you want.
then your process will be spawn with this name as well.
I have been thinking about this and searching this for ages without finding anything, so I am going to assume I hit the XY problem.
Let me describe my issue, this sounds common enough.
We use capistrano to deploy our web app and db. The relevant part is that we have a dedicated server for delayed job and we use capistrano to deploy to it and start/restart the processes. This is a custom number of workers with 2 different Gemfiles and 3 queues.
What I want to do is to start them up on server restart or, more to the point, on server clone + start.
I have tried calling cap production delayed_job:custom_start from the server itself.. didn't work. (This is the core of my non XY problem adjusted question). Not sure it even makes sense. But I want to know if it is possible. custom_start is a task that starts our set of workers.
Alternatively I am thinking of abstracting the code into a rake task or a script or something and calling it from both capistrano and where ever I would need to add it to start on startup. Does this make more sense?
Edit: Just found this post.. discouraging..
p.s. I just want to clarify that when I say server I mean my Machine/ec2 instance and not my web app restarting.
My Jenkins deploy jobs are littered with direct tasks developers can call such as cap dev app:fetch_logs, cap qa sanitize_environment, etc.
This feature of Capistrano is easy and verified.
What I am guessing you want to do is use Capistrano to setup rc.d files. Yes, you can do this. Should you use chef/puppet at this point? Consider. Scalr/Rightscale are funs things to look at too.
You can add a bash script as an .erb template for all your worker variables, then put upload the script into the deploy_to directory. Finally, you can setup another task (#{sudo}) to inject rc.d wrappers into rc.d. Or you rather than rc.d wrappers to you bash script, just call the bash script from rc.d-local. You can use sed to append to rc.d-local.
I ended up moving the delayed job related logic to its own script that accepts start/stop, and delegating to this script from capistrano. This means I can add this script to my rc.local.
Our current production deployment uses jenkins to deploy a warble generated war file to Tomcat. The whole thing works like a charm. The problem I'm running into however is how to kick start up sidekiq's workers on this machine via "bundle exec sidekiq [options]". Ideally I'd love to avoid setting up a whole seperate ruby environment on this machine just to do this, but either way to run properly, sidekiq needs access to the exploded/installed apps environment etc.
Is there an accepted way to do something like this? Is there a better way to startup sidekiq in instances like this beyond bundle?
This project may be of help. It allows you to package anything that can be a rake task into a jar file. Their documentation has some specific notes about warbler use. Have a look!
For notes on how to run Sidekiq from outside of the command line run something like this from your project root:
cat $(bundle show sidekiq)/bin/sidekiq
You should see some lines:
cli = Sidekiq::CLI.instance
cli.parse
cli.run
If you read into the CLI class, you'll notice that parse takes either ARGV as the default argument, but you can override it with your own arguments:
cli.parse "-q myqueue -e production".split(' ')
I have a rake task which parses a streaming API and enters data into database. The streaming API is live feed and the rake task should run continuously for the live data to enter the database. The rake task once called will run continuously and parse the data. Now i have started the rake task and it is running. The problem is that if i close the terminal or reboot the server, the rake task wil be stopped. So, i want a script in linux (something like the one used to start, or stop apache server), which does the following:
1. start the rake task by calling rake command (rake parse:stream) from the RAILS-ROOT (application directory of Rails app)
2. stop the rake task by killing the process.
3. start the rake task automatically when the server reboots.
i am not familiar to linux scripts and i dont know where to start. i am using ubuntu server. can anyone help me?
Here's an article that might help you also. It discussed various options for managing Ruby applications and their related processes:
http://michaelvanrooijen.com/articles/2011/06/08-managing-and-monitoring-your-ruby-application-with-foreman-and-upstart/
You need to run your script as a daemon. When I create this kind of startup scripts I usually make 2 files, one that stays in /etc/init.d and handles the start/stop/status/restart commands and another one that actually does the job and gets called by the first script.
Here is one solution, and although the daemon script is written in perl, you want to run some command lines only, so daemonizing a perl script could do your job easily.
If you want, there are also ruby gems for daemonizing scripts, so you can write a script in ruby that does the rake tasks.
And if you want to go hardcore, there are solutions for writing bash scripts that can daemonize, but I'm not sure I would recommend a solution like that; at least I find them pretty difficult to use.
Take a look at how Github's Resque project does it.
Essentially they create tasks for starting/restarting/stopping a particular task, in this case resque:work. Note that the restart_workers task simply invokes the other tasks, stop and start. It should be really easy to change this for what you want.