I was wondering if it was possible to run migrations automatically during deployment with Google App Engine. I have been using AWS Elasticbeanstalk for a while and they were ran automatically but now I am considering moving to the Google App Engine for my future projects.
Right now, I must run this command manually:
bundle exec rake appengine:exec -- bundle exec rake db:migrate GAE_CONFIG=app.yml
Thank you
WARNING: As discussed in comments, there is a race condition in migrations if deployment is done on multiple containers in parallel, because it will try to run migration on all containers. Solution is being discussed in comments, i will update this answer when we land on something.
Disclaimer: This answer is not exactly what was asked for, but it solves same problem and it works. And from what i can tell from question, doing it with some appengine config is not a requirement, rather he just want the migrations to run automatically.
I will expand on my comment on question, here is something i tried and it works. I am strong believer of KISS(keep it simple and stupid). So instead of trying to figure out appengine(which i have never used anyway) if i were you, i would take a generic approach. Which is, to plug into rails server booting process and trigger migrations. For this we have multiple approaches.
With my understanding of appengine and suggested by this official doc link appengine has a app.yaml file, this file has an entry something like:
entrypoint: rails server
So we will use this entry point to plug in our code to run migrations before starting server. For this i did this:
Make a new file in bin directory, i named it
rails_with_migrations.sh you can name it whatever you like.
Give it execute permissions with chmod +x bin/rails_with_migrations.sh
Put this code inside it:
#!/bin/bash
bundle exec rake db:migrate
bundle exec rails $#
Of course you can give whatever RAILS_ENV you want to give these.
Now in app.yaml on the entrypoint section, instead of rails server give it bin/rails_with_migrations.sh server and it should be it. It worked on local, should work everywhere.
NOTE: In entrypoint: i have bin/rails_with_migrations.sh server here, server is rails command parameter, you can pass as much parameters as you like these all will be passed to rails server command with $#'s magic. It is there to allow you to pass port and any other parameters you may need to provide for your environment. Also it allows you to run rails console locally with bin/rails_with_migrations.sh console which will also cause migrations to get triggered.
UPDATE1: As per comment, i checked what happens if migration fails, and it starts server even if migration fail. We can alter this behavior of-course in our sh file.
UPDATE2: The shell-script with migration error code handling will look something like:
#!/bin/bash
bundle exec rake db:migrate
if [ $? -eq 0 ]
then
bundle exec rails $#
else
echo "Failure: migrations failed, please check application logs for more details." >&2
exit 1
fi
This update will prevent server from starting and causing a non zero exit code from the script, which should indicate that this command failed.
Related
Please bear in mind, that I'm new to Docker.
I'm using a Mac.
When I followed this official documentation https://github.com/docker/awesome-compose/tree/master/official-documentation-samples/rails/, it all seemed to work, and so after testing that, I created a new directory and followed the example while changing things to incorporate other things. I was making changes to the official documentation with some things shown in this link, https://tihandev.com/how-to-integrate-elasticsearch-with-ruby-on-rails/
But this time when I try to run
docker compose run --no-deps web rails new . --force --database=postgresql
It gave me the error
No Rakefile found
So I was trying to debug what change would cause this and kept backtracking the changes, and even when I backtracked to make all the files be the same as the official documentation that had worked, I still get the "No Rakefile found" error.
Funny thing is that even after I delete Gemfile.lock and recreate it, the Gemfile.lock file gets populated instantly without any console logs about installing dependencies. So clearly, this information is saved somewhere else that I don't know.
Even after I go to the "Docker Destop" GUI client, and click "Delete" for that container among the containers listed, and then delete the Gemfile.lock file, and then rerun that command, it gives the same output as if some previous erroneous state is stored somewhere else.
Can someone please help me with what is going on here?
For a clarification on this question, even though I really appreciate answers on why doing a docker compose run wouldn't work for the first time because that is an information I need as well, my main confusion here is on why even after deleting problematic sections and after returning to what had worked before, I still get the same problem.
Edit 1: Steps to reproduce
These are the steps to reproduce my case.
Follow exactly the same as https://github.com/docker/awesome-compose/tree/master/official-documentation-samples/rails/ except for entrypoint.sh. Modify entrypoint.sh as below
#!/bin/bash
set -e
# Remove a potentially pre-existing server.pid for Rails.
rm -f /myapp/tmp/pids/server.pid
bundle check || bundle install
bundle exec rake db:create
bundle exec rake db:migrate
bundle exec rake db:seed
# Then exec the container's main process (what's set as CMD in the Dockerfile).
exec "$#"
Run docker compose run --no-deps web rails new . --force --database=postgresql
This would produce a message that says "No Rakefile found"
Change the entrypoint.sh file to be the same as how it was in the link.
Delete Gemfile.lock and touch Gemfile.lock
(At this point I doublecheck that every file is the same as how it is in the link)
Run docker compose run --no-deps web rails new . --force --database=postgresql
You still get "No Rakefile found"
Go to Docker Desktop and press on the trash bin icon to delete the container that had been created due to calling docker compose run
Run docker compose run --no-deps web rails new . --force --database=postgresql again, and still get No Rakefile found.
Again, note that if I had followed the link exactly from the beginning, it runs fine.
I really wish that you had been clear about the change that you made to entrypoint.sh in your original post. I spent a lot of time attempting to answer your original question -- before the edit -- with an answer that was 500 lines long with detailed steps. I could have answered your question in 15 minutes if you had been clear about what changes you made.
But it turns out that the problem is that you're editing entrypoint.sh and telling it "you should expect to find a Rails app in this directory" before you've created a Rails app in that directory.
These commands are the problem:
bundle exec rake db:create
bundle exec rake db:migrate
bundle exec rake db:seed
These commands expect:
There is a bundler-managed Ruby app in this directory
Its dependencies include the rake gem
There is an accompanying Rakefile in the directory with a configuration that can be understood by bundle exec rake
There are additional rake tasks defined by the Ruby app in this directory that explain how to execute the commands db:create db:migrate db:seed
None of that exists until after you create the Rails app.
Follow the instructions in the repository that you linked to first. Don't edit entrypoint.sh before you've completed those steps because it cannot and will not work the way you expect for the reasons I've described.
Scope of problem
Rails 4.2.11
Ansible 2.1.1.0
Ubuntu 14
Ubuntu user: deploy
I have Rails app which I deploy to Ubuntu server by Ansible script.
I have problem of understanding why Rails app create log files with root permissions when Ansible script execute rake tasks.
In my example it's running rake db:migrate but also same behaviour with rake assets:precompile
You can see by photo below that application is deployed via user 'deploy' but after run of rake task it create 2 log files with root permission. After restart of web server it crash with permission denied error, so I need manually change ownership to deploy:deploy
Structure of Rails logger is also looks suspicious. You can check #dev=IO:<STDERR> value. I've checked in another project and there I can see something like #dev=#<File:/var/www/.../log/production.log>
I tried to explore source code of ror4 but so far no luck to understand from there what is happening. Only idea is could be that Rails raise exception when create log file and STDERR become output
Please help if you have similar problem or can point out where I can look to.
The rake script runs as root User.
To answer the question you must add the complete ansible script or the bundle script if you manually add a "sudo" commands or something unusual.
There are different possible positions to define the user in ansible.
Read the become section of the ansible documentation https://docs.ansible.com/ansible/latest/user_guide/become.html
Or give a try with
- name: Run db:migrate
shell: ... rake cmd ...
become: yes
become_user: deploy
become_method: su
Change the become_method to your needs e.g become_flags: '-s /bin/sh' or sudo
I have a Rails runner task that I want to run from cron, but of course cron runs as root and so the environment is set up improperly to get RVM to work properly. I've tried a number of things and none have worked thus far. The crontab entry is:
* 0 * * * root cd /home/deploy/rails_apps/supercharger/current/ && /usr/local/rvm/wrappers/ruby-1.9.3-p484/ruby bundle exec rails runner -e production "Charger.start"
Apologies for the super long command line. Anyhow, the error I'm getting from this is:
ruby: No such file or directory -- bundle (LoadError)
So ruby is being found in the RVM directory, but again, the environment is wrong.
I tried rvm alias delete [alias_name] and it seemed to do something, but darn if I know where the wrapper it generated went. I looked in /usr/local/rvm/wrappers and didn't see one with the name I had specified.
This seems like a common problem -- common enough that the whenever gem exists. The runner command I'm using is so simple, it seemed like a slam dunk to just put this entry in the crontab and go, but not so much...
Any help with this is appreciated.
It sounds like you could use a third-party tool to tether your Rails app to cron: Whenever. You already know about it, but it seems you never tried it. This gem includes a simple DSL that could be applied in your case like:
every :day # Or specify another period, or something else, see README
runner "Charger.start"
end
Once you've defined your schedule, you'll need to write it into crontab with whenever command line utility. See README file and whenever --help for details.
It should not cause any performance impact at runtime since all it does is conversion into crontab format upon deployment or explicit command. It's not needed, once the server is running, everything is done by cron after that.
If you don't want an extra gem anyway, you might as well check what command does it issue for executing your task. Still, an automated way of adding a cron task is easier to maintain and to deploy. Sure, just tossing a line into the crontab is easier — just for you and just this once. Then it starts to get repetitive and tiring, not to mention confusion for other potential developers who will have to set up something similar on their own machines.
You can run cron as different user than root. Even in your example the task begins with
* 0 * * * root cd
root is the user that runs the command. You can edit it with crontab -e -u username.
If you insist on running cron task as root or running as other user does not work for some reason, you can switch user with su. For example:
su - username -c "bundle exec rails runner -e production "Charger.start"
With my large application, the Rails console takes a while to load up. Is there a way to single commands more easily?
I'd also like to be able to automate stuff, and echo "query" | rails console isn't a great way to do things.
Thoughts?
EDIT: What about a long-running process that I can ping queries to whenever I have need?
There are two main ways to run commands outside console:
Rake task which depends on :environment
rails runner (previously script/runner), eg:
$ rails runner "query"
Both are pretty well documented on the rails guide: https://guides.rubyonrails.org/command_line.html#bin-rails-runner
Both of these methods will still take the same time as a console to fire up, but they are useful for non-interactive tasks.
Just pipe it in:
echo 'puts Article.count' | bundle exec rails c
It should now be a lot faster than when then the question was originally asked, because of Spring. It's not immediate, but still a lot faster than spinning up the whole app. Use this for the fast lane, it should run in under a second (assuming your required command is fast):
echo 'puts Article.count' | spring rails c
If you really want a single long-running process, you could easily do it by creating a controller action that simply runs whatever you POST to it, then send commands to it using curl behind an alias. The action would of course be completely insecure and should be triple-guarded against running anywhere near production, but it would be easy to setup.
Solution: bundle exec command allows us to run an executable script in the specific context of the project's bundle - making all gems specified in the Gemfile available to require in Ruby application. In addition it eventually avoids any conflicts with other versions of rake installed globally.
echo '<command>' | bundle exec rails c
for more information look at the documentation of bundler
example:
configuration_item=$(echo 'ConfigurationManager.getKey("authentication_method")' | bundle exec rails c )
echo $configuration_item
#output:
MFA_authentication
I have a very simple task called update_feeds:
desc "Update feeds"
task :update_feeds do
run "cd #{release_path}"
run "script/console production"
run "FeedEntry.update_all"
end
Whenever I try to run this task, I get the following message:
[out :: mysite.com] sh: script/console: No such file or directory
I figured it's because I am not in the right directory, but trying
run "cd ~/user/mysite.com/current"
instead of
run "cd #{release_path}"
Also fails. Running the exact same commands manually (through ssh) works perfectly.
Why can't capistrano properly cd (change directory) into the site directory to run the command?
Thanks!
Update: Picked an answer, and thank you so much to all who replied.
The best answer may actually be the one on server fault, though the gist of both (the one on server fault and the one on stack overflow) is the same.
You want to use script/runner. It starts an instance of the app to execute the method you want to call. Its slow though as it has to load all of your rails app.
~/user/mysite.com/current/script/runner -e production FeedEntry.update_all 2>&1
You can run that from the capistrano task.
I cannot imagine that you would be able to remotely log into rails console from capistrano. I suggest you call your model method from a rake task.
How do I run a rake task from Capistrano?
As for the latter part of your question, are you logging into the server with the same user account as capistrano?