I'm trying to write a task for Capistrano 3 that involves executing 'composer install' within the directory of the current release. It looks something like this:
namespace :composer do
desc 'Install dependencies with Composer'
task :install do
on roles(:web) do
within release_path do
execute "#{fetch(:composer_command)} install"
end
end
end
end
composer_command is set in the staging and production files - in my particular case to php /home/user/composer.phar
For some reason this command does not actually run in the current release directory, but instead runs in the parent directory (containing current, shared, releases, etc)
I delved into this a bit further and found that when I ran a single word command, like:
within release_path do
execute "pwd"
end
It works just fine, and runs the command in the current release directory. But... when I run a command with spaces, like:
within release_path do
execute "pwd && ls"
end
It runs in the parent directory, and not the directory set by the within block.
Can someone shed some light on this? Thanks!
Smells like a Cap 3 bug.
I suggest just guaranteeing you are where you want to be from the shell perspective:
execute "cd '#{release_path}'; #{fetch(:composer_command)} install"
You can retain all the niceties of within(), with(), default_env, etc, while still keeping the natural string syntax:
within release_path do
execute *%w[ pip install -r requirements.txt ]
end
A couple of tips:
1) Capistrano uses SSHKit for a lot of things, among which command execution. In order to simplify using Composer you could configure the command map (in deploy.rb or production.rb, etc), here are 2 examples:
SSHKit.config.command_map[:composer] = "#{shared_path.join('composer.phar')}"
SSHKit.config.command_map[:composer] = '/usr/bin/env composer.phar'
Next you can execute it like so:
execute :composer, :install
2) From a security perspective it's wise to disable the php setting allow_url_fopen, but unfortunately Composer needs it enabled to function. You can use this trick to leave it disabled globally:
SSHKit.config.command_map[:composer] = "/usr/bin/env php -d allow_url_fopen=On #{shared_path.join('composer.phar')}"
Check out iniscan for more security advise on php settings.
3) Composer has an option -d, --working-dir, which you can point to the directory containing the composer.json file in order to run Composer from any other directory. This should solve your problem:
execute :composer, '-d', release_path, :install
4) You may want to take a look at the capistrano-composer project :)
Actually, your use of the within function is almost correct. You have supplied it an entire string as a command, but the doc points out that this results in unreliable behaviour (which I have experienced myself).
Let the first argument to execute be a symbol instead of a string (which contains whitespace):
within release_path do
execute fetch(:composer_command).to_sym, "install"
execute :pwd
execute :ls
end
just for reference here is the Capistrano Doc explaining why within {} does not work with arguments with whitespace. I hope this helps.
Related
I'm trying to run a C++ executable in my Rails app that is place in a folder called "algo", like this:
result = `cd algo && ./my_main #{str} -1 -1 #{id}`
In development works flawlessly but in production in the cloud does not run
Consider that:
1) In the cloud, that is a virtual machine, i run the same executable without problems in the console terminal navigate through the Rails application folders, it only fails when i try to run it from the Rails application
2)
Rails.logger.info result
Returns nothing
3)
Rails.logger.info `pwd`
Does return the current folder of the proyect
4)
Rails.logger.info $?
Only returns: pid 35314 exit 127
5)
Rails.logger.info File.exist?("algo/my_main")
Returns true
6)
In the config/environments/production.rb the log level is config.log_level = :info
7)
In the log/production.log does not appear any error like you will see in development in the terminal
8)
I also try to use other commands like: system(), exec(), %x() with the same result
9)
Finally, i run sudo chmod -R 777 in the virtual machine, in the main folder before the Rails folder app, i think that is implicit in the point 1, but for clarify
You should always use absolute paths for any code that will be executed by a script. The PATH variable may be different for the user executing the script than it is for the user that you use, and its much better to be 100% precise about the file path you want than to rely on PATH.
Along the same lines, make sure the user that runs the Rails server have execute permissions on the script. If in doubt, login as that user and attempt to execute the script.
You also need to escape both str and id for security reasons. Even if these variables are not currently derived in any way from submitted parameters, there's always a possibility that whatever function contains this code might be executed with user-submitted variables at some point. Basically, its better to be safe than sorry, because this is the kind of security hole that could allow anyone on the Internet to execute arbitrary code on your server.
I'm using whenever to schedule a task in Rails, and I would like to find a way to test the behavior being executed.
Is there a way to trigger the event in a test (like RSpec) so I can make assertions about the results? The executed task is a class method that by itself works because I've tested it manually in the Rails console, but is there a way to trigger the event so that this behavior happens and I can assert it works the same way within the config/schedule.rb?
I used a simple trick to assert my expectations like below
require 'spec_helper'
require 'whenever'
describe 'Schedule' do
it 'sends email 10th of each month before 08 AM EST' do
expected = "0 7 10 * * /bin/bash -l -c 'cd && script/rails runner -e production '\\''Delayed::Job.enqueue(MyMailerJob.new([2,3,4]), priority: 10)'"
expect(cron_output).to include(expected)
end
end
def cron_output
Whenever::JobList.new(file: Rails.root.join("config", "schedule.rb").to_s).
generate_cron_output.
gsub(Dir.pwd, '')
end
The "whenever event"
The only thing that changes in what whenever wraps (ie, cron) compared to your regular spec run is system environment, and more specifically environment variables.
You can see the commands whenever will install in cron using its executable :
$ whenever
Cron always start with a raw environment. If you check your crontab content (using crontab -e) you'll see on top variables that are set. You can set additional variables, there. For example :
SHELL="/bin/bash"
PATH="/sbin:/bin:/usr/sbin:/usr/bin"
MAILTO="your_user"
HOME="/home/your_user"
RAILS_ENV="production"
You should not need those, though, because whenever uses a nice trick : it calls all commands using /bin/bash -l -c. This ensures it uses a login shell, which loads ~/.bashrc file. So any variable that is exported in your bashrc file will be accessible within cron execution.
Also note some distributions (like gentoo) have a check in the skel bashrc file that quit if shell is not interactive (so you have to set your environment variables before it if they are needed in cron).
Tests with similar env
To answer about testing, this is probably not a good idea. There's nothing related to your codebase here, only related to your system (similarly, you don't test which version of postgres or imagemagick is installed on your system in your specs).
You can somewhat simulate, for debugging sake, what happens in cron while invoking rspec doing such :
env - HOME=/home/<your_user>/ /bin/bash -l -c 'rspec spec/'
This will ensure environment is emptied (env -), then set HOME as a minimal env, then call a login shell just like whenever commands do.
I should change to specific folder so that the output of the script can be exported into a file in that folder (123.txt in the example code below).
but this command, which I run from inside my ruby code
./iw2_broadcast.py
takes 5 minutes or more to complete. I try to append & to make it run in the background, but it seems it does not work.
Any ideas? Thanks
Dir.chdir(#iw2_dir)
` ./iw2_broadcast.py -f 123.txt & `
puts "123"
Create a shell script with the execution command
# py.sh
./iw2_broadcast.py
Execute the above shell script from ruby using system command
# ruby_script.rb
system("./py.sh")
Now your ruby code will be executed without waiting for the output
Adding & works with system(). You can easily test it with:
system("sleep 1 &")
So for this precise example:
system("./iw2_broadcast.py -f 123.txt &")
I have around 50 feature files and I want the tester team to run them. I don't want them to run all the files at once and also don't want them to get in the prerequisites for running cucumber ie..bundle install ...rake db:migrate...service postgresql start...... etc
I want to make a shellscript file so that they can execute it, select the option which file to run (all or other individual file) and then the script would execute the prerequisites and execute the file and output the log to specified folder. I want to make a sh file.
Of course I don't know your development environment or exactly what commands you need to execute to get your tests running, but here's a sample shell script:
#!/bin/sh
bundle install
rake db:migrate
rake db:test:prepare
rake cucumber "$#"
Execute the script like this:
./script_name.sh
With arguments:
./script_name.sh features/registration.feature
The "$#" part will pass any arguments like features/registration.feature to rake cucumber.
I wrote a script that does maintenance tasks for a rails application. The script uses a class that uses models defined in the application. Just an example, let's say application defines model User, and my class (used within the script), sends messages to it, like User.find id.
I am looking for ways to optimize this script, because right now it has to load the application environment: require '../config/environment'. This takes ~15 seconds.
Had the script not use application codebase to do its job, I could have replaced model abstractions with raw SQL. But unfortunatly I can't do that because I would have to repeat the code in the script that is already present in the codebase. Not only would this violate DRY principle and require alot of work, the script would not be very maintainable, in case the model methods that I am using change.
I would like to hear ideas how to approach this problem. The script is not run from the application itself, but from the shell (with Capistrano for instance).
I hope I've described the problem clear enough. Thank you.
Could you write a little daemon that is in a read on a pipe (or named fifo, or unix domain socket, or, with more complexity, a tcp port) that accepts 'commands' that would be run on your database?
#!/usr/bin/ruby
require '../config/environment'
while (true) do
File.open("/tmp/fifo", "r") do |f|
f.each_line do |line|
case line
when "cleanup" then puts "clean!"
when "publish" then puts "published!"
else puts "invalid command, ignoring"
end
end
end
end
You could start this thing up with vixie cron's #reboot specifier, or you could run it via capistrano commands, or run it out of init or init scripts. Then you write your capistrano rules (that you have now) to simply echo commands into the fifo:
First,
mkfifo /tmp/fifo
In one terminal:
$ ./env.rb
In another terminal:
$ echo -n "cleanup" > /tmp/fifo
$ echo -n "publish" > /tmp/fifo
$ echo -n "go away" > /tmp/fifo
The output in the first terminal looks like this:
clean!
published!
invalid command, ignoring
You could make the matching as friendly (perhaps allow plain echo, rather than require echo -n as my example does) or unfriendly as you want. And the commands that get run can of course call into your model files to do their work.
Please make sure you choose a good location for your fifo -- /tmp/ is probably a bad place, as many distributions clear it on reboot. Also make sure you set the fifo owner and permission (chown and chmod) appropriately for your application -- you might not want to allow your Firefox's flash plugin to write to this file and command your database.