Rails 3, Bundler, LoadError - ruby-on-rails

Question

I'm writing an app that will run scripts in a specified folder, and then record the numbers and graph them.
My problem is that if the script is a ruby file, the require statements fail inside the script because bundler seems to have done something funky with the load path.
Running rails runner Datasource.run_jobs fails:
class Datasource < ActiveRecord::Base
def self.run_jobs
require 'aws_sdb'
access_key_id = "REDACTED"
secret_key = "REDACTED" # In all actuality these woudln't be here.
#sdb = AwsSdb::Service.new(:access_key_id => access_key_id, :secret_access_key => secret_key)
Datasource.all.each do |ds|
puts "Updating #{ds.name}..."
unless #sdb.list_domains.include? ds.name
puts "#{ds.name} doesn't exist in SDB, creating a domain for it..."
#sdb.create_domain ds.name
end
#data = "TEST"
data = `#{RAILS_ROOT}/lib/plugins/#{ds.name}`
#sdb.put_attributes(ds.name, Time.now.to_i, data)
puts "Data Collected: #{data.inspect}"
end
end
has_many :graphs
end

Answer 1

(Basing this off your comments on the question)
You will need to add hpricot (and any other gem this needs) to your Gemfile so that they are made available by Bundler. Bundler is by far the easiest way to avoid gem conflicts and tomfoolery.
Imagine this situation: You somehow lose the gems that you have currently. Be this happening through a format or system change or any other reason. Whatever it is, you've lost your gems. How are you going to re-install all your gems? You could keep a list of them somewhere else yourself, but is this truly likely?
Bundler solves this problem by making you state what gems your application requires and only adding those gems to the load path, which is why you can't find hpricot. When you run bundle install the first time, this creates a Gemfile.lock which contains something like this:
GEM
remote: http://rubygems.org/
specs:
abstract (1.0.0)
actionmailer (3.0.0)
...
Because you commit this file to your source control "solution" of choice (be it Git, SVN, FTP, whatever, it's not important) you have a solid way of specifying the precise gems and precise versions of those gems that your application uses.
When/If your gems are wiped, you can simply clone your project again and run bundle install. Because the Gemfile.lock file exists, you'll have exactly the same gems you had originally, even if there were updates.
If you don't want the exact same gems, just run bundle update and this will ignore the specifications in Gemfile.lock and instead revert to depending on Gemfile to define them. This will check for new versions of gems and install them, updating the Gemfile.lock when it's done.
Honestly, I don't understand the Bundler hate. If you could explain in wider terms than "OMG IT SUCKS YEHUDA IS SATAN", I'd be much obliged.
Edit: WedTM asked for a sample Gemfile and related code:
In the Gemfile you'd have this:
group(:scripts) do
gem 'gem1'
end
To require these gems for your scripts:
require 'bundler'
Bundler.require(:scripts)
You may also wish to require the default gems too which you can do by just adding default anywhere to the arguments of require:
Bundler.require(:default, :scripts)
If this for some reason doesn't work I would imagine it would be because it can't locate the Gemfile. This can be fixed by setting the ENV['BUNDLE_GEMFILE'] to the path to the Gemfile.

Answer 2

I wonder if you might be able to use RVM to set up the ruby environment before running your scripts. Maybe something with a gemset like:
data = `rvm gemset use scripts; #{RAILS_ROOT}/lib/plugins/#{ds.name}`