Currently, I'm triggering a SuckerPunch Job via clockwork in my Rails app.
User.find_each do |user|
puts "Found user: #{user.name}!"
end
# Doesn't work, too.
User.all.each do |user|
puts "Found user: #{user.name}!"
end
However, the execution stucks when searching for the users. When printing the Enumerator (User.find_each.inspect) to the console it shows the users. Why won't the block run? May there be a Rails context missing?
It's been driving me crazy for hours. I'm deploying to Heroku, btw.
EDIT: I figured out, there's an error: Primary key not included in the custom select clause.
Thanks in advance!
I suspect you're hitting some sort of limit due to the number of Users in the db. Have you tried using find_in_batch instead?
http://apidock.com/rails/ActiveRecord/Batches/find_each (uses find_in_batch with a default batch_size of 1000)
http://apidock.com/rails/ActiveRecord/Batches/find_in_batches (note the last post)
Related
I have an API which uses a Service, in which I have used Ruby thread to reduce the response time of the API. I have tried to share the context using the following example. It was working fine with Rails 4, ruby 2.2.1
Now, we have upgraded rails to 5.2.3 and ruby 2.6.5. After which service has stopped working. I can call the service from Console, it works fine. But with API call, service becomes unresponsive once it reaches CurrencyConverter.new. Any Idea what can be the issue?
class ParallelTest
def initialize
puts "Initialized"
end
def perform
# Our sample set of currencies
currencies = ['ARS','AUD','CAD','CNY','DEM','EUR','GBP','HKD','ILS','INR','USD','XAG','XAU']
# Create an array to keep track of threads
threads = []
currencies.each do |currency|
# Keep track of the child processes as you spawn them
threads << Thread.new do
puts currency
CurrencyConverter.new(currency).print
end
end
# Join on the child processes to allow them to finish
threads.each do |thread|
thread.join
end
{ success: true }
end
end
class CurrencyConverter
def initialize(params)
#curr = params
end
def print
puts #curr
end
end
If I remove the CurrencyConverter.new(currency), then everything works fine. CurrencyConverter is a service object that I have.
Found the Issue
Thanks to #anothermh for this link
https://guides.rubyonrails.org/threading_and_code_execution.html#wrapping-application-code
https://guides.rubyonrails.org/threading_and_code_execution.html#load-interlock
As per the blog, When one thread is performing an autoload by evaluating the class definition from the appropriate file, it is important no other thread encounters a reference to the partially-defined constant.
Only one thread may load or unload at a time, and to do either, it must wait until no other threads are running application code. If a thread is waiting to perform a load, it doesn't prevent other threads from loading (in fact, they'll cooperate, and each perform their queued load in turn, before all resuming running together).
This can be resolved by permitting concurrent loads.
https://guides.rubyonrails.org/threading_and_code_execution.html#permit-concurrent-loads
Rails.application.executor.wrap do
urls.each do |currency|
threads << Thread.new do
CurrencyConverter.new(currency)
puts currency
end
ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
threads.map(&:join)
end
end
end
Thank you everybody for your time, I appreciate.
Don't re-invent the wheel and use Sidekiq instead. 😉
From the project's page:
Simple, efficient background processing for Ruby.
Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails to make background processing dead simple.
With 400+ contributors, and 10k+ starts on Github, they have build a solid parallel job execution process that is production ready, and easy to setup.
Have a look at their Getting Started to see it by yourself.
I'm in the middle of trying to upgrade pieces of our app to Rails 3, specifically our unit tests. I'm consistently running into an issue with delayed job and mailers, in particular, namely that the mailers aren't getting "sent" when the test is run.
To test emails sent, we run them all through an "assert_email" method in our test helper, which looks more-or-less like this:
def process_delayed_jobs
while Delayed::Worker.new(:quiet => true).send(:reserve_and_run_one_job) do
# nothing
end
end
alias_method :deliver_delayed_emails, :process_delayed_jobs
def assert_emails(number, skip_assertion = false)
deliver_delayed_emails # start with a clean slate
if block_given?
original_count = ActionMailer::Base.deliveries.size
yield
deliver_delayed_emails
new_count = ActionMailer::Base.deliveries.size
assert_equal original_count + number, new_count, "#{number} emails expected, but #{new_count - original_count} were sent" unless skip_assertion
else
assert_equal number, ActionMailer::Base.deliveries.size unless skip_assertion
end
end
We test on this like so:
assert_emails 2 do
Model.action_that_sends_mailer
end
This code works flawlessly in Rails 2, running delayed_job 2.0.5. On Rails 3 we are running 2.1.4.
Basically, this fails every time we test it, because "0" emails get sent. The mailers themselves are upgraded properly, and I've removed the erroneous ".deliver" from the method calls to fire them off. The delayed_jobs log is empty, so it doesn't look like anything is being processed.
I feel like I'm missing some kind of key (but easy) syntax problem here. Any help is appreciated.
Edit: I should also note that I've started the delayed_job process on my local machine with both "rails delayed_job start" and "script/delayed_job start." Neither fix the problem.
Turned out to be something unrelated to delayed_job.
I recently upgraded an app i was working on to rails 3.1rc5..
For the most part it's been great but a few of my tests are having really weird issues..
For example in one of my cucumber specs i create a bunch of fake records using factory girl.. usually this works fine but it seemed that it wasn't creating the records for some reason..
So I commented out all of my factory stuff and replaced it with this:
c = Contact.new(:first_name => "SOMEONE", :last_name => "COOL", :dob => 10.years.ago, :sex => "male")
if c.save
puts "MYCOUNT: #{Contact.count}"
else
puts "EXPLOSIONS!!!"
end
Running this as part of the cucumber suite outputs this:
MYCOUNT: 0
So the contact record is obviously being saved (and passing validations) yet it is still not showing up when i call count??
Why??
I am using:
Rails 3.1rc5
rspec-rails
cucumber-rails
and
factory_girl_rails
I should also probably note i'm indexing my models using sunspot (solr API) https://github.com/outoftime/sunspot
It sounds like you've got a transaction rollback firing within your test:
1) Transaction opened
2) Contact.save succeeds (now have legit Contact instance and db record)
3) Something goes wrong, raises ActiveRecord::Rollback
4) Transactions rolls back, leaving legit Contact instance but no db record, count = 0
I don't know what would cause this related to your Rails upgrade, but perhaps this will help you find what's failing.
EDIT:
If you tail your log/test.log file you should see a Rollback occur if this is the case. You can look at the previous DB activity to get a clue as to what the last successful db operation was before it.
Not an exact science, but may help you decide if this is the case and get a rough idea of where it went sideways.
I've been trying to solve a problem for a few weeks now. I am running rspec tests for my Rails app, and they are working fine except for one error that I can't seem get my head around.
I am using MySQL with the InnoDB engine.
I have set config.use_transactional_fixtures = true in spec_helper.rb
I load my test fixtures manually with the command rake spec:db:fixtures:load.
The rspec test is being written for a BackgrounDRb worker, and it is testing that a record can have its state updated (through the state_machine gem).
Here is my problem:
I have a model called Listings. The rspec test calls the update_sold_items method within a file called listing_worker.rb.
This method calls listing.sell for a particular record, which sets the listing record's 'state' column to 'sold'.
So far, this is all working fine, but when the update_sold_items method finishes, my rspec test fails here:
listing = Listing.find_by_listing_id(listing_id)
listing.state.should == "sold"
expected: "sold",
got: "current" (using ==)
I've been trying to track down why the state change is not persisting, but am pretty much lost. Here is the result of some debugging code that I placed in the update_sold_items method during the test:
pp listing.state # => "current"
listing.sell!
listing.save!
pp listing.state # => "sold"
listing.reload
pp listing.state # => "current"
I cannot understand why it saves perfectly fine, but then reverts back to the original record whenever I call reload, or Listing.find etc.
Thanks for reading this, and please ask any questions if I haven't given enough information.
Thanks for your help,
Nathan B
P.S. I don't have a problem creating new records for other classes, and testing those records. It only seems to be a problem when I am updating records that already exist in the database.
I suspect, like nathan, transaction issues. Try putting a Listing.connection.execute("COMMIT") right before your first save call to break the transaction and see what changes. That will break you out of the transaction so any additional rollback calls will be non-effectual.
Additionally, by running a "COMMIT" command, you could pause the test with a debugger and inspect the database from another client to see what's going on.
The other hypothesis, if the transaction experimentation doesn't yield any results, is that perhaps your model really isn't saving to the database. Check your query logs. (Specifically find the update query).
These kind of issues really stink! Good luck!
If you want to investigate what you have in DB while running tests you might find this helpful...
I have a rspec test where I save #user.save and it works like a charm, but then I wanted to see if it's really saved in the DB.
I opened rails console for test environment
rails c test
ran
User.all
and as expected got nothing
I ran my spec that contains:
user_attr_hash = FactoryGirl.attributes_for(:user)
#user = User.new user_attr_hash
#user.save
binding.pry
I thought that stopping the test after save would mean that it's persisted, but that's not the case. It seems that COMMIT on the connection is fired later (I have no idea when:\ )
So, as #Tim Harper suggests, you have to fire that commit yourself in the pry console:
pry(#<RSpec::Core::ExampleGroup::Nested_1>)> User.connection.execute("COMMIT")
Now, if you run User.all in your rails console you should see it ;)
I have this block of code:
users = Array.new
users << User.find(:all, :conditions => ["email like ?", "%foo%"])
users << User.find(:all, :conditions => ["name like ?", "%bar%"])
users.flatten!
users.uniq!
puts users.to_json :include => [:licenses]
When I run it using script/console, it returns exactly what you would think it should, a JSON representation of the Array of users that I found, flattened, and uniquified. But running that same line of code as part of a search_for_users method, I get this error
TypeError in ControllerName#search_for_users
wrong argument type Hash (expected Data)
and the line referenced is the line with the .to_json call.
It's baffling me because the code is verbatim the same. The only difference is that when I'm running it in the console, I'm entering the conditions manually, but in my method, I'm pulling the query from params[:query]. But, I just tried hardcoding the queries and got the same result, so I don't think that is the problem. If I remove the :include, I don't see the error, but I also don't get the data I want.
Anyone have any idea what the issue might be?
There are a few plugins and gems that can cause .to_json to fail if included in your controller. I believe that the Twitter gem is one of them (ran into a problem with this awhile back).
Do you have "include [anything]" or "require [anything]" in this controller?
If not, I'd suggest temporarily removing any plugins you're using to troubleshoot, etc.
Finally, what happens if you replace that entire controller action with simply:
%w(1 2 3 4 5).to_json
That should help you pin down what is failing.
Whenever code in tests or the console behaves different from production environment (which is a guess... you might be running your site in development mode), this calls for a load order issue. In production environment, all the models and controllers are preloaded, in other environments they are loaded lazily when needed.
Start your console with RAILS_ENV=production ./script/console and see if you can reproduce the error this way.
As cscotta mentioned, there are a couple of gems and librarys, that can interfere with .to_json, first to mention the functionality, that you get when you require 'json'. I personally ran into several issues with that.
Hope this helps
Seb