Sidekiq job is scheduled, not running and just disappears - ruby-on-rails

I'm working on a Rails application that sends emails in the background using Sidekiq and Mandrill.
In my application I have 2 queues, one for user interactions with my service (irrelevant) and the second is a mailer queue.
The mailer queue is used by 2 mailers: One for managing the user profile and the other one is for interactions with our service (irrelevant).
Today I noticed that users are not getting emails for a few days already except the confirmation email that is sent by Devise.
The other queue is working flawlessly and doesn't have any problems. (So maybe Sidekiq configuration is not the problem).
For example lets take the UserMailer that notifies the user of a successful profile update:
user_mailer.rb
def update_user(user)
#user = user
mail(:to => user.email)
end
user_profile_controller.rb
def update_resource(user)
user.save
UserMailer.update_user(user).deliver_later(wait: 1.minute)
end
When I tried testing this function with deliver_now instead of deliver_later it worked and the mail was sent (So the connection with Mandrill is working).
When I test this function as it is, the job is being added to the mailer queue. When it's time to send the email it just disappears from the queue without sending the email and without leaving any exceptions in the logs.
I can see it appear in the Sidekiq Web UI and then disappear.
I guess the confirmation email is the only one being sent because it isn't delayed, but I don't know why this is happenning.
Everything is working great on my staging environment. The difference is that it's working with mailtrap and doesn't have as much jobs as the production.
Some code that might be relevant:
production.rb
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = {
:user_name => 'xxx',
:password => Rails.application.secrets.smtp_password,
:address => 'smtp.mandrillapp.com',
:domain => config.app_domain,
:port => 'xxx',
:authentication => :login
}
...
config.action_mailer.default_url_options = { :host => config.app_domain, :protocol => 'https' }
config.action_mailer.perform_deliveries = true
config.action_mailer.raise_delivery_errors = false
application.rb
config.active_job.queue_adapter = :sidekiq
Walked through these links without any solution:
http://guides.rubyonrails.org/active_job_basics.html
http://api.rubyonrails.org/classes/ActionMailer/Base.html
Haven't found any other links relevant to my issue.
Tried re-deploying
Tried restarting Sidekiq
Tried restarting Redis
Tried restarting the server itself
Tried sending email from the console - Successful
Tried sending email with deliver_later(wait:1min) - Unsuccessful

I'm suspecting it has something to do with lack of memory on the server. When I increased the memory on the server and the max_connections parameter on postgres, all the emails on that queue that were supposed to be sent, were delivered immediately

Related

Devise Confirmation Email Not Being Sent

I am developing a Rails app on Cloud9, I am trying to do something pretty simple, which is to get the Devise confirmation emails to work. For some reason, they are not being sent.
I'm using SendGrid for email functionality. I know that SendGrid is working correctly, because I can make a trivial ActionMailer that sends a test email. But for some reason, the Devise confirmation emails are not being sent. I can see that it is making a call on ActionMailer::Base.mail method, but the mail is never delivered.
I have no idea how to debug this.
Just to prevent suggestions to check stuff that is already working:
The user options include :confirmable
I am setting config.action_mailer.perform_deliveries = true
I have the following lines in config/initializers/setup_mail.rb
Code:
if Rails.env.development? || Rails.env.production?
ActionMailer::Base.delivery_method = :smtp
ActionMailer::Base.smtp_settings = {
address: 'smtp.sendgrid.net',
port: '2525',
authentication: :plain,
user_name: ENV['SENDGRID_USERNAME'],
password: ENV['SENDGRID_PASSWORD'],
domain: "sendgrid.com",
enable_starttls_auto: true
}
I'm guessing that my solution is not the intended solution, but it works (although I'm not sure why).
In the file config/initializers/setup_mail.rb, I added the following two lines:
ActionMailer::Base.layout 'mailer'
ActionMailer::Base.default from: "from#example.com"
These are probably things that should be specified in some configuration file for Devise, but I didn't see any instructions for doing so.
[Edit] It is only line 2 that is important. It doesn't matter that the "from" address is a bogus email address, but if it is missing, the email does not get delivered.

How to set up mailer in Rails app for production environment on Heroku

I need to use a mailer for sending out emails to users to set their passwords to the "recoverable" function of Devise and active admin. On the development environment I have done this by adding the following to these files:
config/environments/development
#Added per active admin install instructions
config.action_mailer.default_url_options = { :host => 'localhost:3000' }
#These settings are for the sending out email for active admin and consequently the devise mailer
ActionMailer::Base.delivery_method = :smtp
ActionMailer::Base.perform_deliveries = true
ActionMailer::Base.raise_delivery_errors = true
ActionMailer::Base.smtp_settings =
{
:address => 'smtp.gmail.com',
:port => 587,
:domain => 'gmail.com', #you can also use google.com
:authentication => :plain,
:user_name => 'XXXXX#gmail.com',
:password => 'XXXXXXX'
}
How do I get the same functionality for the production environment? I want to deploy my app to Heroku. What files and code would I need to add?
All configurations you have set in Development mode will work EXCEPT you will need to reconfigure the default mailer url.
So.
Copy-paste your settings from development.rb.
Point your default mailer to your heroku app:
config.action_mailer.default_url_options = { :host => 'YOURAPPNAME.herokuapp.com' }
Also, be careful of any email limits your smtp may have when moving to production. It's hard to trigger gmail's smtp limits while developing, for example, but they could be more easily triggered in production.
If it works in development mode, then it will work in production mode.
Supposing everything is setup correctly, resetting a password in development will already send an actual email using your gmail account.
Devise only relies on the mailer config setup correctly (which you have done), and configuring devise to allow password reset, and possibly another setting for the From field of the email.
This should work fine!
As long as config/environments/production.rb has the same thing with an exception. The default_url_options should have a :host value of 'localhost' only in development and 'YOURAPPNAME.herokuapp.com' in heroku production.
i.e.
config.action_mailer.default_url_options = { :host => 'YOURAPPNAME.herokuapp.com' }
Remember to unlock captcha on gmail, otherwise it won't send email from heroku (unknown source). You can do that by going to this link: http://www.google.com/accounts/DisplayUnlockCaptcha
Just as a suggestion, I'd say move this from environments.rb
ActionMailer::Base.perform_deliveries = true
ActionMailer::Base.raise_delivery_errors = true
and place is in environments/development.rb as
config.action_mailer.perform_deliveries = true
config.action_mailer.raise_delivery_errors = true
It's not needed in production.
See Net::SMTPAuthenticationError when sending email from Rails app (on staging environment) for more information in regards to gmail seeing heroku as an unknown host.

Rails / Devise / Mailer: Throwing 'config.action_mailer.default_url_options' error on attempted registration

It's just been one problem after another today. I just deployed to my production server and testing it out, with issues whenever anything involves email. Particularly with Devise's confirmable registration email, whenever I sign up for an account, the following error is thrown up in the log...
ActionView::Template::Error (You can no longer call ActionMailer::Base.default_url_options directly. You need to set config.action_mailer.default_url_options. If you are using ActionMailer standalone, you need to include the routing url_helpers directly.):
Looking at my config/environments/production.rb, I have the following set...
config.action_mailer.default_url_options = { :host => 'localhost' }
So config.action_mailer.default_url_options IS being used, but is being completely ignored. I tried changing the host to '127.0.0.1' and my server's IP addy, but it wouldn't have it. Threw it in config/application.rb, but no go. I hunted in my project for any other lines declaring default_url_options, but it's only set in the production.rb file. I Googled and found a post suggesting to set config.cache_classes to false, but negative there as well.
Now here is what's crazy...I got the same error whenever I tried to submit a brand new comment in my project (set up to send an email to confirm the email address if you are a first time commentor). I took OUT config.action_mailer...etc from the production.rb file, and viola - my comment mailer worked and sent the email just fine!
Still, Devise is being absolutely stubborn and keeps throwing the above error at me. Any ideas on why? BTW, all my gems are up to date. Below is my production.rb file.
--- UPDATE ---------------
Have made a little progress, but this is just weird. When I start the server and attempt to do something that will send mail (user registration, comment), I get the ActionMailer::Base error message above. When I try the action AGAIN...it works and the mail gets sent. From that point on (until Passenger and/or the server is rebooted), all mailer actions work fine.
But not without yet another issue (it never ends)...for some reason the email body is completely blank.
I ended up hitting this problem once again when I attempted to test in the test database, and I was determined once and for all to figure out why I was getting this error, and why no one else seemed to have come across it (very little results from Google).
After completely tearing down my project and reconstructing it, I finally pinpointed the issues to the Sitemap-generator plugin I had installed. Once I removed all traces of that, the above error finally ceased.
I'm working on this issue for Fat Free CRM, and I realized that this error happens because Rails 3 lazy-loads classes. All you need to do is 'touch' the ActionMailer::Base class in your environments/**.rb files, and it will load the class. Then you can call ActionMailer::Base.default_url_options from a controller, etc. (We do this so that we can automatically set the mailer host from request.host_with_port)
So my environment files now look like this:
FatFreeCRM::Application.configure do
...
ActionMailer::Base
end
I believe I have solved this issue...though I'm not "exactly" sure what did it. It very well may have been config.cache_classes = true which I commented out, and although I was concerned this would effect my site caching or something else, it doesn't seem to have broke it.
In addition, the empty email body I solved by switching from :sendmail to :smtp.
Here is my final production.rb file, hopefully this might be of use to someone with a similar issue in the future.
MyProject::Application.configure do
# Commented out, causes 'ActionView::Template' error
#config.cache_classes = true
config.whiny_nils = true
config.consider_all_requests_local = true
config.action_view.debug_rjs = true
config.action_controller.perform_caching = true
config.cache_store = :mem_cache_store
config.active_support.deprecation = :log
config.action_dispatch.best_standards_support = :builtin
config.action_mailer.raise_delivery_errors = true
Sunspot.config.solr.url = 'http://127.0.0.1:8080/solr'
Paperclip.options[:command_path] = "/usr/bin/"
config.action_mailer.perform_deliveries = true
config.action_mailer.delivery_method = :smtp
ActionMailer::Base.smtp_settings = {
:address => "smtp.gmail.com",
:enable_starttls_auto => true,
:port => 587,
:authentication => :plain,
:user_name => "user#domain.com",
:password => 'password'
}
config.action_mailer.default_url_options = { :host => 'dev.mydomain.com' }
config.time_zone = "Central Time (US & Canada)"
end

Rails 3/delayed_job - Wanted: Basic example of delayed mail

I've been trying to figure out how to send delayed mail using delayed_job with rails 3. I've tried pretty much every combination of feasible possibilities I can think of - I can get the mail to run in the background, I just can't get it to delay the sending to a future time. The delayed_jobs table in the db clears the tasks, the log says 'sent', the delayed_job task processor picks up the task & says sent without failure...but the mail is either:
sent immediately, or
simply doesn't arrive
if I try to send in the future.
If anyone could offer a bare-bones example of a rails 3 delayed_job that sends mail in the future, I'd be really appreciative. I'm sure lotsa folks do this so I suspect I'm missing something obvious. One (of countless) combinations I've tried below:
delayed_job: 2.1.2
rails: 3.0.3
actionmailer: 3.0.3
Controller:
class TestmailerController < ApplicationController
def index
Testmailer.delay.test_mail
end
end
Mailer:
class Testmailer < ActionMailer::Base
def test_mail
mail(:to => '(myemailaddress#removedforprivacy.com', :from => '(removedforprivacy)#gmail.com', :subject => 'Testing Delayed Job', :content_type => 'text/plain').deliver
end
handle_asynchronously :test_mail, :run_at => Proc.new { 2.minutes.from_now }
end
relevant mail part of config/environments/development.rb:
# Don't care if the mailer can't send
config.action_mailer.raise_delivery_errors = true
# Print deprecation notices to the Rails logger
config.active_support.deprecation = :log
config.action_mailer.default_url_options = { :host => 'localhost:3000' }
ActionMailer::Base.smtp_settings = {
:address => "smtp.gmail.com",
:port => 587,
:domain => "gmail.com",
:user_name => "(removedforprivacy)",
:password => "(removedforprivacy)",
:authentication => "plain",
:enable_starttls_auto => true
}
Job command:
rake jobs:work
I agree with andrea - I was having this exact problem, and after switching my local development database from sqlite to mysql, I can run code like
Emailer.delay({:run_at => 5.minutes.from_now}).welcome(#user)
and it sends the email 5 minutes later. Note that you might need a bigger delay than five minutes (in case of time zone weirdness) to be sure it is working.
I found in Rails 3 with mongoid that removing the handle_asynchronously line gets it to work. I was having all kinds of problems, and it appeared that delayed_job wasn't recognizing any objects within my Emailer class. Removing handle_asynchronously fixed it.
Both using the .delay method and setting handle_asynchronously :test_mail is redundant. Try removing the .delay method from your code. use simply
Testmailer.test_mail # without .deliver due to a delayed_job issue
However, I ran some test on your configuration and when using sqlite, run_at is simply ignored (do not know why), but when using mysql2 everything works fine.

How do I configure Rails to disable sending real emails out while in staging?

I'm on Heroku, and emails don't get sent out in development, but are properly being sent in production. I'd like to run a seperate staging instance on Heroku, but don't want emails being sent out (just to a log).
This line in test.rb tells ActionMailer not to deliver emails:
config.action_mailer.delivery_method = :test
Instead, they are accumulated in the ActionMailer::Base.deliveries array.
You'll need to set up a staging environment for your application and configure Heroku to use that environment on your staging instance.
Applications that use the Mail gem (including rails >= 3.0 projects) can use the safety_mailer gem. Specify a domain (or set of domains, or magic word in email address) email is allowed to go to, and email to all other domains is silently dropped.
https://github.com/cluesque/safety_mailer
Add the gem to your Gemfile, specifying groups (probably not production) to include it in.
gem "safety_mailer", :group => :development
Don't forget to bundle install to install
In your environment file config/environments/development.rb configure it, and some regular expressions.
config.action_mailer.delivery_method = :safety_mailer
config.action_mailer.safety_mailer_settings = {
allowed_matchers: [ /mydomain.com/, /mytestacct#gmail.com/, /super_secret_test/ ],
delivery_method: :smtp,
delivery_method_settings: {
:address => "smtp.mydomain.com",
:port => 25,
:domain => "mydomain.com",
:authentication => :plain,
:user_name => "mydomain_mailer#mydomain.com",
:password => "password"
}
}
... and now, email to anyone#mydomain.com, mytestacct#gmail.com, bob+super_secret_test#yahoo.com all get sent
and email to other recipients (like the real users in the production database you copied to a test server) is suppressed.
You might be interested in mailtrap.io (disclaimer: I am affiliated with this product). It is a perfect tool to test email deliveries in development and production. All you have to do is set mailtrap.io as an smtp server in your staging environment config:
config.action_mailer.smtp_settings = {
:address => "mailtrap.io",
:port => 2525,
:authentication => :plain,
:user_name => "LOGIN",
:password => "PASSWORD"
}
Having this all your test emails sent in staging env will be stored in mailtrap for view and sharing. But non of them will be sent to the real addresses. You can use it in development as well.
And by way - it's totally free!
put this in your environment.rb file
config.action_mailer.delivery_method = :test
It should stop sending mail to the mail server, I think there is a :log option, but I have not tried it out.
I see people suggest using Mailtrap.io. Good alternative is Debug Mail. Using is quite simple.
We use maildev, which you can install locally. Great for development and staging environments, easy to install in a variety of tech stacks.
Depending on your choices
If you want a convenient way of receiving emails for debugging, etc. I recommend https://github.com/fgrehm/letter_opener_web, which will save emails locally, and provide an URL to browse emails that were sent. No email is sent outside, and you can very conveniently see the output in your browser
If you want to be able to open email files with your email clients, you should choose a :file adapter for ActionMailer (configure in config/environments/your_env.rb)
If you want a real production-like environment, I'd suggest to configure an email interceptor that would rewrite the TO/CC/BCC to a real mailbox of your choice, this way you can keep and test your original ActionMailer adapter
if Rails.env.staging?
class TestEmailsInterceptor
def self.delivering_email(mail)
mail.to = ['My Test Box <test-box#example.com>']
# remove bcc, cc, etc.
end
end
ActionMailer::Base.register_interceptor(TestEmailsInterceptor)
end

Resources