Devise "Confirmation token is invalid" when user signs up - ruby-on-rails

Question

Using Rails 4 and Devise 3.1.0 on my web app. I wrote a Cucumber test to test user sign up; it fails when the "confirm my account" link is clicked from the e-mail.
Scenario: User signs up with valid data # features/users/sign_up.feature:9
When I sign up with valid user data # features/step_definitions/user_steps.rb:87
Then I should receive an email # features/step_definitions/email_steps.rb:51
When I open the email # features/step_definitions/email_steps.rb:76
Then I should see the email delivered from "no-reply#mysite.com" # features/step_definitions/email_steps.rb:116
And I should see "You can confirm your account email through the link below:" in the email body # features/step_definitions/email_steps.rb:108
When I follow "Confirm my account" in the email # features/step_definitions/email_steps.rb:178
Then I should be signed in # features/step_definitions/user_steps.rb:142
expected to find text "Logout" in "...Confirmation token is invalid..." (RSpec::Expectations::ExpectationNotMetError)
./features/step_definitions/user_steps.rb:143:in `/^I should be signed in$
This error is reproducible when I sign up manually through the web server as well, so it doesn't appear to be a Cucumber issue.
I would like:
The user to be able to one-click confirm their account through this e-mail's link
Have the user stay signed in after confirming their account
I have setup:
The latest Devise code, from GitHub (3.1.0, ref 041fcf90807df5efded5fdcd53ced80544e7430f)
A User class that implements confirmable
Using the 'default' confirmation controller (I have not defined my own custom one.)
I have read these posts:
Devise confirmation_token is invalid
Devise 3.1: Now with more secure defaults
GitHub Issue - Devise confirmation_token invalid
And have tried:
Setting config.allow_insecure_tokens_lookup = true in my Devise initializer, which throws an 'unknown method' error on startup. Plus it sounds like this is only supposed to be a temporary fix, so I'd like to avoid using it.
Purged my DB and started from scratch (so no old tokens are present)
Update:
Checking the confirmation token stored on the User after registering. The emails token matches the DBs token. According to the posts above, the new Devise behavior says not supposed to, and that instead it is should generate a second token based on the e-mail's token. This is suspicious. Running User.confirm_by_token('[EMAIL_CONFIRMATION_TOKEN]') returns a User who has errors set "#messages={:confirmation_token=>["is invalid"]}", which appears to be the source of the issue.
Mismatching tokens seems to be the heart of the issue; running the following code in console to manually change the User's confirmation_token causes confirmation to succeed:
new_token = Devise.token_generator.digest(User, :confirmation_token, '[EMAIL_TOKEN]')
u = User.first
u.confirmation_token = new_token
u.save
User.confirm_by_token('[EMAIL_TOKEN]') # Succeeds
So why is it saving the wrong confirmation token to the DB in the first place? I am using a custom registration controller... maybe there's something in it that causes it to be set incorrectly?
routes.rb
devise_for :users,
:path => '',
:path_names => {
:sign_in => 'login',
:sign_out => 'logout',
:sign_up => 'register'
},
:controllers => {
:registrations => "users/registrations",
:sessions => "users/sessions"
}
users/registrations_controller.rb:
class Users::RegistrationsController < Devise::RegistrationsController
def create
# Custom code to fix DateTime issue
Utils::convert_params_date_select params[:user][:profile_attributes], :birthday, nil, true
super
end
def sign_up_params
# TODO: Still need to fix this. Strong parameters with nested attributes not working.
# Permitting all is a security hazard.
params.require(:user).permit!
#params.require(:user).permit(:email, :password, :password_confirmation, :profile_attributes)
end
private :sign_up_params
end

Answer 1

So upgrading to Devise 3.1.0 left some 'cruft' in a view that I hadn't touched in a while.
According to this blog post, you need to change your Devise mailer to use #token instead of the old #resource.confirmation_token.
Find this in app/views/<user>/mailer/confirmation_instructions.html.erb and change it to something like:
<p>Welcome <%= #resource.email %>!</p>
<p>You can confirm your account email through the link below:</p>
<p><%= link_to 'Confirm my account', confirmation_url(#resource, :confirmation_token => #token) %></p>
This should fix any token-based confirmation problems you're having. This is likely to fix any unlock or reset password token problems as well.

Answer 2

A friend of mine just found this question and emailed me asking if I had figured this out, which reminded me that I never submitted my own answer, so here goes :)
I ended up resetting the token & using send to get the raw token. It's ugly, but it works in a punch for devise (3.5.1).
26 it "should auto create org" do
27 email = FG.generate :email
28 visit new_user_registration_path
29 fill_in :user_name, with: 'Ryan Angilly'
30 fill_in :user_user_provided_email, with: email
31 fill_in :user_password, with: '1234567890'
32
33 expect do
34 click_button 'Continue'
35 end.to change { Organization.count }.by(1)
36
37 expect(page.current_path).to eq(confirmation_required_path)
38 u = User.where(email: email).first
39 u.send :generate_confirmation_token
40 email_token = u.instance_variable_get(:#raw_confirmation_token)
41 u.save!
42 os = u.organizations
43 expect(os.size).to eq(1)
44 visit user_confirmation_path(confirmation_token: email_token)
45 o = os.first
46
47 u.reload
48 expect(u.confirmed?)
49 expect(page.current_url).to eq(organization_getting_started_url(o))
50 end

Answer 3

As of devise 3.5.2, the confirmation token is no longer digested during the confirmation process. This means that the token in the email will match the token in the database.
I was still having trouble with confirmations after figuring this out, but in my case it turned out to be a bug I introduced when I overrode find_first_by_auth_conditions. By fixing the bug I introduced in that method, I fixed my errors with confirmation.