"ActionController::InvalidAuthenticityToken" error when using form_with - ruby-on-rails

Question

I have a form that looks like this:
<%= form_with(url: star.starname, method: :post, local: true) do |f| %>
<% star.availabilities.each do |avail| %>
<%= f.label avail.time_slot %>
<%= radio_button_tag(:time_slot, avail.time_slot) %> <br>
<% end %>
<%= f.submit "Create" %>
<% end %>
Immediately after form submission:
Notes:
This is occurring in an app (not an API), so sessions are important, hence CSRF protection must be left on.
The problem occurs in chrome, incognito, and safari.
I have tried logging in with different users and clearing cookies (in case it was being caused by a stale token)
Some more of the error message:
Started POST "/talljohn" for ::1 at 2020-09-16 10:06:21 +1000
Processing by StarsController#book as HTML
Parameters: {"authenticity_token"=>"P++4a+giwUBqZgCLfMwqKpMu0EGitd8zTOi5RWsnxpKlNcjiuU6hd3ebbIC/IOxlL74RJIvrq+yDuA1ZtfcvFw==", "time_slot"=>"2020-09-16 01:00:00 UTC", "commit"=>"Create", "starname"=>"talljohn"}
Can't verify CSRF token authenticity.
Completed 422 Unprocessable Entity in 1ms (ActiveRecord: 0.0ms | Allocations: 655)
ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken):
actionpack (6.0.3.2) lib/action_controller/metal/request_forgery_protection.rb:215:in `handle_unverified_request'
actionpack (6.0.3.2) lib/action_controller/metal/request_forgery_protection.rb:247:in `handle_unverified_request'
devise (4.7.2) lib/devise/controllers/helpers.rb:255:in `handle_unverified_request'
actionpack (6.0.3.2) lib/action_controller/metal/request_forgery_protection.rb:242:in `verify_authenticity_token'
activesupport (6.0.3.2) lib/active_support/callbacks.rb:428:in `block in make_lambda'
activesupport (6.0.3.2) lib/active_support/callbacks.rb:200:in `block (2 levels) in halting'
actionpack (6.0.3.2) lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'
activesupport (6.0.3.2) lib/active_support/callbacks.rb:201:in `block in halting'
Update
I reverted back to the last working version of the form, which was exactly the same as above but without , local: true. Then it suddenly works! (no errors).
I thought local: true (or remote: false) simply turns off ajax form submission. So I don't understand why that would make any difference (or have anything to do with CSRF), it seems that those two aspects are unrelated and it isn't clear why these two concepts would have any affect on eachother
Update 2
I later realised that another untouched previously working form also produced this error. It had not been changed in any way. I tried it in chrome incognito, and it produced the error. Half an hour later (without changing any code) I tried it again in the same browser and it worked. This (very) strange behaviour makes me think it's something to do with sessions, cookies or caching. I will report back if I learn anything further
Update 3
After reading Sarah's solution adding protect_from_forgery prepend: true to the application controller (I tried both before and after before_action :authenticate_user!), the same error message appears in the logs, the POST request isn't actioned, but the app redirects to the home page. I.e. upon POST I see:
Can't verify CSRF token authenticity.
Completed 401 Unauthorized in 1ms (ActiveRecord: 0.0ms | Allocations: 444)
Started GET "/users/sign_in" for ::1 at 2020-09-17 21:08:42 +1000
Processing by Devise::SessionsController#new as HTML
User Load (0.5ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 1], ["LIMIT", 1]]
Redirected to http://localhost:3000/
Filter chain halted as :require_no_authentication rendered or redirected
Completed 302 Found in 3ms (ActiveRecord: 0.5ms | Allocations: 1900)
Update 4
I attempted to manually clear the rails fragment cache (with Rails.cache.clear ). But the result is exactly the same before/after clearing the fragment cache.

Answer 1

I remember running into something like this and adding protect_from_forgery prepend: true before any user authentication to the ApplicationController solved it.

Answer 2

TL;DR: after 2 weeks' of debugging attempts, I turned off turbolinks and the problem went away.
Aside from turning off turbolinks, another solution appears to be (mentioned here) adding this to application.js
$(document).on('turbolinks:load', function(){ $.rails.refreshCSRFTokens(); });
Previous answer
The issue kept reemerging. I have tried the following six things but it still hasn't fixed it
1. Clear the fragment cache
Rails.cache.clear (warning because it clears the cache, it will also remove things like sidekiq jobs etc). This will remove the stale token and refreshing the app in the browser will return things to normal, and the form should submit (a simple 'resubmit' won't work, so go back to the form page, refresh, then submit and it should work)
2. Hard refresh page
Press cmd + opt + j to bring up the developer console, then right click on refresh and select 'Empty Cache and Hard Reload'
3. Delete site cookies
Right click on the tiny icon to the immediate left of the url (it will be a lock if using https, or the letter 'i' if using http). Go into each of categories listed (e.g. 'Cookies', 'Site Settings' etc) and delete them all
4. Delete cookies for other urls that point to the same site
For example, if your site is www.example.com, and it's hosted on heroku at www.example.herokuapp.com, then delete cookies for that second url as well
5. Delete cookies for localhost
I deleted localhost cookies just to be sure
6. Testing in completely isolated instances of chrome

Answer 3

For me, the solution was to switch to a more specific Cookie Strategy:
# config/initializers/new_framework_defaults_6_1.rb
# Before: i've used None, but this leads to broken Cookies somehow. Strict or Lax seems to work.
Rails.application.config.action_dispatch.cookies_same_site_protection = :strict