Last week I upgrade Fedora to the brand new 28 release, which came with a mongodb upgrade to 3.6. See How to repair mongodb service after an upgrade to Fedora 28? for how I managed to resolve my first problem which was that mongod would no longer start. Now I'm facing an other problem on the Rails application that use this same database.
This most probably is unrelated to the mongodb upgrade, but I thought it might worth providing that context and don't miss a solution for not providing enough of it.
So since the system upgrade any login attempt on this Rails project will fail with a BCrypt::Errors::InvalidHash in Devise::SessionsController#create
error, raised at bcrypt (3.1.11) lib/bcrypt/password.rb:60:ininitialize'`. Analyzing further in a Rails console of the project, it seems any call to this method will fail:
> BCrypt::Password.create('TestPassword')
BCrypt::Errors::InvalidHash: invalid hash
from /home/psychoslave/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bcrypt-3.1.11/lib/bcrypt/password.rb:60:in `initialize'
I tried to bundle uninstall/reinstall bcrypt, and even use the github repository version of the bcrypt gem instead, but it didn't change anything.
Looking at /home/psychoslave/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bcrypt-3.1.11/lib/bcrypt/password.rb:60:ininitialize'`, the problem seems that the hash is not valid.
# Initializes a BCrypt::Password instance with the data from a stored hash.
def initialize(raw_hash)
if valid_hash?(raw_hash)
self.replace(raw_hash)
#version, #cost, #salt, #checksum = split_hash(self)
else
raise Errors::InvalidHash.new("invalid hash")
end
end
And the corresponding test is as follow:
def valid_hash?(h)
h =~ /^\$[0-9a-z]{2}\$[0-9]{2}\$[A-Za-z0-9\.\/]{53}$/
end
The hash itself is created through BCrypt::Engine.hash_secret(secret, BCrypt::Engine.generate_salt(cost)), which in the platform I use call __bc_crypt(secret.to_s, salt), which seems to be calling bcrypt-3.1.11/ext/mri/bcrypt_ext.c.
More importantly, adding a binding.pry in the valid_hash? method, it's possible to see what the hash value returned for a call to BCrypt::Password.create('TestPassword'), it's actually a rather long string whose start seems usual, but end up with what is most likely misgenerated sequence:
"$2a$10$Eb1f8DSkGh4G1u5GicyTYujBk6SwFXKYCH.nqxapmBlqJ0eFYdX32\x00\x00\x00\x00\xD1F\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00T\xBD\x02\x00\x00\x00\x00\x00\xF1V\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\xE2\xB0\x02\x00\x00\x00\x
00\x00AW\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00 \x04\x00\x00\x00\x00\x00\x00\x86\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\xB5\xF8\x0E\x00\x00\x00\x00\x00q\xD8\x01\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00…"
I can provide a dump of a whole hash if it might be of any interest (around 32Ko!).
Here is a circumvent solution which make rspec of bcrypt pass all tests successfully again.
This is really a uggly hack while waitting a proper solution, but does the job until then. Just change ~/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/bcrypt-3.1.11/lib/bcrypt/engine.rb (path to adapt, of course), line 51 from:
- __bc_crypt(secret.to_s, salt)
+ __bc_crypt(secret.to_s, salt).gsub(/(\n|\x00).*/, '')
That is, trunk the string starting at the first "\x00" or "\n" occurrence, if any.
Credit note: this version of the hack was proposed by Andrey Sitnik, and I replaced the one I proposed here independently, before discovering it.
After that, BCrypt::Password#create will function again:
> BCrypt::Password.create('TestPassword')
=> "$2a$10$YPRnQF3ZihXHpa9kSx7Mpu.j28PlbdwaNs2umSQvAGkS.JJ.syGye"
I had this issue with a (very) old application and BCrypt 3.1.10. Upgrading to 3.1.12 resolved the issue. :)
Related
I built a pretty small Rails 5.1.4 (Ruby 2.3.1) application. Once I've deployed it to production, I'm getting this particular error from time to time:
RuntimeError: can't add a new key into hash during iteration
Pointing here:
# rack/request.rb, line 67
def set_header(name, v)
#env[name] = v
end
I understand, this error happens when you're attempting to add new key to the hash while iterating over that hash. Since #env is a hash, it makes sense. But:
in a stacktrace I found nothing related to iterations over #env, it's a dead simple chain of app.call(env) calls.
this error occurs not always, but just once per hour or two, so this is also super weird to me
I can't reproduce it locally: I've tried to load server with multiple request hits, assuming this might be thread-safety issue, but locally it works like a charm...
Full stacktrace only consists of rack middlewares can be found here:
https://gist.github.com/Nattfodd/e513122400b4115a653ea38d69917a9a
Gemfile.lock:
https://gist.github.com/Nattfodd/a9015e9204544302bf3959cec466b715
Server is running with puma, config is very simple: just amount of threads and workers:
threads 0, 5
workers 5
My current ideas are:
one of monitoring gems has a bug (sentry-raven, new_relic)
concurrent-ruby has a bug (I read about one, but it was fixed in 1.0.2, and actual version I'm using for Puma is 1.0.5)
something super stupid, I missed, but I have no idea where to look, since the controller's action contains 3 lines of code and application config is mostly default...
this is something config-related, since backtrace does not contain the controller at all...
can you can paste the full stack-trace?
My assumption is set_header is getting called from method which iterating env.
When replicating an app to production, my POSTGIS table columns started misbehaving, with Rails informing me there was an "unknown OID 26865" and that the fields would be treated as String.
Instead of current_pos yielding e. g.
#<RGeo::Geographic::SphericalPointImpl:0x22fabdc "POINT (13.39318248760133 52.52908798020595)"> I would get 0101000020E6100000FFDD958664C92A403619DEE6B2434A40. It looked like the activerecord-postgis-adapter was not installed, or installed badly, but I eliminated that possibility by testing for the existence of data type RGeo::Feature::Point and by test-assigning
current_pos = "POINT (13.39318248760133 52.52908798020595)"
to the field - which proceeded without error but then yielded another incomprehensible hex string like the above.
Also, strangely enough, POSTGIS was working correctly within the database, e.g. giving correct results for a ST_DISTANCE query. A very limited problem thus, where writing, writing-parsing (from Point to hex format), manipulating by SQL and reading all worked, only the parsing upon read didn't.
When I tried to use migrations to ensure the database column would have the correct type, the migrations failed, giving
undefined method `st_point' for #<ActiveRecord::ConnectionAdapters::PostgreSQL::TableDefinition:0x00000005cb80b8>
I spent several hours trying all kinds of solutions, even re-installing the server from scratch, double-checking version numbers of everything, installing a slightly newer version of Ruby and a slightly older version of POSTGIS (to match my other environment), exporting the database and starting with a clean one, and so on. After I had done migrations and arrived at the "undefined method st_point" error, I was finally able to find the solution via Google, way down in a Github issue, and it's really simple:
In config/database.yml, swap out postgres:// for postgis:// in the database url. If you're using Heroku, this may require some ugly manipulation:
production:
url: <%= ENV.fetch('DATABASE_URL', '').sub(/^postgres/, "postgis") %>
So silly...
Do not forget to add activerecord-postgis-adapter to your Gemfile so #Sprachprofi's solution can run.
Since upgraded to Ruby 2.2.0 I get the following message in my tests:
invalid salt (BCrypt::Errors::InvalidSalt)
I didn't find any upgrade notice helping me to understand the problem. I'm using Rails 4.1.8 and Sorcery 0.8.6.
Anybody else having this problem?
MORE Details:
I'm using Sorcery and not Devise. The encrypted data is the password.
It all started in Cucumber tests, in 2 cases:
When I used to send the #user to the mailer to prepare the data for the mails. Here was the code:
UserMailer.passphrase_reset_notification(#user).deliver
Which generated the exception with the message I wrote in the initial message. As a workaround instead of sending the #user I sent the fields I needed and it worked. Here's the new code:
UserMailer.passphrase_reset_notification(#user.name, #user.email).deliver
But the second case is the sign up. It failed in dev and I had to add :salt to user_params to fix it. But it does not fix the thing in the test env.
There's no stack trace, just that one liner message with the lines of my scenario leading to the error.
And I press "Sign up"
invalid salt (BCrypt::Errors::InvalidSalt)
./app/controllers/users_controller.rb:66:in block in create'
./app/controllers/users_controller.rb:64:increate'
./app/controllers/application_controller.rb:120:in scope_current_tenant'
./features/step_definitions/web_steps.rb:53:in/^(?:|I )press "([^"]*)"$/'
features/users/sign_up.feature:149:in `And I press "Sign up"'
I removed the "null: false" for the field "salt" in the user table, as suggested by a community member in a post on a more or less similar issue, it didn't help either.
My main question is still the same: what the Ruby new version (2.2.0) has to do with this? And what might be the other surprises if I upgrade the prod?
I just fixed this. Turned out it had to do with serializing an object with has_secure_password (which uses bcrypt-ruby)
More specifically, something like the following was causing the issue with Sidekiq as it tried to serialize arguments into objects for Redis queueing.
#user = User.new(
:firstname => 'Scott',
:lastname => 'Klein',
:password => 'mypass',
:password_confirmation => 'mypass'
)
#user.save!
# broken
# note that #user.password can still be called here
# and sidekiq will attempt to serialize this whole object using YAML
# and this is the serialization issue that barfs (in the depths of YAML)
UserMailer.delay.new_user_signup(#user)
# fixed
# i just passed the id and then recalled the user record in the mailer class
UserMailer.delay.new_user_signup(#user.id)
I've had similar problem. Investigation made me conclude that it's bcrypt not playing well with Psych (that's the Ruby system library for generating and parsing YAML).
There's an open bcrypt issue now. Waiting for gem author to fix it.
** FIXED **
The problem, at least mine, is fixed. I just upgraded the bcrypt gem from 3.1.
9 to 3.1.10 and it was it! Thanks Oleg to have created an issue on bcrypt account.
I am having many problems running DJ. Primarily, I cannot get delayed_job running any methods that change the database. I am testing locally with a sqlite3 database, DJ 3.0.0, and I even added the delayed_job_active_record gem.
I have, for example, tried to run the following method in the background:
#user = User.find(1)
#user.delay.recorder_method
Where this method is:
def recorder_method
self.relevant_field +=1
update
end
This creates a delayed job, and the handler has the appropriate info for the user and the appropriate method name. The script runs the job, thinks it has succeeded and thus deletes the record from the delayed_job table. BUT the user is unchanged (the database is unchanged).
What on earth is going wrong? Note that, when I run the same code ("#user.delay.recorder_method") from rails console, it works.. and the difference is the handler created by the rails console call is:
--- !ruby/object:Delayed::PerformableMethod
object: !ruby/ActiveRecord:User
attributes:
... (attribute info and rest of file)
Whereas the one created in by the call in a controller action is:
--- !ruby/struct:Delayed::PerformableMethod
object: !ruby/ActiveRecord:User
attributes:
Not the difference in line 1 of both things (ruby/object vs ruby/struct.. the former works and the latter doesn't). Maybe this is something that might signal what is going wrong. Any ideas, anybody?
Your Rails app is running an old version of the DJ gem and your console is running a new one. About six months ago, Delayed::PerformableMethod was refactored from a Struct into a regular class: https://github.com/collectiveidea/delayed_job/commit/7b8a79a72c0ee5d8bac4bc0b183d1cce9cedff85 (So your Rails app is running a gem at least six months old and your console is running one newer than that).
This is an easy fix. First update the appropriate line in your Rails Gemfile. You'll see a line like this:
gem 'delayed_job_active_record'
If it has a version specification, make sure it's updated to the newest. Then from the command line (in the Rails root), run:
bundle update delayed_job_active_record
For anyone interested, removing the following lines from boot.rb solved this issue for me:
require 'yaml'
YAML::ENGINE.yamler = 'syck'
I am apparently having a huge problem switching from the plugin version of Paperclip to the gem version in my app. It's been my impression that there should be no difference whatsoever between a plugin and a gem of a specified version. However, I'm not seeing this as an easy transition at all.
Rails 2.3.11, Ruby 1.8.7
The plugin version I am using is version 2.3.3 and was upgraded on August 2, 2010. Attempting to update this to the gem of the same version basically killed all my tests, not being able to load a factory model which did not have its attachment loaded. It appeared that validate_attachment_content_type was also attempting to validate the attachment presence, and couldn't find it, so everything just started breaking. Again, with the plugin there are no problems and I haven't had any problems in all this time we've been using it. On the other hand, this problem seems to not occur past version 2.3.4. That's a whole other set of problems.
Basically, in all versions from 2.3.4 and up I get the problem below:
can't convert nil into String
/home/joshua/.rvm/gems/ruby-1.8.7-p334#paperclip_upgrade/gems/paperclip-2.3.15/lib/paperclip/storage/s3.rb:163:in `extname'
/home/joshua/.rvm/gems/ruby-1.8.7-p334#paperclip_upgrade/gems/paperclip-2.3.15/lib/paperclip/storage/s3.rb:163:in `to_file'
/home/joshua/.rvm/gems/ruby-1.8.7-p334#paperclip_upgrade/gems/paperclip-2.3.15/lib/paperclip/attachment.rb:94:in `assign'
/home/joshua/.rvm/gems/ruby-1.8.7-p334#paperclip_upgrade/gems/paperclip-2.3.15/lib/paperclip.rb:279:in `avatar='
/home/joshua/railscamp/app/app/models/organization.rb:311:in `copy_membership'
in all my tests that access my organization model.
The apparent offending code in this case is attempting to clone a membership model from one organization to another, with the * line being the offending call.
def copy_membership(membership)
m = membership.clone
u = m.user.clone
u.organization = self
m.organization = self
begin
m.avatar = membership.avatar *
rescue RuntimeError
m.avatar = nil
end
m.user = u
m.save
m
end
Does this make any sense to anyone? Why would the plugin work, but the gem of the same version just wrecks everything?
Update: I also don't appear to have any paperclip rake tasks available. Any ideas?
As it turns out, we should have been checking whether the filename is valid or not, rather than depending on a generic runtime error for detecting avatar presence.