I'm trying to debug stale entries in a cached view in a Rails (5.0.0.beta2) running on Heroku. I'd like to look at the entries in the cache to confirm that they are named the way that I expect and are getting expired when they should.
Is there any way to do this? I found this question, HOW do i see content of rails cache, which suggests Rails.cache.read("your_key"). So, using bin/rails c (on Heroku) I tried:
Rails.cache.read(User.find(19).cache_key) => nil
Where 19 is the :id of one of the users for whom I'm seeing stale data. This has me kind of stumped...
If I try:
User.find(19).cache_key => "users/19-20160316151228266421"
But when a cache entry is supposedly expired the log line looks like:
Expire fragment views/users/19-20160316151228266421 (0.2ms)
So I tried doing a Rails.cache.read on that path, this also returned nil – I also tried doing the same with a user that had not be expired, and got nil again.
I'm wondering if that difference in path signals a problem, or if there is a way to see the path of the key that is created (I've been assuming that it matches at least the part after the slash).
Cache has the following instance variables:
[:#options, :#data, :#key_access, :#max_size, :#max_prune_time, :#cache_size, :#monitor, :#pruning]
You can examine the data with:
Rails.cache.instance_variable_get(:#data)
Related
I've got an application that's been running in production for many months now, with tens of thousands of attachments. This morning, I tried to do an operation on one of these attachments, and got the following error:
Azure::Core::Http::HTTPError: BlobNotFound (404): The specified blob does not exist.
I can easily recreate this blob, but this situation makes me want to write a script to check the integrity of all of my attachments, and verify that no others have gone missing. (I expect that this was a transitory network error, and I expect to find very few, but I need the peace of mind.)
I can see that there is a method to call that seems to do exactly what I need: exist?(key), which is documented here:
https://github.com/rails/rails/blob/master/activestorage/lib/active_storage/service/disk_service.rb
However, I can't figure out how I'm supposed to call it. According to this, it's implemented as an instance method. So how do I reference my Rails application's active ActiveStorage instance (depending on environment) to use this method?
I wondered if it were possible to backtrack to get the instance of ActiveStorage::Service::AzureStorageService (in production), and found this answer on finding each instance of a class.
From there, I found that I could:
asass = ObjectSpace.each_object(ActiveStorage::Service::AzureStorageService).first
Which then allowed me to:
2.5.5 :015 > asass.exist?(c.json.blob.key)
AzureStorage Storage (313.3ms) Checked if file exists at key: ....PTLWNbEFLgeB8Y5x.... (yes)
=> true
Further digging around in the bowels of Rails' GitHub issues led me to see that the reference to the ActiveStorage service instance can be reached through an instance of a stored object, and that I could always be using the send() method to call one of its methods:
2.5.5 :045 > c.json.blob.service.send(:exist?, c.json.blob.key)
AzureStorage Storage (372.4ms) Checked if file exists at key: ....PTLWNbEFLgeB8Y5x.... (yes)
=> true
I'm sure both approaches are lighting up the same code paths, but I'm not sure which one I prefer at the moment. I still think there must exist some way to traverse the Rails instance through app. or Rails.application. to get to the ActiveStorage instance, but I haven't been able to suss that out.
I have a Rails 5.2 app that's configured to use the new much touted feature of recyclable cache keys.
I can confirm the setting is enabled in the console:
Rails.application.config.active_record.cache_versioning
=> true
ActiveRecord::Base.cache_versioning
=> true
BlogPost.cache_versioning
=> true
With this setting, blog_post.cache_key now returns a stable string, because the cache_version is actually stored inside the cache entry (as this article details):
blog_post.cache_key
=> "blog_posts/10317"
blog_post.cache_version
=> "20190417193345000000"
But the problem is, even tough everything works as expected in the console, I can't seem to see this working watching the server logs, because it keeps generating cache_keys that contain the cache_version:
In my view:
<% cache(['blog_post_list_item_v2', blog_post, I18n.locale, browser.device.mobile?]) do %>
...
<% end %>
In the server logs:
Rendered blog/blog_posts/_blog_post_list_item.html.erb (2.5ms) [cache miss]
Read fragment views/blog/blog_posts/_blog_post_list_item:0bdff42a9193ea497e5ed4a9cc2f51e8/blog_post_list_item_v2/blog_posts/10317-20190417193345000000/pt-br/ (0.5ms)
As you see, the cache key should be .../blog_posts/10317/, but it actually contains the timestamp.
After debugging through the Rails code, I could confirm that the key was actually stable. What gets printed in the server log includes the version for debugging purposes only, but the key being stored on your cache doesn't actually contain the version.
The version is stored instead within the serialized object in the cache, which is an instance of ActiveSupport::Cache::Entry and contains an attr_reader :version. So, if you're like me, you'd assume that the cache (for instance, raw HTML) was stored directly in memcached, but it actually is stored in the value attribute of that ActiveSupport::Cache::Entry (which also has the version attribute if you have cache_versioning turned on), and that entire object is saved serialized into the cache.
If you want to confirm it yourself, you can check your own memcached realtime log. If you're on a mac, first stop it (I'm assuming it was installed with homebrew) with brew services stop memcached, start it on the foreground with verbose mode with memcached -vv and take a look at the keys requested by rails. After you finish your study, brew services start memcached will re-enable memcached as a daemon.
Also, if you are migrating from the old way (without recyclable cache keys), you should wipe your cache first with Rails.cache.clear in the console. Remember to do that in production as well.
If you want to understand more about how this works, a good read is https://dzone.com/articles/cache-invalidation-complexity-rails-52-and-dalli-c, but debugging through the rails code with binding.pry was what got things clear to me.
In a nutshell, it's a very brilliant implementation in my opinion, and the cache recycling just makes it so much better (the article quotes DHH saying that 'We went from only being able to keep 18 hours of caching to, I believe, 3 weeks. It was the single biggest performance boost that Basecamp 3 has ever seen.')
I have this setup
config.cache_store = :redis_store, ENV['REDIS_CACHE_URL']
$ redis-cli
127.0.0.1:6379> set random_key 1
OK
Now I go to the console and do Rails.cache.clear which returns nil
And I am still able to access the key random_key in the redis-cli. It did not clear the cache.
I could not read what Rails.cache returns here too ruby/2.3.4/lib/ruby/gems/2.3.0/gems/railties-4.2.8/lib/rails.rb
Is Rails.cache.clear is supposed to return true?
Can someone please help me out if my understanding is wrong?
redis-cache stores data under a particular namespace.
For example, if you've configured redis-store according to Documentation, then cache keys will be stored under cache namespace. That means, that when you Rails.cache.write("random_key", "key") a key cache:random_key will appear in the Redis. Therefore, when you Rails.cache.clear, only keys under cache namespace will be deleted.
Hence, if you manually create random_key in Redis, Rails.cache.clear won't remove it. But if you manually create cache:random_key, it will.
Be careful when using Rails.cache.clear it will invalidate all the keys for the application (source)
[~Not sure if this is the best place for this answer~]
This helpful article was a great way for me to understand caching when changing versions of Rails from 5.1+ to Rails 6.1+. The article talks about options for generating a cache key with or without versioning.
In the instance of my application, versioning was needed but not turned on when upgraded to Rails 6.1:
#in application.rb
config.active_record.collection_cache_versioning = true
Then within the application code where object.cache_key is called, I had to change it to object.cache_key_with_version (source)
I am having a very strange problem with loading a CSV file that is driving my absolutely crazy and doesn't make any sense to me! I am loading a CSV file into my database with the following code:
CSV.foreach(Rails.root.join('public','uploads', '0', csv_file.file_name), :headers => true, :header_converters => lambda { |h| h.try(:downcase)}) do |row|
Exclusion.create!(row.to_hash)
end
I have a file with 14,808 entries. If I try to load the file at once, for some reason it adds all 14,808 entries into the database as expected, but then starts over again from entry 1. It continues to do this in a recursive manner until I stop the server or it crashes. If I break the file down into two files, the individual files get added to the database as expected. I thought it may be a problem with the number of records, but I was able to load a csv file with about 100,000 records without this problem. I am very baffled as to why this is occurring. Oddly, if I comment out the create statement and just put a counter there, it stops at 14,808. Also, if I create a view that prints each row.to_hash, it stops at 14,808. I can't figure out what about saving it into the database would cause it to continue to repeat itself? I am using SQLite3, but again, I dont have a problem with a CSV file with 100,000 records.
Update:
Going through the log, it looks like the CSV file is loaded properly and all records are added to the database. Ruby even redirects to the proper url afterwards, but then seems to receive the "load file" request again. Since my screen has already timed out waiting for the server to process the CSV file, could that be causing an error and leading to either a duplicate request to load the file or the server thinking it hasn't processed the request and starting over?
Browsers sometimes repeat requests. If you have a catch-all route that goes to one action (the one that loads CSV file) then it could also be triggered by the browsers requesting favicon.ico.
However, I'd ask, why are you using a web request for this? Does csv_file come from the user? If not (i.e. if you already have the CSV file you want to load) I'd recommend putting this in a Rake task and just running it manually.
I have this block of code:
users = Array.new
users << User.find(:all, :conditions => ["email like ?", "%foo%"])
users << User.find(:all, :conditions => ["name like ?", "%bar%"])
users.flatten!
users.uniq!
puts users.to_json :include => [:licenses]
When I run it using script/console, it returns exactly what you would think it should, a JSON representation of the Array of users that I found, flattened, and uniquified. But running that same line of code as part of a search_for_users method, I get this error
TypeError in ControllerName#search_for_users
wrong argument type Hash (expected Data)
and the line referenced is the line with the .to_json call.
It's baffling me because the code is verbatim the same. The only difference is that when I'm running it in the console, I'm entering the conditions manually, but in my method, I'm pulling the query from params[:query]. But, I just tried hardcoding the queries and got the same result, so I don't think that is the problem. If I remove the :include, I don't see the error, but I also don't get the data I want.
Anyone have any idea what the issue might be?
There are a few plugins and gems that can cause .to_json to fail if included in your controller. I believe that the Twitter gem is one of them (ran into a problem with this awhile back).
Do you have "include [anything]" or "require [anything]" in this controller?
If not, I'd suggest temporarily removing any plugins you're using to troubleshoot, etc.
Finally, what happens if you replace that entire controller action with simply:
%w(1 2 3 4 5).to_json
That should help you pin down what is failing.
Whenever code in tests or the console behaves different from production environment (which is a guess... you might be running your site in development mode), this calls for a load order issue. In production environment, all the models and controllers are preloaded, in other environments they are loaded lazily when needed.
Start your console with RAILS_ENV=production ./script/console and see if you can reproduce the error this way.
As cscotta mentioned, there are a couple of gems and librarys, that can interfere with .to_json, first to mention the functionality, that you get when you require 'json'. I personally ran into several issues with that.
Hope this helps
Seb