We have an online store running on Rails 3 Spree platform. Recently customers started reporting weird errors during checkout and after analyzing production logs I found the following error:
Errno::ENAMETOOLONG (File name too long - /var/www/store/tmp/cache/UPS-R43362140-US-NJ-FlorhamPark07932-1025786194_1%7C1025786087_1%7C1025786089_15%7C1025786146_4%7C1025786147_3%7C1025786098_3%7C1025786099_4%7C1025786100_2%7C1025786114_1%7C1025786120_1%7C1025786121_1%7C1025786181_1%7C1025786182_1%7C1025786208_120110412-2105-1e14pq5.lock)
I'm not sure why this file name is so long and if this error is specific to Rails or Spree. Also I'm not very familiar with Rails caching system. I would appreciate any help on how I can resolve this problem.
I'm guessing you are using spree_active_shipping, as that looks like a cache id for a UPS shipping quote. This will happen when someone creates an order that has a lot of line items in it. With enough line items this will of course create a very large filename for the cache, thus giving you the error.
One option would be to use memcache or redis for your Rails.cache instead of using the filesystem cache. Another would be to modify the algorithm that generates the cache_key within app/models/active_shipping.rb in the spree_active_shipping gem.
The latter option would probably be best, and you could simply have the generated cache key run through a hash like MD5 or SHA1. This way you'll get predictable cache key lengths.
Really this should be fixed within spree_active_shipping though, it shouldn't be generating unpredictably long cache keys, even if you a key-value store is used, that's wasted memory.
It is more related to your file system. Either set up a file system which supports longer file names or change the software to make better (md5?timestamp?unique id?) file names.
May be this help:
config.assets.digest and config.assets.debug can't both be true
It's a bug : https://github.com/rails/jquery-rails/issues/33
I am using rails 3.2.x and having same issue. I end up generating MD5 digest in the view helper method used to generate cache key.
FILENAME_MAX_SIZE = 200
def cache_key(prefix, params)
params = Array.wrap(params) if params.instance_of?(String)
key = "#{prefix}/" << params.entries.sort { |a,b| a[0].to_s <=> b[0].to_s }.map { |k,v| "#{k}:#{v}"}.join(',').to_s
if URI.encode_www_form_component(key).size > FILENAME_MAX_SIZE
key = Digest::MD5.hexdigest(key)
end
key
end
Here I have to check length of URI encoded key value using URI.encode_www_form_component(key).size because as you can see in my case, cache key is generated using : and , separators. Rails encodes the key before caching the results.
I took reference from the pull request.
Are you using paperclip gem? If yes, this issue is solved: https://github.com/thoughtbot/paperclip/issues/1246.
Please update your paperclip gem to the latest version.
Related
This issue relates to a need to set a Rails config variable as the application boots, and the value of that variable needs to come from data in the database (which are then modified). So, I have an initializer with something like this:
require "#{Rails.root}/lib/modules/facet_altering.rb"
include FacetAltering
Rails.application.config.reject_subjects = FacetAltering.reject
The reject method is potentially slow and calls the Subject model (which includes some concerns).
If I try to require subject.rb, application_rb and the relevant concerns from app/models then I progress a bit further, but eventually get stuck on uninitialized constant Subject::MySpecialConcern.
There might be some better way to set the reject_subjects value; I'd prefer not to run FacetAltering.reject each time the value of reject_subjects is used, though this might be an easy 'fix' if no other solution arises (at the cost of slowing things down). Or, is there another way to access these classes as the application boots?
Edit: Following on from the comment below, this is in config/application.rb:
%W[#{Rails.root}/lib/modules #{Rails.root}/test/mailers/previews].each do |path|
config.eager_load_paths << path
end
This post offered a useful clue:
Rails Model no longer available in initializer after upgrade to Rails 7.0
So, putting my code in config/application.rb as follows did the trick:
config.after_initialize do
Rails.application.config.reject_subjects = FacetAltering.reject
end
Now to find the answer to RuntimeError: Foreign key violations found in your fixture data. Ensure you aren't referring to labels that don't exist on associations. and I might be able to complete this Rails 6 -> 7 upgrade!
I would like to migrate from Paperclip to Carrier Wave or Refile because of this. The solution written here is impressive, but strikes me as complex and perhaps brittle.
My Rails4 app has 100's of images in production that were uploaded with Paperclip. Files are stored on production server. I have looked for a complete set of steps to follow to migrate, but keep coming up empty.
Is there a set of steps one can follow that allows for migration without necessitating application code re-write?
Alternatively, is there another way to persist uploaded files in Paperclip when form validation fails?
What am I missing here?
UPDATE:
Tried the solution detailed here by https://stackoverflow.com/users/646389/galatians . My paperclip :path and :url interpolations make use of :id_partition. I don't see a way this can be reconciled with an uploaded Image that is staged, but not yet saved.
I migrated to Carrierwave. Here are the relevant stats:
Time to work on and fail at coding a solution for persistent files
across form reloading with Paperclip - 4 hours. See OP update for the issue I could not overcome.
Time to Migrate to carrierwave, adjust the relevant models, controllers, and forms, and test. - 2 hours. Not so bad.
This key info helped me adjust the paths correctly. Keeping the path identical was important to me for avoiding having to move images to a new location in production:
Carrierwave code for generating paperclip-like :path and :url info here.
Paperclip interpolation info here.
This link got me on the right track, although my default :path made use of :id_partition not :id.
UPDATE:
Migration breaks this paradigm:
#protocol.images.each do |i|
tmp=i.dup
tmp.avatar = File.open(i.avatar.current_path)
tmp.save!
#dest.images << tmp
end
See: Duplicating a record that contains a carrierwave avatar : Getting "can't convert nil into Integer" error
You don't need to do anything because the only data needed to get images is already stored in DB.
In my rails app, I'm using Carrierwave to upload images on Amazon S3. I'd like to point to existing Amazon S3 images without having to re-upload the image. For example, if I have an existing Amazon S3 image at http://test.s3.amazonaws.com/image/path/0001/image.jpg, can I update an image's path to point to this image? I don't want to use the remote upload option because I really just want to use the same exact image that's already there (but save it in my record's "path" attribute).
In the console, I've tried:
image.update_attributes(:path=> "http://test.s3.amazonaws.com/image/path/0001/image.jpg")
but this fails to override the image's path.
Chiming in, better late than never! Caveat: This is for rails 4, and I am testing on rails 4.1 only at the moment.
This is harder than it should be, methinks! The reason this was absolutely crucial to me was that I am attaching 100MB+ MP3 files, which I cannot receive on my host, due to CloudFlare SSL limitations (and common sense). Fortunately, AWS supports preauthorized uploads, and I got carrierwave to do the right thing for me:
Step 1: get carrierwave to tell me where it would store a file if it could:
m.raw_write_attribute('file','file.mp3');
url = m.file.url
signed = aws_presigned_url(url)
raw_write_attribute does not save anything, it just bypasses carrierwave when setting the value. This makes the object act as if it read 'file.mp3' out of the database. Then you can ask Carrierwave "where the file lives". I then upload the file directly from the client to S3. When that's done, I make another API call to Rails, which performs the following code:
m.raw_write_attribute('file','file.mp3');
m.update_attribute('file','file.mp3');
These two paired get around Carrierwave. The first makes carrierwave think that the 'file' column is set to 'file.mp3', the second explicitly tells rails to persist 'file.mp3' to the DB. Because of the raw_write_attribute call, Carrierwave allows the second through un-changed.
In my case update_column and update_columns worked great:
model.update_columns file_1: 'filename.txt'
Update column is with comma:
model.update_column :file_1, 'filename.txt'
This will not run any callback and set column to filename.txt.
When I do model.file_1.url I get the right S3 URL.
I am a bit late to the party, but you can use Active Record's raw_write_attribute method something like:
#image.raw_write_attribute(:path, "http://test.s3.amazonaws.com/image/path/0001/image.jpg")
I found that you can actually do this, for example if your mount_uploader is :path, then:
image.remote_path_url = "http://test.s3.amazonaws.com/image/path/0001/image.jpg"
image.save
On redmine 1.2/rails 2.3.11 I'm rendering a repository markdown file as html (as redmine_markdown_extra_viewer does), and now I'am trying to cache the result, which should be updated on each commit.
So I have a git hook that fetch the repo changes, and i'd like it to also clear the corresponding cache entries.
Cache generation (in a RepositoriesController::entry override):
cache_key =['repositories_md', #project.id.to_s, #path.to_s].join('/')
puts cache_key
#content = cache_store.fetch cache_key do
Kramdown::Document.new(#repository.cat(#path, #rev)).to_html
end
render :action => "entry_markdown"
The hook that should clear the cache, but has no effect:
# This is ok
ruby script/runner "Repository.fetch_changesets"
# This not
ruby script/runner "Rails.cache.delete_matched(/repositories_md\/.*/)"
So it doesn't work and i don't even know if i've taken the right direction to implement that. Any input much appreciated!
Which cache backend are you using?
If it's memcached or anything other than the FileStore or the MemoryStore, the delete_matched method is not supported.
You're probably better off letting them expire and just replace their cached contents as they get updated.
The problem is when using a Regular Expression as a fragment name, try using a String as a fragment name. Maybe get verbose. I had a similar problem with Dalli(with Memcached), and that was the reason.
I'm trying to re-create all thumbs. I'm not sure why is saying the key does not exist. I have AWS-S3 configured properly and it's working well (I can upload pictures with no problems.)
>> Attachment.all.each {|x|x.attachment.reprocess!}
AWS::S3::NoSuchKey: The specified key does not exist.
/app/d999782b-a789-4763-ac86-e8c65fa781eb/home/.bundle/gems/ruby/1.8/gems/aws-s3- 0.6.2/lib/aws/s3/error.rb:38:in `raise'
/app/d999782b-a789-4763-ac86-e8c65fa781eb/home/.bundle/gems/ruby/1.8/gems/aws-s3-0.6.2/lib/aws/s3/base.rb:72:in `request'
/app/d999782b-a789-4763-ac86-e8c65fa781eb/home/.bundle/gems/ruby/1.8/gems/aws-s3-0.6.2/lib/aws/s3/base.rb:88:in `get'
/app/d999782b-a789-4763-ac86-e8c65fa781eb/home/.bundle/gems/ruby/1.8/gems/aws-s3-0.6.2/lib/aws/s3/object.rb:134:in `value'
When I tried to do the same to a single object seems to do it well, so the problem seems to be related to generating with a collection.
>> Attachment.last.attachment.reprocess!
=> true
UPDATE: I'm pretty sure it's related to the fact that there are uploaded files such as .htm that should be valid image files. Any idea how to skip them?
Though I am not sure, but I hope this might help you.
Attachment.all.each { |x| x.attachment.reprocess! if ['.jpeg','.jpg','.png','.gif'].include?(File.extname(file_name))}
where file_name => Name of the uploaded file
Best of Luck
Not sure where you've put your key for the AWS-S3, but you may have to specify that you want to run this under the production environment.
heroku rake paperclip:refresh CLASS=Attachment RAILS_ENV=production
I don't know how your validations are set up but is it possible that some attachment objects can have a blank attachment? If so, try:
Attachment.all.each { |x| x.attachment.reprocess! rescue nil }
This error can also refer to when an object(key) no longer exists on S3 but you have a record pointing to it in your database. This only happens if someone has made changes to the S3 bucket that don't mesh with what you have in your DB.
If that's the case you can use the ".exists?" method on the attachment to check if that key exists on Amazon's server's first, not that this will issue a read request.
This would change your reprocess command to something like this:
Attachment.all.each { |x| x.attachment.reprocess! if x.attachment.exists? }
Have you considered using:
rake paperclip:refresh
Instead?