I have a task in Rails that exports an xml, I've one line in it though
tmp_filename="#{Rails.root}/tmp/orders-#{o.id}-#{DateTime.now}.xml"
and this outputs the xml file with a filename like
orders-42-2015-01-28T17:22:35+00-00.xml
This is the way it shows up when its uploaded directly to amazon s3, the problem is I need to get rid of the colons and just have dashes because the system thats taking these files doesn't work properly with the colon in the filename.
The strange thing is that when I download the file from s3 it downloads as dashes.
I'm not sure how or if I can use strftime on #{}
Could anybody help with what I'm trying to do. Or if this is just an amazon s3 thing and the file is actually being generated with the - and not : already.
Strftime doesn't seem to work on amazon s3, the file still uploads as the original format even after adding
tmp_filename="#{Rails.root}/tmp/orders-#{o.id}-#{DateTime.now.strftime('%d-%m-%Y-%H%M%S')}.xml"
and it also adds an extra +00:00 at the end for some reason that I can't get rid of
Can't you just format the DateTime without colons, for example:
tmp_filename="#{Rails.root}/tmp/orders-#{o.id}-#{DateTime.now.strftime('%Y-%m-%d-%H-%M-%S')}.xml"
With this you'll get the time in format like below, without colons:
irb(main):010:0> DateTime.now.strftime('%Y-%m-%d-%H-%M-%S')
=> "2015-01-29-10-50-30"
Related
I have a CSV file that I'm uploading which runs into an issue when importing rows into the database:
Encoding::UndefinedConversionError ("\xCC" from ASCII-8BIT to UTF-8)
What would be the most efficient way to ensure each column is properly encoded for being placed in the database or ignored?
The most basic approach is to go through each row and each field and force encoding on the string but that seems incredibly inefficient. What would be a better way to handle this?
Currently it's just uploaded as a parameter (:csv_file). I then access it as follows:
CSV.parse(csv_file.download) within the model.
I'm assuming there's a way to force the encoding when CSV.parse is called on the activestorage file but not sure how. Any ideas?
Thanks!
The latest version ActiveStorage (6.0.0.rc1) adds an API to be able to download the file to a temp file, which you can then read from. I'm assuming that Ruby will read from the file using the correct encoding.
https://edgeguides.rubyonrails.org/active_storage_overview.html#downloading-files
If you don't want to upgrade to the RC of Rails 6 (like I don't) you can use this method to convert the string to UTF-8 while getting rid of the byte order mark that may be present in your file:
wrongly_encoded_string = active_record_model.attachment.download
correctly_encoded_string = wrongly_encoded_string.bytes.pack("c*").force_encoding("UTF-8")
I am making a rails app with video uploads. In testing any medium to big sized video totally wrecks my server logs since it tries to print out the video file. It sent the response but 10 minutes later it is still trying to print out the video file. Can I prevent rails from logging the params?
There's a way of suppressing arbitrary parameters in your configuration, so just include the name you want to suppress. For example:
Rails.application.config.filter_parameters += [ :video ]
This is described in the documentation and is by default loaded in config/initializers/filter_parameter_logging.rb. You could either edit that and add your params, or to make it modular, add another initializer file of the same format.
Note this won't block logging of SQL queries, so if you're inserting large BLOB type objects your log will be noisy. This is why it's usually better to dump the video on an object-store like Amazon S3.
I gonna upload files to rackspace(video, audio and images) in rails with paperclip or carrierwave, I need to Know the kind of file, to show in the view with image_tag, or video_tag or audio_tag, rackspace tell me the kind of file? or I have to store in my database? thanks
You can query/set the file type by using the 'content_type' function located in the 'ruby-cloudfiles' library.
See here: https://github.com/rackerlabs/ruby-cloudfiles/blob/master/lib/cloudfiles/storage_object.rb#L80-L82
Something like this should work for creating the object:
container = conn.create_container('new_container')
obj = container.create_object('new_obj.txt')
obj.load_from_filename('./obj.txt')
obj.content_type = 'text/plain'
And to retrieve the object:
obj = container.object('new_obj.txt')
puts obj.content_type # text/plain
Even if rackspace would tell you the file type, you don't want it to, since it would take so long to run roundtrips from your server to theirs.
My code examples below assume carrierwave, but I'm sure paperclip has similar options. Two options:
Interpret the file extension
Something like: File.extname(user.avatar), which you then have to interpret however you like.
Record & interpret the mime type.
The carrierwave readme explains how to get carrierwave to calculate it in the first place, and then you should probably store it to your database manually or using carrierwave-meta. Then user.avatar.content_type would be something like image/jpeg which you could easily interpret as a particular file type.
I'm using Ruby on Rails on my local WEBrick server.
I'm generating some images of websites from urls, writing them to my local filesystem, and displaying them. I name the images by their url but replace all \ with -. Some images seem to not be loading because it cannot find the images on my filesystem, and I get the broken image icon. However, I see that all the images are there when I check my filesystem.
This is the error I get in my logs:
Started GET "/images/image_site/http:--www.urbandictionary.com-define.php?term=slim%20shady.jpg?1309465379" for 127.0.0.1 at Thu Jun 30 13:23:06 -0700 2011
ActionController::RoutingError (No route matches "/images/image_site/http:--www.urbandictionary.com-define.php"):
This is my html code:
<img alt="Http:--www.urbandictionary.com-define.php?term=slim%20shady" class="site_image" src="/images/image_site/http:--www.urbandictionary.com-define.php?term=slim%20shady.jpg?1309465379">
What going on and how can I fix this? Please let me know if I need to provide more information.
Looks like you're not properly encoding your image names in the src attribute. I'd guess that you have a file with a name like this:
http:--www.urbandictionary.com-define.php?term=slim%20shady.jpg
But when you have this:
src="/images/image_site/http:--www.urbandictionary.com-define.php?term=slim%20shady.jpg?1309465379"
The filename looks like this:
/images/image_site/http:--www.urbandictionary.com-define.php
because everything after the first ? is considered to be part of the query string.
Replacing the slashes with hyphens is not good enough, you're still leaving all sorts of holes for confusion and nefarious intent. Instead, you should generate the image file names completely, something like id.jpg where id is the image's ID in your database. Then, store the original filename in your database and only show that filename (properly encoded!) to people, don't use it in your local file system.
A quick fix would be to properly URL encode your src attributes. But you really should fix up how you handle the images or you will leave yourself open to all sorts of trouble.
You need to place images in your public/ folder and then you can access them from there. By default Rails will server static assets primarily from public/.
I have a system where the user uploads a file and I have to read the file and display its contents on a form without storing it either on the server side or in the database
When the file is uploaded Rails will automatically read it in and make it an instance of Tempfile so it's already stored, it won't however be stored forever on the system.
You can access the file using the normal params[:field_name] syntax as if the file were any other field (don't forget to set content-type of the form to multipart/form-data - i.e.
form_for #mything, :html => {:multipart => true})
and you will get back the tempfile. The tempfile can be read from like any other file.
Rails (Or Maybe Rack I'm not 100% up to date) determines whether to do this or not to uploaded content based on the attachment part of the mulitpart/form-data element containing the file.
It might be possible to override things if you need to though to stop this storage from happening. Common practice however is to just work with the file and then let Ruby deal with the temp file.