I need to save image from file to PostgreSQL database as binary. Trying to do it like this:
image_file = File.open("image.png", "rb") { |file| file.read }
Image.create(product_id: product_id, image: image_file)
This code gives me the following error:
string contains null byte
Using paperclip or similar, or converting binary to base64 are not an option.
Try providing us more information, if you're able. I know there was a recent PR for Rails that fixed more superfluous bytea escaping in PostgreSQL. So definitely try updating your system to see if that helps. If it doesn't, please provide us more information if you're able.
Here's a link to the PR
Related
Given the following situation:
document source of type PDF freshly received via a file transfer
document target which is saved as an attachment blob using active_storage and versionned
I want to check whether any existing version of target is binary-equal to source. Without active_storage, I'd have done a SHA256 sum of any blob that enters the DB. In order to compare, I'd have compared the fresh SHA256 sum of source to each checksum stored for any version of target.
However the method .checksum of active_storage attachments and blobs appears to be neither a MD5 or SHA265 sum. For instance I get Cr4IxYNF7v7cJao1EiiBEw== for some file.
A solution would be to use something like Digest::SHA256.hexdigest(Person.find(46).photo.download) however the performance would be terrible.
How can I efficiently search my active_storage "database" ?
According to the ActiveStorage source, the checksum is in fact MD5. But it has been base64 encoded.
From the source at: https://github.com/rails/rails/blob/8da6ba9cae21beae1ee3c379db7b7113d2731c9b/activestorage/app/models/active_storage/blob.rb#L313
def compute_checksum_in_chunks(io)
Digest::MD5.new.tap do |checksum|
while chunk = io.read(5.megabytes)
checksum << chunk
end
io.rewind
end.base64digest
end
So hopefully you should be able to just base64 encode your own MD5 hashes for a comparison in the database.
I have a CSV file that I'm uploading which runs into an issue when importing rows into the database:
Encoding::UndefinedConversionError ("\xCC" from ASCII-8BIT to UTF-8)
What would be the most efficient way to ensure each column is properly encoded for being placed in the database or ignored?
The most basic approach is to go through each row and each field and force encoding on the string but that seems incredibly inefficient. What would be a better way to handle this?
Currently it's just uploaded as a parameter (:csv_file). I then access it as follows:
CSV.parse(csv_file.download) within the model.
I'm assuming there's a way to force the encoding when CSV.parse is called on the activestorage file but not sure how. Any ideas?
Thanks!
The latest version ActiveStorage (6.0.0.rc1) adds an API to be able to download the file to a temp file, which you can then read from. I'm assuming that Ruby will read from the file using the correct encoding.
https://edgeguides.rubyonrails.org/active_storage_overview.html#downloading-files
If you don't want to upgrade to the RC of Rails 6 (like I don't) you can use this method to convert the string to UTF-8 while getting rid of the byte order mark that may be present in your file:
wrongly_encoded_string = active_record_model.attachment.download
correctly_encoded_string = wrongly_encoded_string.bytes.pack("c*").force_encoding("UTF-8")
I am using the Carrierwave gem to upload attachments to my model. I added elasticsearch with the mapper attachments plugin to allow for full text search of the attachments.
Carrierwave and elasticsearch work fine, but in order to get the full text search working I need to encode the attachment as base64.
I have followed this tutorial (http://rny.io/rails/elasticsearch/2013/08/05/full-text-search-for-attachments-with-rails-and-elasticsearch.html) but I assume there has been some changes to either Rails or Carrierwave as I can't get it to work. Specifically, when I am trying to encode the attachment as base64, I get the following Type error:
no implicit conversion of CarrierWave::SanitizedFile into String
The error is in the following line of the model:
File.open(Base64.encode64(File.read(document.file)))
If I replace the path with a url to an actual file it works fine.
I have searched SO and the only relevant answer I can find gives me the same error: Carrierwave encode file to base64 as process
I am a complete rails newbie and hopefully this is something that's obvious to everyone except me, but we're all beginners at first, right?
Thanks!
CarrierWave's read method returns the content of the file. So assuming Document is your model and file is your uploader attribute, this should work:
Base64.encode64(document.file.read)
Characters coming out of my database are encoded differently than the same characters written directly in the source. For exmaple, the word Permissões shows a different result when the string is written directly in the HTML, than when the string is output from a db record.
# From the source
Addressable::URI.encode("Permissões.pdf") #=> "Permiss%C3%B5es.pdf"
# From the db
Addressable::URI.encode("Permissões.pdf") #=> "Permisso%CC%83es.pdf"
The encodings are different. But my database is set to UTF-8, and I am using HTML5. What could be causing this?
I am unable to download files I upload to S3 because of this issue. I tried to force the encoding attachment.path.encode("UTF-8") but that makes no diffrence.
To solve this, since I am using Rails, I used ActiveSupport::Multibyte::Unicode to normalize any unicode characters before they get inserted into the database.
before_save do
self.path = ActiveSupport::Multibyte::Unicode.normalize(path)
end
I gonna upload files to rackspace(video, audio and images) in rails with paperclip or carrierwave, I need to Know the kind of file, to show in the view with image_tag, or video_tag or audio_tag, rackspace tell me the kind of file? or I have to store in my database? thanks
You can query/set the file type by using the 'content_type' function located in the 'ruby-cloudfiles' library.
See here: https://github.com/rackerlabs/ruby-cloudfiles/blob/master/lib/cloudfiles/storage_object.rb#L80-L82
Something like this should work for creating the object:
container = conn.create_container('new_container')
obj = container.create_object('new_obj.txt')
obj.load_from_filename('./obj.txt')
obj.content_type = 'text/plain'
And to retrieve the object:
obj = container.object('new_obj.txt')
puts obj.content_type # text/plain
Even if rackspace would tell you the file type, you don't want it to, since it would take so long to run roundtrips from your server to theirs.
My code examples below assume carrierwave, but I'm sure paperclip has similar options. Two options:
Interpret the file extension
Something like: File.extname(user.avatar), which you then have to interpret however you like.
Record & interpret the mime type.
The carrierwave readme explains how to get carrierwave to calculate it in the first place, and then you should probably store it to your database manually or using carrierwave-meta. Then user.avatar.content_type would be something like image/jpeg which you could easily interpret as a particular file type.