TL;DR
How would I specify the mode of encoding on File.write, or how would one save image binary to a file in a similar fashion?
More Details
I'm trying to download an image from a Trello card and then upload that image to S3 so it has an accessible URL. I have been able to download the image from Trello as binary (I believe it is some form of binary), but I have been having issues saving this as a .jpeg using File.write. Every time I attempt that, it gives me this error in my Rails console:
Encoding::UndefinedConversionError: "\xFF" from ASCII-8BIT to UTF-8
from /app/app/services/customer_order_status_notifier/card.rb:181:in `write'
And here is the code that triggers that:
def trello_pics
#trello_pics ||=
card.attachments.last(config_pics_number)&.map(&:url).map do |url|
binary = Faraday.get(url, nil, {'Authorization' => "OAuth oauth_consumer_key=\"#{ENV['TRELLO_PUBLIC_KEY']}\", oauth_token=\"#{ENV['TRELLO_TOKEN']}\""}).body
File.write(FILE_LOCATION, binary) # doesn't work
run_me
end
end
So I figure this must be an issue with the way that File.write converts the input into a file. Is there a way to specify encoding?
AFIK you can't do it at the time of performing the write, but you can do it at the time of creating the File object; here an example of UTF8 encoding:
File.open(FILE_LOCATION, "w:UTF-8") do
|f|
f.write(....)
end
Another possibility would be to use the external_encoding option:
File.open(FILE_LOCATION, "w", external_encoding: Encoding::UTF_8)
Of course this assumes that the data which is written, is a String. If you have (packed) binary data, you would use "wb" for openeing the file, and syswrite instead of write to write the data to the file.
UPDATE As engineersmnky points out in a comment, the arguments for the encoding can also be passed as parameter to the write method itself, for instance
IO::write(FILE_LOCATION, data_to_write, external_encoding: Encoding::UTF_8)
I am using Rails action mailer. I need to send a email with document(doc, docx, txt) as attachment. When I am attaching .txt files, the content in that txt file are viewing properly. But when I am trying to attach .doc or docx files, the content in that files are not displaying(it is converting into encoding format) Why is that happening. Is there any way to attach document files in email using rails action mailer.
def send_email(email, file)
#email = email
attachments['document.docx'] = File.read(file.path)
mail(from: #email, to: "aa#gmail.com", subject: "Document attachment")
end
original document attached
document received in email
Please guide me, if I am going wrong in some place.
Thanks in advance.
Try reading the file in binary mode (see binary file mode & :mode option:
attachments['document.docx'] = File.read(file.path, mode: 'rb')
In my case, Rails was adding a carriage return (aka "CR", or "\r") to every line in a CSV file that ended with either a "\n" (newline) or "\r".
I am trying to set up some rake tasks. It requires me to connect to gmail and download a Zip file which is sent as an attachment.
I have written the following code(which works fine for downloading csv) -
gmail = Gmail.connect(ENV["USERNAME"], ENV["PASSWORD"])
msg = gmail.inbox.find(from: ENV["REC_USER"],
subject: args[:subject])
dir_path = "lib/mfu_payment_data/"
Dir.mkdir dir_path unless File.exists?(dir_path)
if msg.first
msg.first.attachments.each do |attachment|
File.write(File.join(dir_path,attachment.filename),attachment.body.decoded)
end
end
It throws the following error -
rake aborted!
Encoding::UndefinedConversionError: "\xED" from ASCII-8BIT to UTF-8
I assume that this has got something to do with the attachment.body.decoded, but I do not know how else to do this.
You can try writing the file in binary mode:
File.open('/path/to/file;, 'wb') { |file| file.write(attachment.body.decoded) }
"b" Binary file mode
Suppresses EOL <-> CRLF conversion on Windows. And
sets external encoding to ASCII-8BIT unless explicitly
specified.
The modes are described in the IO class which File inherits from.
I think you have many option to generate zip file
Download and unzip
I'm having serious issues trying to parse some CSV in rails right now.
Basically my app gets a user to upload a CSV file. The app then converts the file to ensure it is in UTF-8 format, then attempts to parse it and process it. Whenever the app attempts to parse it however, I get the MalformedCSVError stating "Illegal quoting on line 1"
Now what I don't get, is if I copy the original file into a new document and save it, then I can parse it on a rails console without a problem.
If I attempt to parse the original file, it complains about an invalid character for UTF-8 encoding (the file isn't in UTF-8 hence the app converts it)
If I attempt to parse the file which the app has converted to UTF-8 and changed the line endings to LF, it fails to parse.
If I do a file diff between the version the app has produced, and the copy/paste version that I have made (which works) there are 0 differences so I really can't figure out why one is parsable, and one is not.
Any suggestions? My app is processing the file as follows :
def create
#survey = Survey.new(params[:survey])
# Now we need to try and convert this to UTF-8 if it isn't already
encoded = File.read(#survey.survey_data.current_path)
encoding = CharlockHolmes::EncodingDetector.detect(encoded)
# We've got a guess at the encoding,
# so we can try and convert it but it
# may still fail so we need to handle
# that
begin
re_encoded = CharlockHolmes::Converter.convert(encoded, encoding[:encoding], 'UTF-8')
re_encoded = re_encoded.gsub(/\r\n?/, "\n")
# Now replace the uploaded file
File.open(#survey.survey_data.current_path, 'w') { |f|
f.write(re_encoded)
}
rescue ArgumentError
puts "UH OH!!!!!"
end
puts "#{#survey.survey_data.current_path}"
#parsed = CSV.read(#survey.survey_data.current_path)
end
The file uploading gem is CarrierWave if that makes any difference.
Please can someone help me as this is driving me insane!
Edit
The error says it's on line 1. Line 1 (assuming it doesn't index from 0) is
"Survey","RD","GarrysMDs","NigelsMDs","PaulsMDs","StephensMDs","BrinleyJ","CarolineP","DaveL","GrantR","GregS","Kent","NeilC","NicolaP","AndyC","DarrenS","DeanB","KarenF","PaulR","RichardF","SteveG","BrianG","GordonA","NickD","NickR","NickT","RayL","SimonH","EdmondH","JasonF","MikeS","SamanthaN","TimB","TravisF","AlanS","Q1","Q2","Q3","Q4","Q5","Q6","Q7","Q8PM","Q8N","Q9","Q10","Q11","Q12","Q13","Q14","Q15","Q16PM","Q16N","Q17PM","Q17N","Q18PM","Q18N","Q19","Q20","Q21","Q22","comment","Q23.1","Q23.2","Q23.3","TQ23.1","TQ23.2","VPM","VN","VQ1","VQ2","VQ3","VQ4","VQ5","VQ6","VQ7","VQ8N","VQ8PM","VQ9","VQ10","VQ11","VQ12","VQ13","VQ14","VQ15","VQ16","VQ16N","VQ16PM","VQ17","VQ17N","VQ17PM","VQ18","VQ18N","VQ18PM","VQ19","VQ20","VQ21","VQ22","VQ23.1","VQ23.2","VQ23.3","VRD","XQ16","XQ17","XQ18"
Well that was irritating!
Turns out the file had a BOM which was causing the CSV parser to break. Loading the file with
CSV.open("path/to/file.csv", "rb:bom|encoding")
allowed it to parse it perfectly! So annoyed how long it took to track down but it's now working and with no need to convert to UTF-8 now either!
I am trying to send an email containing a file attachment created by the user.
attachments[document.display_name] = File.read(document.public_filename)
This works in most conditions (including .docx, but fails for .xlsx files with the error:
invalid byte sequence in UTF-8
I am using attachment_fu to upload the attachments, and delayed_job to defer sending the emails, however the file I am trying to attach looks ok, and I can open it successfully outside the application.
I also saw a suggestion to change the code as follows, but it does not seem to help:
include an extra.
attachments[document.display_name] = { :content => File.read(document.public_filename), :transfer_encoding => :binary }
How can I make the code work for all attachment types?
You need to specify the mode or encoding to let it read the file as binary:
attachments[document.display_name] = File.read(document.public_filename, :mode => 'rb')
or
attachments[document.display_name] = File.read(document.public_filename, :encoding => 'BINARY')