My application stores a lot of XML files. A background job periodically sends some of those XML files to a specific mailbox. The mailer code is dead-simple:
class MailSender < ActionMailer::Base
default :from => AppConfig.mail_from
smtp_settings :address => AppConfig.smtp_host,
:username => AppConfig.smtp_user,
:password => AppConfig.smtp_pass
def send_xml(record)
f = record.filename.gsub("\\", "/") # converts \ to /
f_short = arq.gsub(/.*\//, "") # extracts only the filename
f_phys = "#{AppConfig.xml_root}#{arq}" # builds the physical filename
headers["Return-Receipt-To"] = AppConfig.return_receipt
attachments[f_short] = File.read(f_phys) if File.exists?(f_phys)
mail :subject => "...",
:to => AppConfig.mail_to
end
end
However, for some reason, those XML are getting corrupted on transmission: the first line break gets added at column 987, and the following are added at column 990. After each break, a space is inserted. I think the picture says for itself:
col 1 col 990
|.................................................|
<?xml version="1.0" ... </IE><IM>321505493301<
/IM><CNAE>4744001< ... 00</pCOFINS><vCOFINS>0.00
</vCOFINS></COFINS ... /prod><imposto><ICMS><ICM
S40><orig>0</orig> ... <infAdic><infCpl>Permite
I tried calling File.read myself on rails console, it works fine, no line breaks are added. So I assume the error should lie on the ActionMailer. Any tips?
Edit for clarification: Most of the XML document lie on a big, single line. I can't change it, since the XML are digitally signed - any change, including adding line breaks and indentation, breaks the digital signature.
Answering the question that gave me the 'Thumbleweed' badge :)
I ended up encoding the file myself, and it's now working fine:
attachments[f_short] = {
:encoding => 'base64',
:content => Base64.encode64( File.read(f_phys) ).chomp
} if File.exists?(f_phys)
Related
I have some code that runs every week using Whenever gem. It creates a file and sends a download link.
ApplicationMailer
def weekly_email
file = CSVData::Report.new(time).create_csv
#filename = File.basename(file.path)
mail(
:from => "from",
:to => "to",
:subject => "Data #{time.strftime("%m/%d/%Y")}"
)
end
The file is stored in downloads/. The issue is, it works in testing, it works sometimes, but sometimes it generate a Missing File error:
Error:
ActionController::MissingFile occurred in file_downloads#download:
Cannot read file downloads/data_9_17_2015.csv
actionpack (3.2.13) lib/action_controller/metal/data_streaming.rb:71:in `send_file'
Here is the download code:
def download
send_file "downloads/#{params[:filename]}.csv", type: "application/csv", x_sendfile: true
end
Here is a shortened version of the file creation:
def create_file
file_data = some_cool_data
file = create_file(File.join(Dir.pwd, "/downloads/#{#file_name}.csv"))
file.write(file_data)
file.close
file
end
def create_file(path)
FileUtils.mkdir_p(File.dirname(path))
File.new(path, "w+")
end
I dont think its an issue with the file reference (i.e. /dowloads vs downloads), because it works fine in testing and sometimes in production. It just seems that the file is getting deleted. What could be causing the missing file error?
I am using ActionMailer in a Ruby on Rails app to read emails (ruby 1.9.3, rails 3.2.13).
I have an email that has a winmail.dat file attached to it (ms-tnef) and I am using the tnef gem to extract its contents.
The problem is that when I read the attachment from the mail, it gets corrupted and tnef can not extract files from it.
$ tnef winmail.dat
ERROR: invalid checksum, input file may be corrupted
Extracting the winmail.dat attachment using any mail app, the extracted winmail.dat works fine with tnef and I got it's content.
Comparing the two files I noticed that:
- original file is bigger (76k against 72k)
- they differ on line breaks: Orginal file has the windows format (0D 0A) and the file saved by rails has the linux format (0A)
I wrote this test:
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
That fails with these messages:
WARNING: invalid checksum, input file may be corrupted
Invalid RTF CRC, input file may be corrupted
WARNING: invalid checksum, input file may be corrupted
Assertion failed: ((attr->lvl_type == LVL_MESSAGE) || (attr->lvl_type == LVL_ATTACHMENT)), function attr_read, file attr.c, line 240.
Errno::EPIPE: Broken pipe
anexo = message.attachments[0]
=> #<Mail::Part:2159872060, Multipart: false, Headers: <Content-Type: application/ms-tnef; name="winmail.dat">, <Content-Transfer-Encoding: quoted-printable>, <Content-Disposition: attachment; filename="winmail.dat">>
I tried to save it to disk as bynary, and read it again, but I got the same result
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
tmpfile_name = "#{::Rails.root}/tmp/#{anexo.filename}"
File.open(tmpfile_name, 'w+b', 0644) { |f| f.write anexo.body.decoded }
anexo = File.open(tmpfile_name)
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
How should I read the attachment?
The method anexo.body.decoded calls the decode method of the best suited encoding (Mail::Encodings) for the attachment, in your case quoted_printable.
Some of these encodings (7bit, 8bit and quoted_printable), perform a conversion, changing different types of line breaks to the platform specific line break.
the *quoted_printable" call .to_lf that corrupt the winmail.dat file
# Decode the string from Quoted-Printable. Cope with hard line breaks
# that were incorrectly encoded as hex instead of literal CRLF.
def self.decode(str)
str.gsub(/(?:=0D=0A|=0D|=0A)\r\n/, "\r\n").unpack("M*").first.to_lf
end
mail/core_extensions/string.rb:
def to_lf
to_str.gsub(/\n|\r\n|\r/) { "\n" }
end
To solve it you have perform the same encoding without the last .to_lf.
To do that you can create a new encoding that does not corrupt your file and use it to encode you attachment.
create the file:
lib/encodings/tnef_encoding.rb
require 'mail/encodings/7bit'
module Mail
module Encodings
# Encoding to handle Microsoft TNEF format
# It's pretty similar to quoted_printable, except for the 'to_lf' (decode) and 'to_crlf' (encode)
class TnefEncoding < SevenBit
NAME='tnef'
PRIORITY = 2
def self.can_encode?(str)
EightBit.can_encode? str
end
def self.decode(str)
# **difference here** removed '.to_lf'
str.gsub(/(?:=0D=0A|=0D|=0A)\r\n/, "\r\n").unpack("M*").first
end
def self.encode(str)
# **difference here** removed '.to_crlf'
[str.to_lf].pack("M")
end
def self.cost(str)
# These bytes probably do not need encoding
c = str.count("\x9\xA\xD\x20-\x3C\x3E-\x7E")
# Everything else turns into =XX where XX is a
# two digit hex number (taking 3 bytes)
total = (str.bytesize - c)*3 + c
total.to_f/str.bytesize
end
private
Encodings.register(NAME, self)
end
end
end
To use your custom encoding you must, first, register it:
Mail::Encodings.register('tnef', Mail::Encodings::TnefEncoding)
And then, set it as your preferred encoding for the attachment:
anexo.body.encoding('tnef')
Your test would, then, become:
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
tmpfile_name = "#{::Rails.root}/tmp/#{anexo.filename}"
Mail::Encodings.register('tnef', Mail::Encodings::TnefEncoding)
anexo.body.encoding('tnef')
File.open(tmpfile_name, 'w+b', 0644) { |f| f.write anexo.body.decoded }
anexo = File.open(tmpfile_name)
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
Hope it helps!
In the rails console:
ActionDispatch::Http::UploadedFile.new tempfile: 'tempfilefoo', original_filename: 'filename_foo.jpg', content_type: 'content_type_foo', headers: 'headers_foo'
=> #<ActionDispatch::Http::UploadedFile:0x0000000548f3a0 #tempfile="tempfilefoo", #original_filename=nil, #content_type=nil, #headers=nil>
I can write a string to #tempfile, and yet #original_filename, #content_type and #headers remain as nil
Why is this and how can I write information to these attributes?
And how can I read these attributes from a file instance?
i.e.
File.new('path/to/file.png')
It's not documented (and doesn't make much sense), but it looks like the options UploadedFile#initialize takes are :tempfile, :filename, :type and :head:
def initialize(hash) # :nodoc:
#tempfile = hash[:tempfile]
raise(ArgumentError, ':tempfile is required') unless #tempfile
#original_filename = encode_filename(hash[:filename])
#content_type = hash[:type]
#headers = hash[:head]
end
Changing your invocation to this ought to work:
ActionDispatch::Http::UploadedFile.new tempfile: 'tempfilefoo',
filename: 'filename_foo.jpg', type: 'content_type_foo', head: 'headers_foo'
Or you can set them after initialization:
file = ActionDispatch::Http::UploadedFile.new tempfile: 'tempfilefoo', filename: 'filename_foo.jpg'
file.content_type = 'content_type_foo'
file.headers = 'headers_foo'
I'm not sure I understand your second question, "And how can I read these attributes from a file instance?"
You can extract the filename (or last component) from any path with File.basename:
file = File.new('path/to/file.png')
File.basename(file.path) # => "file.png"
If you want to get the Content-Type that corresponds to a file extension, you can use Rails' Mime module:
type = Mime["png"] # => #<Mime::Type:... #synonyms=[], #symbol=:png, #string="text/png">
type.to_s # => "text/png"
You can put this together with File.extname, which gives you the extension:
ext = File.extname("path/to/file.png") # => ".png"
ext = ext.sub(/^\./, '') # => "png" (drop the leading dot)
Mime[ext].to_s # => "text/png"
You can see a list of all of the MIME types Rails knows about by typing Mime::SET in the Rails console, or looking at the source, which also shows you how to register other MIME types in case you're expecting other types of files.
the following should help you:
upload = ActionDispatch::Http::UploadedFile.new({
:tempfile => File.new("#{Rails.root}/relative_path/to/tempfilefoo") , #make sure this file exists
:filename => "filename_foo" # use this instead of original_filename
})
upload.headers = "headers_foo"
upload.content_type = "content_type_foo"
I didn't understand by "And how can I read these attributes from a file instance?", what you exactly want to do.
Perhaps if you want to read the tempfile, you can use:
upload.read # -> content of tempfile
upload.rewind # -> rewinds the pointer back so that you can read it again.
Hope it helps :) And let me know if I have misunderstood.
I'm trying to send a file attachment using Rubys ActionMailer. However, when I send my file, the carriage returns "\r" that I've added are removed.
string = "the quick brown\r\nfox jumped over\r\nthe bridge"
File.open(file = "attachment_#{Time.now.to_i}.txt", "w+") do |f|
f.write(string)
end
attachments['test_file.txt'] = {
mime_type: 'text/plain',
content: string
}
mail(
:to => 'somebody#example.com',
:from => 'somebody#example.com',
:subject => 'Message Test'
).deliver
The file that is written has the proper line endings, but the attached file has the carriage returns removed. How I can prevent this from happening?
So just wanted to post my solution in case anyone else ends up with this issue...
After checking the base64 encoded attachment from the email, I found that the string, did in fact, not have the carriage return.
1.9.3-p448 :001 > Base64.decode64('dGhlIHF1aWNrIGJyb3duCmZveCBqdW1wZWQgb3Zlcgp0aGUgYnJpZGdlCg==')
=> "the quick brown\nfox jumped over\nthe bridge\n"
This led me to believe that the ActionMailer was in fact reformatting my email before it was encoded. I figured that I could just encode the message body manually and send it over ....
encoded = Base64.encode64(string)
attachments['test_file.txt'] = {
mime_type: 'text/plain;charset=utf-8',
encoding: 'base64',
content: encoded
}
And that seems to have done the trick. My attachment now contains carriage return and line feed endings ("\r\n")
I'm not sure if this is expected functionality for the ActionMailer. I definitely didn't expect it.
I've written a basic Rails 3 application that shows a form and an upload form on specific URLs. It was all working fine yesterday, but now I'm running into several problems that require fixing. I'll try to describe each problem as best as I can. The reason i'm combining them, is because I feel they're all related and preventing me from finishing my task.
1. Cannot run the application in development mode
For some unknown reason, I cannot get the application to run in development mode. Currently i've overwritten the production.rb file from the environment with the settings from the development environment to get actuall stacktraces.
I've added the RailsEnv production setting to my VirtualHost setting in apache2, but it seems to make no difference. Nor does settings ENV variable to production.
2. ArgumentError on all calls
Whatever call I seem to make, results in this error message. The logfile tells me the following:
Started GET "/" for 192.168.33.82 at
Thu Apr 07 00:54:48 -0700 2011
ArgumentError (wrong number of
arguments (1 for 0)):
Rendered
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.6/lib/action_dispatch/middleware/templates/rescues/_trace.erb
(1.0ms) Rendered
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.6/lib/action_dispatch/middleware/templates/rescues/_request_and_response.erb
(4.1ms) Rendered
/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.6/lib/action_dispatch/middleware/templates/rescues/diagnostics.erb
within rescues/layout (8.4ms)
This means nothing to me really. I have no clue what's going wrong. I currently have only one controller which looks like this:
class SearchEngineController < ApplicationController
def upload
end
def search
#rows = nil
end
# This function will receive the query string from the search form and perform a search on the
# F.I.S.E index to find any matching results
def query
index = Ferret::Index::Index.new :path => "/public/F.I.S.E", :default_field => 'content'
#rows = Array.New
index.search_each "content|title:#{params[:query]}" do |id,score, title|
#rows << {:id => id, :score => score, :title => title}
end
render :search
end
# This function will receive the file uploaded by the user and process it into the
# F.I.S.E for searching on keywords and synonims
def process
index = Ferret::Index::Index.new :path => "public/F.I.S.E", :default_field => 'content'
file = File.open params[:file], "r"
xml = REXML::Document.new file
filename = params[:file]
title = xml.root.elements['//body/title/text()']
content = xml.root.elements['normalize-space(//body)']
index << { :filename => filename, :title => title, :content => content}
file.close
FileUtils.rm file
end
end
The routing of my application has the following setup: Again this is all pretty basic and probably can be done better.
Roularta::Application.routes.draw do
# define all the url paths we support
match '/upload' => 'search_engine#upload', :via => :get
match '/process' => 'search_engine#process', :via => :post
# redirect the root of the application to the search page
root :to => 'search_engine#search'
# redirect all incoming requests to the query view of the search engine
match '/:controller(/:action(/:id))' => 'search_engine#search'
end
If anyone can spot what's wrong and why this application is failing, please let me know. If needed I can edit this awnser and include additional files that might be required to solve this problem.
EDIT
i've managed to get further by renaming one of the functions on the controller. I renamed search into create and now I'm getting back HAML errors. Perhaps I used a keyword...?
woot, finally found the solutions....
Seems I used keywords to define my actions, and Rails didn't like this. This solved issue 2.
Issue 1 got solved by adding Rails.env= 'development' to the environment.rb file