In rails 4.2.4, I am using gem 'delayed_job_active_record' & gem 'daemons' for sending an emails.
Also I am importing a data from .csv and .xls file to my database. After saving an object I am trying to send an email using delay, but right now mail is sending only for data which is imported from .csv file. Mail is not sending for the data which is imported from .xls file.
I am using gem 'roo' for extracting xls file data.
In delayed_jobs table for spreadsheet data, handler field contains like below,
raw_attributes:
name: User1
email: &5 !ruby/string:Spreadsheet::Link
str: user1#gmail.com
url: mailto:user1#gmail.com
fragment:
In the same delayed_jobs table for csv data, handler field contains like below,
raw_attributes:
name: "User2\tuser2#gmail.com\t111111333"
id: 7
email:
mobile_number:
I am extracting xls file data like below,
def extract_asset_data_from_xls(asset)
require 'spreadsheet'
filename = "#{Rails.root}/#{asset.document.path}"
if File.exist?(filename)
file = File.open(filename)
if file
worksheet = Spreadsheet.open(file).worksheet(0)
1.upto worksheet.last_row_index do |index|
row = worksheet.row(index)
user = User.new
user.name = row[0]
user.email = row[1]
user.save
end
end
else
puts "#{filename} doesn't exists".red
return
end
end
How can I send an email to spreadsheet's data using delay?
Related
I have a data backup system for customers of my app. I gather up all associated csv files and zip them. Once that zip file is complete, I attach it in an email. This process breaks on heroku due to their file system. I thought since heroku-16 we could write to the app/tmp directory and that this process might occur within the same transaction and the files would be fine, but that doesn't seem to be the case. I don't even seem to be writing the files to the tmp directory in production (in Dev I am).
So, what I would like to do instead is just write the csv files directly to S3, then Zip those files and also save the .zip to S3...then, pull that file as an email attachment. To do this, I need to generate the csv files and write them to S3 from inside ActiveJob. I use S3 already as part of ActiveStorage, but this process will not utilize ActiveStorage.
Is there's a command for me to manually direct upload to an S3 bucket. I've been digging around in the docs, etc but don't see what I'm after.
The Job (using /tmp)
def perform(company_id, recipient_id)
company = Company.find(company_id)
source_folder = "#{ Rails.root }/tmp"
zipfile_name = "company_#{ company.id }_archive.zip"
zipfile_path = "#{ Rails.root }/tmp/#{ zipfile_name }"
input_filenames = []
# USERS: create a new empty csv file,
# ... then add rows to it
# ... and, add the file name to the list of files array
users_file_name = "#{ company.name.parameterize.underscore }_users_list.csv"
input_filenames << users_file_name
users_csv_file = File.new("#{ Rails.root.join('tmp') }/#{ users_file_name }", 'w')
users_csv_file << company.users.to_csv
users_csv_file.close
...
# gather up the created files and zip them
Zip::File.open(zipfile_path, create: true) do |zipfile|
input_filenames.uniq.each do |filename|
zipfile.add(filename, File.join(source_folder, filename))
end
end
puts "attaching data_export".colorize(:red)
company.data_exports.attach(
io: StringIO.new("#{ Rails.root }/tmp/company_14_#{ Time.current.to_date.to_s }_archive.zip"),
filename: 'company_14_archive.zip',
content_type: 'application/zip'
)
last_id = company.data_exports.last.id
puts "sending mail using company.id: #{ company.id }, recipient_id: #{ recipient_id }, company.data_exports.last.id: #{ last_id }".colorize(:red)
CompanyMailer.mail_data_export(
company.id,
recipient_id,
last_id
)
end
You can upload file like this on S3
key = "file_name.zip"
file_path = "tmp/file_name.zip"
new_s3_client = Aws::S3::Resource.new(region: 'eu-west-1', access_key_id: '123', secret_access_key: '456')
new_bucket = new_s3_client.bucket('public')
obj = new_bucket.object(key)
obj.upload_file(file_path)
In ActionMailer, I am trying to convert an array of array to a CSV and make sure the file can be read as if it were converted in UTF8 with BOM.
Previously, I was copying the content to a new file in Sublime Text, and clicking File > Save With Encoding > UTF8 with BOM otherwise the characters would end up messed up.
How can I achieve the same encoding while sending an in-memory CSV through ActionMailer (I am never writing the file to my disk)
Here is my sample code for sending the email
class CSVMailer < ApplicationMailer
def csv(csv_as_array_of_array,
to:,
cc: [],
from: 'messages-noreply#example.com',
reply_to: 'me#example.com',
subject: 'Here is your CSV made with love 😘',
filename: 'your_csv_made_with_love.csv'
)
attach_csv(csv_as_array_of_array, filename: filename)
mail(
to: to,
cc: cc,
subject: subject
)
end
private
def attach_csv(array_of_arrays, filename:)
attachments[filename] = {
mine_type: 'text/csv',
content: CSV.generate(col_sep: ';') do |csv|
array_of_arrays.each do |row|
csv << row
end
end
}
end
end
Alright here is how I did it, assuming you are sending the following array of arrays to the mailer function
csv_as_array_of_array = [
['header1', 'header2'],
['row1cell1', 'row1cell2'],
...
]
a simple utility function that prepends the UTF8 bom to the csv is enough
# Small utility method
def Utility.with_utf8_bom(content)
"\uFEFF" + content
end
So in the ActionMailer class
class CSVMailer < ApplicationMailer
# app/mailers/csv_mailer.rb
def send_email_with_csv(csv_as_array_of_array)
...
attach_csv(csv_as_array_of_array, filename: filename)
mail(to: ...)
end
private
def attach_csv(array_of_arrays, filename:)
attachments[filename] = {
mine_type: 'text/csv',
content: Utility.with_utf8_bom(
CSV.generate(col_sep: ';', encoding: Encoding::UTF_8) do |csv|
array_of_arrays.each do |row|
csv << row
end
end
)
}
end
I have a Rails 3.2 app that uses gem 'wicked_pdf', and gem 'combine_pdf'.
They both work and I can create PDFs which get emailed.
But, I have run into a situation where the email would be too big.
So, I'm trying to save the created pdf to Amazon S3. The app already has the gem 'aws-sdk'.
This is my code:
def self.saveallpdf
#costprojects = Costproject.where("client_id = 2")
pdf = CombinePDF.new
#costprojects.each do |costproject|
#costproject = costproject
controller = CostprojectsController.new
controller.instance_variable_set(:"#costproject", #costproject)
pdf2 = controller.render_to_string(pdf: "Captital Projects.pdf",
template: "costprojects/viewproject",
encoding: "UTF-8")
pdf << CombinePDF.parse(pdf2)
end
#s3 = AWS::S3.new
#bucket = #s3.buckets['ndeavor3-pdf']
#obj = #bucket.objects['filename'].write(pdf, acl: :public_read)
end
The error I'm getting is:
:data must be provided as a String, Pathname, File, or an object that responds to #read and #eof?
/app/vendor/bundle/ruby/1.9.1/gems/aws-sdk-1.8.3.1/lib/aws/s3/data_options.rb:125:in `validate_data!'
/app/vendor/bundle/ruby/1.9.1/gems/aws-sdk-1.8.3.1/lib/aws/s3/data_options.rb:32:in `compute_write_options'
/app/vendor/bundle/ruby/1.9.1/gems/aws-sdk-1.8.3.1/lib/aws/s3/s3_object.rb:594:in `write'
/app/app/models/costproject.rb:167:in `saveallpdf
'
I guess was-sdk doesn't like the "pdf" as the file??
PS - I can email the "pdf" - if it was smaller in size.
Thanks for your help!
At this point in time:
#obj = #bucket.objects['filename'].write(pdf, acl: :public_read)
pdf is a CombinePDF object and not a File, String, or Pathname
pdf.to_s might work, or you will have to create a new file from the CombinePDF object
File.new(CombinePDF) # pseduo code only
I am using ActionMailer in a Ruby on Rails app to read emails (ruby 1.9.3, rails 3.2.13).
I have an email that has a winmail.dat file attached to it (ms-tnef) and I am using the tnef gem to extract its contents.
The problem is that when I read the attachment from the mail, it gets corrupted and tnef can not extract files from it.
$ tnef winmail.dat
ERROR: invalid checksum, input file may be corrupted
Extracting the winmail.dat attachment using any mail app, the extracted winmail.dat works fine with tnef and I got it's content.
Comparing the two files I noticed that:
- original file is bigger (76k against 72k)
- they differ on line breaks: Orginal file has the windows format (0D 0A) and the file saved by rails has the linux format (0A)
I wrote this test:
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
That fails with these messages:
WARNING: invalid checksum, input file may be corrupted
Invalid RTF CRC, input file may be corrupted
WARNING: invalid checksum, input file may be corrupted
Assertion failed: ((attr->lvl_type == LVL_MESSAGE) || (attr->lvl_type == LVL_ATTACHMENT)), function attr_read, file attr.c, line 240.
Errno::EPIPE: Broken pipe
anexo = message.attachments[0]
=> #<Mail::Part:2159872060, Multipart: false, Headers: <Content-Type: application/ms-tnef; name="winmail.dat">, <Content-Transfer-Encoding: quoted-printable>, <Content-Disposition: attachment; filename="winmail.dat">>
I tried to save it to disk as bynary, and read it again, but I got the same result
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
tmpfile_name = "#{::Rails.root}/tmp/#{anexo.filename}"
File.open(tmpfile_name, 'w+b', 0644) { |f| f.write anexo.body.decoded }
anexo = File.open(tmpfile_name)
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
How should I read the attachment?
The method anexo.body.decoded calls the decode method of the best suited encoding (Mail::Encodings) for the attachment, in your case quoted_printable.
Some of these encodings (7bit, 8bit and quoted_printable), perform a conversion, changing different types of line breaks to the platform specific line break.
the *quoted_printable" call .to_lf that corrupt the winmail.dat file
# Decode the string from Quoted-Printable. Cope with hard line breaks
# that were incorrectly encoded as hex instead of literal CRLF.
def self.decode(str)
str.gsub(/(?:=0D=0A|=0D|=0A)\r\n/, "\r\n").unpack("M*").first.to_lf
end
mail/core_extensions/string.rb:
def to_lf
to_str.gsub(/\n|\r\n|\r/) { "\n" }
end
To solve it you have perform the same encoding without the last .to_lf.
To do that you can create a new encoding that does not corrupt your file and use it to encode you attachment.
create the file:
lib/encodings/tnef_encoding.rb
require 'mail/encodings/7bit'
module Mail
module Encodings
# Encoding to handle Microsoft TNEF format
# It's pretty similar to quoted_printable, except for the 'to_lf' (decode) and 'to_crlf' (encode)
class TnefEncoding < SevenBit
NAME='tnef'
PRIORITY = 2
def self.can_encode?(str)
EightBit.can_encode? str
end
def self.decode(str)
# **difference here** removed '.to_lf'
str.gsub(/(?:=0D=0A|=0D|=0A)\r\n/, "\r\n").unpack("M*").first
end
def self.encode(str)
# **difference here** removed '.to_crlf'
[str.to_lf].pack("M")
end
def self.cost(str)
# These bytes probably do not need encoding
c = str.count("\x9\xA\xD\x20-\x3C\x3E-\x7E")
# Everything else turns into =XX where XX is a
# two digit hex number (taking 3 bytes)
total = (str.bytesize - c)*3 + c
total.to_f/str.bytesize
end
private
Encodings.register(NAME, self)
end
end
end
To use your custom encoding you must, first, register it:
Mail::Encodings.register('tnef', Mail::Encodings::TnefEncoding)
And then, set it as your preferred encoding for the attachment:
anexo.body.encoding('tnef')
Your test would, then, become:
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
tmpfile_name = "#{::Rails.root}/tmp/#{anexo.filename}"
Mail::Encodings.register('tnef', Mail::Encodings::TnefEncoding)
anexo.body.encoding('tnef')
File.open(tmpfile_name, 'w+b', 0644) { |f| f.write anexo.body.decoded }
anexo = File.open(tmpfile_name)
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
Hope it helps!
i'm want to generate CSV data and send it via mail to some email-address. For the generation of the CSV i'm using FasterCSV with the following code:
csv_data = FasterCSV.generate(:col_sep => ";") do |csv|
csv << ["timestamp", "staff_firstname", "staff_lastname", "message"]
log.each do |log_entry|
csv << [log_entry.timestamp, log_entry.staff_firstname, log_entry.staff_lastname, log_entry.message]
end
end
The csv_data i want to send via a ActionMailer method and therefore i'm using the following code:
def log_csv_export(log_csv, email)
mail.attachments["log.csv"] = log_csv
mail(:to => email, :subject => 'Export Log' )
end
To call the ActionMailer method i'm using:
AccountMailer.log_csv_export(csv_data, email).deliver
If I test it, the mail was send to the transmitted email address, but without an attachment. The csv-data is shown as plain text in the email, but not as attachment to save.
This problem only occurs if i send the mail via heroku mailgun. If i'm testing it with
ActionMailer::Base.delivery_method = :sendmail in the config, then it works.
Did someone knows what the issue is or what i need to change that it works?
Thank you.