How do you parse out just the body of an email using Ruby on Rails? Below is what I have come up with. I have down to the subject with the body, but all I want is the body. I am trying to parse this without using TMail or email based libraries.
show
<p>Body: <%= #text.parse_body %></p>
controller
def show
#text = Text.find(params[:id])
end
model
class Text < ActiveRecord::Base
attr_accessible :email
def text_input
#raw_email = email.to_s
end
def parse_body
#parse_email = #raw_email.match(/(\r\n\r\n[\W\w\S\s\D\d\b]*\r\n)/i)
#parse_body = #parse_email.to_s.gsub(/<[\W\w\S\s\D\d\b]*?>/, "")
end
result #parse_email
\r\n\r\n<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2//EN\">\r\n<HTML>\r\n<HEAD>\r\n<META
HTTP-EQUIV=3D\"Content-Type\" CONTENT=3D\"text/html; =\r\ncharset=3Diso-8859-1\">\r\n<META
NAME=3D\"Generator\" CONTENT=3D\"MS Exchange Server version
=\r\n6.5.7654.12\">\r\n<TITLE>Test for long subject</TITLE>\r\n</HEAD>\r\n<BODY>\r\n<!--
Converted from text/plain format -->\r\n\r\n<P><FONT
SIZE=3D2>Test<BR>\r\n<BR>\r\n</FONT>\r\n</P>\r\n\r\n</BODY>\r\n" 1:"\r\n\r\n<!DOCTYPE HTML
PUBLIC \"-//W3C//DTD HTML 3.2//EN\">\r\n<HTML>\r\n<HEAD>\r\n<META HTTP-EQUIV=3D\"Content-
Type\" CONTENT=3D\"text/html; =\r\ncharset=3Diso-8859-1\">\r\n<META NAME=3D\"Generator\"
CONTENT=3D\"MS Exchange Server version =\r\n6.5.7654.12\">\r\n<TITLE>Test for long
subject</TITLE>\r\n</HEAD>\r\n<BODY>\r\n<!-- Converted from text/plain format --
>\r\n\r\n<P><FONT SIZE=3D2>Body<BR>\r\n<BR>\r\n</FONT>\r\n</P>\r\n\r\n</BODY>\r\n
Result of #parse_body
Body: Test for long subject Body
Result I am trying to get
Body
You should use a gem that does this for you, such as mail! You could do this:
mail = Mail.read(#raw_email)
mail.subject
mail.body
etc. Best to avoid writing your own parser in situations like this, if possible!
Note: I guess you say that you want to avoid mail libraries, but why? It will help a lot. This one is pure ruby, too.
Related
Rails 4.5 Ruby 2.3.1
I am getting json from an API and trying to store the following into a model CLrates
1. timestamp as unix time (date)
2. Currency_code (string)
3. quote (decimal monetary value)
I can use the following in irb after parsing the json and know how to get the elements individually using: response["quotes"]. How can I generate params to be saved in the model above when the body is as follows:
irb(main):036:0> puts response.body
{
"success":true,
"terms":"https:\/\/xxxxx.com\/terms",
"privacy":"https:\/\/xxxxx.com\/privacy",
"timestamp":1504817289,
"source":"USD",
"quotes":{
"USDAED":3.672703,
"USDAFN":68.360001,
"USDCUC":1,
"USDCUP":26.5,
"USDCVE":91.699997,
"USDCZK":21.718701,
............ many more lines removed for brevity
"USDZWL":322.355011
}
I can do this using a separate associated model but have very little idea how to create the params to save to a single table.
The following links got me to this point and well worth a read if you need info on httparty GET (client):
1. http://www.rubydoc.info/github/jnunemaker/httparty/HTTParty/
2. http://eric-price.net/blog/rails-api-wrapper/
3. https://www.driftingruby.com/episodes/playing-with-json
The class and method in lib/clayer.rb:
class clayer
include HTTParty
format :json
read_timeout 10
def self.get_quotes
response = HTTParty.get('http://www.nnnnnnn.net/api/live?
access_key=nnnnnnnnnn&format=1')
end
end
I used irb as I am still learning how to run this through rails c. This will be called in the controller and saved however need to work out how to get the params from the json
Thanks for the help
OK: after digging I think I am on the right track
I get the response["QUOTES"], loop through them and build the params required saving each at the end of the loop
rates = response["QUOTES"]
rates.each do |k,v|
clrate = Realtimerates.new
clrate.date = response["timestamp"]
clrate.countrycode = "#{k}"
clrate.price = "#{v}"
clrate.save
end
Going to give this a whirl
In model
class Realtimerate < ActiveRecord::Base
include HTTParty
format :json
read_timeout 5
def self.get_cl_rates
response = HTTParty.get('http://www.mmmmm.net/api/live?access_key="key"&format=1')
rates = response["quotes"]
rates.each do |k,v|
create!(
date: Time.at(response["timestamp"]),
country_code: "#{k}",
price: "#{v}")
end
end
end
In the controller:
def index
Realtimerate.get_cl_rates
#realtimerates = Realtimerate.all
end
This is working and shows latest GET.
You already have a hash in your response.body. All you need to do now is to assign the relevant key-value to your model's attributes. e.g.
clrate = ClRate.new
res = response.body
clate.time = res["timestamp"]
...
clate.save
In my app, I store emails.
I want to parse those emails for email addresses in the text, on the fly and replace them with a link (so that we send the email through the app).
e.g.
#email.body = "Hi Tom, Drop me a line at jerry#cheese.com."
I want some sort of helper that will translate that on the fly to:
#email.sanitized_body
"Hi Tom, Drop me a line at #{link_to "Email", email_send_email_path("jerry#cheese.com")}."
I've been around a few circles.
e.g. in a model
Class Email
def sanitized_body
text = self.body
emails = text.scan(/\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b/i)
emails.each do |email|
text.gsub!("jerry#cheese.com", helper.link_to("email", "http://www.google.com"))
end
text
end
I'm sure there's a sensible way of doing this, probably with a helper but can't quite work it out...
module EmailsHelper
def sanitized_body(email_body)
text = email_body
emails = text.scan(/\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b/i)
emails.each do |email|
text.gsub!("jerry#cheese.com", "#{link_to("email", "http://www.google.com")}")
end
text
end
end
Gets me almost there. But the text comes out as text in the string when displayed.
Any help much appreciated.
You should use html_safe for your text to be rendered as HTML code instead of a simple string.
module EmailsHelper
def sanitized_body(email_body)
text = email_body
emails = text.scan(/\b[A-Z0-9._%+-]+#[A-Z0-9.-]+\.[A-Z]{2,4}\b/i)
replace_text = "You text %s" % helper.link_to("email", "http://www.google.com")
emails.each do |email|
text.gsub!("jerry#cheese.com", replace_text.html_safe)
end
text
end
end
What I want to achieve is the following:
Send an email with delayed_job containing:
plain-text
html (will be displayed by regular clients which don't understand the inline ical)
"inline" ical which is recognized by Outlook and Thunderbird (with Lightning).
a "regular" ical attachment (for #2)
What works so far/what does'nt:
I am able to send the email via delayed_job with all parts, however:
in Apple's Mail 2 attachments show up (instead of one):
(the html is displayed fine)
in Thunderbird (Lightning) I do get an invitation, just like I want. But the Alarm does not show up.
I have to do some REALLY disgusting gsubs on the rendered iCal in order for the ATTENDEES to show up. (see code snippet)
My thinking:
The first thing to keep in mind is: in order to send an email with attachments from delayed_job
To fix this, remember to add this line to your mailer: content_type "multipart/mixed"
As far as I understand the correct MIME-Type hierarchy would therefore be:
multipart/mixed
multipart/alternative
text/plain
text/html
text/calendar (with: method=REQUEST)
application/ics
Warning! code incoming.
I currently construct this email in the following manner:
Edit: I updated the mailer for Rails 4.2 (attachments must be placed before mail)
in my mailer.rb
def invitation_email(...)
subject = "I suck at email..."
attachments["invite.ics"] = { mime_type: "application/ics",
content: ical_attachment }
email = mail(from: me, to: you, subject: subject)
add_ical_part_to(email)
email
end
def add_ical_part_to(mail)
outlook_body = ical_attachment
mail.add_part(Mail::Part.new do
content_type "text/calendar; method=REQUEST"
body outlook_body
end)
end
and this is how I construct the ical attachments:
def ical_attachment
params_participant = {
"ROLE" => "REQ-PARTICIPANT",
"RSVP" => "FALSE",
"PARTSTAT" => "ACCEPTED"
}
params_invite = {
"CUTYPE" => 'INDIVIDUAL',
"ROLE" => "REQ-PARTICIPANT",
"PARTSTAT" => "NEEDS-ACTION",
"RSVP" => "TRUE"
}
cal = Icalendar::Calendar.new
event = Icalendar::Event.new
event.dtstart #party.from.to_datetime, { "VALUE" => "DATE" }
event.dtend #party.to.to_datetime, { "VALUE" => "DATE" }
event.summary #party.title
event.description #party.description
event.klass "PRIVATE"
event.organizer "cn=#{#user.name} #{#user.surname}:mailto:#{#user.email}"
# THIS DOES NOT WORK
event.alarm.trigger = "-PT5M" # 5 Minutes before...
#party.participations.each do |participation|
str = "cn=#{participation.user.name} #{participation.user.surname}:mailto:#{participation.user.email}"
event.add_attendee(str, params_participant)
end
#party.invitations.each do |invitee|
event.add_attendee("mailto:#{invitee.email}", params_invite)
end
cal.add_event(event)
cal.publish
# I KNOW THIS IS HORRIBLE AND I HATE IT, BUT OTHERWISE THE ATTENDEES DO NOT SHOW UP
cal.to_ical.gsub("ORGANIZER:", "ORGANIZER;").gsub("ACCEPTED:", "ACCEPTED;").gsub("TRUE:", "TRUE;").gsub("PUBLISH", "REQUEST")
end
Any help would be really appreciated!
The email that is being generated: http://pastebin.com/patf05zd
Oh and I'm on:
Rails 3.2.13
The Icalendar gem I'm using
In case someone else happens to come across this, here is what I did:
Instead of the icalendar gem I now use ri_cal. Although I was skeptical because the last commit to that repo was 3 years ago, the google group was a very helpful resource.
Here is how I generate the ical attachment (both inline and normal), which seems to be working fine (although it obviously needs some refactoring :))
def to_ical
# this is horrible
klass = self
cal = RiCal.Calendar do
event = event do
organizer "CN=#{klass.user.name} #{klass.user.surname}:mailto:#{klass.user.email}"
summary klass.party.title
description klass.ical_description
dtstart klass.party.from.utc.to_datetime
dtend klass.party.to.utc.to_datetime
location "See url in description"
security_class klass.security_class
# this is horrible
h = self
klass.party.participations.each do |participation|
h.add_attendee klass.prepare_participant(participation)
end
klass.party.invitations.each do |invitee|
h.add_attendee klass.prepare_invitee(invitee.email)
end
unless klass.party.reminder == 0
alarm do
description "Alarm description"
trigger klass.convert_trigger # -PT1H
action "DISPLAY"
end
end
end
end
# THE HORROR
cal.to_s.gsub("ATTENDEE:", "ATTENDEE")
.gsub("ORGANIZER:", "ORGANIZER;")
.gsub("CALSCALE:GREGORIAN", "CALSCALE:GREGORIAN\nMETHOD:REQUEST\n")
end
The 2 Attachments in Apples Mail still show up, I don't think that can be fixed.
Your second B64 encoded attachment contains a lot of garbage towards the end (attendee field).
That would explain the Thunderbird issue.
Please note that some clients will ignore any alarm you may set on a REQUEST: As an organizer, you should not dictate when each attendee should be reminded of the meeting. That would be a rather rude thing to do.
Regarding the Apple iCal issue, there is not much you can do I'm afraid: Some clients want the ics within, some as an attachment so you have to provide both. Does it show the accept/decline panel on iCal ?
I'm using the Mailman gem to process incoming email for my Rails app. My application looks for a YAML document in the plain-text email and then loads it into a Ruby object for further manipulation by the app.
However, I want to be able to plan ahead for email clients that might respond with a multi-part email. I need to get the plain-text part of the email and pass it into the YAML parser.
For some reason, it's still having problems parsing the YAML. I'm guessing because it's not really getting the plain text part here.
Is there a better way to get the text/plain part of an email with Mailman? Should I scrap Mailman and just get down and dirty with ActionMailer instead?
Mailman::Application.run do
default do
begin
message.parts.each do |part|
Mailman.logger.info part.content_type
if part.content_type == 'text/plain; charset=ISO-8859-1' # My poor way of getting the text part
the_yaml = part.body.decoded.scan(/(\-\-\-.*\.\.\.)/m).first.last # Find the YAML doc in the email and assign it to the_yaml
ruby_obj = YAML::load(the_yaml.sub(">", "")) # Remove any >'s automatically added by email clients
if ruby_obj['Jackpots']
ruby_obj['Jackpots'].each do |jackpot|
jp = Jackpot.find(jackpot['jackpot']['id'])
jp.prize = jackpot['jackpot']['prize']
jp.save
end
end
end
end
rescue Exception => e
Mailman.logger.error "Exception occurred while receiving message:\n#{message}"
Mailman.logger.error [e, *e.backtrace].join("\n")
end
end
end
I was able to find a little bit better way to handle getting the text part of the email.
Mailman::Application.run do
default do
begin
if message.multipart?
the_message = message.text_part.body.decoded
else
the_message = message.body.decoded
end
the_yaml = the_message.sub(">", "").scan(/(\-\-\-.*\.\.\.)/m).first.last
ruby_obj = YAML::load(the_yaml)
if ruby_obj['Jackpots']
ruby_obj['Jackpots'].each do |jackpot|
jp = Jackpot.find(jackpot['jackpot']['id'])
jp.prize = jackpot['jackpot']['prize']
jp.save
end
end
rescue Exception => e
Mailman.logger.error "Exception occurred while receiving message:\n#{message}"
Mailman.logger.error [e, *e.backtrace].join("\n")
end
end
end
And then after running it through the debugger and inspecting after the text part was successfully parsed. It would get hung up on the YAML loading. Turns out, a couple of my lines were too long, to the email client inserted a newline, breaking a comment in my YAML, and thus breaking the whole YAML document.
I have used the pop mailer for ruby (net/pop)
The problem I am having is that some of the emails are in HTML format is there a way to specify that I want plain text ?
Thanks, Alex
Emails can come in different formats. The most common is MIME which allows an email to contain multiple "parts". Commonly an HTML and a plain-text part. However, you can not control which parts the email actually contains. This can only the sender for obvious reasons.
You can however use ruby to get the plain text part if one is present or try to generate some representation of that from the HTML part.
The following condensed example to get the plain text part of an email is from the MailHandler model and the POP3 module of Redmine (licensed under GPLv2).
def plain_text_body(email)
parts = email.parts.collect {|c| (c.respond_to?(:parts) && !c.parts.empty?) ? c.parts : c}.flatten
if parts.empty?
parts << email
end
plain_text_part = parts.detect {|p| p.content_type == 'text/plain'}
if plain_text_part.nil?
# no text/plain part found, assuming html-only email
# strip html tags and remove doctype directive
plain_text_body = strip_tags(email.body.to_s)
plain_text_body.gsub! %r{^<!DOCTYPE .*$}, ''
else
plain_text_body = plain_text_part.body.to_s
end
plain_text_body.strip
end
pop = Net::POP3.APOP(true).new(host,port)
pop.start(username, password) do |pop_session|
if pop_session.mails.empty?
puts "No email to process"
else
puts "#{pop_session.mails.size} email(s) to process..."
pop_session.each_mail do |msg|
message = msg.pop
plain_text = plain_text_body(message)
#
# Now do something with the plain text body
#
end
end
end