I am needing to extract specific lines from a raw email that i have coming into my webhook.site url. I figured it would be an xpath extract or some custom action but my knowledge only goes so far.
i need to extract: Subject: which is the header about the content-type: text/plain.
Whats my path and layout to do so??
thanks
Return-Path: <leadnotification-noreply2#ylopo.com>
Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53])
by inbound-smtp.eu-west-1.amazonaws.com with SMTP id nqoso8q41s14sho14modqhpqbqvvvcqb0e3hea81
for OsirsYlopoPriorityEmail#email.webhook.site;
Wed, 27 Oct 2021 19:21:34 +0000 (UTC)
X-SES-Spam-Verdict: PASS
X-SES-Virus-Verdict: PASS
Received-SPF: pass (spfCheck: domain of ylopo.com designates 209.85.167.53 as permitted sender) client-ip=209.85.167.53; envelope-from=leadnotification-noreply2#ylopo.com; helo=mail-lf1-f53.google.com;
Authentication-Results: amazonses.com;
spf=pass (spfCheck: domain of ylopo.com designates 209.85.167.53 as permitted sender) client-ip=209.85.167.53; envelope-from=leadnotification-noreply2#ylopo.com; helo=mail-lf1-f53.google.com;
dkim=pass header.i=#ylopo.com;
dmarc=pass header.from=ylopo.com;
X-SES-RECEIPT: AEFBQUFBQUFBQUFHdlpaMkNVNS9wd25QYmtqSU9xbGJBVzZJa0tSV1dCNWcwWFFwZFNUS1lweHpxY1A1NlRoSlZyU1NEM0drMlp3Q0Jpd0d2ZG1RUC9VYVRCbWt6UUhMdkFwOUJLS1NGYnFCSzQyVGpQK1loZzU0SkpIcy9pNnQ4aHhScnV2dG9sV2M5b2VnSzY4MU0vMm9JaWx5VDJOcW5WWllPRzhvNkp6VHdJSWNmbmJBd1lvZlF1WHdEQUNFRzkyM0dQQVkxdGgwS0NwUHAzcVo5dit5clgvOXFzdG8xMFJaTVpuMVRLcXUwVDNNMVRod3lWSDB0NStRNjNUcEMrZ2RwZ0dSZFBQcEQ2OXZ2bUFOMkI3UUhJYUNZdG5iM2hqMUZhV1NST0FBbHdwSUM5TDIySk11dWh2alN3VE9BbVhGWjlKditMUzQ9
X-SES-DKIM-SIGNATURE: a=rsa-sha256; q=dns/txt; b=AXzUmXzoB1/c89r/6eJBBuEUriK6kdQMMJRXPnBwlUC5oCKD1Apk+Av3zpg5yxj+4djsbxeCsSPIwkcYX6xlXUfWUQ/mi7pgHViuVh+r/NrEKptMjb5efdeH7/mls8tRzyaQQF+12LbSb0wBo2bTkcQLXkP3WvKP5OSFde3B620=; c=relaxed/simple; s=uku4taia5b5tsbglxyj6zym32efj7xqv; d=amazonses.com; t=1635362495; v=1; bh=5WqfRbOb11+ge2mx0Egi4AqA6n1lyVIU1IpAVg4H0dA=; h=From:To:Cc:Bcc:Subject:Date:Message-ID:MIME-Version:Content-Type:X-SES-RECEIPT;
Received: by mail-lf1-f53.google.com with SMTP id y26so8295478lfa.11
for <OsirsYlopoPriorityEmail#email.webhook.site>; Wed, 27 Oct 2021 12:21:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=ylopo.com; s=google;
h=mime-version:from:date:message-id:subject:to;
bh=nrZwBjOzEgZN0tQ61MtDEDWEj3XJn2q+y4Qx1/acY5A=;
b=gtj7GzFH9q0O2S1ynt3Qvhp5zgomrYmufqbSQ0qIjEalwk9Dd0lSI7MeOMrgNtjDFL
sGwRBO9L4ZW3yE5ZKmP/wSYKmVlerL51ZlTQQhuTXsxioymJto3j0ERWirJQj+BapzGT
HBxScQEwYkpqZqWX6KkCTjCzCGZqW+fp9vitHmgfqt1/nLiyZp+7WEbluw+rPQO0G7dR
CGObjTeYa0Fd+Dc8h/k/a7suZ2umrqqnl/HYaoY7BeMxhAJDP5TuaoAsjQh1EU9zqHY8
TJxcJZoo83n+7f8qVMSNpAstVynlmsH6h7nzW1q27pfeWWY6LgMRUjkKqrYIc4F8lsqR
98SQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
bh=nrZwBjOzEgZN0tQ61MtDEDWEj3XJn2q+y4Qx1/acY5A=;
b=D/VtUGyzcVfTxi2x63V/3OC7ckHj1uuLlILVUNmOkMrvW6GWUJq7mKI9/D4UYXWc5i
Ybds0/Dktx6cDJUZbAiAYWOiYd56XMMc2O+Yoe36u+eREry29IJQXOLTcRj2KFcLGSWa
Me5GjcFPVBTuhjtxlPb41wCKhGmDevYEHDkbGIcoNp5w3weGobSPg8bLQNsEO/Hspn6y
4Q18s7uNIJ8X2o5DEevA8DrZfibThQ3X5HUFmpuaRT3089Qm3H92wiHv3rkn3fQFDnVY
4p/PcWw9bxK6pU47bBtO/qtJ2ce/3Q7OQq/NCvJ5ZjD83lRai1mKIaCtiyt3gccQrd5S
hc5g==
X-Gm-Message-State: AOAM531woVO34G4bG9aVBLV9ae6QgX8pklYV4DDYs4VALPxuz7W9GStL
QM3YDolGrDOXbzaZghT/1+So8DR3WqejlOCELMsa+zO2Xxs=
X-Google-Smtp-Source: ABdhPJylxjs2zEdxsuBr2M1l9CnsOHrWvk+53vOLw2bs77bdl75s4022uZaKlqYx/GG+UyEsRrt8DT6mRRCAQZSebbE=
X-Received: by 2002:a05:6512:acf:: with SMTP id n15mr578213lfu.222.1635362493750;
Wed, 27 Oct 2021 12:21:33 -0700 (PDT)
Received: from 927538837578 named unknown by gmailapi.google.com with
HTTPREST; Wed, 27 Oct 2021 12:21:33 -0700
MIME-Version: 1.0
From: leadnotification-noreply2#ylopo.com
Date: Wed, 27 Oct 2021 12:21:33 -0700
Message-ID: <CAN2r-3o0aD_98y2GrdxdBW7W6UHi8RMS9-JmvsHrftheurwMeQ#mail.gmail.com>
Subject: Ylopo Priority Alert - Party: Daniel Askew 19293 -
PRIORITY_LEAD_EVENT - massaquoimartha#yahoo.com - 8562838525
To: OsirsYlopoPriorityEmail#email.webhook.site, qojfsghi#mailparser.io
Content-Type: multipart/alternative; boundary="00000000000084f8be05cf5a80dc"
--00000000000084f8be05cf5a80dc
Content-Type: text/plain; charset="UTF-8"
Lead Name: Martha Mansaray
Lead Email: massaquoimartha#yahoo.com
Lead Phone: 8562838525
Text:
Ylopo PRIORITY LEAD ALERT: Martha Mansaray (856) 283-8525
Martha Mansaray VIEWED 6185 Old Highway 31E, Bethpage, TN
<https://andrea.livetn.com/listing-detail/124037148> 29 TIMES.
Recommend actions:
I think you need regex because XPath is used to find a node from XML/HTML type text. But regex can be used on any text. To get the value of header name "Subject" you can use regex \nSubject: (.*).
Python example:
import re
sample = """
...
<text you want to parse>
...
Subject: Ylopo Priority Alert - Party: Daniel Askew 19293 -
"""
if match := re.search(r"\nSubject: (.*)", sample):
print(match.group(1)) # output: Ylopo Priority Alert - Party: Daniel Askew 19293 -
When I send a message with attachment using the Gmail API, the recipient receives the message without the attachment.
What is strange though is that:
1: in the sent folder of the sender, I do see the attachment properly
2: if I send to myself, both message are fine (in sent folder and in inbox folder)
3: if I use with GMail SMTP with the same raw message, it works fine
4: if I use a 3rd party SMTP with the same raw message, it works fine.
Point number 1+2 super puzzling.
Here is the source of the original message in the sent folder:
Received: from 13936824666 named unknown by gmailapi.google.com with HTTPREST; Wed, 25 Jan 2017 18:44:30 -0500
Date: Wed, 25 Jan 2017 18:44:30 -0500
From: Jeremy Chatelaine <source#gmail.com>
To: Jeremy <target#domain.com>
Message-Id: <CABX8Avad0vTtu8=jotRD5HM1r0My-ZKeV7RFRo0TmTxf5PNd0g#mail.gmail.com>
Subject: Export
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="--==_mimepart_5889385be5066_a133fd0f785e20837629"; charset=UTF-8
Content-Transfer-Encoding: 7bit
----==_mimepart_5889385be5066_a133fd0f785e20837629
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Please find attached the message
----==_mimepart_5889385be5066_a133fd0f785e20837629
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit
Please find attached the message
----==_mimepart_5889385be5066_a133fd0f785e20837629
Content-Type: text/csv; charset=UTF-8; filename=export.csv
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename=export.csv
Content-ID: <5889385be6fd7_a133fd0f785e20837735#jeremy.mail>
Some text...
----==_mimepart_5889385be5066_a133fd0f785e20837629--
Here is the source of the original message in the recipient inbox folder:
Delivered-To: target#domain.com
Received: by 10.55.110.193 with SMTP id j184csp1999707qkc;
Wed, 25 Jan 2017 16:30:42 -0800 (PST)
X-Received: by 10.107.34.213 with SMTP id i204mr540101ioi.203.1485390642385;
Wed, 25 Jan 2017 16:30:42 -0800 (PST)
Return-Path: <source#domain.com>
Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com. [2607:f8b0:4001:c0b::22b])
by mx.google.com with ESMTPS id 88si336719ioq.54.2017.01.25.16.30.42
for <target#domain.com>
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Wed, 25 Jan 2017 16:30:42 -0800 (PST)
Received-SPF: pass (google.com: domain of source#domain.com designates 2607:f8b0:4001:c0b::22b as permitted sender) client-ip=2607:f8b0:4001:c0b::22b;
Authentication-Results: mx.google.com;
dkim=pass header.i=#domain.com;
spf=pass (google.com: domain of source#domain.com designates 2607:f8b0:4001:c0b::22b as permitted sender) smtp.mailfrom=source#domain.com
Received: by mail-it0-x22b.google.com with SMTP id 203so119411500ith.0
for <target#domain.com>; Wed, 25 Jan 2017 16:30:42 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=domain.com; s=domain;
h=from:mime-version:date:message-id:subject:to;
bh=wx26/V0bJk9VItDp3TAvKl28UAn7IRQq4NITJZDM+Co=;
b=L3KTzPTCoIJUfAacuJy+PE8jHnY9iwGuXUWSpZzneRs5bvMysigSMyPGn1YicyIvQ6
d/LvbEJPlsu+S0zElhIVPITjAmXKDKNIKwLQDHpkcKnnI3btBUrENN923fMtS1fDdHyV
3At0QenKrb34uQqYYoHtX2WU4nyYrISbYKL62=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:mime-version:date:message-id:subject:to;
bh=wx26/V0bJk9VItDp3TAvKl28UAn7IRQq4NITJZDM+Co=;
b=tGPSHjI3iRzjrI0jGHVX76uTw7OXYC4B2bP0qQKgQotBWru+Pn5Ci9A1Qop9oGN1Ys
fnxCgLOLG8ZJU165ODBNX1DGjPa8ud9SWg18FTsxIjNw9qTr1yJqbWr0LToJi7HdQUr8
7Aaiqil7PbPUf5SdxLCqwBNf660Rn9Sd/ADZeT1Bc2+iYQcizjiK/rOPPX+X1ZndvqxP
Ok3Ac2yyIWxi+m4xaPEztcF4JXFZDlJWFdclUDv4s5Jdc0eb1HmB5d2r1qroGLo5MTjd
d7jO1nRsKTO5I9I69p9AgC+LpDiWBxgzZMBVsU6vVpeZ03/pCroyk9DHDUAjn3ijtlFh
O2vw==
X-Gm-Message-State: AIkVDXJo2tPJlkHkthgEjnxYp7vz5Rfpn91pWCS7zEurkSiDhJtyzLpUoSDORq37K/7ATCRSyLAypKIYrdYwEiL2
X-Received: by 10.36.84.148 with SMTP id t142mr20263701ita.90.1485390641934; Wed, 25 Jan 2017 16:30:41 -0800 (PST)
Received: from 13936824667 named unknown by gmailapi.google.com with HTTPREST; Wed, 25 Jan 2017 19:30:41 -0500
From: Jeremy Chatelaine <source#domain.com>
Mime-Version: 1.0
Date: Wed, 25 Jan 2017 19:30:41 -0500
Message-ID: <CABK5xHonVstaJHWHfvBJFF1BK5Y0B8NJsAJTPFokPNAUcawhGA#mail.gmail.com>
Subject: Export
To: Jeremy <target#domain.com>
Content-Type: multipart/alternative; boundary=001a1143a6cc90c9040546f4753f
--001a1143a6cc90c9040546f4753f
Content-Type: text/plain; charset=UTF-8
Please find attached your requested export
--001a1143a6cc90c9040546f4753f
Content-Type: text/html; charset=UTF-8
Please find attached your requested export
--001a1143a6cc90c9040546f4753f--
As you can see, the mime part where I had my text attachment vanished.
Here is how the message is produced (cut for clarity)
msg = Mail.new
html_part = Mail::Part.new do
content_type 'text/html; charset=UTF-8'
body html_body
end
msg.html_part = html_part
new_text = plan_text
text_part = Mail::Part.new do
body new_text
end
msg.text_part = text_part
file_paths.each do |file_path|
msg.add_file(file_path)
# I also tried like that, same result
#open(file_path) do |file|
# msg.attachments[file_path] = file.read
#end
end
raw_message = msg.to_s
Here is how I send with the gmail api
client = Google::APIClient.new(:application_name => "app", :application_version => "1")
client.authorization.client_id = "someverylongnumbers.apps.googleusercontent.com"
client.authorization.client_secret = "morerandomletters"
client.authorization.access_token = token
client.authorization.scope = [
"https://www.googleapis.com/auth/gmail.modify"
]
gmail_api = client.discovered_api('gmail', 'v1') # https://www.googleapis.com/auth/gmail.modify
result = client.execute(
:api_method => gmail_api.users.messages.to_h['gmail.users.messages.send'],
:parameters => {
'userId' => "me"
},
:body_object => {
'raw' => Base64.urlsafe_encode64(raw_message)
}
)
What is wrong with this?
According to the documentation, we should be looking at a parameter called "message", which doesn't not come in. What does come in is a parameter called "mandrill_events".
require 'mail'
class InboxController < ApplicationController
skip_before_filter :verify_authenticity_token, only: :create
def create
mail = Mail.new(params[:message])
....
No dice, but I do see this in the console log when I use the following syntax (note, it's so much JSON that SO has trouble viewing it with formatting):
puts JSON.parse(params[:mandrill_events])
{"mandrill_events"=>"[{\"event\":\"inbound\",\"ts\":1426188360,\"msg\":{\"raw_msg\":\"Received: from mail-ob0-f181.google.com (unknown [209.85.214.181])\\n\\tby ip-10-39-136-77 (Postfix) with ESMTPS id 25B0E2C0509\\n\\tfor <test#inbound.diabetesdelivery.com>; Thu, 12 Mar 2015 19:26:00 +0000 (UTC)\\nReceived: by obcvb8 with SMTP id vb8so16123307obc.10\\n for <test#inbound.diabetesdelivery.com>; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed\\/relaxed;\\n d=gmail.com; s=20120113;\\n h=mime-version:date:message-id:subject:from:to:content-type;\\n bh=jyd6a9aMNBrnuDpbsOEokXnACKw\\/lLnh72zkG7LCs3c=;\\n b=wzt6xiplBtL6dq7uGR8RV6tRR9Lcsmj5zmIktZ7KXFcxbwezN+uEhK\\/XFw6ZjIfhp6\\n LVoEH5ljLjOwK4X4nrtHaPoNzHZizwQUzxk0qXGabTXHW6tqOLZUs3FeDwtt2ekeLvlb\\n bDuU4VcBnNZ5VkXDpA5hQeysCfGECBilAtMy\\/EBjdOngcgk4Fsp8u11\\/eeB2xqBPUcrR\\n JYeu\\/99XossdoeJW3avfxFDBw7ngc98oRS2ZPpyL7MY36XoCZCM25U9ue9kBcJ82n92i\\n ctbNdl+9ikVPdcmUokI0WIvBIBGl5yAcrvOQffxmaTz0c31fkq8+iyprDMP\\/GexVfQzN\\n SAKA==\\nMIME-Version: 1.0\\nX-Received: by 10.202.89.135 with SMTP id n129mr34093639oib.60.1426188359432;\\n Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\\nReceived: by 10.60.119.68 with HTTP; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\\nDate: Thu, 12 Mar 2015 12:25:59 -0700\\nMessage-ID: <CAMf1_47jvgW6QMwDx8R9JU6C5YiUMaPjvFz3muxhM5v4HJyoSg#mail.gmail.com>\\nSubject: Subject FTW?\\nFrom: Dee diabetic <Deediabetic#gmail.com>\\nTo: test#inbound.diabetesdelivery.com\\nContent-Type: multipart\\/alternative; boundary=001a113d39288bfe8505111c5a88\\n\\n--001a113d39288bfe8505111c5a88\\nContent-Type: text\\/plain; charset=UTF-8\\n\\nBody here\\n\\n--001a113d39288bfe8505111c5a88\\nContent-Type: text\\/html; charset=UTF-8\\n\\n<div dir=\\\"ltr\\\">Body here<\\/div>\\n\\n--001a113d39288bfe8505111c5a88--\",\"headers\":{\"Received\":[\"from mail-ob0-f181.google.com (unknown [209.85.214.181]) by ip-10-39-136-77 (Postfix) with ESMTPS id 25B0E2C0509 for <test#inbound.diabetesdelivery.com>; Thu, 12 Mar 2015 19:26:00 +0000 (UTC)\",\"by obcvb8 with SMTP id vb8so16123307obc.10 for <test#inbound.diabetesdelivery.com>; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\",\"by 10.60.119.68 with HTTP; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\"],\"Dkim-Signature\":\"v=1; a=rsa-sha256; c=relaxed\\/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=jyd6a9aMNBrnuDpbsOEokXnACKw\\/lLnh72zkG7LCs3c=; b=wzt6xiplBtL6dq7uGR8RV6tRR9Lcsmj5zmIktZ7KXFcxbwezN+uEhK\\/XFw6ZjIfhp6 LVoEH5ljLjOwK4X4nrtHaPoNzHZizwQUzxk0qXGabTXHW6tqOLZUs3FeDwtt2ekeLvlb bDuU4VcBnNZ5VkXDpA5hQeysCfGECBilAtMy\\/EBjdOngcgk4Fsp8u11\\/eeB2xqBPUcrR JYeu\\/99XossdoeJW3avfxFDBw7ngc98oRS2ZPpyL7MY36XoCZCM25U9ue9kBcJ82n92i ctbNdl+9ikVPdcmUokI0WIvBIBGl5yAcrvOQffxmaTz0c31fkq8+iyprDMP\\/GexVfQzN SAKA==\",\"Mime-Version\":\"1.0\",\"X-Received\":\"by 10.202.89.135 with SMTP id n129mr34093639oib.60.1426188359432; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\",\"Date\":\"Thu, 12 Mar 2015 12:25:59 -0700\",\"Message-Id\":\"<CAMf1_47jvgW6QMwDx8R9JU6C5YiUMaPjvFz3muxhM5v4HJyoSg#mail.gmail.com>\",\"Subject\":\"Subject FTW?\",\"From\":\"Dee diabetic <Deediabetic#gmail.com>\",\"To\":\"test#inbound.diabetesdelivery.com\",\"Content-Type\":\"multipart\\/alternative; boundary=001a113d39288bfe8505111c5a88\"},\"text\":\"Body here\\n\\n\",\"text_flowed\":false,\"html\":\"<div dir=\\\"ltr\\\">Body here<\\/div>\\n\\n\",\"from_email\":\"Deediabetic#gmail.com\",\"from_name\":\"Dee diabetic\",\"to\":[[\"test#inbound.diabetesdelivery.com\",null]],\"subject\":\"Subject FTW?\",\"spf\":{\"result\":\"pass\",\"detail\":\"sender SPF authorized\"},\"spam_report\":{\"score\":0.5,\"matched_rules\":[{\"name\":\"RCVD_IN_DNSWL_LOW\",\"score\":-0.7,\"description\":\"RBL: Sender listed at http:\\/\\/www.dnswl.org\\/, low\"},{\"name\":null,\"score\":0,\"description\":null},{\"name\":\"listed\",\"score\":0,\"description\":\"in list.dnswl.org]\"},{\"name\":\"FREEMAIL_FROM\",\"score\":0,\"description\":\"Sender email is commonly abused enduser mail provider\"},{\"name\":\"HTML_MESSAGE\",\"score\":0,\"description\":\"BODY: HTML included in message\"},{\"name\":\"DKIM_VALID_AU\",\"score\":-0.1,\"description\":\"Message has a valid DKIM or DK signature from author's\"},{\"name\":\"DKIM_SIGNED\",\"score\":0.1,\"description\":\"Message has a DKIM or DK signature, not necessarily valid\"},{\"name\":\"DKIM_VALID\",\"score\":-0.1,\"description\":\"Message has at least one valid DKIM or DK signature\"},{\"name\":\"RDNS_NONE\",\"score\":1.3,\"description\":\"Delivered to internal network by a host with no rDNS\"}]},\"dkim\":{\"signed\":true,\"valid\":true},\"email\":\"test#inbound.diabetesdelivery.com\",\"tags\":[],\"sender\":null,\"template\":null}}]"}
Mar 12 12:26:046666 app/web.1: {"event"=>"inbound", "ts"=>1426188360, "msg"=>{"raw_msg"=>"Received: from mail-ob0-f181.google.com (unknown [209.85.214.181])\n\tby ip-10-39-136-77 (Postfix) with ESMTPS id 25B0E2C0509\n\tfor ; Thu, 12 Mar 2015 19:26:00 +0000 (UTC)\nReceived: by obcvb8 with SMTP id vb8so16123307obc.10\n for ; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\nDKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=gmail.com; s=20120113;\n h=mime-version:date:message-id:subject:from:to:content-type;\n bh=jyd6a9aMNBrnuDpbsOEokXnACKw/lLnh72zkG7LCs3c=;\n b=wzt6xiplBtL6dq7uGR8RV6tRR9Lcsmj5zmIktZ7KXFcxbwezN+uEhK/XFw6ZjIfhp6\n LVoEH5ljLjOwK4X4nrtHaPoNzHZizwQUzxk0qXGabTXHW6tqOLZUs3FeDwtt2ekeLvlb\n bDuU4VcBnNZ5VkXDpA5hQeysCfGECBilAtMy/EBjdOngcgk4Fsp8u11/eeB2xqBPUcrR\n JYeu/99XossdoeJW3avfxFDBw7ngc98oRS2ZPpyL7MY36XoCZCM25U9ue9kBcJ82n92i\n ctbNdl+9ikVPdcmUokI0WIvBIBGl5yAcrvOQffxmaTz0c31fkq8+iyprDMP/GexVfQzN\n SAKA==\nMIME-Version: 1.0\nX-Received: by 10.202.89.135 with SMTP id n129mr34093639oib.60.1426188359432;\n Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\nReceived: by 10.60.119.68 with HTTP; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)\nDate: Thu, 12 Mar 2015 12:25:59 -0700\nMessage-ID: \nSubject: Subject FTW?\nFrom: Dee diabetic \nTo: test#inbound.diabetesdelivery.com\nContent-Type: multipart/alternative; boundary=001a113d39288bfe8505111c5a88\n\n--001a113d39288bfe8505111c5a88\nContent-Type: text/plain; charset=UTF-8\n\nBody here\n\n--001a113d39288bfe8505111c5a88\nContent-Type: text/html; charset=UTF-8\n\nBody here\n\n--001a113d39288bfe8505111c5a88--", "headers"=>{"Received"=>["from mail-ob0-f181.google.com (unknown [209.85.214.181]) by ip-10-39-136-77 (Postfix) with ESMTPS id 25B0E2C0509 for ; Thu, 12 Mar 2015 19:26:00 +0000 (UTC)", "by obcvb8 with SMTP id vb8so16123307obc.10 for ; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)", "by 10.60.119.68 with HTTP; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)"], "Dkim-Signature"=>"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=jyd6a9aMNBrnuDpbsOEokXnACKw/lLnh72zkG7LCs3c=; b=wzt6xiplBtL6dq7uGR8RV6tRR9Lcsmj5zmIktZ7KXFcxbwezN+uEhK/XFw6ZjIfhp6 LVoEH5ljLjOwK4X4nrtHaPoNzHZizwQUzxk0qXGabTXHW6tqOLZUs3FeDwtt2ekeLvlb bDuU4VcBnNZ5VkXDpA5hQeysCfGECBilAtMy/EBjdOngcgk4Fsp8u11/eeB2xqBPUcrR JYeu/99XossdoeJW3avfxFDBw7ngc98oRS2ZPpyL7MY36XoCZCM25U9ue9kBcJ82n92i ctbNdl+9ikVPdcmUokI0WIvBIBGl5yAcrvOQffxmaTz0c31fkq8+iyprDMP/GexVfQzN SAKA==", "Mime-Version"=>"1.0", "X-Received"=>"by 10.202.89.135 with SMTP id n129mr34093639oib.60.1426188359432; Thu, 12 Mar 2015 12:25:59 -0700 (PDT)", "Date"=>"Thu, 12 Mar 2015 12:25:59 -0700", "Message-Id"=>"", "Subject"=>"Subject FTW?", "From"=>"Dee diabetic ", "To"=>"test#inbound.diabetesdelivery.com", "Content-Type"=>"multipart/alternative; boundary=001a113d39288bfe8505111c5a88"}, "text"=>"Body here\n\n", "text_flowed"=>false, "html"=>"Body here\n\n", "from_email"=>"Deediabetic#gmail.com", "from_name"=>"Dee diabetic", "to"=>[["test#inbound.diabetesdelivery.com", nil]], "subject"=>"Subject FTW?", "spf"=>{"result"=>"pass", "detail"=>"sender SPF authorized"}, "spam_report"=>{"score"=>0.5, "matched_rules"=>[{"name"=>"RCVD_IN_DNSWL_LOW", "score"=>-0.7, "description"=>"RBL: Sender listed at http://www.dnswl.org/, low"}, {"name"=>nil, "score"=>0, "description"=>nil}, {"name"=>"listed", "score"=>0, "description"=>"in list.dnswl.org]"}, {"name"=>"FREEMAIL_FROM", "score"=>0, "description"=>"Sender email is commonly abused enduser mail provider"}, {"name"=>"HTML_MESSAGE", "score"=>0, "description"=>"BODY: HTML included in message"}, {"name"=>"DKIM_VALID_AU", "score"=>-0.1, "description"=>"Message has a valid DKIM or DK signature from author's"}, {"name"=>"DKIM_SIGNED", "score"=>0.1, "description"=>"Message has a DKIM or DK signature, not necessarily valid"}, {"name"=>"DKIM_VALID", "score"=>-0.1, "description"=>"Message has at least one valid DKIM or DK signature"}, {"name"=>"RDNS_NONE", "score"=>1.3, "description"=>"Delivered to internal network by a host with no rDNS"}]}, "dkim"=>{"signed"=>true, "valid"=>true}, "email"=>"test#inbound.diabetesdelivery.com", "tags"=>[], "sender"=>nil, "template"=>nil}}
So I am getting a JSON string, but how exactly is the best way to grab the TO, FROM, and SUBJECT?
Update: I just tried the following:
tester = JSON.parse(params[:mandrill_events])
puts tester[:msg][:subject]
#and even this
puts tester["msg"]["subject"]
Did not work. Got an error: **TypeError (no implicit conversion of String into Integer): **
After a day of banging my head, I came up with the solution. By the way, accessing the values directly doesn't work. I had to loop into it.
incomingMail = params[:mandrill_events]
mail_from = "Cannot read From"
mail_originally_to = "Cannot read To"
mail_subject = "No subject"
mail_date = ""
mail_body = "No email body"
if incomingMail != nil
incomingMail = JSON.parse(incomingMail)
incomingMail.each do |item|
mail_from = item["msg"]["from_email"]
mail_originally_to = item["msg"]["headers"]["To"]
mail_subject = item["msg"]["subject"]
mail_date = item["msg"]["headers"]["Date"]
mail_body = item["msg"]["text"]
mail_body_html = item["msg"]["html"]
end
#Confirm
else
puts "Email not sent. Parameter invalid."
end
I have an IMAP mail with a message string that looks like this:
message = #<Mail::Message:70152447148720, Multipart: false, Headers: <Return-Path: <apache#mail.gameseek.co.uk>>, <Received: by 10.86.68.12 with SMTP id q12cs352558fga; Mon, 9 Mar 2009 04:23:05 -0700 (PDT)>, <Received: by 10.210.137.14 with SMTP id k14mr2429643ebd.46.1236597783700; Mon, 09 Mar 2009 04:23:03 -0700 (PDT)>, <Received: from exproxy-2.exserver.dk (exproxy-2.exserver.dk [195.69.129.163]) by mx.google.com with ESMTP id 27si3500694ewy.75.2009.03.09.04.23.03; Mon, 09 Mar 2009 04:23:03 -0700 (PDT)>, <Received: by exproxy-2.exserver.dk (Postfix, from userid 65534) id DF2F6106EF3; Mon, 9 Mar 2009 12:13:26 +0100 (CET)>, <Received: from exsmtp01.exserver.dk (exsmtp01.exserver.dk [195.69.129.177]) by exproxy-2.exserver.dk (Postfix) with ESMTP id C2CEE106ED0 for <support_email.com#exfwd01.scannet.dk>; Mon, 9 Mar 2009 12:13:26 +0100 (CET)>, <Received: from exsmtp02.exserver.dk ([10.10.10.32]) by exsmtp01.exserver.dk with Microsoft SMTPSVC(6.0.3790.1830); Mon, 9 Mar 2009 12:22:19 +0100>, <Received: from front08.exserver.dk ([195.69.129.93]) by exsmtp02.exserver.dk with Microsoft SMTPSVC(6.0.3790.1830); Mon, 9 Mar 2009 12:22:19 +0100>, <Received: from localhost (front08.exserver.dk [127.0.0.1]) by front08.exserver.dk (Postfix) with ESMTP id F1B2BC4028 for <support#email.com>; Mon, 9 Mar 2009 12:46:22 +0100 (CET)>, <Received: from front08.exserver.dk ([127.0.0.1]) by localhost (front08.exserver.dk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mrYGo4G2pt13 for <support#email.com>; Mon, 9 Mar 2009 12:46:16 +0100 (CET)>, <Received: from mail.gameseek.co.uk (78.109.164.42.srvlist.ukfast.net [78.109.164.42]) by front08.exserver.dk (Postfix) with ESMTP id 99022C4021 for <support#email.com>; Mon, 9 Mar 2009 12:46:16 +0100 (CET)>, <Received: by mail.gameseek.co.uk (Postfix, from userid 48) id D321218DD2D; Mon, 9 Mar 2009 11:22:55 +0000 (GMT)>, <Date: Mon, 09 Mar 2009 11:22:55 +0000>, <From: myorder#gameseek.co.uk>, <Reply-To: myorder#gameseek.co.uk>, <To: support#email.com>, <Message-ID: <20090309112255.D321218DD2D#mail.gameseek.co.uk>>, <Subject: Gameseek Order Refunded: Gh68y1235386413>, <Delivered-To: my#email.com>, <Received-SPF: neutral (google.com: 195.69.129.163 is neither permitted nor denied by best guess record for domain of apache#mail.gameseek.co.uk) client-ip=195.69.129.163;>, <Authentication-Results: mx.google.com; spf=neutral (google.com: 195.69.129.163 is neither permitted nor denied by best guess record for domain of apache#mail.gameseek.co.uk) smtp.mail=apache#mail.gameseek.co.uk>, <X-Exserver-To: support_email.com#exfwd01.scannet.dk>, <X-Virus-Scanned: amavisd-new at exserver.dk>, <X-OriginalArrivalTime: 09 Mar 2009 11:22:19.0838 (UTC) FILETIME=[4F6005E0:01C9A0A9]>, <X-ScanNet-Forward: TTL=5>>
I now wish to give it a proper encoding:
unless message.multipart?
charset = message.charset # => "UTF-8"
if charset != nil
body = message.body.decoded.force_encoding(charset).encode("UTF-8") # => "\n\nHello you,\n\nYour order or part of it has been refunded by Gameseek. The refund will be present on the same payment method you used when purchasing. If no other items are due to be posted to you the postage charge will also be refunded.\n\nPlease allow upto four working days for this refund to process.\n\nIf you have not contacted us about this order then it is most likely you are being refunded for an item we cannot currently get hold of.\n\nWe do apologise if this is the case, we would rather refund customers rather than having them wait weeks and weeks for an item.\n\nIf you have contacted us about this order then you will know why you are being refunded.\nMay we apologise if we have not met your requirements on this occassion.\n\nYour Order: Product | Category | Quantity | Cost\n---------------------------------------------------\nDragon Ball Z - Supersonic Warriors 2 | NintendoDS | 1 | \xA326.97\n\n\nFor all order enquires please contact myorder#gameseek.co.uk\n\nThank you for using Gameseek.\n"
end
end
body = body.split(/Sent from my iPhone/)[0]
The last line raises the following error:
invalid byte sequence in UTF-8
Any idea how to fix this?
The text contains the invalid sequence \xA3. This represents a pound sign in Latin-1 (ISO-8859-1).
"\xA3".force_encoding('ISO-8859-1').encode('UTF-8')
#=> "£"
The quick fix is to replace invalid byte sequences in body with String#scrub, but that will remove them:
"\xA326.97".scrub('')
#=> "26.97"
However, to solve the "real" problem you should look earlier in the pipeline. The supplied charset seems to be wrong. Apparently the message is encoded in Latin-1, although the charset suggests something different. Maybe the problem is on the side of the sender.
Like the title says, nothing shows up in real time events nor in any other event
The request being sent is done with ruby's faraday gem
require 'faraday'
require 'json'
GOOGLE_ANALYTICS_SETTINGS = {}
class GoogleAnalyticsApi
def event(client_id = '555', category, action, label, value, user_agent)
return unless !GOOGLE_ANALYTICS_SETTINGS["tracking_code"].empty?
params = {
v: GOOGLE_ANALYTICS_SETTINGS["version"],
tid: GOOGLE_ANALYTICS_SETTINGS["tracking_code"],
cid: client_id,
t: "event",
ec: category,
ea: action,
el: label,
ev: value
}
begin
puts params
c = Faraday.new() do |f|
f.request :url_encoded
f.response :logger
f.adapter Faraday.default_adapter
end
response = c.post GOOGLE_ANALYTICS_SETTINGS["endpoint"], params, { "User-Agent" => user_agent || "Subscribing Worker theSkimm" }
puts response.headers
puts response.body
return true
rescue Exception => rex
return false
end
end
end
From the console:
{:v=>1, :tid=>"UA-XXXXXXXX-1", :cid=>"1999999999.1389999972", :t=>"event", :ec=>"test-event", :ea=>"test-action", :el=>nil, :ev=>nil}
I, [2014-01-15T18:04:04.555555 #7666] INFO -- : post http://www.google-analytics.com/collect
D, [2014-01-15T18:04:04.555667 #7666] DEBUG -- request: User-Agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
Content-Type: "application/x-www-form-urlencoded"
I, [2014-01-15T18:04:04.670273 #7666] INFO -- Status: 200
D, [2014-01-15T18:04:04.670512 #7666] DEBUG -- response: pragma: "no-cache"
expires: "Mon, 07 Aug 1995 23:30:00 GMT"
cache-control: "private, no-cache, no-cache=Set-Cookie, proxy-revalidate"
access-control-allow-origin: "*"
last-modified: "Sun, 17 May 1998 03:00:00 GMT"
x-content-type-options: "nosniff"
content-type: "image/gif"
date: "Wed, 15 Jan 2014 23:04:04 GMT"
server: "Golfe2"
content-length: "35"
alternate-protocol: "80:quic"
connection: "close"
{"pragma"=>"no-cache", "expires"=>"Mon, 07 Aug 1995 23:30:00 GMT", "cache-control"=>"private, no-cache, no-cache=Set-Cookie, proxy-revalidate", "access-control-allow-origin"=>"*", "last-modified"=>"Sun, 17 May 1998 03:00:00 GMT", "x-content-type-options"=>"nosniff", "content-type"=>"image/gif", "date"=>"Wed, 15 Jan 2014 23:04:04 GMT", "server"=>"Golfe2", "content-length"=>"35", "alternate-protocol"=>"80:quic", "connection"=>"close"}
GIF89a�����,D;
For whomever encounters this the problem was setting nil value for el and ev. When the params hash is moved through Faraday the nil valued items get converted to the key name without any value and google didn't like that.
IE a query string with nil values going through Faraday would be:
...&el&ev
Not good for google's Measurement Protocol.