Send email with log file as attachment - shell

I am using Hadoop (CDH 5.4.8) to process the unstructured data and after successful processing I want to send a mail notification to the concerned team with log file as attachment.
CDH 5.4.8 Oozie does not support attachment feature in email action. So I want to do this using shell script. Please let me know the best way to do this.

You can easily send an email from within a shell by piping a complete mail message (header and body) into sendmail. This assumes that the host you're doing this is properly configured with a mail transfer agent (e.g. sendmail or postfix) to send email messages.
The easiest way to send email with an attachment is to create a simple template message in your mail user agent (e.g. Thunderbird), and copy its contents as a template with the view source command. Modify that template to suit your needs and place it in the shell script.
Here is an example:
#!/bin/sh
cat <<\EOF |
To: Ramesh <ramesh#example.com>
From: Diomidis Spinellis <dds#aueb.gr>
Subject: Here are your Hadoop results
Message-ID: <5700BF28.2070500#aueb.gr>
Date: Sun, 3 Apr 2016 09:58:48 +0300
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="------------030303090406090809090501"
This is a multi-part message in MIME format.
--------------030303090406090809090501
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
The attachment contains your Hadoop results.
--------------030303090406090809090501
Content-Type: application/octet-stream;
name="data"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="data"
HviDNR105+2Tr0+0fsx3OyzNueQqPuAXl9IUrafOi7Y=
--------------030303090406090809090501--
EOF
sendmail ramesh#example.com
To configure a fixed message with actual data, replace the parts you want to modify with commands that generate them. (Note the missing backslash from the here document EOF marker.)
#!/bin/sh
cat <<EOF |
To: Ramesh <ramesh#example.com>
From: Diomidis Spinellis <dds#aueb.gr>
Subject: Here are your Hadoop results
Message-ID: <5700BF28.2070500#aueb.gr>
Date: $(date -R)
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="------------030303090406090809090501"
This is a multi-part message in MIME format.
--------------030303090406090809090501
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
The attachment contains your Hadoop results.
--------------030303090406090809090501
Content-Type: application/octet-stream;
name="data"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="data"
$(base64 binary-data-file.dat)
--------------030303090406090809090501--
EOF
sendmail ramesh#example.com

Related

Load testing server with http multipart/related request which contains audio content

We built a server which handles speech recorded by user using an app. The audio data is sent through http post in real time. The body looks like this:
--BOUNDARY
Content-Disposition: form-data; name="metadata"
Content-Type: application/json; charset="UTF-8"
<JSON FORMATTED METADATA HERE>
--BOUNDARY
Content-Disposition: form-data; name="audio"
Content-Type: application/octet-stream
<AUDIO BYTES HERE>
--BOUNDARY--
Now, I need to do load testing for the server. I am thinking of using ApacheBench and just do consistent uploading requests but I wish to use the same format as above for each request. How could that be setup in AB?
I was able to solve the problem by using the following command:
ab -p test -T "multipart/form-data; boundary=BOUNDARY" -c 1000 -n 1000 -l http://someipaddress.com/
where test is a file containing the post content.

Is it possible to send RingCentral SMS / MMS using multipart/form-data?

The OpenAPI spec for the Create SMS Message endpoint includes the following request content types:
consumes:
- application/json
- multipart/mixed
- multipart/form-data
https://netstorage.ringcentral.com/dpw/api-reference/specs/rc-platform.yml?v=2019082420190816-0828
I found the SMS / MMS instructions to include a multipart/mixed example in the API Reference, but don't see any information on using multipart/form-data. I'm specifically interested in sending files.
https://developers.ringcentral.com/api-reference/SMS/createSMSMessage
The same API Reference shows support for both multipart/form-data and multipart/mixed for sending faxes.
https://developers.ringcentral.com/api-reference/Fax/createFaxMessage
Since both APIs send files and metadata so I'm wondering if the SMS API also supports multipart/form-data and, if so, how to send it?
No, it does not appear so.
The example you'd linked for the SMS message uses multipart/mixed to separate the API call itself (which is in turn sent as application/json) from the payload being sent as an MMS (image/png).
The use of multipart/form-data in the fax API is specific to the way that particular metadata is included, but there isn't an equivalent system for SMS/MMS as they both need that particular meta information encoded either as a single JSON document or as the JSON element of a multipart/mixed message.
To send a file, though, multipart/mixed is fine. Your request would then be something like:
POST /restapi/v1.0/account/403391985008/extension/403391985008/sms
Content-Type: multipart/mixed; boundary=Boundary_1_14413901_1361871080888
--Boundary_1_14413901_1361871080888
Content-Type: application/json; charset=UTF-8
Content-Transfer-Encoding: 8bit
{"to" :[{"phoneNumber": "+18772004569"},{"phoneNumber": "+18772094569"}],
"text" :"hello",
"from" :{"phoneNumber": "+18882004237"}}
--Boundary_1_14413901_1361871080888
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="filename.zip"
[Some encoded binary stream here ...]
--Boundary_1_14413901_1361871080888--
It'd be up to you to set the file's mime type properly and ensure things are encoded. The key points here are that the message information is encoded in the first JSON component in your multipart message, while the file attached to the MMS is encoded in the second.
multipart/form-data can be sent as shown in the following example:
POST / HTTP/1.1
HOST: platform.ringcentral.com/restapi/v1.0/account/~/extension/~/sms
Authorization: Bearer <MyToken>
Content-Type: multipart/form-data; boundary=12345
--12345
Content-Disposition: form-data; name="to"
+16505550101
--12345
Content-Disposition: form-data; name="to"
+16505550102
--12345
Content-Disposition: form-data; name="from"
+16505550100
--12345
Content-Disposition: form-data; name="text"
Hello World
--12345
Content-Disposition: form-data; name="attachment" filename="picture.jpg"
content of picture.jpg ...
--12345--
This can be done using curl as follows:
curl -XPOST https://platform.ringcentral.com/restapi/v1.0/account/~/extension/~/sms \
-H 'Authorization: Bearer <MyToken>' \
-F 'to=+16505550101' \
-F 'to=+16505550102' \
-F 'from=+16505550100' \
-F 'text=Hello World' \
-F 'attachment=#picture.jpg'

Send S/MIME encrypted html email with bash

How do you send an encrypted and html-formatted email through the command line? Here is the code I have so far:
# Encrypt email with a certificate
openssl cms -encrypt -in "/tmp/email_to_be_sent.html" -out "/tmp/encrypted.txt" -from $SENDER -to $RECEIVER -subject "Test: Encrypted message" -des3 "/tmp/$CERT.pem"
# Send the encrypted email
cat "/tmp/encrypted.txt" | sendmail -f $SENDER $RECEIVER
The generated encrypted email /tmp/encrypted.txt is as follow
To: recipient#mail.com
From: sender#mail.com
Subject: Test: Encrypted message
MIME-Version: 1.0
Content-Disposition: attachment; filename="smime.p7m"
Content-Type: application/pkcs7-mime; smime-type=enveloped-data;name="smime.p7m"
Content-Transfer-Encoding: base64
MIIDjAYJKoZIhvcNAQcDoIIDfTCCA3kCAQAxggFZMIIBVQIBADA9MDcxHDAaBgNVBAoME0V1cm9wZWFu
AxAlApQsmjzCwQoonT57JetCp7DHJdHWU1bkLIZWPPBRwa2EB0ZdxOXIvtg7rJavnnbxeTghblM45Pur
A+6BDKJbWvXFyxb...
The problem is, once in the recipient inbox and decrypted, the message is not html formatted and html code like <html><body></body></html> is still readable inside the message.
S/MIME requires the original message to be enveloped. This means that the original message is encrypted and this fact and the type of encryption is added to the outer message headers, so the client knows how to handle the message contents.
Because of this, the message headers that define the original message format need to be inside the S/MIME envelope, so the client knows which content type it is after decrypting the message.
The correct way is to extract these headers from the original message, then add them before the original message body. Note that these headers must start on the first line, and that after these headers a blank line is required before the original message body starts.
Headers that should be moved into the enveloped message data are
MIME-Version (optional)
Content-Type
Content-Transfer-Encoding
Content-Disposition (if exists)
"Moved" means that they should be included in the enveloped message data and removed from the outer message headers.
The remaining headers should be left in the envelope message. The openssl cms -encrypt command will then add the above headers as required for S/MIME encrypted messages.
Example
Original message
From: someone#somedomain.net
To: receipient#otherdomain.net
Subject: It's a test
MIME-Version: 1.0
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Custom-Header: Additional data
This is the message text.
Good night.
Moved headers before encryption (note the additional blank line)
From: someone#somedomain.net
To: receipient#otherdomain.net
Subject: It's a test
X-Custom-Header: Additional data
MIME-Version: 1.0
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
This is the message text.
Good night.
Message after encryption
From: someone#somedomain.net
To: receipient#otherdomain.net
Subject: It's a test
X-Custom-Header: Additional data
MIME-Version: 1.0
Content-Disposition: attachment; filename="smime.p7m"
Content-Type: application/pkcs7-mime; smime-type=enveloped-data; name="smime.p7m"
Content-Transfer-Encoding: base64
MIJ5lAYJKoZIhvcNAQcDoIJ5hTCCeYECAQAxggHZMIIB1QIBADCBvDCBtjEaMBgG
A1UEAwwRc2F2aWduYW5vIENFUlQtaTIxJTAjBgNVBAoMHHNhdmlnbmFubyBzb2Z0
d2FyZSBzb2x1dGlvbnMxHjAcBgNVBAsMFUNlcnRpZmljYXRpb24gU2VydmljZTEL
(more encrypted data removed)
So the comments from Stefan leaded me to the solution.
The unencrypted email /tmp/email_to_be_sent.html should have a header like this before encryption:
To: recipient#mail.com
From: sender#mail.com
Subject: Test: Encrypted message
MIME-Version: 1.0
Content-Type: text/html; charset=utf-8
<html><body><p> test message </p></body></html>
Beware that a newline is needed between the email header and the html code.

need to send two attachments as email body one as plain text and other as html

The code below is a shell script which is working for one attachment, I need to send second attachment in the same email but that should be in text/plain. both need to go as body text in the email. Please advice.
I have tried the below and is working for one attachment, need to embed the second attachment in the same email as message body plain text.
cat <<'EOF' - /usr/local/oracle/wls1036/owbmonitor/owbmonitor.log | /usr/sbin/sendmail -t
To:s.a#yahoo.com
Subject: PREPROD MONITOR ${DATE}
Content-Type: text/html
EOF
As per the suggestion to use multiform types:, I have tried the below which is not working
(
cat <<!
Subject: OWB PREPROD MONITOR AT ${DATE}
To: s.ad#yahoo.com.com
MIME-Version: 1.0
Content-Type: multipart/mixed;boundary="nextfile"
--nextfile
Content-Type: text/html
`cat /usr/local/oracle/wls1036/domains/mydomain/bin/owbmonitor/owbmonitor.log`
--nextfile
Content-Type: text/plain
`cat /usr/local/oracle/wls1036/domains/mydomain/bin/owbmonitor/top.log`
!
) | /usr/sbin/sendmail -t

email encoding and sending through SMTP - Ruby

I have run across an interesting problem. I am sending email with attachments through the NET::SMTP class in ruby through Apple's me.com SMTP servers and I am running into some funny issues.
I am trying to send a series of jpg files through the SMTP server. I am encoding them in ruby and when I send to another me.com email all five jpg images show up at the other end in perfect condition. When I send to my gmail address the files truncate at 90k (they are normally around 500k). When I open the two emails in textmate I see the encoding on the text portion of the email is 8bit on the email sent to the .me address and 7bit in the email sent to the gmail server. I'm not sure if this is my problem or not.
Here is a brief of the code I am using:
file1Content = File.read(directory +'/Photo_1.jpg')
file1 = [file1Content].pack("m")
marker = "AUNIQUEMARKER"
body =<<EOF
#{emailbody}
EOF
# Define the main headers.
part1 =<<EOF
From: #{from}
To: #{donor}
Subject: #{subject}
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=#{marker}
--#{marker}
EOF
# Define the message action
part2 =<<EOF
Content-Transfer-Encoding:8bit
Content-Type: text/plain
#{body}
--#{marker}
EOF
# Define the attachment section
part3 =<<EOF
Content-Type: image/jpeg; name=\"Photo_1.jpg\"
Content-Transfer-Encoding:base64
Content-Disposition: attachment; filename="Photo_1.jpg"
#{file1}
--#{marker}
EOF
(etc to 5 files where I end the marker with --#{marker}--
I would really appreciate any help you could give. I'm completely stumped. A couple of other notes. I am using MacRuby and not all Gems work on it, especially for embeded MacRuby. I have had some success with small libraries but I haven't had any luck with ActionMailer.
I had a friend come in and we worked through it and here is the result.
In the email encodings the line breaks are extremely important. Some mail servers appear to be more forgiving (apple's) which is why I didn't see the problem initially.
Here is the working code:
marker = "AUNIQUEMARKER"
body =<<EOF
#{emailbody}
EOF
# Define the main headers.
part1 =<<EOF
From: #{from}
To: #{donor}
Subject: #{subject}
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=#{marker}
--#{marker}
EOF
# Define the message action
part2 =<<EOF
Content-Type: text/plain
Content-Transfer-Encoding:8bit
#{body}
--#{marker}
EOF
# Define the attachment section
part3 =<<EOF
Content-Type: image/jpeg; name=Photo_1.jpg
Content-Transfer-Encoding:base64
Content-Disposition: attachment; filename=Photo_1.jpg
#{file1}
--#{marker}
EOF
part4 =<<EOF
Content-Type: image/jpeg; name=Photo_2.jpg
Content-Transfer-Encoding:base64
Content-Disposition: attachment; filename=Photo_2.jpg
#{file2}
--#{marker}
EOF
I have no experience with attachment encoding, but I think 7bit is still the standard.
I recommend using a mail lib that does all this for you, like the one from Mikel. Re-inventing the wheel is not really useful, unless you only want to learn inventing wheels.
Link to Mikel's mail lib: http://github.com/mikel/mail

Resources