I successfully fetched a list of email headers using the sample code from this url: https://godoc.org/code.google.com/p/go-imap/go1/imap#example-Client . However, I still haven't been able to fetch the body of the emails. Can anyone show some working sample code that could fetch the body of the emails from a imap server in Golang?
I figured out how to get the body text now.
cmd, _ = c.UIDFetch(set, "RFC822.HEADER", "RFC822.TEXT")
// Process responses while the command is running
fmt.Println("\nMost recent messages:")
for cmd.InProgress() {
// Wait for the next response (no timeout)
c.Recv(-1)
// Process command data
for _, rsp = range cmd.Data {
header := imap.AsBytes(rsp.MessageInfo().Attrs["RFC822.HEADER"])
uid := imap.AsNumber((rsp.MessageInfo().Attrs["UID"]))
body := imap.AsBytes(rsp.MessageInfo().Attrs["RFC822.TEXT"])
if msg, _ := mail.ReadMessage(bytes.NewReader(header)); msg != nil {
fmt.Println("|--", msg.Header.Get("Subject"))
fmt.Println("UID: ", uid)
fmt.Println(string(body))
}
}
cmd.Data = nil
c.Data = nil
}
The example code you've linked to demonstrates the use of the IMAP FETCH command to fetch the RFC822.HEADER message data item for a message. The RFC contains a list of standard data items you can fetch from a message.
If you want the entire mime formatted message (both headers and body), then requesting BODY should do. You can get the headers and message body separately by requesting BODY[HEADER] and BODY[TEXT] respectively. Modifying the sample program to use one of these data items should get the data you are after.
Related
I am going through Gmail API docs(https://developers.google.com/gmail/api), I am able to read all the user emails which are present in the inbox.
eg snippet (reading complete list of emails):
for {
req := svc.Users.Messages.List("me")
r, _ := req.Do()
for _, m := range r.Messages {
msg, _ := svc.Users.Messages.Get("me", m.Id).Do()
date := ""
for _, h := range msg.Payload.Headers {
if h.Name == "Date" {
date = h.Value
break
}
}
msgs = append(msgs, message{
...
})
}
}
Now, when new emails come I want to read them as well (either immediately or after some time). I can write a scheduled job for that purpose, But I am not sure if I can fetch email after a particular timestamp or after an email identifier. I don't want to read a whole bunch of emails, again and again, to figure out the new emails, in this way, there is a lot of unnecessary computation being involved. Is there any way I can make this task easier?
Looking at the docs, it looks like it supports a query parameter, q.
The query parameter supports the same options as available in the Gmail search bar:
Only return messages matching the specified query. Supports the same query format as the Gmail search box. For example, "from:someuser#example.com rfc822msgid:somemsgid#example.com is:unread". Parameter cannot be used when accessing the api using the gmail.metadata scope.
This means you can do something like "after:YYYY/MM/DD" or with a timestamp "after:1616819452".
req := svc.Users.Messages.List("me").Q("after:2021/01/01")
See the full usage here https://pkg.go.dev/google.golang.org/api/gmail/v1#UsersMessagesListCall.Q
I have to fetch 250K chromebooks from google workspace (Gsuite), I am using Admin Directory API to retrieve JSON data from Google.
The response returns in chunks of 200 records, in the response is included a nextPageToken, I use that next page token to retrieve the next 200 and so on.
After an hour, of using the nextPageToken attached from the previous request, However Google returns with error 400,
{error_code: 400, "message"=>"Invalid Input: CMiJhq7-5ewCEp0BCm737N8GN......"},
Note: This string 'CMiJhq7-5ewCEp0BCm737N8GN......' which google is calling as invalid is the nextPageToken.
Why is this happening? Does nextPageToken expire after 1 hour?
My code snippet:
query_list = {
'maxResults' => 200,
'access_token' => access_token,
'pageToken' => next_page_token
}
HTTParty.get(endpoint_url, query: query_list)
The nextPage token is created when the initial request is sent. This token is used in order to get the next batch of rows from the request.
This token is intended to be used immediately as the data associated with the initial request may be changed if you wait to long.
So yes next page tokens do expire i would actually expect them to expire in a lot less than an hour. I also wonder if the next page token wouldn't just expire after you used it the first time.
If you want to make the same request again i suggest you do that and get new next page tokens built for you after the hour.
I had to change my approach, initially, I fetched 200 chunks from Google API, performed some time-taking processing and then made entries into my database (database-intensive tasks) and then requested the next 200 chunks and so so. After an hour, the last nextpagetoken sent by Google became invalid.
So, now I fetch 200 chunks, save them to my database in JSON format without performing any database-intensive tasks, request the next 200, and so on. I was able to fetch 300K Chromebooks JSON data from google in around 56 Minutes before the nextPageToken became invalid.
I am now processing that JSON data present in my database, without having network overhead or any google API dependency.
I'm encountering error Error 400: Invalid Value, invalid when I tried to use pageToken parameter of the Google Calendar Events API.
The code (in Go) I was using.
call := service.Events.List(calendarID).SingleEvents(true)
events, err := call.Do()
if err != nil {
return err
}
var allEvents []*calendar.Event
allEvents = append(allEvents, events.Items...)
if events.NextPageToken != "" {
nextPageToken := events.NextPageToken
nextCall := service.Events.List(calendarID).SingleEvents(true).PageToken(nextPageToken)
events, err = nextCall.Do()
if err != nil {
return err
}
allEvents = append(allEvents, events.Items...)
}
As illustrated in Paging through lists of resources, the next API call has to be exactly the same as the previous one. Thus, the following code works with the pageToken (where no nextCall is being used).
call := service.Events.List(calendarID).SingleEvents(true)
events, err := call.Do()
if err != nil {
return err
}
var allEvents []*calendar.Event
allEvents = append(allEvents, events.Items...)
if events.NextPageToken != "" {
nextPageToken := events.NextPageToken
call := call.PageToken(nextPageToken)
events, err = call.Do()
if err != nil {
return err
}
allEvents = append(allEvents, events.Items...)
}
I'm attempting to retrieve the full message body of messages using the Gmail API in Go. Currently when I do so, I only get the first three characters of the message body which are "<ht". I'm pretty sure my issue lies with the decoding of the message body but I can't seem to figure out what I'm doing wrong.
I've looked at examples in several other languages and have tried to translate them to Go with no success. The encoded message body is rather large so I'm fairly certain some data is getting lost somewhere.
Here is an (abridged) code snippet illustrating how I've been attempting to go about this:
req := svc.Users.Messages.List("me").Q("from:someone#somedomain.com,label:inbox")
r, _ := req.Do()
for _, m := range r.Messages {
msg, _ := svc.Users.Messages.Get("me", m.Id).Format("full").Do()
for _, part := range msg.Payload.Parts {
if part.MimeType == "text/html" {
data, _ := base64.StdEncoding.DecodeString(part.Body.Data)
html := string(data)
fmt.Println(html)
}
}
}
Need to use Base64 URL encoding (slightly different alphabet than standard Base64 encoding).
Using the same base64 package, you should use:
base64.URLEncoding.DecodeString instead of
base64.StdEncoding.DecodeString.
To get URL Base64 from standard Base64, replace:
+ to - (char 62, plus to dash)
/ to _ (char 63, slash to underscore)
= to * padding
from body string (source here: Base64 decoding of MIME email not working (GMail API) and here: How to send a message successfully using the new Gmail REST API?).
Here is the code snippet to send the email via a local postfix server:
from := r.FormValue("from")
to := strings.Split(r.FormValue("to"), ";")
body := r.FormValue("body")
mime := "MIME-version:1.0;\nContent-Type:text/html;charset=\"UTF-8\";\n\n"
subject := fmt.Sprintf("Subject: %s\n", r.FormValue("subject"))
msg := []byte(subject + mime + body)
err := smtp.SendMail("localhost:25", nil, from, to, msg)
The email was sent/received fine. However, it is missing the receipt address in the To field of received email. I also tried it on an exchange server. The receipt addresses are missing as well. Here is what it shows in the email source.
To: Undisclosed recipients:;
Any suggestions to fix it? thanks!
You're setting the values for the mail envelope, but you haven't put any headers in the email itself except for Subject:. You should also be using \r\n as a newline for email.
A minimal example might look like:
headers := make(map[string]string)
headers["Subject"] = "this is a test"
headers["From"] = "me#example.com"
headers["To"] = "you#example.com"
body := "hello,\nthis is a test"
var msg bytes.Buffer
for k, v := range headers {
msg.WriteString(k + ": " + v + "\r\n")
}
msg.WriteString("\r\n")
msg.WriteString(body)
Some other helpful stdlib packages:
net/textproto for MIME header handling
net/mail for address handling (though the package is really only for parsing email)
http://gopkg.in/gomail.v1 for a more complete solution (there are probably many others)
I wrote a cookie getter and setter. Now I want to test it, and wrote following test function.
func TestAuthorizationReader(t *testing.T) {
tw := httptest.NewServer(testWriter())
tr := httptest.NewServer(Use(testReader()))
defer tw.Close()
defer tr.Close()
c := &http.Client{}
rs, err := c.Get(tw.URL)
assert.NoError(t, err, "Should not contain any error")
// Assign cookie to client
url, err := rs.Location()
fmt.Print(url)
assert.NoError(t, err, "Should not contain any error")
//c.Jar.SetCookies(url, rs.Cookies())
}
The test fail at the second part, as output message I've got
- FAIL: TestAuthorizationReader (0.05s)
Location: logged_test.go:64
Error: No error is expected but got http: no Location header in response
Messages: Should not contain any error
I can not get the URL location pointer, what do I wrong here?
The Response.Location method returns the value of the Location response header. You would usually only expect to see this header for redirect responses, so it isn't surprising you got this error.
If you want to know the URL used to retrieve a particular response, try rs.Request.URL.String(). Even if the HTTP library followed a redirect to retrieve the document, this will look at the request used for this particular response, which is what you'd be after when determining a cookie's origin.
If you just want the client to keep track of cookies set by requests it processes though, all you should need to do is set the Jar attribute on your client. Something like this:
import "net/http/cookiejar"
...
c := &http.Client{
Jar: cookiejar.New(nil),
}
Now cookies set in earlier responses should be set in future requests to the same origin.