golang s3 download to buffer using s3manager.downloader - go

Question

I'm using the Amazon s3 SDK to download files like below:
file, err := os.Create("/tmp/download_file")
downloader := s3manager.NewDownloader(session.New(&aws.Config{
Region: aws.String("us-west-2")}))
numBytes, err := downloader.Download(file,
&s3.GetObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(fileName),
})
It downloads to a file.
How do I get the download content into a []byte slice (buffer) directly.
I tried something like:
var tBuf bytes.Buffer
tBufIo := bufio.NewWriter(&tBuf)
instead of "file". But I get an error for io.WriterAt interface
cannot use *tBufIo (type bufio.Writer) as type io.WriterAt in argument to downloader.Download

Answer 1

Found it from the link
https://groups.google.com/forum/#!topic/Golang-Nuts/4z8rcWEZ8Os
buff := &aws.WriteAtBuffer{}
downloader := s3manager.NewDownloader(session.New(&aws.Config{
Region: aws.String(S3_Region)}))
numBytes, err := downloader.Download(buff,....
data := buff.Bytes() // now data is my []byte array
Works and fits the need.
See also: https://docs.aws.amazon.com/sdk-for-go/api/aws/#WriteAtBuffer

Answer 2

If you are running AWS SDK v2 then you can create a WriterAt like this
import (
"context"
"errors"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/credentials"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/aws/aws-sdk-go-v2/feature/s3/manager"
)
func DownloadS3File(objectKey string, bucket string, s3Client *s3.Client) ([]byte, error) {
buffer := manager.NewWriteAtBuffer([]byte{})
downloader := manager.NewDownloader(s3Client)
numBytes, err := downloader.Download(context.TODO(), buffer, &s3.GetObjectInput{
Bucket: aws.String(bucket),
Key: aws.String(objectKey),
})
if err != nil {
return nil, err
}
if numBytes < 1 {
return nil, errors.New("zero bytes written to memory")
}
return buffer.Bytes(), nil
}