I am using FastImage for caching image and it loads image very fast after caching data as expected. But my server is generating new uri (s3 presigned url) each time for same image.
So, FastImage is considering it as new image and tries to download everytime which affects my app performance.
My question is, Is there any optimistic way to render images fast as possible without caching it?
If you have chance to modify the server side application, you can create Authorization headers instead of creating presigned urls.
This function should help.
import aws4 from 'aws4';
export function getURIWithSignedHeaders(imagePath) {
if(!imagePath){
return null;
}
const expires = 86400; // 24 hours
const host = `${process.env.YOUR_S3_BUCKET_NAME}.s3.${process.env.YOUR_S3_REGION}.amazonaws.com`;
// imagePath should be something like images/3_profileImage.jpg
const path = `/${imagePath}?X-Amz-Expires=${expires}`;
const opts = {
host,
path,
headers: {
'Content-Type': 'image/jpeg'
}
};
const { headers } = aws4.sign(opts, {accessKeyId: process.env.YORU_ACCESS_KEY_ID, secretAccessKey: process.env.YOUR_SECRET_ACCESS_KEY});
return {
uri: `https://${host}${path}`,
headers: {
Authorization: headers['Authorization'],
'X-Amz-Content-Sha256': headers['X-Amz-Content-Sha256'],
'X-Amz-Date': headers['X-Amz-Date'],
'Content-Type': 'image/jpeg',
}
}
}
See: 609974221
Related
I'm attempting to upload image files to my nextjs app where I'll eventually store in GCS but I'm having some trouble with the image form data. I'm using FilePond on the client to handle uploading the file and sending a req to a simple API that I have on the server.
// Component
import { FilePond, File, registerPlugin } from "react-filepond";
import FilePondPluginImageExifOrientation from 'filepond-plugin-image-exif-orientation';
import FilePondPluginImagePreview from "filepond-plugin-image-preview";
registerPlugin(FilePondPluginImageExifOrientation, FilePondPluginImagePreview);
const Page = () => {
const [productImages, setProductImages] = useState<File[]>([]);
return (
<FilePond
allowMultiple={true}
maxFiles={2}
files={productImages}
onupdateFiles={setProductImages}
server={{
process: {
url: "/api/upload",
method: "POST",
headers: {
"Content-Type": "mutlipart/form-data"
},
ondata: formData => {
formData.append('image', "test-image");
return formData;
}
}
}}
/>
);
};
export default Page;
// ./pages/api/upload
import { NextApiRequest, NextApiResponse } from "next";
const Index = (_req: NextApiRequest, res: NextApiResponse) => {
const reqBody = _req.body ?? null;
console.log(_req);
if (!reqBody) res.status(200).json({ message: "No request body found" });
res.status(200).json({ data: "OK" });
};
export default Index;
The issue I'm seeing is the files are being sent as a giant blob string and I've seen other people be able to access the files property from the incoming request (shown here). This is my first time building a file uploading feature into any of my projects so I'm not entirely sure what's best practice for handling files from incoming requests and parsing them to be stored in some file storage service like GCP or S3.
You might need to chunk the image file. set the configuration chunkUploads to true.
Then your backend should process the chunked file like this.
I'm trying to upload a file and send data from a React frontend to a S3 bucket using an API Gateway/ Lambda function setup using the Serverless framework and I've been struggling with it for the last couple of days.
From the frontend I am using axios and creating a formdata to send a post request to the API like the following:
let formData = new FormData();
formData.append('imageFile', selectedImage);
formData.append('itemId', clubIdRef.current.value);
formData.append('itemDescription', itemDescRef.current.value);
axios.post(
baesURL+"/item/create", formData,
{headers: {
'Content-Type': 'multipart/form-data'
}}
).then((response) => {
console.log("response" + response)
console.log("response.data" + response.data)
})
Appending string attributes to the formdata feels off but the only way I could find to send data and an image at the same time was like the above.
Then to receive this data in the backend I've been using lambda-multipart-parser like the following:
const createItem = async (event) => {
const result = await multipartParser.parse(event);
const imageFile = result.imageFile;
const itemDescription = result.itemDescription;
where the result console logs as:
{
files: [],
imageFile: '[object File]',
itemId: '12',
itemDescription: "Description"
}
I can then store the imageFile successfully in S3 and generate the URL. Next, I create an Item object with the S3 url and id and description to store in dynamoDB. Everything works fine but when I open the S3 url the file is corrupted and just opens as a grey box instead of the actual image I uploaded.
This is how I am uploading the file using the s3 sdk
const AWS = require("aws-sdk");
const s3 = new AWS.S3();
const params = {
Bucket: BUCKET_NAME,
Key: `images/${directoryPath}/${id}.png`,
Body: imageFile,
ContentType: "image/png",
ACL : "public-read"
}
uploadResult = await s3.putObject(params).promise();
These are the things I've tried but still don't have any success uploading the correct image to my S3 bucket:
Looking and changing the BinaryMediaType of the API gateway but I can't find the settings under the API...
Tried using aws-lambda-multipart-parser but still wasn't able to add multipart/form-data binary media type and parse the full form data correctly
I know that I could first try to send a request directly from React to S3 to upload the image using aws-sdk in react to get a preSignedURL and attach that URL and make a POST request to my API Gateway simply parse the event.body without having to use a multipart form parser, but I want to avoid sending multiple requests if needed and handle everything in the backend.
Any suggestions would be highly appreciated!
It is quite hard to understand where is the problem with given context.
We have no idea which image format you are uploading, no idea how you store this image to S3.
My answer will try to cover these missing informations as it is a common mistake on S3 uploads.
S3 files are stored and returned with given ContentType.
You might check your S3 file's ContentType on AWS console.
Console > S3 > Select object (image) > Metadata > ContentType
I will suppose that image format is PNG and image data is correct and might be posted to S3 as is (from result).
S3Service.ts
import AWS, {S3} from "aws-sdk";
import {PutObjectRequest} from "aws-sdk/clients/s3";
import {PutObjectResponse} from "aws-sdk/clients/mediastoredata";
AWS.config.update({region: 'eu-west-3' });
const s3: S3 = new AWS.S3();
export class S3Service {
public static async putImage(key: string, data: string, contentType: string): Promise<PutObjectResponse> {
const s3Params: PutObjectRequest = {
Bucket: process.env.S3_BUCKET,
Key: key,
Body: data,
ContentType: contentType // <== I draw your attention here
}
return await s3.putObject(s3Params).promise()
}
}
index.ts
import { S3Service } from "service/aws/s3-service";
await S3Service.putImage(result.itemId + ".png", result.imageFile, "image/png");
A common mistake, which I assume might be the cause of your problem, is to forget content-type resulting in incorrect download format.
I'm creating a simple Blog to exploring Sveltekit and how it works.
I created an endpoint to manage the Upload of an image, which is stored in a folder placed in the root folder (same level of src).
Now I'm trying to get this image and shows in the front end when the post is loaded.
It's pretty simple but I can't manage how to do it. In Nodejs normally I create an API to serve the image when is called like (ex. the API url is /api/v1/images/):
function get(req, res, next) {
...
var fileStream = fs.createReadStream(imagePath);
res.writeHead(200, { "Content-Type": "image/" + extensionName });
fileStream.pipe(res);
...
}
In the frontend I call it:
<img src={getImageFromBackend("example.jpg")} alt="Example" />
But in Sveltekit I can't do the same.
Any ideas?
Thanks
You should create another endpoint that serves the image. The endpoint will look like this:
import {promises as fs} from "fs";
export async function get() {
const asset = await fs.readFile("sample.jpg");
return {
headers: {
"Content-Type": "image/jpeg"
},
body: asset
};
}
Here is a example of an endpoint sending an image:
import {promises as fs} from 'fs'
export async function GET() {
const asset = await fs.readFile("sample.jpg")
return new Response(asset, {
headers: {
"Content-Type": "image/jpg"
}
})
}
I am using minio to manage the files
const getMinioClient = () => {
const minioClient = new Minio.Client({
endPoint: '127.0.0.1',
port: 9000,
useSSL: false,
accessKey: 'minioadmin',
secretKey: 'minioadmin'
});
return minioClient;
};
uploadFile(bucketName, newFileName, localFileLocation,metadata={}) {
return new Promise((resolve, reject) => {
const minioClient = getMinioClient();
//'application/octet-stream'
minioClient.fPutObject(bucketName, newFileName, localFileLocation, metadata , (err, etag) => {
if (err) return reject(err);
return resolve(etag);
});
});
}
with the following code I can upload the file, after successfully uploading it returns me only with etag, but I want to get the download link, how would I get it directly without searching the filename again.
You won't be able to get something like Public URL/Link for accessing images unless you ask for it to manually generate a time limited download URL using something like:
https://min.io/docs/minio/linux/reference/minio-mc/mc-share-download.html#generate-a-url-to-download-object-s
One workaround is to let nginx directly access the location you are uploading your files to:
https://gist.github.com/harshavardhana/f05b60fe6f96803743f38bea4b565bbf
After you have successfully written your file with your code above, you can use presignedUrl method to generate the link to your image.
An example for Javascript is here: https://min.io/docs/minio/linux/developers/javascript/API.html#presignedUrl:~:text=//%20presigned%20url%20for%20%27getObject%27%20method.%0A//%20expires%20in%20a%20day.%0AminioClient.presignedUrl(%27GET%27%2C%20%27mybucket%27%2C%20%27hello.txt%27%2C%2024*60*60%2C%20function(err%2C%20presignedUrl)%20%7B%0A%20%20if%20(err)%20return%20console.log(err)%0A%20%20console.log(presignedUrl)%0A%7D)
In any case you have to set an expiration time. Here or you set a very long time, which is suitable to your app or if you have a backend, require the images from Frontend through the backend with the getObject method: getObject(bucketName, objectName, getOpts[, callback]).
https://min.io/docs/minio/linux/developers/javascript/API.html#presignedUrl:~:text=getObject(bucketName%2C%20objectName%2C%20getOpts%5B%2C%20callback%5D)
If you have only a few number of static images to show in your app, (which are not uploaded by your app), you can also create the links manually with tme minio client or from the Minio-UI.
So we get a file (an image file) in the front-end like so:
//html
<input type="file" ng-change="onFileChange">
//javascript
$scope.onFileChange = function (e) {
e.preventDefault();
let file = e.target.files[0];
// I presume this is just a binary file
// I want to HTTP Post this file to a server
// without using form-data
};
What I want to know is - is there a way to POST this file to a server, without including the file as form-data? The problem is that the server I am send a HTTP POST request to, doesn't really know how to store form-data when it receives a request.
I believe this is the right way to do it, but I am not sure.
fetch('www.example.net', { // Your POST endpoint
method: 'POST',
headers: {
"Content-Type": "image/jpeg"
},
body: e.target.files[0] // the file
})
.then(
response => response.json() // if the response is a JSON object
)
You can directly attach the file to the request body. Artifactory doesn't support form uploads (and it doesn't look like they plan to)
You'll still need to proxy the request somehow to avoid CORS issues, and if you're using user credentials, you should be cautious in how you treat them. Also, you could use a library like http-proxy-middleware to avoid having to write/test/maintain the proxy logic.
<input id="file-upload" type="file" />
<script>
function upload(data) {
var file = document.getElementById('file-upload').files[0];
var xhr = new XMLHttpRequest();
xhr.open('PUT', 'https://example.com/artifactory-proxy-avoiding-cors');
xhr.send(file);
}
</script>
Our front-end could not HTTP POST directly to the JFrog/Artifactory server. So we ended up using a Node.js server as a proxy, which is not very ideal.
Front-end:
// in an AngularJS controller:
$scope.onAcqImageFileChange = function (e) {
e.preventDefault();
let file = e.target.files[0];
$scope.acqImageFile = file;
};
// in an AngularJS service
createNewAcqImage: function(options) {
let file = options.file;
return $http({
method: 'POST',
url: '/proxy/image',
data: file,
headers: {
'Content-Type': 'image/jpeg'
}
})
},
Back-end:
const express = require('express');
const router = express.Router();
router.post('/image', function (req, res, next) {
const filename = uuid.v4();
const proxy = http.request({
method: 'PUT',
hostname: 'engci-maven.nabisco.com',
path: `/artifactory/cdt-repo/folder/${filename}`,
headers: {
'Authorization': 'Basic ' + Buffer.from('cdt-deployer:foobar').toString('base64'),
}
}, function(resp){
resp.pipe(res).once('error', next);
});
req.pipe(proxy).once('error', next);
});
module.exports = router;
not that we had to use a PUT request to send an image to Artifactory, not POST, something to do with Artifactory (the engci-maven.nabisco.com server is an Artifactory server). As I recall, I got CORS issues when trying to post directly from our front-end to the other server, so we had to use our server as a proxy, which is something I'd rather avoid, but oh well for now.