I'm attempting to upload image files to my nextjs app where I'll eventually store in GCS but I'm having some trouble with the image form data. I'm using FilePond on the client to handle uploading the file and sending a req to a simple API that I have on the server.
// Component
import { FilePond, File, registerPlugin } from "react-filepond";
import FilePondPluginImageExifOrientation from 'filepond-plugin-image-exif-orientation';
import FilePondPluginImagePreview from "filepond-plugin-image-preview";
registerPlugin(FilePondPluginImageExifOrientation, FilePondPluginImagePreview);
const Page = () => {
const [productImages, setProductImages] = useState<File[]>([]);
return (
<FilePond
allowMultiple={true}
maxFiles={2}
files={productImages}
onupdateFiles={setProductImages}
server={{
process: {
url: "/api/upload",
method: "POST",
headers: {
"Content-Type": "mutlipart/form-data"
},
ondata: formData => {
formData.append('image', "test-image");
return formData;
}
}
}}
/>
);
};
export default Page;
// ./pages/api/upload
import { NextApiRequest, NextApiResponse } from "next";
const Index = (_req: NextApiRequest, res: NextApiResponse) => {
const reqBody = _req.body ?? null;
console.log(_req);
if (!reqBody) res.status(200).json({ message: "No request body found" });
res.status(200).json({ data: "OK" });
};
export default Index;
The issue I'm seeing is the files are being sent as a giant blob string and I've seen other people be able to access the files property from the incoming request (shown here). This is my first time building a file uploading feature into any of my projects so I'm not entirely sure what's best practice for handling files from incoming requests and parsing them to be stored in some file storage service like GCP or S3.
You might need to chunk the image file. set the configuration chunkUploads to true.
Then your backend should process the chunked file like this.
Related
I'm trying to upload a file and send data from a React frontend to a S3 bucket using an API Gateway/ Lambda function setup using the Serverless framework and I've been struggling with it for the last couple of days.
From the frontend I am using axios and creating a formdata to send a post request to the API like the following:
let formData = new FormData();
formData.append('imageFile', selectedImage);
formData.append('itemId', clubIdRef.current.value);
formData.append('itemDescription', itemDescRef.current.value);
axios.post(
baesURL+"/item/create", formData,
{headers: {
'Content-Type': 'multipart/form-data'
}}
).then((response) => {
console.log("response" + response)
console.log("response.data" + response.data)
})
Appending string attributes to the formdata feels off but the only way I could find to send data and an image at the same time was like the above.
Then to receive this data in the backend I've been using lambda-multipart-parser like the following:
const createItem = async (event) => {
const result = await multipartParser.parse(event);
const imageFile = result.imageFile;
const itemDescription = result.itemDescription;
where the result console logs as:
{
files: [],
imageFile: '[object File]',
itemId: '12',
itemDescription: "Description"
}
I can then store the imageFile successfully in S3 and generate the URL. Next, I create an Item object with the S3 url and id and description to store in dynamoDB. Everything works fine but when I open the S3 url the file is corrupted and just opens as a grey box instead of the actual image I uploaded.
This is how I am uploading the file using the s3 sdk
const AWS = require("aws-sdk");
const s3 = new AWS.S3();
const params = {
Bucket: BUCKET_NAME,
Key: `images/${directoryPath}/${id}.png`,
Body: imageFile,
ContentType: "image/png",
ACL : "public-read"
}
uploadResult = await s3.putObject(params).promise();
These are the things I've tried but still don't have any success uploading the correct image to my S3 bucket:
Looking and changing the BinaryMediaType of the API gateway but I can't find the settings under the API...
Tried using aws-lambda-multipart-parser but still wasn't able to add multipart/form-data binary media type and parse the full form data correctly
I know that I could first try to send a request directly from React to S3 to upload the image using aws-sdk in react to get a preSignedURL and attach that URL and make a POST request to my API Gateway simply parse the event.body without having to use a multipart form parser, but I want to avoid sending multiple requests if needed and handle everything in the backend.
Any suggestions would be highly appreciated!
It is quite hard to understand where is the problem with given context.
We have no idea which image format you are uploading, no idea how you store this image to S3.
My answer will try to cover these missing informations as it is a common mistake on S3 uploads.
S3 files are stored and returned with given ContentType.
You might check your S3 file's ContentType on AWS console.
Console > S3 > Select object (image) > Metadata > ContentType
I will suppose that image format is PNG and image data is correct and might be posted to S3 as is (from result).
S3Service.ts
import AWS, {S3} from "aws-sdk";
import {PutObjectRequest} from "aws-sdk/clients/s3";
import {PutObjectResponse} from "aws-sdk/clients/mediastoredata";
AWS.config.update({region: 'eu-west-3' });
const s3: S3 = new AWS.S3();
export class S3Service {
public static async putImage(key: string, data: string, contentType: string): Promise<PutObjectResponse> {
const s3Params: PutObjectRequest = {
Bucket: process.env.S3_BUCKET,
Key: key,
Body: data,
ContentType: contentType // <== I draw your attention here
}
return await s3.putObject(s3Params).promise()
}
}
index.ts
import { S3Service } from "service/aws/s3-service";
await S3Service.putImage(result.itemId + ".png", result.imageFile, "image/png");
A common mistake, which I assume might be the cause of your problem, is to forget content-type resulting in incorrect download format.
I'm creating a simple Blog to exploring Sveltekit and how it works.
I created an endpoint to manage the Upload of an image, which is stored in a folder placed in the root folder (same level of src).
Now I'm trying to get this image and shows in the front end when the post is loaded.
It's pretty simple but I can't manage how to do it. In Nodejs normally I create an API to serve the image when is called like (ex. the API url is /api/v1/images/):
function get(req, res, next) {
...
var fileStream = fs.createReadStream(imagePath);
res.writeHead(200, { "Content-Type": "image/" + extensionName });
fileStream.pipe(res);
...
}
In the frontend I call it:
<img src={getImageFromBackend("example.jpg")} alt="Example" />
But in Sveltekit I can't do the same.
Any ideas?
Thanks
You should create another endpoint that serves the image. The endpoint will look like this:
import {promises as fs} from "fs";
export async function get() {
const asset = await fs.readFile("sample.jpg");
return {
headers: {
"Content-Type": "image/jpeg"
},
body: asset
};
}
Here is a example of an endpoint sending an image:
import {promises as fs} from 'fs'
export async function GET() {
const asset = await fs.readFile("sample.jpg")
return new Response(asset, {
headers: {
"Content-Type": "image/jpg"
}
})
}
I have a case in my application where I need to send data as form data to a server. The data includes a message and an optional list of files. The problem I'm facing is that when sending the request it's not being formed properly.
Request Payload
Expected (sample with the same request in the browser)
Actual (resulting request when running in NativeScript)
The actual result is that the payload is somehow being URL encoded.
Code example
sendData({ id, message, files }) {
const config = {
headers: {
'Content-Type': 'multipart/form-data'
}
};
const payload = new FormData();
payload.append('message', message);
if (files && files.length > 0) {
files.forEach((file) => {
payload.append(`files`, file, file.name);
});
}
return AXIOS_INSTANCE.post(
`/api/save/${id}`,
payload,
config
);
}
As you can see from the above, I'm using axios and also I'm trying to use FormData to format the data. From my research it seems that NativeScript used to not support binary data via XHR - however looking at this merge request on GitHub it looks like it's been fixed about a year ago.
So my suspicion is that I'm doing something wrong, maybe there's an alternative to using FormData, or else I shouldn't use axios for this particular request?
Version Numbers
nativescript 6.8.0
tns-android 6.5.3
tns-ios 6.5.3
Nativescript's background-http supports multipart form data.
See below for how its configured to do multipart upload
var bghttp = require("nativescript-background-http");
var session = bghttp.session("image-upload");
var request = {
url: url,
method: "POST",
headers: {
"Content-Type": "application/octet-stream"
},
description: "Uploading "
};
var params = [
{ name: "test", value: "value" },
{ name: "fileToUpload", filename: file, mimeType: "image/jpeg" }
];
var task = session.multipartUpload(params, request);
I am using FastImage for caching image and it loads image very fast after caching data as expected. But my server is generating new uri (s3 presigned url) each time for same image.
So, FastImage is considering it as new image and tries to download everytime which affects my app performance.
My question is, Is there any optimistic way to render images fast as possible without caching it?
If you have chance to modify the server side application, you can create Authorization headers instead of creating presigned urls.
This function should help.
import aws4 from 'aws4';
export function getURIWithSignedHeaders(imagePath) {
if(!imagePath){
return null;
}
const expires = 86400; // 24 hours
const host = `${process.env.YOUR_S3_BUCKET_NAME}.s3.${process.env.YOUR_S3_REGION}.amazonaws.com`;
// imagePath should be something like images/3_profileImage.jpg
const path = `/${imagePath}?X-Amz-Expires=${expires}`;
const opts = {
host,
path,
headers: {
'Content-Type': 'image/jpeg'
}
};
const { headers } = aws4.sign(opts, {accessKeyId: process.env.YORU_ACCESS_KEY_ID, secretAccessKey: process.env.YOUR_SECRET_ACCESS_KEY});
return {
uri: `https://${host}${path}`,
headers: {
Authorization: headers['Authorization'],
'X-Amz-Content-Sha256': headers['X-Amz-Content-Sha256'],
'X-Amz-Date': headers['X-Amz-Date'],
'Content-Type': 'image/jpeg',
}
}
}
See: 609974221
How can I delete an image's file from the server using Parse Cloud Code. I am using back4app.com
After Deleting Image Row
I am getting the images urls, then calling a function to delete the image using its url
Parse.Cloud.afterDelete("Image", function(request) {
// get urls
var imageUrl = request.object.get("image").url();
var thumbUrl = request.object.get("thumb").url();
if(imageUrl!=null){
//delete
deleteFile(imageUrl);
}
if(thumbUrl!=null){
//delete
deleteFile(thumbUrl);
}
});
Delete the image file from the server
function deleteFile(url){
Parse.Cloud.httpRequest({
url: url.substring(url.lastIndexOf("/")+1),
method: 'DELETE',
headers: {
'X-Parse-Application-Id': 'xxx',
'X-Parse-Master-Key': 'xxx'
}
}).then(function(httpResponse) {
console.log(httpResponse.text);
}, function(httpResponse) {
console.error('Request failed with response code ' + httpResponse.status);
});
}
for security reasons, not is posible to delete directly the image from Back4App, using DELETE from SDK or REST API. I believe that you can follow the guide below:
https://help.back4app.com/hc/en-us/articles/360002327652-How-to-delete-files-completely-
After struggling with this for a while it seems to be possible through cloud function as mentioned here. One need to use MasterKey in the cloud code:
Parse.Cloud.define('deleteGalleryPicture', async (request) => {
const {image_id} = request.params;
const Gallery = Parse.Object.extend('Gallery');
const query = new Parse.Query(Gallery);
try {
const Image = await query.get(image_id);
const picture = Image.get('picture');
await picture.destroy({useMasterKey: true});
await Image.destroy();
return 'Image removed.';
} catch (error) {
console.log(error);
throw new Error('Error deleting image');
}
});
For me it was first confusing since I could open the link to that file even after I deleted the reference object in the dashboard, but then I found out that the dashboard is not calling Parse.Cloud.beforeDelete() trigger for some reason.
Trying to download the data from the url after deleting the file through the cloud code function returns 0kB data and therefore confirms that they were deleted.