I have uploaded some images and videos in the minio server (https://play.minio.io:9000) from my android client. Now I have been able to download the uploaded content from that folder. What I can see that the content is not the actual image/video, instead it shows me some encrypted format. My question is how can I get the actual image/video that I have uploaded. Is there any such tool or anything else I could do?
Any help would be appreciated..
Thanks in advance.
Here is my solution for Image preview in minio
# set alias
mc alias set myminio HOST:9000 MINIO_CLIENT MINIO_SECRET
# create bucket
mc mb myminio/image
# set policy for the bucket
mc policy set download myminio/image
My uploader function
const fs = require('fs');
const file = './cat.jpg';
const fileStream = fs.createReadStream(file);
const metadata = {
'Content-type': 'image',
};
const upload = async () => {
const etag = await minioClient.putObject(
'image',
'cat.jpg',
fileStream,
metadata // this metadata in important to preview image from browser instead of download
);
console.log({ etag });
};
Now navigate to http://localhost:9000/image/cat.jpg
You will see your cat image from direct link.
Related
I'm trying to upload a file and send data from a React frontend to a S3 bucket using an API Gateway/ Lambda function setup using the Serverless framework and I've been struggling with it for the last couple of days.
From the frontend I am using axios and creating a formdata to send a post request to the API like the following:
let formData = new FormData();
formData.append('imageFile', selectedImage);
formData.append('itemId', clubIdRef.current.value);
formData.append('itemDescription', itemDescRef.current.value);
axios.post(
baesURL+"/item/create", formData,
{headers: {
'Content-Type': 'multipart/form-data'
}}
).then((response) => {
console.log("response" + response)
console.log("response.data" + response.data)
})
Appending string attributes to the formdata feels off but the only way I could find to send data and an image at the same time was like the above.
Then to receive this data in the backend I've been using lambda-multipart-parser like the following:
const createItem = async (event) => {
const result = await multipartParser.parse(event);
const imageFile = result.imageFile;
const itemDescription = result.itemDescription;
where the result console logs as:
{
files: [],
imageFile: '[object File]',
itemId: '12',
itemDescription: "Description"
}
I can then store the imageFile successfully in S3 and generate the URL. Next, I create an Item object with the S3 url and id and description to store in dynamoDB. Everything works fine but when I open the S3 url the file is corrupted and just opens as a grey box instead of the actual image I uploaded.
This is how I am uploading the file using the s3 sdk
const AWS = require("aws-sdk");
const s3 = new AWS.S3();
const params = {
Bucket: BUCKET_NAME,
Key: `images/${directoryPath}/${id}.png`,
Body: imageFile,
ContentType: "image/png",
ACL : "public-read"
}
uploadResult = await s3.putObject(params).promise();
These are the things I've tried but still don't have any success uploading the correct image to my S3 bucket:
Looking and changing the BinaryMediaType of the API gateway but I can't find the settings under the API...
Tried using aws-lambda-multipart-parser but still wasn't able to add multipart/form-data binary media type and parse the full form data correctly
I know that I could first try to send a request directly from React to S3 to upload the image using aws-sdk in react to get a preSignedURL and attach that URL and make a POST request to my API Gateway simply parse the event.body without having to use a multipart form parser, but I want to avoid sending multiple requests if needed and handle everything in the backend.
Any suggestions would be highly appreciated!
It is quite hard to understand where is the problem with given context.
We have no idea which image format you are uploading, no idea how you store this image to S3.
My answer will try to cover these missing informations as it is a common mistake on S3 uploads.
S3 files are stored and returned with given ContentType.
You might check your S3 file's ContentType on AWS console.
Console > S3 > Select object (image) > Metadata > ContentType
I will suppose that image format is PNG and image data is correct and might be posted to S3 as is (from result).
S3Service.ts
import AWS, {S3} from "aws-sdk";
import {PutObjectRequest} from "aws-sdk/clients/s3";
import {PutObjectResponse} from "aws-sdk/clients/mediastoredata";
AWS.config.update({region: 'eu-west-3' });
const s3: S3 = new AWS.S3();
export class S3Service {
public static async putImage(key: string, data: string, contentType: string): Promise<PutObjectResponse> {
const s3Params: PutObjectRequest = {
Bucket: process.env.S3_BUCKET,
Key: key,
Body: data,
ContentType: contentType // <== I draw your attention here
}
return await s3.putObject(s3Params).promise()
}
}
index.ts
import { S3Service } from "service/aws/s3-service";
await S3Service.putImage(result.itemId + ".png", result.imageFile, "image/png");
A common mistake, which I assume might be the cause of your problem, is to forget content-type resulting in incorrect download format.
I am using minio to manage the files
const getMinioClient = () => {
const minioClient = new Minio.Client({
endPoint: '127.0.0.1',
port: 9000,
useSSL: false,
accessKey: 'minioadmin',
secretKey: 'minioadmin'
});
return minioClient;
};
uploadFile(bucketName, newFileName, localFileLocation,metadata={}) {
return new Promise((resolve, reject) => {
const minioClient = getMinioClient();
//'application/octet-stream'
minioClient.fPutObject(bucketName, newFileName, localFileLocation, metadata , (err, etag) => {
if (err) return reject(err);
return resolve(etag);
});
});
}
with the following code I can upload the file, after successfully uploading it returns me only with etag, but I want to get the download link, how would I get it directly without searching the filename again.
You won't be able to get something like Public URL/Link for accessing images unless you ask for it to manually generate a time limited download URL using something like:
https://min.io/docs/minio/linux/reference/minio-mc/mc-share-download.html#generate-a-url-to-download-object-s
One workaround is to let nginx directly access the location you are uploading your files to:
https://gist.github.com/harshavardhana/f05b60fe6f96803743f38bea4b565bbf
After you have successfully written your file with your code above, you can use presignedUrl method to generate the link to your image.
An example for Javascript is here: https://min.io/docs/minio/linux/developers/javascript/API.html#presignedUrl:~:text=//%20presigned%20url%20for%20%27getObject%27%20method.%0A//%20expires%20in%20a%20day.%0AminioClient.presignedUrl(%27GET%27%2C%20%27mybucket%27%2C%20%27hello.txt%27%2C%2024*60*60%2C%20function(err%2C%20presignedUrl)%20%7B%0A%20%20if%20(err)%20return%20console.log(err)%0A%20%20console.log(presignedUrl)%0A%7D)
In any case you have to set an expiration time. Here or you set a very long time, which is suitable to your app or if you have a backend, require the images from Frontend through the backend with the getObject method: getObject(bucketName, objectName, getOpts[, callback]).
https://min.io/docs/minio/linux/developers/javascript/API.html#presignedUrl:~:text=getObject(bucketName%2C%20objectName%2C%20getOpts%5B%2C%20callback%5D)
If you have only a few number of static images to show in your app, (which are not uploaded by your app), you can also create the links manually with tme minio client or from the Minio-UI.
I have the #pdftron node module installed in a test Node (Hapi) application on my Mac. I'm trying to generate a PDF file from a HTML string but the saved file is 0 KB. I've tried this two ways:
By implementing the example code in the GET handler for a route
configured in index.js
By running the sample scripts that are
installed at
node_modules/#pdftron/pdfnet-node/samples/HTML2PDFTest/NODEJS/HTML2PDFTest.js
In both cases, any new PDFs that are saved have a size of 0KB and any existing PDFs that are supposed to be modified by the process remain unchanged.
I've checked that the html2pdf module library path is being set correctly.
The route handler code is as follows, which is in accordance with example code provided by PDFTRon.
server.route({
method: 'GET',
path: '/pdftron/html',
handler: (request, h) => {
const { PDFNet } = require('#pdftron/pdfnet-node/lib/pdfnet.js')
const main = async () => {
await PDFNet.initialize()
await PDFNet.HTML2PDF.setModulePath('node_modules/#pdftron/pdfnet-node/lib')
const output_path = '/tmp/'
try {
const html2pdf = await PDFNet.HTML2PDF.create();
const doc = await PDFNet.PDFDoc.create();
const html = '<html><body><h1>Heading</h1><p>Paragraph.</p></body></html>'
html2pdf.insertFromHtmlString(html);
await html2pdf.convert(doc);
doc.save(output_path.concat('pdf2html.pdf'), PDFNet.SDFDoc.SaveOptions.e_linearized);
} catch (err) {
console.log(err)
}
}
PDFNet.runWithCleanup(main, 0).then(function () { PDFNet.shutdown(); })
return 'PDF2HTML Test'
}
})
Any thoughts/suggestions would be much appreciated.
Update on this - I had feedback from PDFTron to say that there is currently an issue with the current stable release of the Mac OS build and that they are addressing it.
In the meantime, they pointed me to the nightly experimental builds here. I downloaded and installed a recent experimental build and the PDF files now save OK using that.
How can I delete an image's file from the server using Parse Cloud Code. I am using back4app.com
After Deleting Image Row
I am getting the images urls, then calling a function to delete the image using its url
Parse.Cloud.afterDelete("Image", function(request) {
// get urls
var imageUrl = request.object.get("image").url();
var thumbUrl = request.object.get("thumb").url();
if(imageUrl!=null){
//delete
deleteFile(imageUrl);
}
if(thumbUrl!=null){
//delete
deleteFile(thumbUrl);
}
});
Delete the image file from the server
function deleteFile(url){
Parse.Cloud.httpRequest({
url: url.substring(url.lastIndexOf("/")+1),
method: 'DELETE',
headers: {
'X-Parse-Application-Id': 'xxx',
'X-Parse-Master-Key': 'xxx'
}
}).then(function(httpResponse) {
console.log(httpResponse.text);
}, function(httpResponse) {
console.error('Request failed with response code ' + httpResponse.status);
});
}
for security reasons, not is posible to delete directly the image from Back4App, using DELETE from SDK or REST API. I believe that you can follow the guide below:
https://help.back4app.com/hc/en-us/articles/360002327652-How-to-delete-files-completely-
After struggling with this for a while it seems to be possible through cloud function as mentioned here. One need to use MasterKey in the cloud code:
Parse.Cloud.define('deleteGalleryPicture', async (request) => {
const {image_id} = request.params;
const Gallery = Parse.Object.extend('Gallery');
const query = new Parse.Query(Gallery);
try {
const Image = await query.get(image_id);
const picture = Image.get('picture');
await picture.destroy({useMasterKey: true});
await Image.destroy();
return 'Image removed.';
} catch (error) {
console.log(error);
throw new Error('Error deleting image');
}
});
For me it was first confusing since I could open the link to that file even after I deleted the reference object in the dashboard, but then I found out that the dashboard is not calling Parse.Cloud.beforeDelete() trigger for some reason.
Trying to download the data from the url after deleting the file through the cloud code function returns 0kB data and therefore confirms that they were deleted.
I need to upload an image, and display it, as well as save it so that I don't lose it when I refresh the localhost. This needs to be done using an "Upload" button, which prompts for a file-selection.
I am using node.js and express for the server-side code.
First of all, you should make an HTML form containing a file input element. You also need to set the form's enctype attribute to multipart/form-data:
<form method="post" enctype="multipart/form-data" action="/upload">
<input type="file" name="file">
<input type="submit" value="Submit">
</form>
Assuming the form is defined in index.html stored in a directory named public relative to where your script is located, you can serve it this way:
const http = require("http");
const path = require("path");
const fs = require("fs");
const express = require("express");
const app = express();
const httpServer = http.createServer(app);
const PORT = process.env.PORT || 3000;
httpServer.listen(PORT, () => {
console.log(`Server is listening on port ${PORT}`);
});
// put the HTML file containing your form in a directory named "public" (relative to where this script is located)
app.get("/", express.static(path.join(__dirname, "./public")));
Once that's done, users will be able to upload files to your server via that form. But to reassemble the uploaded file in your application, you'll need to parse the request body (as multipart form data).
In Express 3.x you could use express.bodyParser middleware to handle multipart forms but as of Express 4.x, there's no body parser bundled with the framework. Luckily, you can choose from one of the many available multipart/form-data parsers out there. Here, I'll be using multer:
You need to define a route to handle form posts:
const multer = require("multer");
const handleError = (err, res) => {
res
.status(500)
.contentType("text/plain")
.end("Oops! Something went wrong!");
};
const upload = multer({
dest: "/path/to/temporary/directory/to/store/uploaded/files"
// you might also want to set some limits: https://github.com/expressjs/multer#limits
});
app.post(
"/upload",
upload.single("file" /* name attribute of <file> element in your form */),
(req, res) => {
const tempPath = req.file.path;
const targetPath = path.join(__dirname, "./uploads/image.png");
if (path.extname(req.file.originalname).toLowerCase() === ".png") {
fs.rename(tempPath, targetPath, err => {
if (err) return handleError(err, res);
res
.status(200)
.contentType("text/plain")
.end("File uploaded!");
});
} else {
fs.unlink(tempPath, err => {
if (err) return handleError(err, res);
res
.status(403)
.contentType("text/plain")
.end("Only .png files are allowed!");
});
}
}
);
In the example above, .png files posted to /upload will be saved to uploaded directory relative to where the script is located.
In order to show the uploaded image, assuming you already have an HTML page containing an img element:
<img src="/image.png" />
you can define another route in your express app and use res.sendFile to serve the stored image:
app.get("/image.png", (req, res) => {
res.sendFile(path.join(__dirname, "./uploads/image.png"));
});