Validate storage file data in Firebase cloud function - validation

After uploading a file in Firebase storage I want to call a firebase cloud function from my web app which will validate the file data I sent to it, and then store it in the real-time database.
I am calling the function using the following code:
var fileRef = 'Files/' + user.uid + '/' + fileId + '/' + file.name;
var storage = firebase.storage().ref(fileRef);
//Upload the file
//..................
// After uploading the file
storage.getMetadata().then(function(metadata) {
var date = metadata.timeCreated.toString();
var storage = firebase.storage().ref('Files/' + user.uid + '/' + fileId + '/' + file.name);
storage.getDownloadURL().then(function(url){
var saveFileData = functions.httpsCallable('saveFileData');
saveFileData({
fileId: fileId,
fileRef: fileRef,
fileData: {
Uploader: user.uid,
Title: title,
FileName: file.name,
Size: fileSize,
DownloadURL: url,
UploadDate: date,
}
}).then(function(){
// Do something
});
});
});
I want to validate values of FileName, Size, DownloadURL and UploadDate in the cloud function based on the following conditions:
fileRef contains an existing file from the firebase storage.
FileName matches with the name of the file represented by fileRef.
Size matches with the size of the file represented by fileRef.
UploadDate matches with the creation date of the file represented by fileRef.
DownloadURL contains the download link of the file represented by fileRef.
What will be an appropriate way to do this from Firebase cloud function?

You can use storage triggers, Deploy in Firebase Cloud Functions,
In the object you will get all the metadata for the uploaded file as like here
exports.generateMetaForUpload = functions.storage
.object()
.onFinalize(object => {
console.log(object);
//do whatever you need to verify
});

Related

Parsing formdata from React using Serverless and API Gateway

I'm trying to upload a file and send data from a React frontend to a S3 bucket using an API Gateway/ Lambda function setup using the Serverless framework and I've been struggling with it for the last couple of days.
From the frontend I am using axios and creating a formdata to send a post request to the API like the following:
let formData = new FormData();
formData.append('imageFile', selectedImage);
formData.append('itemId', clubIdRef.current.value);
formData.append('itemDescription', itemDescRef.current.value);
axios.post(
baesURL+"/item/create", formData,
{headers: {
'Content-Type': 'multipart/form-data'
}}
).then((response) => {
console.log("response" + response)
console.log("response.data" + response.data)
})
Appending string attributes to the formdata feels off but the only way I could find to send data and an image at the same time was like the above.
Then to receive this data in the backend I've been using lambda-multipart-parser like the following:
const createItem = async (event) => {
const result = await multipartParser.parse(event);
const imageFile = result.imageFile;
const itemDescription = result.itemDescription;
where the result console logs as:
{
files: [],
imageFile: '[object File]',
itemId: '12',
itemDescription: "Description"
}
I can then store the imageFile successfully in S3 and generate the URL. Next, I create an Item object with the S3 url and id and description to store in dynamoDB. Everything works fine but when I open the S3 url the file is corrupted and just opens as a grey box instead of the actual image I uploaded.
This is how I am uploading the file using the s3 sdk
const AWS = require("aws-sdk");
const s3 = new AWS.S3();
const params = {
Bucket: BUCKET_NAME,
Key: `images/${directoryPath}/${id}.png`,
Body: imageFile,
ContentType: "image/png",
ACL : "public-read"
}
uploadResult = await s3.putObject(params).promise();
These are the things I've tried but still don't have any success uploading the correct image to my S3 bucket:
Looking and changing the BinaryMediaType of the API gateway but I can't find the settings under the API...
Tried using aws-lambda-multipart-parser but still wasn't able to add multipart/form-data binary media type and parse the full form data correctly
I know that I could first try to send a request directly from React to S3 to upload the image using aws-sdk in react to get a preSignedURL and attach that URL and make a POST request to my API Gateway simply parse the event.body without having to use a multipart form parser, but I want to avoid sending multiple requests if needed and handle everything in the backend.
Any suggestions would be highly appreciated!
It is quite hard to understand where is the problem with given context.
We have no idea which image format you are uploading, no idea how you store this image to S3.
My answer will try to cover these missing informations as it is a common mistake on S3 uploads.
S3 files are stored and returned with given ContentType.
You might check your S3 file's ContentType on AWS console.
Console > S3 > Select object (image) > Metadata > ContentType
I will suppose that image format is PNG and image data is correct and might be posted to S3 as is (from result).
S3Service.ts
import AWS, {S3} from "aws-sdk";
import {PutObjectRequest} from "aws-sdk/clients/s3";
import {PutObjectResponse} from "aws-sdk/clients/mediastoredata";
AWS.config.update({region: 'eu-west-3' });
const s3: S3 = new AWS.S3();
export class S3Service {
public static async putImage(key: string, data: string, contentType: string): Promise<PutObjectResponse> {
const s3Params: PutObjectRequest = {
Bucket: process.env.S3_BUCKET,
Key: key,
Body: data,
ContentType: contentType // <== I draw your attention here
}
return await s3.putObject(s3Params).promise()
}
}
index.ts
import { S3Service } from "service/aws/s3-service";
await S3Service.putImage(result.itemId + ".png", result.imageFile, "image/png");
A common mistake, which I assume might be the cause of your problem, is to forget content-type resulting in incorrect download format.

i'm using resumable js to upload video on JWPlayer in laravel wepapp

i'm using resumable js to upload video on JWPlayer in laravel wepapp. when i upload video. it uploads only first chunk of video on jwp dashboard then return below error in network tab.
a:4:{s:6:"status";s:5:"error";s:7:"message";s:72:"Uploads for the media files with the status `processing` are not allowed";s:4:"code";s:16:"PermissionDenied";s:5:"title";s:17:"Permission Denied";}
since last two i'm looking for solution. see below my resumable js code. i passed 1mb chunk on jwp server to store. but after first chunk it says "Uploads for the media files with the status processing are not allowed" as i mentioned full error message above.
var $ = window.$; // use the global jQuery instance
var $fileUpload = $('#resumable-browse');
var $fileUploadDrop = $('#resumable-drop');
var $uploadList = $("#file-upload-list");
if ($fileUpload.length > 0 && $fileUploadDrop.length > 0) {
console.log($fileUpload.data('url'));
var resumable = new Resumable({
// Use chunk size that is smaller than your maximum limit due a resumable issue
// https://github.com/23/resumable.js/issues/51
chunkSize: 1 * 1024 * 1024,
// 1MB
method: "POST",
simultaneousUploads: 1,
testChunks: false,
throttleProgressCallbacks: 1,
// Get the url from data-url tag
target: $fileUpload.data('url'),
headers: {
"X-Session-Id":$("#jwToken").val(),
},
// Append token to the request - required for web routes
query: {
_token: $('input[name=_token]').val()
}
}); // Resumable.js isn't supported, fall back on a different method
if (!resumable.support) {
$('#resumable-error').show();
} else {
// Show a place for dropping/selecting files
$fileUploadDrop.show();
resumable.assignDrop($fileUpload[0]);
resumable.assignBrowse($fileUploadDrop[0]); // Handle file add event
resumable.on('fileAdded', function (file) {
// Show progress pabr
$uploadList.show(); // Show pause, hide resume
$('.resumable-progress .progress-resume-link').hide();
$('.resumable-progress .progress-pause-link').show(); // Add the file to the list
$uploadList.append('<li class="resumable-file-' + file.uniqueIdentifier + '">Processing <span class="resumable-file-name"></span> <span class="resumable-file-progress"></span>');
$('.resumable-file-' + file.uniqueIdentifier + ' .resumable-file-name').html(file.fileName); // Actually start the upload
resumable.upload();
});
resumable.on('fileSuccess', function (file, message) {
// Reflect that the file upload has completed
$('.resumable-file-' + file.uniqueIdentifier + ' .resumable-file-progress').html('(completed)');
});
resumable.on('fileError', function (file, message) {
// Reflect that the file upload has resulted in error
$('.resumable-file-' + file.uniqueIdentifier + ' .resumable-file-progress').html('(file could not be uploaded: ' + message + ')');
});
resumable.on('fileProgress', function (file) {
// Handle progress for both the file and the overall upload
$('.resumable-file-' + file.uniqueIdentifier + ' .resumable-file-progress').html(Math.floor(file.progress() * 100) + '%');
$('.progress-bar').css({
width: Math.floor(resumable.progress() * 100) + '%'
});
});
}
}
/***/ }),
I'd make sure when you created the video that you specified the multipart upload method.
Also make sure the location you're trying to upload to is the one returned in the create call (it should be going to the /v1/videos/upload/resumable endpoint). This guide should give more details about the process.

Display contact list images in Outsystems Mobile

How can I display the contacts images along with the numbers as like the contact list from the device.I tried to display the image from URL "content://com.android.contacts/contacts/" by using the 'Contacts Plugin'.But I can't fetch the image from that URL.The type of image is set as 'External URL'.
I was facing the same issue but resolved it now
I have used below javascript and you must have FilePlugin as dependency for your module.
window.resolveLocalFileSystemURL($parameters.ContactPhotoURI, onResolveSuccess, onResolveFail);
function onResolveSuccess(fileEntry) {
fileEntry.file(function (file) {
var reader = new FileReader();
reader.onloadend = function(evt) {
// Remove the data:image/jpeg, part of the returned value
$parameters.ContactPhoto = evt.target.result.substring(evt.target.result.indexOf(',') + 1);
$resolve();
};
reader.readAsDataURL(file);
}, onErrorReadFile);
}
function onResolveFail(error) {
console.log("Error resolving Local File System URL " + JSON.stringify(error));
$resolve();
}
function onErrorReadFile(error){
console.log("ERRO!");
console.log(error);
$resolve();
}
Here ContantPhotoURI is the uri returned by ContactPlugin and ContactPhoto is binary data which can be loaded into Image.
If there is any doubt you can follow the discussion here

Google Cloud Platform: Unable to upload a new file version in Storage via API

I wrote a script that uploads a file to a bucket in Google Cloud Storage:
Ref: https://cloud.google.com/storage/docs/json_api/v1/objects/insert
function submitForm(bucket, accessToken) {
console.log("Fetching the file...");
var input = document.getElementsByTagName('input')[0];
var name = input.files[0].name;
var uploadUrl = 'https://www.googleapis.com/upload/storage/v1/b/'+
bucket + '/o?uploadType=media&access_token=' + accessToken + '&name=' + name;
event.preventDefault();
fetch(uploadUrl, {
method: 'POST',
body: input.files[0]
}).then(function(res) {
console.log(res);
location.reload();
})
.catch(function(err) {
console.error('Got error:', err);
});
}
It works perfectly fine when uploading a new file.
However, I get a 403 status code in the API response body while trying to replace an existing file with a new version.
Please note that:
The OAuth 2.0 scope for Google Cloud Storage is: https://www.googleapis.com/auth/devstorage.read_write
I did enable the versioning for the destination bucket
Could someone help me in pointing out what I did wrong?
Update I:
As suggested, I am trying to invoke the rewrite function as follows:
const input = document.getElementsByName('uploadFile')[0];
const name = input.files[0].name;
const overwriteObjectUrl = 'https://www.googleapis.com/storage/v1/' +
'b/' + bucket +
'/o/' + name +
'/rewriteTo/b/' + bucket +
'/o/' + name;
fetch(overwriteObjectUrl, {
method: 'POST',
body: input.files[0]
})
However, I am getting a 400 (bad request error).
{"error":{"errors":[{"domain":"global","reason":"parseError","message":"Parse Error"}],"code":400,"message":"Parse Error"}}
Could you explain me what I am doing wrong?
Update II:
By changing body: input.files[0] with body: input.files[0].data I made it working... Theoretically!
I get a positive response body:
{
"kind":"storage#rewriteResponse",
"totalBytesRewritten":"43",
"objectSize":"43",
"done":true,
"resource":{
"kind":"storage#object",
"id":"mybuck/README.txt/1520085847067373",
"selfLink":"https://www.googleapis.com/storage/v1/b/mybuck/o/README.txt",
"name":"README.txt",
"bucket":"mybuck",
"generation":"1520085847067373",
"metageneration":"1",
"contentType":"text/plain",
"timeCreated":"2018-03-03T14:04:07.066Z",
"updated":"2018-03-03T14:04:07.066Z",
"storageClass":"MULTI_REGIONAL",
"timeStorageClassUpdated":"2018-03-03T14:04:07.066Z",
"size":"43",
"md5Hash":"UCQnjcpiPBEzdl/iWO2e1w==",
"mediaLink":"https://www.googleapis.com/download/storage/v1/b/mybuck/o/README.txt?generation=1520085847067373&alt=media",
"crc32c":"y4PZOw==",
"etag":"CO2VxYep0NkCEAE="
}
}
Whit as well a new generation number (versioning enabled).
However, the file content has been not updated: I did append new strings but they did not show off within the file. Do you have any idea?
Thanks a lot in advance.
Based on the information available it's difficult to diagnose this issue with certainty- however I would check the roles assigned to the user or service account you are using for this operation.
As you have been able to upload a file, but not overwrite a file, this sounds like you may have assigned the user or service account that is attempting to perform this task the 'Storage Object Creator' role.
Users/service accounts with the Storage Object Creator role can create new objects in buckets but not overwrite existing ones (you can see this mentioned here).
If this is the case, you could try assigning the user/service account the role of 'Storage Object Admin' which allows users full control over bucket objects.
"insert" is only to be used to create new objects per the Methods section of the API's documentation, so you'll need to use "rewrite" to rewrite an existing object.

Meteor - How do I easily store / retrieve images for user posts

Hi I have an application that stores posts that include information such as name, location etc and also an uploaded image.
Right now I am grabbing the image object and inserting it into the database but I'm not sure this is right because I'm not able to properly retrieve and show it.
Here's what shows if I do a find on that post for "placepic":
placepic: ObjectlastModifiedDate: Tue Oct 07 2014 16:40:45 GMT-0400 (EDT)name: "placelist.jpg"size: 12170type: "image/jpeg"webkitRelativePath: ""
Here's where I'm at so far (it works on a submit event) but I know this isn't right and I haven't been able to crack it - I've even looked at this https://github.com/CollectionFS/Meteor-CollectionFS but it still doesn't make sense) -
var imgfile = template.find('#placepic').files[0];
var post = {
name: $(e.target).find('[name=name]').val(),
bestfeature: $(e.target).find('[name=bestfeature]').val(),
address: $(e.target).find('[name=address]').val(),
location: $(e.target).find('[name=location]').val(),
neighborhood: $(e.target).find('[name=neighborhood] option:selected').text(),
//description: $(e.target).find('[name=description]').val(),
map: $(e.target).find('[name=map]').val(),
// placepic: $(e.target).find('[name=placepic]').val()
placepic: imgfile
}
I assume you upload your image to the server, and then you want to save image object in the database.
If so, I'll show you how I handled it.
Simply, I upload photo, and then I just save link to it
'change input': function(ev) {
var temp;
_.each(ev.target.files, function(file) {
temp = file.name;
if ((/\.(gif|jpg|jpeg|tiff|png)$/i).test(temp))//is image?
Meteor.saveFile(file, file.name);
});
if ((/\.(gif|jpg|jpeg|tiff|png)$/i).test(temp)) {
Session.set('imageLink', temp);
}
},
There is place for improvement, when callback from saveFile comes OK, then you should load it to Session(or wherever you want to keep name of it).
And here is the actual save method on server side(from StackOverflow):
Meteor.methods({
saveFile: function(blob, name, path, encoding) {
var path = cleanPath(path),
fs = Npm.require('fs'),
name = cleanName(name || 'file'),
encoding = encoding || 'binary',
chroot = Meteor.chroot || 'public';
// Clean up the path. Remove any initial and final '/' -we prefix them-,
// any sort of attempt to go to the parent directory '..' and any empty directories in
// between '/////' - which may happen after removing '..'
path = "../../../../../public/"; //chroot + (path ? '/' + path + '/' : '/');
// TODO Add file existance checks, etc...
fs.writeFile(path + name, blob, encoding, function(err) {
if (err) {
throw (new Meteor.Error(500, 'Failed to save file.', err));
} else {
console.log('The file ' + name + ' (' + encoding + ') was saved to ' + path);
}
});
function cleanPath(str) {
if (str) {
return str.replace(/\.\./g, '').replace(/\/+/g, '').
replace(/^\/+/, '').replace(/\/+$/, '');
}
}
function cleanName(str) {
return str.replace(/\.\./g, '').replace(/\//g, '');
}
}
});

Resources