Use of HEIC files with filepond? - filepond

I'm trying to upload an HEIC file with filepond. Which file type should I specify?
At the moment I have this:
accepted-file-types="image/jpeg, image/png, image/gif, image/jpg"
I can't find anything in the docs about this, and my experimentation doesn't work.
Here's the test file I'm trying to upload:
https://github.com/tigranbs/test-heic-images/raw/master/image1.heic

Thank to #Rik for pointers. Here is some code which does this using filepond in Vue. Feels slightly hacky but does the job.
Accept the image/heic content type and add a custom validator:
<file-pond
v-if="supported"
ref="pond"
name="photo"
:allow-multiple="multiple"
accepted-file-types="image/jpeg, image/png, image/gif, image/jpg, image/heic"
:file-validate-type-detect-type="validateType"
:files="myFiles"
image-resize-target-width="800"
image-resize-target-height="800"
image-crop-aspect-ratio="1"
label-idle="Drag & Drop photos or <span class="btn btn-white ction"> Browse </span>"
:server="{ process, revert, restore, load, fetch }"
#init="photoInit"
#processfile="processed"
#processfiles="allProcessed"
/>
Then in the validator check for the filename, to handle browsers that don't set the correct MIME type:
validateType(source, type) {
const p = new Promise((resolve, reject) => {
if (source.name.toLowerCase().indexOf('.heic') !== -1) {
resolve('image/heic')
} else {
resolve(type)
}
})
return p
}
Then in the process callback, spot the HEIC file and use heic2any to convert it to PNG and use that data in the upload.
async process(fieldName, file, metadata, load, error, progress, abort) {
await this.$store.dispatch('compose/setUploading', true)
const data = new FormData()
const fn = file.name.toLowerCase()
if (fn.indexOf('.heic') !== -1) {
const blob = file.slice(0, file.size, 'image/heic')
const png = await heic2any({ blob })
data.append('photo', png, 'photo')
} else {
data.append('photo', file, 'photo')
}
data.append(this.imgflag, true)
data.append('imgtype', this.imgtype)
data.append('ocr', this.ocr)
data.append('identify', this.identify)
if (this.msgid) {
data.append('msgid', this.msgid)
} else if (this.groupid) {
data.append('groupid', this.groupid)
}
const ret = await this.$axios.post(process.env.API + '/image', data, {
headers: {
'Content-Type': 'multipart/form-data'
},
onUpLoadProgress: e => {
progress(e.lengthComputable, e.loaded, e.total)
}
})
if (ret.status === 200 && ret.data.ret === 0) {
this.imageid = ret.data.id
this.imagethumb = ret.data.paththumb
this.image = ret.data.path
if (this.ocr) {
this.ocred = ret.data.ocr
}
if (this.identify) {
this.identified = ret.data.items
}
load(ret.data.id)
} else {
error(
ret.status === 200 ? ret.data.status : 'Network error ' + ret.status
)
}
return {
abort: () => {
// We don't need to do anything - the server will tidy up hanging images.
abort()
}
}
},
The server is then blissfully unaware that the original file was HEIC at all, and everything proceeds as normal.

The format is image/heic, I tested this using this tool:
https://codepen.io/rikschennink/pen/NzRvbj
It's possible that not all browsers assign the correct mime type. You can work around that by using the fileValidateTypeDetectType property see here for an example:
https://pqina.nl/filepond/docs/patterns/plugins/file-validate-type/#custom-type-detection
Please note that uploading will work, but previewing the image won't.

Related

Shopify 2.0 Dawn - How to hide unavailable variants?

PROBLEM
Unavailable variant combinations are different from sold-out because customers don't understand it's the selects which make certain combinations 'not possible'.
Shopify's way of handling this is to display 'Unavailable' in the buy button. But customers think this means sold-out when in reality, they've chosen the wrong combination of variants...
The previous JS workarounds to remove unavailable or 'not possible' variants don't work in Shopify 2.0's new default/flagship theme, Dawn because the JS is different.
As far as I can tell, Dawn's variant JS was recently moved from /asstes/variants.js to line 497 in /assets/global.js.
SKILL
My CSS is decent but my JS is lame, I'm a designer sorry.
QUESTIONS
Based on user interaction with the first variant, how do you hide unavailable variants (not sold-out) in Shopify 2.0 Dawn?
How do you make one variant option set a checkbox instead of a radio button or radio?
What's the best way to add custom text as the first option in selects? e.g. 'Choose a size...' or 'Choose a color...' etc. Is it best to hard-code or use JS for this as well?
RESOURCES/EXAMPLES
Here's a pull request which grabs sold-out from the new Dawn JS but I don't understand how to adapt it for 'Unavailable' sorry (which is a different exception from sold-out): https://github.com/Shopify/dawn/pull/105
Here's an example of how to hide unavailable variants in the older Debut theme which doesn’t seem to work in the newer Dawn JS: https://www.youtube.com/watch?v=vspWDu_POYA
Here's a link to the JS gist referenced in that video: https://gist.github.com/jonathanmoore/c0e0e503aa732bf1c05b7a7be4230c61
And finally, here's the new code from Dawn at line 497 in /assets/global.js
class VariantSelects extends HTMLElement {
constructor() {
super();
this.addEventListener('change', this.onVariantChange);
}
onVariantChange() {
this.updateOptions();
this.updateMasterId();
this.toggleAddButton(true, '', false);
this.updatePickupAvailability();
if (!this.currentVariant) {
this.toggleAddButton(true, '', true);
this.setUnavailable();
} else {
this.updateMedia();
this.updateURL();
this.updateVariantInput();
this.renderProductInfo();
}
}
updateOptions() {
this.options = Array.from(this.querySelectorAll('select'), (select) => select.value);
}
updateMasterId() {
this.currentVariant = this.getVariantData().find((variant) => {
return !variant.options.map((option, index) => {
return this.options[index] === option;
}).includes(false);
});
}
updateMedia() {
if (!this.currentVariant || !this.currentVariant?.featured_media) return;
const newMedia = document.querySelector(
`[data-media-id="${this.dataset.section}-${this.currentVariant.featured_media.id}"]`
);
if (!newMedia) return;
const parent = newMedia.parentElement;
parent.prepend(newMedia);
window.setTimeout(() => { parent.scroll(0, 0) });
}
updateURL() {
if (!this.currentVariant) return;
window.history.replaceState({ }, '', `${this.dataset.url}?variant=${this.currentVariant.id}`);
}
updateVariantInput() {
const productForms = document.querySelectorAll(`#product-form-${this.dataset.section}, #product-form-installment`);
productForms.forEach((productForm) => {
const input = productForm.querySelector('input[name="id"]');
input.value = this.currentVariant.id;
input.dispatchEvent(new Event('change', { bubbles: true }));
});
}
updatePickupAvailability() {
const pickUpAvailability = document.querySelector('pickup-availability');
if (!pickUpAvailability) return;
if (this.currentVariant?.available) {
pickUpAvailability.fetchAvailability(this.currentVariant.id);
} else {
pickUpAvailability.removeAttribute('available');
pickUpAvailability.innerHTML = '';
}
}
renderProductInfo() {
fetch(`${this.dataset.url}?variant=${this.currentVariant.id}&section_id=${this.dataset.section}`)
.then((response) => response.text())
.then((responseText) => {
const id = `price-${this.dataset.section}`;
const html = new DOMParser().parseFromString(responseText, 'text/html')
const destination = document.getElementById(id);
const source = html.getElementById(id);
if (source && destination) destination.innerHTML = source.innerHTML;
document.getElementById(`price-${this.dataset.section}`)?.classList.remove('visibility-hidden');
this.toggleAddButton(!this.currentVariant.available, window.variantStrings.soldOut);
});
}
toggleAddButton(disable = true, text, modifyClass = true) {
const addButton = document.getElementById(`product-form-${this.dataset.section}`)?.querySelector('[name="add"]');
if (!addButton) return;
if (disable) {
addButton.setAttribute('disabled', true);
if (text) addButton.textContent = text;
} else {
addButton.removeAttribute('disabled');
addButton.textContent = window.variantStrings.addToCart;
}
if (!modifyClass) return;
}
setUnavailable() {
const addButton = document.getElementById(`product-form-${this.dataset.section}`)?.querySelector('[name="add"]');
if (!addButton) return;
addButton.textContent = window.variantStrings.unavailable;
document.getElementById(`price-${this.dataset.section}`)?.classList.add('visibility-hidden');
}
getVariantData() {
this.variantData = this.variantData || JSON.parse(this.querySelector('[type="application/json"]').textContent);
return this.variantData;
}
}
customElements.define('variant-selects', VariantSelects);
class VariantRadios extends VariantSelects {
constructor() {
super();
}
updateOptions() {
const fieldsets = Array.from(this.querySelectorAll('fieldset'));
this.options = fieldsets.map((fieldset) => {
return Array.from(fieldset.querySelectorAll('input')).find((radio) => radio.checked).value;
});
}
}
customElements.define('variant-radios', VariantRadios);
Any help or pointers in the right direction would be much appreciated. Cheers

Filepond, DELETE request, and filename

I've got a server that accepts the DELETE request from Filepond just fine. However, I am having a hard time trying to find the filename in the request. After reading, I found that it is in the Body of the DELETE request. Is that true? If so, how does my server access it? req.filename?
if(typeofreq === 'DELETE'){
var nameoffile = req.body; //<---??? How do I pull out the filename?
if(nameoffile === undefined){
res.err;
}else{
var fs = require('fs');
fs.unlinkSync('./assets/images/' + req.params(filename));
}
}
---------------------FilePond React -----------
<> <div className="Fileupload">
<FilePond ref={ref => this.pond = ref}
files={this.state.files}
name="avatar"
allowImageCrop={true}
allowMultiple={true}
method="POST"
maxFiles={5}
allowImageExifOrientation={true}
allowImagePreview={true}
imageResizeTargetWidth={600}
imageCropAspectRatio={1}
allowFileTypeValidation={true}
maxFileSize={10485760}
fetch={null}
revert={null}
allowFileEncode={true}
imageTransformVariants={{
"thumb_medium_": transforms => {
transforms.resize.size.width = 384;
return transforms;
},
'thumb_small_': transforms => {
transforms.resize.size.width = 128;
return transforms;
}
}
}
fileRenameFunction={ file => new Promise(resolve => {
resolve(window.prompt('Enter new filename', file.name)) })}
acceptedFileTypes="image/jpeg, image/png"
server= {{
url: "/photocontroller",
process: {
headers: {
'X-CSRF-TOKEN': '{{ csrf_token() }}'
//"X-CSRF-TOKEN": $('input[name="_token"]').val()
}
},
load:'/user/getphoto'
}
}
oninit={() => this.handleInit() }
onupdatefiles={(fileItems) => {
// Set current file objects to this.state
this.setState({
files: fileItems.map(fileItem => fileItem.file)
});
}}>
</FilePond>
</div>
The problem was due to not using/understanding that axios doesn't use the body of the request but uses "data".

Upload to S3 from React Native with AWS Amplify

I'm trying to upload an image to S3 from React Native using Amplify. I am able to upload a text file SUCCESSFULLY. But not an image.
Here is my code:
import React from 'react';
import {View, Text, Button, Image} from 'react-native';
import {identityPoolId, region, bucket} from '../auth';
import image from '../assets/background.png';
import Amplify, {Storage} from 'aws-amplify';
Amplify.configure({
Auth: {identityPoolId,region},
Storage : {bucket,region}
})
const upload = () => {
Storage.put('logo.jpg', image, {contentType: 'image/jpeg'})
.then(result => console.log('result from successful upload: ', result))
.catch(err => console.log('error uploading to s3:', err));
}
const get = () => { //this works for both putting and getting a text file
Storage.get('amir.txt')
.then(res => console.log('result get', res))
.catch(err => console.log('err getting', err))
}
export default function ImageUpload(props) {
return (
<View style={{alignItems : 'center'}}>
<Image style={{width: 100, height: 100}} source={image} />
<Text>Click button to upload above image to S3</Text>
<Button title="Upload to S3" onPress={upload}/>
<Button title="Get from S3" onPress={get}/>
</View>
)
}
the error message is:
error uploading to s3: [Error: Unsupported body payload number]
I ended up using the react-native-aws3 library to upload the images to S3.
I wish it could be more straight forward to find answers with how to upload an image directly using AWS amplify, but it wasn't working. So here is what I did:
(the wrapper of this function is a React Component. I'm using ImagePicker from 'expo-image-picker', Permissions from 'expo-permissions' and Constants from 'expo-constants' to set up the Image uploading from the Camera Roll)
import {identityPoolId, region, bucket, accessKey, secretKey} from '../auth';
import { RNS3 } from 'react-native-aws3';
async function s3Upload(uri) {
const file = {
uri,
name : uri.match(/.{12}.jpg/)[0],
type : "image/png"
};
const options = { keyPrefix: "public/", bucket, region,
accessKey, secretKey, successActionStatus: 201}
RNS3.put(file, options)
.progress(event => {
console.log(`percentage uploaded: ${event.percent}`);
})
.then(res => {
if (res.status === 201) {
console.log('response from successful upload to s3:',
res.body);
console.log('S3 URL', res.body.postResponse.location);
setPic(res.body.postResponse.location);
} else {
console.log('error status code: ', res.status);
}
})
.catch(err => {
console.log('error uploading to s3', err)
})
}
const pickImage = async () => {
let result = await ImagePicker.launchImageLibraryAsync({
mediaTypes : ImagePicker.MediaTypeOptions.All,
allowsEditing : true,
aspect : [4,3],
quality : 1
});
console.log('image picker result', result);
if (!result.cancelled) {
setImage(result.uri);
s3Upload(result.uri);
}
}
In our recent Single Page Application (SPA) style web application, from React, we used "S3 Signed URLs" for efficient upload/download of files and I felt this has resulted in a cleaner design as compared to direct upload/download.
What is the back-end services implemented in?
You can do it on your node server, save it the URL and send it back to the app.
It'll look like:
const AWS = require('aws-sdk');
var s3 = new AWS.S3({accessKeyId:'XXXXXXXXXXXX', secretAccessKey:'YYYYYYYYYYYY', region:'REGION'});
var params = {Bucket: 'yourBucket', Key: 'your_image/your_image.jpg', ContentType: 'image/jpeg'};
s3.getSignedUrl('putObject', params, function (err, url) {
// send it back to the app
});
On the app side you'll have something like:
const xhr = new XMLHttpRequest()
xhr.onreadystatechange = function() {
if (xhr.readyState === 4) {
if (xhr.status === 200) {
Alert.aler("Upload", "Done");
} else {
Alert.aler("Upload", "Failed");
}
}
}
xhr.open('PUT', presignedUrlReceiveFromServer)
xhr.setRequestHeader('Content-Type', fileType)
xhr.send({ uri: imageFilePath, type: fileType, name: imageFileName })
Hope it helps

Recording and sending audio in nativescript/nativescript-audio

Im working with angular/nativescript with the plugin 'nativescript-audio', I need record audio with best quality possible and send to an API with HttpClient.
There are a few things I need to do, like converting audio to base64 and recording it with a good quality, but I do not have a good knowledge of native audio.
Currently, using this plugin, I can record and play the audio in the library itself, but when sent in base 64 it arrives unrecognizable in the API.
What Im doing:
private async _startRecord(args) {
if (!this._recorder.hasRecordPermission()) await this._askRecordPermission(args);
try {
this._isRecording = true;
await this._recorder.start(this._getRecorderOptions());
} catch (err) {
console.log(err);
}
}
private _getRecorderOptions(): AudioRecorderOptions {
let audioFileName = 'record'
let audioFolder = knownFolders.currentApp().getFolder('audio');
let recordingPath = `${audioFolder.path}/${audioFileName}.${this.getPlatformExtension()}`;
let androidFormat, androidEncoder;
if (platform.isAndroid) {
androidFormat = 4;
androidEncoder = 2;
}
return {
filename: recordingPath,
format: androidFormat,
encoder: androidEncoder,
metering: true,
infoCallback: info => { console.log(JSON.stringify(info)); },
errorCallback: err => this._isRecording = false
}
}
Then:
let audioFileName = 'record';
let audioFolder = knownFolders.currentApp().getFolder('audio');
let file: File = audioFolder.getFile(`${audioFileName}.${this.getPlatformExtension()}`);
let b = file.readSync();
var javaString = new java.lang.String(b);
var encodedString = android.util.Base64.encodeToString(
javaString.getBytes(),
android.util.Base64.DEFAULT
);
this.service.sendFile(encodedString)
.subscribe(e => {
console.log(e);
}, error => {
console.log('ERROR ////////////////////////////////////////////');
console.log(error.status);
})
The service:
sendFile(fileToUpload: any): Observable<any> {
let url: string = `myapi.com`
let body = { "base64audioFile": fileToUpload }
return this.http.post<any>(url, body, {
headers: new HttpHeaders({
// 'Accept': 'application/json',
// 'Content-Type': 'multipart/form-data',
// "Content-Type": "multipart/form-data"
}),
observe: 'response'
});
}
I've already tried changing the recording options in several ways, but I do not know which one is right for the best audio quality and which formats and encodings I need:
androidFormat = 3;
androidEncoder = 1;
channels: 2,
sampleRate: 96000,
bitRate: 1536000,
The result base64 varies a lot with the type of encode I make, but so far I have not managed to get anything recognizable, just some hissing and unrecognizable noises.
First of all, you may just pass the bytes directly to encodeToString(...) method. You don't have to create a string from bytes, it's not valid.
Also use NO_WRAP flag instead of DEFAULT.
var encodedString = android.util.Base64.encodeToString(
b,
android.util.Base64.NO_WRAP
);
Here is a Playground Sample which I wrote a while ago to test base64 encode on iOS & Android. You might have to update the file url, the one in the source gives 404 (I had just picked a random mp3 file form internet for testing).

Spot the difference between these two images

Programmatically, my code is detecting a difference between two classes of images, and always rejecting one class, while always allowing the other.
I have yet to find any difference between the images that yield the error and the ones that don't an yield error. But there has to be some difference, because the ones that yield an error do so 100% of the time, and the others work as expected 100% of the time.
In particular, I have inspected color format: RGB in both groups; size: no notable difference; datatype: uint8 in both; magnitude of pixel values: similar in both.
Below are two images that never work, followed by two images that always work:
This image never works: https://www.colourbox.com/preview/11906131-maple-tree-and-grass-silhouette.jpg
This image never works: http://feldmanphoto.com/wp-content/uploads/awe-inspiring-house-clipart-black-and-white-disney-coloring-pages-big-clipartxtras-illistration-background-housewives-bouncy.jpeg
This image always works: http://www.spacedesign.us/wp-content/uploads/landscape-with-old-tree-and-grass-over-white-background-black-and-black-and-white-trees.jpg
This image always works: http://www.modernhouse.co/wp-content/uploads/2017/07/1024px-RoseSeidlerHouseSulmanPrize.jpg
How can I spot the difference?
The scenario is that I am using Firebase with Swift iOS front end to send these images to a Google Cloud ML-engine hosted convnet. Some images work all the time and certain others never work as above. Further, all images work when I use the gcloud versions predict CLI. To me the issue is necessarily something in the images. Hence I am posting here for the imaging group. Code is included as requested for completeness.
CODE of index.js file is included:
'use strict';
const functions = require('firebase-functions');
const gcs = require('#google-cloud/storage');
const admin = require('firebase-admin');
const exec = require('child_process').exec;
const path = require('path');
const fs = require('fs');
const google = require('googleapis');
const sizeOf = require('image-size');
admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
const rtdb = admin.database();
const dbRef = rtdb.ref();
function cmlePredict(b64img) {
return new Promise((resolve, reject) => {
google.auth.getApplicationDefault(function (err, authClient) {
if (err) {
reject(err);
}
if (authClient.createScopedRequired && authClient.createScopedRequired()) {
authClient = authClient.createScoped([
'https://www.googleapis.com/auth/cloud-platform'
]);
}
var ml = google.ml({
version: 'v1'
});
const params = {
auth: authClient,
name: 'projects/myproject-18865/models/my_model',
resource: {
instances: [
{
"image_bytes": {
"b64": b64img
}
}
]
}
};
ml.projects.predict(params, (err, result) => {
if (err) {
reject(err);
} else {
resolve(result);
}
});
});
});
}
function resizeImg(filepath) {
return new Promise((resolve, reject) => {
exec(`convert ${filepath} -resize 224x ${filepath}`, (err) => {
if (err) {
console.error('Failed to resize image', err);
reject(err);
} else {
console.log('resized image successfully');
resolve(filepath);
}
});
});
}
exports.runPrediction = functions.storage.object().onChange((event) => {
fs.rmdir('./tmp/', (err) => {
if (err) {
console.log('error deleting tmp/ dir');
}
});
const object = event.data;
const fileBucket = object.bucket;
const filePath = object.name;
const bucket = gcs().bucket(fileBucket);
const fileName = path.basename(filePath);
const file = bucket.file(filePath);
if (filePath.startsWith('images/')) {
const destination = '/tmp/' + fileName;
console.log('got a new image', filePath);
return file.download({
destination: destination
}).then(() => {
if(sizeOf(destination).width > 224) {
console.log('scaling image down...');
return resizeImg(destination);
} else {
return destination;
}
}).then(() => {
console.log('base64 encoding image...');
let bitmap = fs.readFileSync(destination);
return new Buffer(bitmap).toString('base64');
}).then((b64string) => {
console.log('sending image to CMLE...');
return cmlePredict(b64string);
}).then((result) => {
console.log(`results just returned and is: ${result}`);
let predict_proba = result.predictions[0]
const res_pred_val = Object.keys(predict_proba).map(k => predict_proba[k])
const res_val = Object.keys(result).map(k => result[k])
const class_proba = [1-res_pred_val,res_pred_val]
const opera_proba_init = 1-res_pred_val
const capitol_proba_init = res_pred_val-0
// convert fraction double to percentage int
let opera_proba = (Math.floor((opera_proba_init.toFixed(2))*100))|0
let capitol_proba = (Math.floor((capitol_proba_init.toFixed(2))*100))|0
let feature_list = ["houses", "trees"]
let outlinedImgPath = '';
let imageRef = db.collection('predicted_images').doc(filePath.slice(7));
outlinedImgPath = `outlined_img/${filePath.slice(7)}`;
imageRef.set({
image_path: outlinedImgPath,
opera_proba: opera_proba,
capitol_proba: capitol_proba
});
let predRef = dbRef.child("prediction_categories");
let arrayRef = dbRef.child("prediction_array");
predRef.set({
opera_proba: opera_proba,
capitol_proba: capitol_proba,
});
arrayRef.set({first: {
array_proba: [opera_proba,capitol_proba],
brief_description: ["a","b"],
more_details: ["aaaa","bbbb"],
feature_list: feature_list},
zummy1: "",
zummy2: ""});
return bucket.upload(destination, {destination: outlinedImgPath});
});
} else {
return 'not a new image';
}
});
Issue was that the bad images were grayscale, not RGB as expected by my model. I initially had checked this first by looking at the shape. But the 'bad' images had 3 color channels, each of those 3 channels stored the same number --- so my model was refusing to accept them. Also, as expected and contrary to what I initially thought I observed, turns out the gcloud ML-engine predict CLI actually also failed for these images. Took me 2 days to figure this out!

Resources