I have a wasm file fib.wasm which contains function fib(n).
If run it in the browser, we can do
var module, functions = {};
fetch('fib.wasm')
.then(response => response.arrayBuffer())
.then(buffer => new Uint8Array(buffer))
.then(binary => {
var moduleArgs = {
wasmBinary: binary,
onRuntimeInitialized: function () {
functions.fib =
module.cwrap('fib',
'number',
['number']);
onReady();
}
};
module = Module(moduleArgs);
});
If in Node, since fetch is not implemented, we can do
const fs = require('fs')
const buf = fs.readFileSync('fib.wasm')
(async () => { res = await WebAssembly.instantiate(buf); })()
const { fib } = res.instance.exports
However, in d8 shell, both ways involves functions undefined. How can we run wasm in d8?
The d8 shell has a function read() which reads a file from disk. It takes an optional parameter to specify binary mode. So the following should work:
const buf = read('fib.wasm', 'binary');
let res;
WebAssembly.instantiate(buf).then((x) => res = x, (error) => console.log(error));
const { fib } = res.instance.exports;
Related
I have been trying to get the base64 of a File of Nativescript, so far I have tried the next:
const documents: Folder = <Folder>knownFolders.documents();
const folder: Folder = <Folder>documents.getFolder("Download");
const file: File = <File>folder.getFile("Vista general Aguas.pdf");
const toBase64 = (fle: File) => new Promise((resolve, reject) => {
const reader = new FileReader();
const newFile = fle.readSync(err => console.log(err));
// I know that I have to convert the binary to base64, but I'm getting this [B#9b1d878 in newFile
reader.readAsDataURL(newFile);
reader.onload = () => resolve(reader.result);
reader.onerror = error => reject(error);
});
toBase64(file).then(result => console.log('64', result));
Any help is appreciated!
I have created function using native implementation. This function return base64 string.
public getBase64String = (path) => {
const sourceFile: fs.File = fs.File.fromPath(path);
const data = sourceFile.readSync();
if (isIOS) {
return data.base64EncodedStringWithOptions(0);
} else {
return android.util.Base64.encodeToString(
data,
android.util.Base64.NO_WRAP
);
}
};
I have a simple saga of this form:
const getAccountDetails = function * () {
const { url } = yield select(state => state.appConfig)
const accountDetails = yield call(apiFetchAccountDetails, url)
}
I'm trying to write a unit test:
describe('getAccountDetails', () => {
const iterator = getAccountDetails()
it("should yield an Effect 'select(state=> state.appConfig)'", () => {
const effect = iterator.next().value
const expected = select(state => state.appConfig)
expect(effect).to.deep.eql(expected)
})
This test fails. Although effect and expected are very similar, they are not identical.
At least one of the differences is buried in payload.selector.scopes, where the yielded effect and expected are as follows:
As the scopes of these two will always be different, how can these tests ever be made to work?
eta: this pattern is adapted from the example linked to from the redux-saga docs
Cracked it after finding this issue from way back.
The fix is to create a named function to do the select and export it from the module where the saga under test lives, and then use this same function in tests. All is well.
export const selectAppConfig = state => state.appConfig
const getAccountDetails = function * () {
const { url } = yield select(selectAppConfig)
const accountDetails = yield call(apiFetchAccountDetails, url)
}
import {selectAppConfig} from './sagaToTest'
describe('getAccountDetails', () => {
const iterator = getAccountDetails()
it("should yield an Effect 'select(state=> state.appConfig)'", () => {
const effect = iterator.next().value
const expected = select(selectAppConfig)
expect(effect).to.deep.eql(expected)
})
Programmatically, my code is detecting a difference between two classes of images, and always rejecting one class, while always allowing the other.
I have yet to find any difference between the images that yield the error and the ones that don't an yield error. But there has to be some difference, because the ones that yield an error do so 100% of the time, and the others work as expected 100% of the time.
In particular, I have inspected color format: RGB in both groups; size: no notable difference; datatype: uint8 in both; magnitude of pixel values: similar in both.
Below are two images that never work, followed by two images that always work:
This image never works: https://www.colourbox.com/preview/11906131-maple-tree-and-grass-silhouette.jpg
This image never works: http://feldmanphoto.com/wp-content/uploads/awe-inspiring-house-clipart-black-and-white-disney-coloring-pages-big-clipartxtras-illistration-background-housewives-bouncy.jpeg
This image always works: http://www.spacedesign.us/wp-content/uploads/landscape-with-old-tree-and-grass-over-white-background-black-and-black-and-white-trees.jpg
This image always works: http://www.modernhouse.co/wp-content/uploads/2017/07/1024px-RoseSeidlerHouseSulmanPrize.jpg
How can I spot the difference?
The scenario is that I am using Firebase with Swift iOS front end to send these images to a Google Cloud ML-engine hosted convnet. Some images work all the time and certain others never work as above. Further, all images work when I use the gcloud versions predict CLI. To me the issue is necessarily something in the images. Hence I am posting here for the imaging group. Code is included as requested for completeness.
CODE of index.js file is included:
'use strict';
const functions = require('firebase-functions');
const gcs = require('#google-cloud/storage');
const admin = require('firebase-admin');
const exec = require('child_process').exec;
const path = require('path');
const fs = require('fs');
const google = require('googleapis');
const sizeOf = require('image-size');
admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
const rtdb = admin.database();
const dbRef = rtdb.ref();
function cmlePredict(b64img) {
return new Promise((resolve, reject) => {
google.auth.getApplicationDefault(function (err, authClient) {
if (err) {
reject(err);
}
if (authClient.createScopedRequired && authClient.createScopedRequired()) {
authClient = authClient.createScoped([
'https://www.googleapis.com/auth/cloud-platform'
]);
}
var ml = google.ml({
version: 'v1'
});
const params = {
auth: authClient,
name: 'projects/myproject-18865/models/my_model',
resource: {
instances: [
{
"image_bytes": {
"b64": b64img
}
}
]
}
};
ml.projects.predict(params, (err, result) => {
if (err) {
reject(err);
} else {
resolve(result);
}
});
});
});
}
function resizeImg(filepath) {
return new Promise((resolve, reject) => {
exec(`convert ${filepath} -resize 224x ${filepath}`, (err) => {
if (err) {
console.error('Failed to resize image', err);
reject(err);
} else {
console.log('resized image successfully');
resolve(filepath);
}
});
});
}
exports.runPrediction = functions.storage.object().onChange((event) => {
fs.rmdir('./tmp/', (err) => {
if (err) {
console.log('error deleting tmp/ dir');
}
});
const object = event.data;
const fileBucket = object.bucket;
const filePath = object.name;
const bucket = gcs().bucket(fileBucket);
const fileName = path.basename(filePath);
const file = bucket.file(filePath);
if (filePath.startsWith('images/')) {
const destination = '/tmp/' + fileName;
console.log('got a new image', filePath);
return file.download({
destination: destination
}).then(() => {
if(sizeOf(destination).width > 224) {
console.log('scaling image down...');
return resizeImg(destination);
} else {
return destination;
}
}).then(() => {
console.log('base64 encoding image...');
let bitmap = fs.readFileSync(destination);
return new Buffer(bitmap).toString('base64');
}).then((b64string) => {
console.log('sending image to CMLE...');
return cmlePredict(b64string);
}).then((result) => {
console.log(`results just returned and is: ${result}`);
let predict_proba = result.predictions[0]
const res_pred_val = Object.keys(predict_proba).map(k => predict_proba[k])
const res_val = Object.keys(result).map(k => result[k])
const class_proba = [1-res_pred_val,res_pred_val]
const opera_proba_init = 1-res_pred_val
const capitol_proba_init = res_pred_val-0
// convert fraction double to percentage int
let opera_proba = (Math.floor((opera_proba_init.toFixed(2))*100))|0
let capitol_proba = (Math.floor((capitol_proba_init.toFixed(2))*100))|0
let feature_list = ["houses", "trees"]
let outlinedImgPath = '';
let imageRef = db.collection('predicted_images').doc(filePath.slice(7));
outlinedImgPath = `outlined_img/${filePath.slice(7)}`;
imageRef.set({
image_path: outlinedImgPath,
opera_proba: opera_proba,
capitol_proba: capitol_proba
});
let predRef = dbRef.child("prediction_categories");
let arrayRef = dbRef.child("prediction_array");
predRef.set({
opera_proba: opera_proba,
capitol_proba: capitol_proba,
});
arrayRef.set({first: {
array_proba: [opera_proba,capitol_proba],
brief_description: ["a","b"],
more_details: ["aaaa","bbbb"],
feature_list: feature_list},
zummy1: "",
zummy2: ""});
return bucket.upload(destination, {destination: outlinedImgPath});
});
} else {
return 'not a new image';
}
});
Issue was that the bad images were grayscale, not RGB as expected by my model. I initially had checked this first by looking at the shape. But the 'bad' images had 3 color channels, each of those 3 channels stored the same number --- so my model was refusing to accept them. Also, as expected and contrary to what I initially thought I observed, turns out the gcloud ML-engine predict CLI actually also failed for these images. Took me 2 days to figure this out!
I am trying to make two ajax call one after the other , i.e with the result of the first call data , i am making the second call. I am trying to use thunk , but it is not happening , i am getting errors.
actions.js
const fetchedStateNameSucess = (json) => {
return {
type: 'FETCH_STATE_NAME_SUCCESS',
json
}
}
const fetchProvidersDetailsSuccess = (providersData) => {
return {
type: 'FETCH_PROVIDER_DETAILS_SUCCESS',
providersData
}
}
export const fetchProvidersDetails = (providerId) => {
return (dispatch) => {
const providerUrl = `http://someUrl`;
const stateClientCountUrl = `http://someUrl/${state}`;
fetch(providerUrl)
.then(response => response.json())
.then(json => dispatch(fetchProvidersDetailsSuccess(json)))
.then((stateValue = json[0].stateVal)
fetch(stateClientCountUrl)
dispatch(fetchedStateNameSucess(response)));
};
}
In the above call , fetch(providerUrl) , i am getting response where i am getting the stateval , how to use that in making the second call to fetch(stateClientCountUrl) which takes stateval as a parameter.
As Miguel said you can do your second query in .then() clause as well as you can use async/await syntax, something like this:
export const fetchProvidersDetails = providerId => {
return async dispatch => {
const providerUrl = `http://someUrl`;
try {
const response = await fetch(providerUrl);
const json = await response.json();
dispatch(fetchProvidersDetailsSuccess(json))
const stateClientCountUrl = `http://someUrl/${json[0].stateVal}`;
const response2 = await fetch(stateClientCountUrl);
const json2 = await response2.json();
dispatch(fetchedStateNameSucess(json2));
} catch (error) {
console.log('Error', error);
}
}
If you want to use values from the first call response for the second fetch call, you need to do the second fetch after the first one has succeeded, more or less like this:
export const fetchProvidersDetails = (providerId) => {
return (dispatch) => {
const providerUrl = `http://someUrl`;
const stateClientCountUrl = `http://someUrl/${state}`;
fetch(providerUrl)
.then(response => response.json())
.then(json => {
dispatch(fetchProvidersDetailsSuccess(json));
const stateValue = json[0].stateVal;
fetch(stateClientCountUrl)
.then(response => dispatch(fetchProvidersDetailsSuccess(response)));
})
}
Don't forget to add error handling there, both for HTTP status codes and for JavaScript errors (by adding corresponding catch clauses).
I have a Koa v2 app with koa-views#next as a renderer and nunjucks templating engine. Here is my working setup, which don't have any problem, I just confused with the double declaration of the views folder:
const Koa = require('koa');
const nunjucks = require('nunjucks');
const path = require('path');
const router = require('koa-router')();
const views = require('koa-views');
const app = new Koa();
const index = require('./routes/index');
app.use(views(path.join(__dirname, 'views'), {
extension: 'njk',
map: { njk: 'nunjucks' },
}));
nunjucks.configure(path.join(__dirname, 'views'), {
autoescape: true,
});
router.use('/', index.routes(), index.allowedMethods());
app
.use(router.routes())
.use(router.allowedMethods());
app.listen(3000);
But if I don't do this, the rendering doesn't work. If I uncommenting the nunjucks.configure block, I'm getting the following error:
Template render error: (unknown path)
Error: template not found: layout.njk
Is there any problem with my setup?
I come up a solution to use nunjucks without any other renderer library in koa v2:
/*
USAGE:
import njk from './nunjucks';
// Templating - Must be used before any router
app.use(njk(path.join(__dirname, 'views'), {
extname: '.njk',
noCache: process.env.NODE_ENV !== 'production',
throwOnUndefined: true,
filters: {
json: function (str) {
return JSON.stringify(str, null, 2);
},
upperCase: str => str.toUpperCase(),
},
globals: {
version: 'v3.0.0',
},
}));
*/
// Inspired by:
// https://github.com/ohomer/koa-nunjucks-render/blob/master/index.js
// https://github.com/beliefgp/koa-nunjucks-next/blob/master/index.js
const Promise = require('bluebird');
const nunjucks = require('nunjucks');
function njk(path, opts) {
const env = nunjucks.configure(path, opts);
const extname = opts.extname || '';
const filters = opts.filters || {};
//console.time('benchmark');
const f = Object.keys(filters).length;
let i = 0;
while (i < f) {
env.addFilter(Object.keys(filters)[i], Object.values(filters)[i]);
i += 1;
}
//console.timeEnd('benchmark');
const globals = opts.globals || {};
const g = Object.keys(globals).length;
let j = 0;
while (j < g) {
env.addFilter(Object.keys(globals)[j], Object.values(globals)[j]);
j += 1;
}
return (ctx, next) => {
ctx.render = (view, context = {}) => {
context = Object.assign({}, ctx.state, context);
return new Promise((resolve, reject) => {
env.render(`${view}${extname}`, context, (err, res) => {
if (err) {
return reject(err);
}
ctx.body = res;
return resolve();
});
});
};
return next();
};
}
module.exports = njk;
Gist