I am doing a visual regression :
code: plugins/indexts
on('task', {
graphicsMatch: ({ img1Path, img2Path, diffImgPath }) => {
const img1 = PNG.sync.read(fs.readFileSync(img1Path));
const img2 = PNG.sync.read(fs.readFileSync(img2Path));
const { width, height } = img1;
const diff = new PNG({ width, height });
let pixelCount = pixelmatch(img1.data, img2.data, diff.data);
fs.writeFileSync(diffImgPath, PNG.sync.write(diff));
return pixelCount;
},
});
stepdef:
And('I capture the screenshot of {string}', (editorDocument: string) => {
cy.wait(5000);
cy.captureScreen(editorDocument).as('generatedFileName');
});
Then(
'the generated pdf is as expected {string} and {string}',
(nameOfFeatureFile: string, editorScreenshot: string) => {
filesApi.comparePixelDiff(
nameOfFeatureFile,
editorScreenshot,
Cypress.currentTest.title,
);
},
);
API's:
comparePixelDiff(
nameOfFeaturefile: string,
actualFileName: string,
expectedFileName: string,
) {
cy.get('#generatedFileName').then((receivedFileName) => {
const sourceFilePath = `${FilesApi.screesnShotsFolder}${FilesApi.visualTestingFeaturesFolder}/${nameOfFeaturefile}/${actualFileName}${FilesApi.graphicExt}`;
const expectedFilePath = `${FilesApi.expectedExportsFolder}${expectedFileName}${FilesApi.graphicExt}`;
const diffFilePath = `${FilesApi.actualExportsFolder}${actualFileName}-${FilesApi.graphicsDiffSufix}${FilesApi.graphicExt}`;
cy.task('graphicsMatch', {
img1Path: sourceFilePath,
img2Path: expectedFilePath,
diffImgPath: diffFilePath,
}).then((pixelsDiff) => {});
});
}
}
i am using pixelmatch here,i have used viewport as 1280x720 in json file locally it works fine but fails on CI as the resolution of screenshot is not persistent
i have even tried cy.viewport(1280, 720) before capturescreenshot it didnt work as well.
How do i fix this issue, please help.
Related
Problem: Whenever I click 't' key my theme changed as I expected. But for some reason this cause isModalVisible state reset. This cause one problem - When I have my modal open and click t - the modal disapear. I don't know why. Maybe BehaviorSubject cause the problem? Additionally, if I create another useState for test, which initial value is string 'AAA', and on open modal I set this state to string 'BBB'. Then I click 't' to change theme this test useState back to beginning value 'AAA'
My code looks like this:
const Component1 = () => {
const [mode, setMode] = useRecoilState(selectedModeState);
const { switcher, status } = useThemeSwitcher();
const toggleMode = (newTheme: MODE_TYPE) => {
theme$.next({ theme: newTheme });
localStorage.setItem('theme', newTheme);
};
useEffect(() => {
const mode_sub = theme$.subscribe(({ theme }) => {
switcher({ theme: theme });
setMode(theme);
});
return () => {
mode_sub.unsubscribe();
};
}, []);
return <Component2 toggleMode={toggleMode} currentMode={mode} />
}
const Component2 = ({toggleMode, currentMode}) => {
const [mode, setMode] = useState(currentMode);
const [savedTheme, setSavedTheme] = useRecoilState(selectedThemeState);
const getTheme = mode === MODE_TYPE.LIGHT ? MODE_TYPE.DARK : MODE_TYPE.LIGHT;
const themeListener = (event: KeyboardEvent) => {
switch (event.key) {
case 't':
setMode(getTheme);
toggleMode(getTheme);
break;
}
};
useEffect(() => {
document.addEventListener('keydown', themeListener);
document.body.setAttribute('data-theme', savedTheme);
localStorage.setItem('color', savedTheme);
return () => {
document.removeEventListener('keydown', themeListener);
};
}, [savedTheme]);
const [isModalVisible, setIsModalVisible] = useState<boolean>(false);
return (
<Modal isModalVisible={isModalVisible} setIsModalVisible={setIsModalVisible} />
)
}
UTILS:
export const selectedModeState = atom<MODE_TYPE>({
key: 'selectedThemeState',
default: (localStorage.getItem('theme') as MODE_TYPE) || MODE_TYPE.LIGHT,
});
export const selectedThemeState = atom<string>({
key: 'selectedColorState',
default: (localStorage.getItem('color') as string) || 'blue',
});
export const theme$ = new BehaviorSubject({
theme: (localStorage.getItem('theme') as THEME_TYPE) || THEME_TYPE.LIGHT,
});
I would like the theme change not to set visibleModal to false which causes the modal to close
I'm using quill editor as rich text editor component and in development everything works fine but as soon as I launch my web app in a production environment, the component throws following error:
TypeError: Cannot read properties of undefined (reading 'className')
at new t (cms-editor.58b2a676.js:1:4281)
at T.value (main.5b8d6e17.js:809:22276)
at T.value (main.5b8d6e17.js:810:2735)
at main.5b8d6e17.js:809:22151
at Array.forEach (<anonymous>)
at T.value (main.5b8d6e17.js:809:22109)
at new Y (main.5b8d6e17.js:788:5408)
at o (main.5b8d6e17.js:829:1661)
at main.5b8d6e17.js:829:1411
at Kt (main.5b8d6e17.js:4:656)
Gv # main.5b8d6e17.js:4
or # main.5b8d6e17.js:4
Kt # main.5b8d6e17.js:4
wt # main.5b8d6e17.js:4
t.__weh.t.__weh # main.5b8d6e17.js:4
Do # main.5b8d6e17.js:4
te # main.5b8d6e17.js:4
mount # main.5b8d6e17.js:4
t.mount # main.5b8d6e17.js:4
setup # main.5b8d6e17.js:831
(anonymous) # main.5b8d6e17.js:12
Promise.then (async)
zh # main.5b8d6e17.js:12
(anonymous) # main.5b8d6e17.js:831
The top log message suggests that cms-editor seems to be the origin of the error:
<template>
<div id="cms-editor" class="cms-editor">
<quill-editor ref="quill" :modules="modules" :toolbar="toolbar" v-model:content="content" contentType="html"/>
</div>
</template>
<script setup>
import BlotFormatter from 'quill-blot-formatter'
import {ref, watchEffect} from 'vue'
import {Quill} from "#vueup/vue-quill";
const props = defineProps({
body: String,
triggerEmit: Boolean
})
const content = ref(props.body || '')
const emit = defineEmits(['emitBody'])
watchEffect(async () => {
if (props.triggerEmit) {
const body = await compressHtml(content.value)
emit('emitBody', body)
}
})
const quill = ref(null)
Quill.debug('error')
const compressImage = (dataUrl, width, mime, resize) => {
return new Promise((resolve) => {
const img = new Image();
img.src = dataUrl
img.onload = () => {
const height = Math.round(img.height / img.width * width)
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
canvas.width = width;
canvas.height = height;
ctx.drawImage(img, 0, 0, width, height);
resolve(canvas.toDataURL(mime, resize));
}
})
}
const compressHtml = async (content) => {
let body = content.split('<')
let count = 1
for (const id in body) {
count = count + 1
let el = body[id]
if (el.substr(0, 3) == 'img') {
const dataUrl = el.split('"')[1]
const src = el.split('"')[1].split(',')[1]
const mime = el.split('data:')[1].split(';')[0]
const size = atob(src).length;
if (size >= 250000) {
let img_el = await compressImage(dataUrl, 600, mime, .9)
.then(res => {
return 'img src="' + res + '">';
})
body[id] = img_el
}
}
}
return body.join('<')
}
const toolbar = [
[{header: [1, 2, 3, false]}],
[{size: ['small', false, 'large', 'huge']}],
['bold', 'italic', 'underline', 'strike'],
['blockquote', 'code-block'],
[{align: []}],
[{list: 'ordered'}, {list: 'bullet'}],
[{color: []}, {background: []}],
['link', 'image'],
['clean'],
]
const modules = {
module: BlotFormatter,
}
var BaseImageFormat = Quill.import('formats/image');
const ImageFormatAttributesList = [
'alt',
'height',
'width',
'style'
];
//make quill image alignment work
class ImageFormat extends BaseImageFormat {
static formats(domNode) {
return ImageFormatAttributesList.reduce(function (formats, attribute) {
if (domNode.hasAttribute(attribute)) {
formats[attribute] = domNode.getAttribute(attribute);
}
return formats;
}, {});
}
format(name, value) {
if (ImageFormatAttributesList.indexOf(name) > -1) {
if (value) {
this.domNode.setAttribute(name, value);
} else {
this.domNode.removeAttribute(name);
}
} else {
super.format(name, value);
}
}
}
Quill.register(ImageFormat, true);
</script>
The error is thrown instantly, when the component is loaded. The props do not seem to play a role because I tried commenting them out and the error still gets thrown. Any idea how I could debug this?
I'm using inertia.js if that is relevant.
Your blot formatter might be configured in a wrong way. I didn't really use this feature in my App, so I disabled / removed it and the production error disappears.
See issue here:
https://github.com/vueup/vue-quill/issues/315
I have been trying to get the base64 of a File of Nativescript, so far I have tried the next:
const documents: Folder = <Folder>knownFolders.documents();
const folder: Folder = <Folder>documents.getFolder("Download");
const file: File = <File>folder.getFile("Vista general Aguas.pdf");
const toBase64 = (fle: File) => new Promise((resolve, reject) => {
const reader = new FileReader();
const newFile = fle.readSync(err => console.log(err));
// I know that I have to convert the binary to base64, but I'm getting this [B#9b1d878 in newFile
reader.readAsDataURL(newFile);
reader.onload = () => resolve(reader.result);
reader.onerror = error => reject(error);
});
toBase64(file).then(result => console.log('64', result));
Any help is appreciated!
I have created function using native implementation. This function return base64 string.
public getBase64String = (path) => {
const sourceFile: fs.File = fs.File.fromPath(path);
const data = sourceFile.readSync();
if (isIOS) {
return data.base64EncodedStringWithOptions(0);
} else {
return android.util.Base64.encodeToString(
data,
android.util.Base64.NO_WRAP
);
}
};
I am new in WebRTC and i have done client/server connection, from client i choose WebCam and post stream to server using Track and on Server side i am getting that track and assign track stream to video source. Everything till now fine but problem is now i include AI(Artificial Intelligence) and now i want to convert my track stream to URL maybe UDP/RTSP/RTP etc. So AI will use that URL for object detection. I don't know how we can convert track stream to URL.
Although there is a couple of packages like https://ffmpeg.org/ and RTP to Webrtc etc, i am using Nodejs, Socket.io and Webrtc, below you can check my client and server side code for getting and posting stream, i am following thi github code https://github.com/Basscord/webrtc-video-broadcast.
Now my main concern is to make track as a URL for video tag, is it possible or not or please suggest, any help would be appreciated.
Server.js
This is nodejs server code
const express = require("express");
const app = express();
let broadcaster;
const port = 4000;
const http = require("http");
const server = http.createServer(app);
const io = require("socket.io")(server);
app.use(express.static(__dirname + "/public"));
io.sockets.on("error", e => console.log(e));
io.sockets.on("connection", socket => {
socket.on("broadcaster", () => {
broadcaster = socket.id;
socket.broadcast.emit("broadcaster");
});
socket.on("watcher", () => {
socket.to(broadcaster).emit("watcher", socket.id);
});
socket.on("offer", (id, message) => {
socket.to(id).emit("offer", socket.id, message);
});
socket.on("answer", (id, message) => {
socket.to(id).emit("answer", socket.id, message);
});
socket.on("candidate", (id, message) => {
socket.to(id).emit("candidate", socket.id, message);
});
socket.on("disconnect", () => {
socket.to(broadcaster).emit("disconnectPeer", socket.id);
});
});
server.listen(port, () => console.log(`Server is running on port ${port}`));
Broadcast.js
This is the code for emit stream(track)
const peerConnections = {};
const config = {
iceServers: [
{
urls: ["stun:stun.l.google.com:19302"]
}
]
};
const socket = io.connect(window.location.origin);
socket.on("answer", (id, description) => {
peerConnections[id].setRemoteDescription(description);
});
socket.on("watcher", id => {
const peerConnection = new RTCPeerConnection(config);
peerConnections[id] = peerConnection;
let stream = videoElement.srcObject;
stream.getTracks().forEach(track => peerConnection.addTrack(track, stream));
peerConnection.onicecandidate = event => {
if (event.candidate) {
socket.emit("candidate", id, event.candidate);
}
};
peerConnection
.createOffer()
.then(sdp => peerConnection.setLocalDescription(sdp))
.then(() => {
socket.emit("offer", id, peerConnection.localDescription);
});
});
socket.on("candidate", (id, candidate) => {
peerConnections[id].addIceCandidate(new RTCIceCandidate(candidate));
});
socket.on("disconnectPeer", id => {
peerConnections[id].close();
delete peerConnections[id];
});
window.onunload = window.onbeforeunload = () => {
socket.close();
};
// Get camera and microphone
const videoElement = document.querySelector("video");
const audioSelect = document.querySelector("select#audioSource");
const videoSelect = document.querySelector("select#videoSource");
audioSelect.onchange = getStream;
videoSelect.onchange = getStream;
getStream()
.then(getDevices)
.then(gotDevices);
function getDevices() {
return navigator.mediaDevices.enumerateDevices();
}
function gotDevices(deviceInfos) {
window.deviceInfos = deviceInfos;
for (const deviceInfo of deviceInfos) {
const option = document.createElement("option");
option.value = deviceInfo.deviceId;
if (deviceInfo.kind === "audioinput") {
option.text = deviceInfo.label || `Microphone ${audioSelect.length + 1}`;
audioSelect.appendChild(option);
} else if (deviceInfo.kind === "videoinput") {
option.text = deviceInfo.label || `Camera ${videoSelect.length + 1}`;
videoSelect.appendChild(option);
}
}
}
function getStream() {
if (window.stream) {
window.stream.getTracks().forEach(track => {
track.stop();
});
}
const audioSource = audioSelect.value;
const videoSource = videoSelect.value;
const constraints = {
audio: { deviceId: audioSource ? { exact: audioSource } : undefined },
video: { deviceId: videoSource ? { exact: videoSource } : undefined }
};
return navigator.mediaDevices
.getUserMedia(constraints)
.then(gotStream)
.catch(handleError);
}
function gotStream(stream) {
window.stream = stream;
audioSelect.selectedIndex = [...audioSelect.options].findIndex(
option => option.text === stream.getAudioTracks()[0].label
);
videoSelect.selectedIndex = [...videoSelect.options].findIndex(
option => option.text === stream.getVideoTracks()[0].label
);
videoElement.srcObject = stream;
socket.emit("broadcaster");
}
function handleError(error) {
console.error("Error: ", error);
}
RemoteServer.js
This code is getting track and assign to video tag
let peerConnection;
const config = {
iceServers: [
{
urls: ["stun:stun.l.google.com:19302"]
}
]
};
const socket = io.connect(window.location.origin);
const video = document.querySelector("video");
socket.on("offer", (id, description) => {
peerConnection = new RTCPeerConnection(config);
peerConnection
.setRemoteDescription(description)
.then(() => peerConnection.createAnswer())
.then(sdp => peerConnection.setLocalDescription(sdp))
.then(() => {
socket.emit("answer", id, peerConnection.localDescription);
});
peerConnection.ontrack = event => {
video.srcObject = event.streams[0];
};
peerConnection.onicecandidate = event => {
if (event.candidate) {
socket.emit("candidate", id, event.candidate);
}
};
});
socket.on("candidate", (id, candidate) => {
peerConnection
.addIceCandidate(new RTCIceCandidate(candidate))
.catch(e => console.error(e));
});
socket.on("connect", () => {
socket.emit("watcher");
});
socket.on("broadcaster", () => {
socket.emit("watcher");
});
socket.on("disconnectPeer", () => {
peerConnection.close();
});
window.onunload = window.onbeforeunload = () => {
socket.close();
};
rtp-to-webrtc does exactly what you want.
Unfortunately you will need to run some sort of server to make this happen, it can’t all be in the browser. You could also upload via other protocols (captured via MediaRecorder) if you don’t want to use WebRTC.
Programmatically, my code is detecting a difference between two classes of images, and always rejecting one class, while always allowing the other.
I have yet to find any difference between the images that yield the error and the ones that don't an yield error. But there has to be some difference, because the ones that yield an error do so 100% of the time, and the others work as expected 100% of the time.
In particular, I have inspected color format: RGB in both groups; size: no notable difference; datatype: uint8 in both; magnitude of pixel values: similar in both.
Below are two images that never work, followed by two images that always work:
This image never works: https://www.colourbox.com/preview/11906131-maple-tree-and-grass-silhouette.jpg
This image never works: http://feldmanphoto.com/wp-content/uploads/awe-inspiring-house-clipart-black-and-white-disney-coloring-pages-big-clipartxtras-illistration-background-housewives-bouncy.jpeg
This image always works: http://www.spacedesign.us/wp-content/uploads/landscape-with-old-tree-and-grass-over-white-background-black-and-black-and-white-trees.jpg
This image always works: http://www.modernhouse.co/wp-content/uploads/2017/07/1024px-RoseSeidlerHouseSulmanPrize.jpg
How can I spot the difference?
The scenario is that I am using Firebase with Swift iOS front end to send these images to a Google Cloud ML-engine hosted convnet. Some images work all the time and certain others never work as above. Further, all images work when I use the gcloud versions predict CLI. To me the issue is necessarily something in the images. Hence I am posting here for the imaging group. Code is included as requested for completeness.
CODE of index.js file is included:
'use strict';
const functions = require('firebase-functions');
const gcs = require('#google-cloud/storage');
const admin = require('firebase-admin');
const exec = require('child_process').exec;
const path = require('path');
const fs = require('fs');
const google = require('googleapis');
const sizeOf = require('image-size');
admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
const rtdb = admin.database();
const dbRef = rtdb.ref();
function cmlePredict(b64img) {
return new Promise((resolve, reject) => {
google.auth.getApplicationDefault(function (err, authClient) {
if (err) {
reject(err);
}
if (authClient.createScopedRequired && authClient.createScopedRequired()) {
authClient = authClient.createScoped([
'https://www.googleapis.com/auth/cloud-platform'
]);
}
var ml = google.ml({
version: 'v1'
});
const params = {
auth: authClient,
name: 'projects/myproject-18865/models/my_model',
resource: {
instances: [
{
"image_bytes": {
"b64": b64img
}
}
]
}
};
ml.projects.predict(params, (err, result) => {
if (err) {
reject(err);
} else {
resolve(result);
}
});
});
});
}
function resizeImg(filepath) {
return new Promise((resolve, reject) => {
exec(`convert ${filepath} -resize 224x ${filepath}`, (err) => {
if (err) {
console.error('Failed to resize image', err);
reject(err);
} else {
console.log('resized image successfully');
resolve(filepath);
}
});
});
}
exports.runPrediction = functions.storage.object().onChange((event) => {
fs.rmdir('./tmp/', (err) => {
if (err) {
console.log('error deleting tmp/ dir');
}
});
const object = event.data;
const fileBucket = object.bucket;
const filePath = object.name;
const bucket = gcs().bucket(fileBucket);
const fileName = path.basename(filePath);
const file = bucket.file(filePath);
if (filePath.startsWith('images/')) {
const destination = '/tmp/' + fileName;
console.log('got a new image', filePath);
return file.download({
destination: destination
}).then(() => {
if(sizeOf(destination).width > 224) {
console.log('scaling image down...');
return resizeImg(destination);
} else {
return destination;
}
}).then(() => {
console.log('base64 encoding image...');
let bitmap = fs.readFileSync(destination);
return new Buffer(bitmap).toString('base64');
}).then((b64string) => {
console.log('sending image to CMLE...');
return cmlePredict(b64string);
}).then((result) => {
console.log(`results just returned and is: ${result}`);
let predict_proba = result.predictions[0]
const res_pred_val = Object.keys(predict_proba).map(k => predict_proba[k])
const res_val = Object.keys(result).map(k => result[k])
const class_proba = [1-res_pred_val,res_pred_val]
const opera_proba_init = 1-res_pred_val
const capitol_proba_init = res_pred_val-0
// convert fraction double to percentage int
let opera_proba = (Math.floor((opera_proba_init.toFixed(2))*100))|0
let capitol_proba = (Math.floor((capitol_proba_init.toFixed(2))*100))|0
let feature_list = ["houses", "trees"]
let outlinedImgPath = '';
let imageRef = db.collection('predicted_images').doc(filePath.slice(7));
outlinedImgPath = `outlined_img/${filePath.slice(7)}`;
imageRef.set({
image_path: outlinedImgPath,
opera_proba: opera_proba,
capitol_proba: capitol_proba
});
let predRef = dbRef.child("prediction_categories");
let arrayRef = dbRef.child("prediction_array");
predRef.set({
opera_proba: opera_proba,
capitol_proba: capitol_proba,
});
arrayRef.set({first: {
array_proba: [opera_proba,capitol_proba],
brief_description: ["a","b"],
more_details: ["aaaa","bbbb"],
feature_list: feature_list},
zummy1: "",
zummy2: ""});
return bucket.upload(destination, {destination: outlinedImgPath});
});
} else {
return 'not a new image';
}
});
Issue was that the bad images were grayscale, not RGB as expected by my model. I initially had checked this first by looking at the shape. But the 'bad' images had 3 color channels, each of those 3 channels stored the same number --- so my model was refusing to accept them. Also, as expected and contrary to what I initially thought I observed, turns out the gcloud ML-engine predict CLI actually also failed for these images. Took me 2 days to figure this out!