I'm just going crazy on this, can some help me please.
Perhaps someone has already solved this properly?
So I have an app with multiple datasources and I want them to autoupdate properly.
Yet I can't believe that I have to do this... Shouldn't it be like a commun thing ?
I put this script in server/boot/20-autoUpdateDb.js
'use strict';
var cluster = require('cluster');
module.exports = function(app, cb) {
if (cluster.isWorker) {
process.nextTick(cb);
} else {
const updateDS = async dataSources => {
await dataSources.forEach(async (ds, key, arr) => {
if (ds.autoupdate) {
console.log('Proceeding with dataSource:', ds.name);
const x = await new Promise(r => {
ds.autoupdate(err => {
if (err) throw err;
console.log(
// 'Auto Updated Models [' + Object.keys(app.models) + ']:',
'Auto Updated Models in',
ds.name,
'(',
ds.adapter.name,
')',
);
r();
});
});
}
});
};
// I just couldn't find a better way as all datasources
// exist in double ex: db,Db
var appDataSources = [
app.dataSources.db,
app.dataSources.dbData,
app.dataSources.dbPool,
];
updateDS(appDataSources).then(() => {
console.log('Autoupdate of all models done.');
process.nextTick(cb);
});
}
};
as output I get
Proceeding with dataSource: db
Proceeding with dataSource: dbData
Proceeding with dataSource: dbPool
Auto Updated Models in dbPool ( sqlite3 )
Autoupdate of all models done.
Web server listening at: http://localhost:3000
Browse your REST API at http://localhost:3000/explorer
Auto Updated Models in dbData ( sqlite3 )
Auto Updated Models in db ( sqlite3 )
Array.prototype.forEach doesn't wait for promises. Try this:
const appDataSources = [
app.dataSources.db,
app.dataSources.dbData,
app.dataSources.dbPool,
]
(async () => {
for (let i = 0; i < appDataSources.length; i++) {
const ds = appDataSources[ i ]
if (ds.autoupdate) {
await new Promise((resolve, reject) => {
ds.autoupdate(err => {
if (err) return reject(err)
resolve()
})
})
}
}
})()
Related
I'm new to service workers and I'm running into an issue with my implementation. My goal is to create a runtime cache for images and videos. I've looked at the workbox implementation but it hasn't worked for me. I see that my service worker successfully registers at the top-level scope of my app but for some reason, it seems like some of the code in my service worker file doesn't get executed. The main issue is that the event listeners from my service worker don't seem to get called (including registerRoute), and therefore, the Cache doesn't ever get created.
I'm not sure if this is related to the issue I'm having but when I look at the console messages, it seems like the code from sw.js may be run prior to the service worker registration:
console messages
I've been stuck on this problem for a while so I would appreciate some help if anyone has run into this issue, thanks!
// main.js (in a Vue 2 app)
if (process.env.NODE_ENV === "production") {
window.addEventListener("load", () => {
if ("serviceWorker" in navigator) {
navigator.serviceWorker
.register(`/sw.js`)
.then(() => {
console.log("Service worker registered!");
navigator.serviceWorker.ready.then((registration) => {
registration.update();
console.log('Service Worker: ready');
});
})
.catch((error) => {
console.warn("Error registering service worker:");
console.warn(error);
});
}
});
}
// sw.js
import { registerRoute } from "workbox-routing";
import { CacheFirst } from "workbox-strategies";
import { CacheableResponsePlugin } from "workbox-cacheable-response";
import { RangeRequestsPlugin } from "workbox-range-requests";
import { clientsClaim } from "workbox-core";
const CACHE_PREFIX = "background-slideshow-cache";
const CACHE_VERSION = "v1";
const CACHE_RUNTIME = "runtime";
const BACKGROUND_SLIDESHOW_CACHE = `${CACHE_PREFIX}-${CACHE_RUNTIME}-${CACHE_VERSION}`;
clientsClaim();
const addToCache = async (url) => {
const cache = await caches.open(BACKGROUND_SLIDESHOW_CACHE);
if (!(await cache.match(url))) {
await cache.add(url);
}
};
const cacheFirstStrategy = new CacheFirst({
cacheName: BACKGROUND_SLIDESHOW_CACHE,
plugins: [
new CacheableResponsePlugin({
statuses: [200],
}),
new RangeRequestsPlugin(),
],
});
self.addEventListener("message", (event) => {
if (event.data && event.data.message) {
if (event.data.message === "SKIP_WAITING") {
self.skipWaiting();
}
}
});
self.addEventListener("install", (event) => {
console.log('Service worker: fetch event', event);
})
console.log("Service Worker: in file");
registerRoute(
({ request }) => {
const { destination } = request;
console.log("Service Worker:", "request", request);
return destination === "video" || destination === "image";
},
({ event, request }) => {
// console.log("Service Worker: in the 2nd param", event, request);
event.respondWith(async () => {
await addToCache(request.url);
return cacheFirstStrategy.handle({ request });
});
}
);
After many hours of debugging, I realized that the minification of sw.js at build time was the reason this code wasn't able to execute. I decided to use uglifyjs-webpack-plugin in my webpack config and this solved the issue!
Configured my store this way with redux toolkit for sure
const rootReducer = combineReducers({
someReducer,
systemsConfigs
});
const store = return configureStore({
devTools: true,
reducer: rootReducer ,
// middleware: [middleware, logger],
middleware: (getDefaultMiddleware) => getDefaultMiddleware({ thunk: false }).concat(middleware),
});
middleware.run(sagaRoot)
And thats my channel i am connecting to it
export function createSocketChannel(
productId: ProductId,
pair: string,
createSocket = () => new WebSocket('wss://somewebsocket')
) {
return eventChannel<SocketEvent>((emitter) => {
const socket_OrderBook = createSocket();
socket_OrderBook.addEventListener('open', () => {
emitter({
type: 'connection-established',
payload: true,
});
socket_OrderBook.send(
`subscribe-asdqwe`
);
});
socket_OrderBook.addEventListener('message', (event) => {
if (event.data?.includes('bids')) {
emitter({
type: 'message',
payload: JSON.parse(event.data),
});
//
}
});
socket_OrderBook.addEventListener('close', (event: any) => {
emitter(new SocketClosedByServer());
});
return () => {
if (socket_OrderBook.readyState === WebSocket.OPEN) {
socket_OrderBook.send(
`unsubscribe-order-book-${pair}`
);
}
if (socket_OrderBook.readyState === WebSocket.OPEN || socket_OrderBook.readyState === WebSocket.CONNECTING) {
socket_OrderBook.close();
}
};
}, buffers.expanding<SocketEvent>());
}
And here's how my saga connecting handlers looks like
export function* handleConnectingSocket(ctx: SagaContext) {
try {
const productId = yield select((state: State) => state.productId);
const requested_pair = yield select((state: State) => state.requested_pair);
if (ctx.socketChannel === null) {
ctx.socketChannel = yield call(createSocketChannel, productId, requested_pair);
}
//
const message: SocketEvent = yield take(ctx.socketChannel!);
if (message.type !== 'connection-established') {
throw new SocketUnexpectedResponseError();
}
yield put(connectedSocket());
} catch (error: any) {
reportError(error);
yield put(
disconnectedSocket({
reason: SocketStateReasons.BAD_CONNECTION,
})
);
}
}
export function* handleConnectedSocket(ctx: SagaContext) {
try {
while (true) {
if (ctx.socketChannel === null) {
break;
}
const events = yield flush(ctx.socketChannel);
const startedExecutingAt = performance.now();
if (Array.isArray(events)) {
const deltas = events.reduce(
(patch, event) => {
if (event.type === 'message') {
patch.bids.push(...event.payload.data?.bids);
patch.asks.push(...event.payload.data?.asks);
//
}
//
return patch;
},
{ bids: [], asks: [] } as SocketMessage
);
if (deltas.bids.length || deltas.asks.length) {
yield putResolve(receivedDeltas(deltas));
}
}
yield call(delayNextDispatch, startedExecutingAt);
}
} catch (error: any) {
reportError(error);
yield put(
disconnectedSocket({
reason: SocketStateReasons.UNKNOWN,
})
);
}
}
After Debugging I got the following:
The Thing is that when I Provide one Reducer to my store the channel works well and data is fetched where as when providing combinedReducers I am getting
an established connection from my handleConnectingSocket generator function
and an empty event array [] from
const events = yield flush(ctx.socketChannel) written in handleConnectedSocket
Tried to clarify as much as possible
ok so I start refactoring my typescript by changing the types, then saw all the places that break, there was a problem in my sagas.tsx.
Ping me if someone faced such an issue in the future
I am new in WebRTC and i have done client/server connection, from client i choose WebCam and post stream to server using Track and on Server side i am getting that track and assign track stream to video source. Everything till now fine but problem is now i include AI(Artificial Intelligence) and now i want to convert my track stream to URL maybe UDP/RTSP/RTP etc. So AI will use that URL for object detection. I don't know how we can convert track stream to URL.
Although there is a couple of packages like https://ffmpeg.org/ and RTP to Webrtc etc, i am using Nodejs, Socket.io and Webrtc, below you can check my client and server side code for getting and posting stream, i am following thi github code https://github.com/Basscord/webrtc-video-broadcast.
Now my main concern is to make track as a URL for video tag, is it possible or not or please suggest, any help would be appreciated.
Server.js
This is nodejs server code
const express = require("express");
const app = express();
let broadcaster;
const port = 4000;
const http = require("http");
const server = http.createServer(app);
const io = require("socket.io")(server);
app.use(express.static(__dirname + "/public"));
io.sockets.on("error", e => console.log(e));
io.sockets.on("connection", socket => {
socket.on("broadcaster", () => {
broadcaster = socket.id;
socket.broadcast.emit("broadcaster");
});
socket.on("watcher", () => {
socket.to(broadcaster).emit("watcher", socket.id);
});
socket.on("offer", (id, message) => {
socket.to(id).emit("offer", socket.id, message);
});
socket.on("answer", (id, message) => {
socket.to(id).emit("answer", socket.id, message);
});
socket.on("candidate", (id, message) => {
socket.to(id).emit("candidate", socket.id, message);
});
socket.on("disconnect", () => {
socket.to(broadcaster).emit("disconnectPeer", socket.id);
});
});
server.listen(port, () => console.log(`Server is running on port ${port}`));
Broadcast.js
This is the code for emit stream(track)
const peerConnections = {};
const config = {
iceServers: [
{
urls: ["stun:stun.l.google.com:19302"]
}
]
};
const socket = io.connect(window.location.origin);
socket.on("answer", (id, description) => {
peerConnections[id].setRemoteDescription(description);
});
socket.on("watcher", id => {
const peerConnection = new RTCPeerConnection(config);
peerConnections[id] = peerConnection;
let stream = videoElement.srcObject;
stream.getTracks().forEach(track => peerConnection.addTrack(track, stream));
peerConnection.onicecandidate = event => {
if (event.candidate) {
socket.emit("candidate", id, event.candidate);
}
};
peerConnection
.createOffer()
.then(sdp => peerConnection.setLocalDescription(sdp))
.then(() => {
socket.emit("offer", id, peerConnection.localDescription);
});
});
socket.on("candidate", (id, candidate) => {
peerConnections[id].addIceCandidate(new RTCIceCandidate(candidate));
});
socket.on("disconnectPeer", id => {
peerConnections[id].close();
delete peerConnections[id];
});
window.onunload = window.onbeforeunload = () => {
socket.close();
};
// Get camera and microphone
const videoElement = document.querySelector("video");
const audioSelect = document.querySelector("select#audioSource");
const videoSelect = document.querySelector("select#videoSource");
audioSelect.onchange = getStream;
videoSelect.onchange = getStream;
getStream()
.then(getDevices)
.then(gotDevices);
function getDevices() {
return navigator.mediaDevices.enumerateDevices();
}
function gotDevices(deviceInfos) {
window.deviceInfos = deviceInfos;
for (const deviceInfo of deviceInfos) {
const option = document.createElement("option");
option.value = deviceInfo.deviceId;
if (deviceInfo.kind === "audioinput") {
option.text = deviceInfo.label || `Microphone ${audioSelect.length + 1}`;
audioSelect.appendChild(option);
} else if (deviceInfo.kind === "videoinput") {
option.text = deviceInfo.label || `Camera ${videoSelect.length + 1}`;
videoSelect.appendChild(option);
}
}
}
function getStream() {
if (window.stream) {
window.stream.getTracks().forEach(track => {
track.stop();
});
}
const audioSource = audioSelect.value;
const videoSource = videoSelect.value;
const constraints = {
audio: { deviceId: audioSource ? { exact: audioSource } : undefined },
video: { deviceId: videoSource ? { exact: videoSource } : undefined }
};
return navigator.mediaDevices
.getUserMedia(constraints)
.then(gotStream)
.catch(handleError);
}
function gotStream(stream) {
window.stream = stream;
audioSelect.selectedIndex = [...audioSelect.options].findIndex(
option => option.text === stream.getAudioTracks()[0].label
);
videoSelect.selectedIndex = [...videoSelect.options].findIndex(
option => option.text === stream.getVideoTracks()[0].label
);
videoElement.srcObject = stream;
socket.emit("broadcaster");
}
function handleError(error) {
console.error("Error: ", error);
}
RemoteServer.js
This code is getting track and assign to video tag
let peerConnection;
const config = {
iceServers: [
{
urls: ["stun:stun.l.google.com:19302"]
}
]
};
const socket = io.connect(window.location.origin);
const video = document.querySelector("video");
socket.on("offer", (id, description) => {
peerConnection = new RTCPeerConnection(config);
peerConnection
.setRemoteDescription(description)
.then(() => peerConnection.createAnswer())
.then(sdp => peerConnection.setLocalDescription(sdp))
.then(() => {
socket.emit("answer", id, peerConnection.localDescription);
});
peerConnection.ontrack = event => {
video.srcObject = event.streams[0];
};
peerConnection.onicecandidate = event => {
if (event.candidate) {
socket.emit("candidate", id, event.candidate);
}
};
});
socket.on("candidate", (id, candidate) => {
peerConnection
.addIceCandidate(new RTCIceCandidate(candidate))
.catch(e => console.error(e));
});
socket.on("connect", () => {
socket.emit("watcher");
});
socket.on("broadcaster", () => {
socket.emit("watcher");
});
socket.on("disconnectPeer", () => {
peerConnection.close();
});
window.onunload = window.onbeforeunload = () => {
socket.close();
};
rtp-to-webrtc does exactly what you want.
Unfortunately you will need to run some sort of server to make this happen, it can’t all be in the browser. You could also upload via other protocols (captured via MediaRecorder) if you don’t want to use WebRTC.
Next.js provides serverless API routes. By creating a file under ./pages/api you can have your service running, and I want to have a Socket.io service by using this mechanism.
I have created a client:
./pages/client.js
import { useEffect } from 'react';
import io from 'socket.io-client';
export default () => {
useEffect(() => {
io('http://localhost:3000', { path: '/api/filename' });
}, []);
return <h1>Socket.io</h1>;
}
And an API route:
./pages/api/filename.js
const io = require('socket.io')({ path: '/api/filename' });
io.onconnection = () => {
console.log('onconnection');
}
io.on('connect', () => {
console.log('connect');
})
io.on('connection', () => {
console.log('connection');
})
export default (req, res) => {
console.log('endpoint');
}
But I can't get the client to connect to the Socket.io server and succesfully see any of: 'onconnection', 'connect', or 'connection' printed.
The trick is to plug 'socket.io' into the http server only once, so checking every access to the api.
Try something like this:
./pages/api/socketio.js
import { Server } from 'socket.io'
const ioHandler = (req, res) => {
if (!res.socket.server.io) {
console.log('*First use, starting socket.io')
const io = new Server(res.socket.server)
io.on('connection', socket => {
socket.broadcast.emit('a user connected')
socket.on('hello', msg => {
socket.emit('hello', 'world!')
})
})
res.socket.server.io = io
} else {
console.log('socket.io already running')
}
res.end()
}
export const config = {
api: {
bodyParser: false
}
}
export default ioHandler
./pages/socketio.jsx
import { useEffect } from 'react'
import io from 'socket.io-client'
export default () => {
useEffect(() => {
fetch('/api/socketio').finally(() => {
const socket = io()
socket.on('connect', () => {
console.log('connect')
socket.emit('hello')
})
socket.on('hello', data => {
console.log('hello', data)
})
socket.on('a user connected', () => {
console.log('a user connected')
})
socket.on('disconnect', () => {
console.log('disconnect')
})
})
}, []) // Added [] as useEffect filter so it will be executed only once, when component is mounted
return <h1>Socket.io</h1>
}
You have to have the /api/pusher/auth to authenticate with pusher on the frontend. Then you use the key you get from that to communicate with pusher. It's for security purposes. You can do it all through the frontend, but depending on your app, if you're saving data (such as messages, or chats) then probably should authenticate.
You can use custom server and attach sockets to it (just like with express) and provide needed path where socket.io will listen. How to use custom server
You can write something like this server.js
const { createServer } = require('http');
const { parse } = require('url');
const next = require('next');
const { Server } = require('socket.io');
const dev = process.env.NODE_ENV !== 'production';
const hostname = 'localhost';
const port = 3000;
// when using middleware `hostname` and `port` must be provided below
const app = next({ dev, hostname, port });
const handle = app.getRequestHandler();
app.prepare().then(() => {
const server = createServer(async (req, res) => {
try {
// Be sure to pass `true` as the second argument to `url.parse`.
// This tells it to parse the query portion of the URL.
const parsedUrl = parse(req.url, true);
const { pathname, query } = parsedUrl;
if (pathname === '/a') {
await app.render(req, res, '/a', query);
} else if (pathname === '/b') {
await app.render(req, res, '/b', query);
} else {
await handle(req, res, parsedUrl);
}
} catch (err) {
console.error('Error occurred handling', req.url, err);
res.statusCode = 500;
res.end('internal server error');
}
});
const io = new Server(server, {
path: '/socket.io' // or any other path you need
});
io.on('connection', socket => {
// your sockets here
console.log('IO_CONNECTION');
});
server.listen(port, err => {
if (err) throw err;
console.log(`> Ready on http://${hostname}:${port}`);
});
});
You would need to run your server using node server.js
In Mocha test beforeEach hook, I am trying to destroy all table records.
import { db } from '../src/db/models';
export const truncateTable = () => {
const promises = Object.keys(db).map(key => {
if (key !== 'Sequelize' && key !== 'sequelize') {
console.log(key);
return db[key].destroy({ where: {} });
}
});
return Promise.all(promises);
};
Then in the test, I am doing this:
describe.only('application mutations', () => {
beforeEach(() => truncateTable());
...
The error I am getting:
SequelizeDatabaseError: could not serialize access due to concurrent
update
TL/DR: in your tests, if you want a quick way to delete models and reset your DB, use sync.
describe.only('application mutations', () => {
beforeEach(async () => {
await db.sync({force: true})
});
}
If you want to individually destroy your models, you must properly await for your promise to finish before initiating a new one. Currently, your promises are being initiated all at once, hence the Sequelize error.
export const truncateTable = async () => {
const promises = Object.keys(db).map(key => {
if (key !== 'Sequelize' && key !== 'sequelize') {
await db[key].destroy({ where: {} });
}
});
};
// in your test file
describe.only('application mutations', () => {
beforeEach(async () => {
await truncateTable();
});
})