I am using the Node.js ws library, to listen to events in user accounts on a 3rd party API. For each user, I open a websocket to listen to the events in the user's account.
Turns out, the 3rd-party API doesn't provide a userID for each event, so if I have 10 websocket connections to user-accounts, I cannot determine which account an event came from.
I have access to a unique userId prior to starting each of my connections.
Is there a way to append or wrap the websocket connection with the userId identifier, to each connection I make, such that when I receive an event, I can access the custom identifier, and subsequently know which user's account the event came from?
The code below is a mix of real code, and pseudocode (i.e customSocket)
const ws = new WebSocket('wss://thirdparty-api.com/accounts', {
port: 8080,
});
ws.send(
JSON.stringify({
action: 'authenticate',
data: {
oauth_token: access_token,
},
})
);
// wrap and attach data here (pseudocode at top-level)
customSocket.add({userId,
ws.send(
JSON.stringify({
action: 'listen',
data: {
streams: ['action_updates'],
},
})
)
})
// listen for wrapper data here, pseudocode at top level
customSocket.emit((customData) {
ws.on('message', function incoming(data) {
console.log('incoming -> data', data.toString());
})
console.log('emit -> customData', customData);
})
Looking at the socket.io library, the namespace feature may solve for this, but I can't determine if that's true or not. Below is an example in their documentation:
// your application has multiple tenants so you want to dynamically create one namespace per tenant
const workspaces = io.of(/^\/\w+$/);
workspaces.on('connection', socket => {
const workspace = socket.nsp;
workspace.emit('hello');
});
// this middleware will be assigned to each namespace
workspaces.use((socket, next) => {
// ensure the user has access to the workspace
next();
});
I found a solution to this which is fairly simple. First create a message handler function:
const eventHandler = (uid, msg) => {
console.log(`${uid} did ${msg}`);
};
Then, when you create the websocket for the given user, wrap the .on event with the handler:
const createSocketForUser = (uid, eventHandler) => {
const socket = new WebSocket(/* ... */);
socket.onmessage = (msg) => {
eventHandler(uid, msg)
};
return socket;
}
Related
I am planning to use RSocket for my notifications system. I wanted to use Spring Boot RSocket for my backend (Java) while for my frontend, I will be using Angular using rsocket-js.
I was able to quickly spin-up a request-stream interaction model wherein I can pull-in all the notifications within my system. See code snippet for my backend:
#MessageMapping("streams")
public Flux<Notification> requestStream() {
log.info("Streaming to notifications...");
return streamEventService.retrieveAllNotifications().log();
}
Now on my frontend, I have the following code snippet:
export class RsocketClientService {
// backend ws endpoint
private readonly wsURL = 'ws://localhost:7000/notification';
client: any;
socket: any
constructor() {
this.client = new RSocketClient({
serializers: {
data: JsonSerializer,
metadata: IdentitySerializer
},
setup: {
keepAlive: 10000,
lifetime: 180000,
dataMimeType: 'application/json',
metadataMimeType: 'message/x.rsocket.routing.v0',
payload: {
data: 23
}
},
transport: new RSocketWebSocketClient({
url: this.wsURL
}),
responder: new EchoResponder()
});
}
public connect() {
console.log("initializeSocket...")
this.client.connect().subscribe({
onComplete: (socket: any) => {
this.socket = socket;
this.socket.connectionStatus().subscribe( (status: any) => {
console.log("Connection status? ", status);
});
},
onError: (error: any) => {
console.error("Connection onError? " + error);
},
onSubscribe: (cancel: any) => {
console.log("Connection onSubscribe? cancel?");
}
});
}
public retrieveNotifications() {
this.socket.requestStream({
data: null,
metadata: String.fromCharCode('streams'.length) + 'streams'
})
.subscribe({
onComplete: () => {
console.log("onComplete?");
},
onError: (error: any) => {
console.error("onError? error: " + error);
},
onNext: (payload: any) => {
console.log("onNext? payload: ", payload);
},
onSubscribe: (subscription: any) => {
console.log("onSubscribe?");
subscription.request(1000000);
},
});
}
I have a button in the UI that if clicked will call the method retrieveNotifications which will subscribe to the rsocket message mapping method in my backend requestStream.
Everything is working fine and I could see my responses coming in. Now my question would be, what if on my server there is a new data inserted into the database for example, then how can I send a notification message from my backend server to the frontend saying that "Hey! new data was pushed into the database." I am kind of stuck on how the server will be able to use a somehow fire and forget to the client side.
You want to server-side send request to client-side when connect established.
You can get this connect's RSocketRequester from server then using it create one of four method(FNF, Request-reponse, request-stream, stream-stream) to send request to client.
In client-side, you can receive data in EchoResponder class in one of four method above.
It looks like you need to create a new controller function that returns a void and when you insert an object in the DB you pass that object to the front end from this function and in angular you connect to it as you did up...try to check this link for fire and forget approach ... hope this helps https://www.baeldung.com/spring-boot-rsocket
I am trying to integrate socket.io with strapi. But unfortunately I have been unable to do so without any proper tutorial or documentation covering this aspect.
I followed along with the only resource I found online which is:
https://medium.com/strapi/strapi-socket-io-a9c856e915a6
But I think the article is outdated. I can't seem to run the code mentioned in it without running into tonnes of errors.
Below is my attempt to implement it and I have been trying to connect it through a chrome websocket plugin smart websocket client But I am not getting any response when I try to run the server.
I'm totally in the dark. Any help will be appreciated
module.exports = ()=> {
// import socket io
var io = require('socket.io')(strapi.server)
console.log(strapi.server) //undefined
// listen for user connection
io.on('connect', socket => {
socket.send('Hello!');
console.log("idit")
// or with emit() and custom event names
socket.emit('greetings', 'Hey!', { 'ms': 'jane' }, Buffer.from([4, 3, 3, 1]));
// handle the event sent with socket.send()
socket.on('message', (data) => {
console.log(data);
});
// handle the event sent with socket.emit()
socket.on('salutations', (elem1, elem2, elem3) => {
console.log(elem1, elem2, elem3);
});
});
};
So I found the solution. Yay. I'll put it here just in case anybody needs it.
boostrap.js
module.exports = async () => {
process.nextTick(() =>{
var io = require('socket.io')(strapi.server);
io.on('connection', async function(socket) {
console.log(`a user connected`)
// send message on user connection
socket.emit('hello', JSON.stringify({message: await strapi.services.profile.update({"posted_by"})}));
// listen for user diconnect
socket.on('disconnect', () =>{
console.log('a user disconnected')
});
});
strapi.io = io; // register socket io inside strapi main object to use it globally anywhere
})
};
Found this at: https://github.com/strapi/strapi/issues/5869#issuecomment-619508153_
Apparently, socket.server is not available when the server starts. So you have to make use of process.nextTick that waits for the socket.server to initialize.
I'll also add a few questions that I faced when setting this up.
How do i connect from an external client like nuxt,vue or react?
You just have to connect through "http://localhost:1337" that is my usual address for strapi.
I am using nuxt as my client side and this is how set up my socketio on the client side
I first installed nuxt-socket-io through npm
Edited the nuxt.config file as per it's documention
modules:[
...
'nuxt-socket-io',
...
],
io: {
// module options
sockets: [
{
name: 'main',
url: 'http://localhost:1337',
},
],
},
And then i finally added a listener in one of my pages.
created() {
this.socket = this.$nuxtSocket({})
this.socket.on('hello', (msg, cb) => {
console.log('SOCKET HI')
console.log(msg)
})
},
And it works.
A clean way to integrate third-party services into Strapi is to use hooks. They are loaded once during the server boot. In this case, we will create a local hook.
The following example has worked with strapi#3.6.
Create a hook for socket.io at ./hooks/socket.io/index.js
module.exports = strapi => {
return {
async initialize() {
const ioServer = require('socket.io')(strapi.server, {
cors: {
origin: process.env['FRONT_APP_URL'],
methods: ['GET', 'POST'],
/* ...other cors options */
}
})
ioServer.on('connection', function(socket) {
socket.emit('hello', `Welcome ${socket.id}`)
})
/* HANDLE CLIENT SOCKET LOGIC HERE */
// store the server.io instance to global var to use elsewhere
strapi.services.ioServer = ioServer
},
}
}
Enable the new hook in order for Strapi to load it - ./config/hook.js
module.exports = {
settings: {
'socket.io': {
enabled: true,
},
},
};
That's done. You can access the websocket server inside ./config/functions/bootstrap.js or models' lifecycle hooks.
// ./api/employee/models/employee.js
module.exports = {
lifecycles: {
async afterUpdate(result, params, data) {
strapi.services.ioServer.emit('update:employee', result)
},
},
};
For those who are looking the answer using Strapi version 4
var io = require("socket.io")(strapi.server.httpServer)
I'm looking into implementing a "subscription" type using server-sent events as the backing api.
What I'm struggling with is the interface, to be more precise, the http layer of such operation.
The problem:
Using the native EventSource does not support:
Specifying an HTTP method, "GET" is used by default.
Including a payload (The GraphQL query)
While #1 is irrefutable, #2 can be circumvented using query parameters.
Query parameters have a limit of ~2000 chars (can be debated)
which makes relying solely on them feels too fragile.
The solution I'm thinking of is to create a dedicated end-point for each possible event.
For example: A URI for an event representing a completed transaction between parties:
/graphql/transaction-status/$ID
Will translate to this query in the server:
subscription TransactionStatusSubscription {
status(id: $ID) {
ready
}
}
The issues with this approach is:
Creating a handler for each URI-to-GraphQL translation is to be added.
Deploy a new version of the server
Loss of the flexibility offered by GraphQL -> The client should control the query
Keep track of all the end-points in the code base (back-end, front-end, mobile)
There are probably more issues I'm missing.
Is there perhaps a better approach that you can think of?
One the would allow a better approach at providing the request payload using EventSource?
Subscriptions in GraphQL are normally implemented using WebSockets, not SSE. Both Apollo and Relay support using subscriptions-transport-ws client-side to listen for events. Apollo Server includes built-in support for subscriptions using WebSockets. If you're just trying to implement subscriptions, it would be better to utilize one of these existing solutions.
That said, there's a library for utilizing SSE for subscriptions here. It doesn't look like it's maintained anymore, but you can poke around the source code to get some ideas if you're bent on trying to get SSE to work. Looking at the source, it looks like the author got around the limitations you mention above by initializing each subscription with a POST request that returns a subscription id.
As of now you have multiple Packages for GraphQL subscription over SSE.
graphql-sse
Provides both client and server for using GraphQL subscription over SSE. This package has a dedicated handler for subscription.
Here is an example usage with express.
import express from 'express'; // yarn add express
import { createHandler } from 'graphql-sse';
// Create the GraphQL over SSE handler
const handler = createHandler({ schema });
// Create an express app serving all methods on `/graphql/stream`
const app = express();
app.use('/graphql/stream', handler);
app.listen(4000);
console.log('Listening to port 4000');
#graphql-sse/server
Provides a server handler for GraphQL subscription. However, the HTTP handling is up to u depending of the framework you use.
Disclaimer: I am the author of the #graphql-sse packages
Here is an example with express.
import express, { RequestHandler } from "express";
import {
getGraphQLParameters,
processSubscription,
} from "#graphql-sse/server";
import { schema } from "./schema";
const app = express();
app.use(express.json());
app.post(path, async (req, res, next) => {
const request = {
body: req.body,
headers: req.headers,
method: req.method,
query: req.query,
};
const { operationName, query, variables } = getGraphQLParameters(request);
if (!query) {
return next();
}
const result = await processSubscription({
operationName,
query,
variables,
request: req,
schema,
});
if (result.type === RESULT_TYPE.NOT_SUBSCRIPTION) {
return next();
} else if (result.type === RESULT_TYPE.ERROR) {
result.headers.forEach(({ name, value }) => res.setHeader(name, value));
res.status(result.status);
res.json(result.payload);
} else if (result.type === RESULT_TYPE.EVENT_STREAM) {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
Connection: 'keep-alive',
'Cache-Control': 'no-cache',
});
result.subscribe((data) => {
res.write(`data: ${JSON.stringify(data)}\n\n`);
});
req.on('close', () => {
result.unsubscribe();
});
}
});
Clients
The two packages mentioned above have companion clients. Because of the limitation of the EventSource API, both packages implement a custom client that provides options for sending HTTP Headers, payload with post, what the EvenSource API does not support. The graphql-sse comes together with it client while the #graphql-sse/server has companion clients in a separate packages.
graphql-sse client example
import { createClient } from 'graphql-sse';
const client = createClient({
// singleConnection: true, use "single connection mode" instead of the default "distinct connection mode"
url: 'http://localhost:4000/graphql/stream',
});
// query
const result = await new Promise((resolve, reject) => {
let result;
client.subscribe(
{
query: '{ hello }',
},
{
next: (data) => (result = data),
error: reject,
complete: () => resolve(result),
},
);
});
// subscription
const onNext = () => {
/* handle incoming values */
};
let unsubscribe = () => {
/* complete the subscription */
};
await new Promise((resolve, reject) => {
unsubscribe = client.subscribe(
{
query: 'subscription { greetings }',
},
{
next: onNext,
error: reject,
complete: resolve,
},
);
});
;
#graphql-sse/client
A companion of the #graphql-sse/server.
Example
import {
SubscriptionClient,
SubscriptionClientOptions,
} from '#graphql-sse/client';
const subscriptionClient = SubscriptionClient.create({
graphQlSubscriptionUrl: 'http://some.host/graphl/subscriptions'
});
const subscription = subscriptionClient.subscribe(
{
query: 'subscription { greetings }',
}
)
const onNext = () => {
/* handle incoming values */
};
const onError = () => {
/* handle incoming errors */
};
subscription.susbscribe(onNext, onError)
#gaphql-sse/apollo-client
A companion package of the #graph-sse/server package for Apollo Client.
import { split, HttpLink, ApolloClient, InMemoryCache } from '#apollo/client';
import { getMainDefinition } from '#apollo/client/utilities';
import { ServerSentEventsLink } from '#graphql-sse/apollo-client';
const httpLink = new HttpLink({
uri: 'http://localhost:4000/graphql',
});
const sseLink = new ServerSentEventsLink({
graphQlSubscriptionUrl: 'http://localhost:4000/graphql',
});
const splitLink = split(
({ query }) => {
const definition = getMainDefinition(query);
return (
definition.kind === 'OperationDefinition' &&
definition.operation === 'subscription'
);
},
sseLink,
httpLink
);
export const client = new ApolloClient({
link: splitLink,
cache: new InMemoryCache(),
});
If you're using Apollo, they support automatic persisted queries (abbreviated APQ in the docs). If you're not using Apollo, the implementation shouldn't be too bad in any language. I'd recommend following their conventions just so your clients can use Apollo if they want.
The first time any client makes an EventSource request with a hash of the query, it'll fail, then retry the request with the full payload to a regular GraphQL endpoint. If APQ is enabled on the server, subsequent GET requests from all clients with query parameters will execute as planned.
Once you've solved that problem, you just have to make a server-sent events transport for GraphQL (should be easy considering the subscribe function just returns an AsyncIterator)
I'm looking into doing this at my company because some frontend developers like how easy EventSource is to deal with.
There are two things at play here: the SSE connection and the GraphQL endpoint. The endpoint has a spec to follow, so just returning SSE from a subscription request is not done and needs a GET request anyway. So the two have to be separate.
How about letting the client open an SSE channel via /graphql-sse, which creates a channel token. Using this token the client can then request subscriptions and the events will arrive via the chosen channel.
The token could be sent as the first event on the SSE channel, and to pass the token to the query, it can be provided by the client in a cookie, a request header or even an unused query variable.
Alternatively, the server can store the last opened channel in session storage (limiting the client to a single channel).
If no channel is found, the query fails. If the channel closes, the client can open it again, and either pass the token in the query string/cookie/header or let the session storage handle it.
I'm trying to set up a server that can dynamically create many rooms for many namespaces. I'm currently just trying to broadcast to sockets of a room, when a new socket has joined that room.
So far I have been able to broadcast to a specific namespace and my event listeners on the client receives the message. However when I try to broadcast to a room, of a specific namespace, my event listener doesn't receive that message.
I've turned on the Debugger mode and see the socket.io-client:socket emitting the event with the right payload and event type. So I am not sure what I am missing since the documentation also seems fairly straightforward. Any help would be much appreciated. Below is my code.
Server
const colorNs = io.of('/color');
colorNs.on('connection', (socket) => {
const { id } = socket.handshake.query;
const { id:connId } = socket.conn;
if(id) {
socket.join(id);
socket.broadcast.to(id).emit('user:connect', { id: connId });
}
socket.on('disconnect', () => {
const { id } = socket.handshake.query;
const { id:connId } = socket.conn;
socket.broadcast.to(id).emit('user:disconnect', { id: connId });
});
});
Client
const socket = io('/color?id="123"');
socket.on('user:connect', () => console.log('data', data));
Client - Debug Trace
socket.io-parser decoded 2/color,["user:connect",{"id":"IZTTPidF121JCzf9AAAO"}] as {"type":2,"nsp":"/color","data":["user:connect",{"id":"IZTTPidF121JCzf9AAAO"}]} +1ms
browser.js:133
socket.io-client:socket emitting event ["user:connect",{"id":"IZTTPidF121JCzf9AAAO"}] +3ms
How can you detect that you received a message on a socket.io connection that you do not have a handler for?
example:
// client
socket.emit('test', 'message');
// server
io.on('connection', function (socket) {
console.log('connection received...');
// logs all messages
socket.conn.on('message', function(data) {
console.log('this gets every message.');
console.log('how do I get just the ones without explicit handlers?');
});
socket.on('other' function(data) {
console.log('expected message');
});
}
By accessing the internals of the socket object you can determine what events it is currently listening for. You can use this server-side code to see if the current message is being handled.
io.on('connection', (socket) => {
console.log('A user connected.');
socket.on('disconnect', () => {
console.log('A user disconnected.');
});
socket.on('chat', (msg) => {
console.log('message: ' + msg);
io.emit('chat', msg);
});
socket.conn.on('message', (msg) => {
if(!Object.keys(socket._events).includes(msg.split('"')[1])) {
console.log(`WARNING: Unhandled Event: ${msg}`);
}
});
}
Once connected I am handling two events, 'disconnect' and 'chat'. After that I define a handler that catches all messages with socket.conn.on(...).
The message it receives is going to be a string that looks something like this: '2["myEventName","param1","param2"]'. By splitting it along the double quotes we can get the event name.
We then peek into the internals of socket to find all the keys of socket._events, which happen to be the event name strings. If this collection of strings includes our event name, then another handler will take care of it, and we don't have to.
You can test it from the console in the browser. Run socket.emit('some other event') there and you should see your warning come up in the server console.
IMPORTANT NOTE: Normally you should not attempt to externally modify any object member starting with an underscore. Also, expect that any data in it is unstable. The underscore indicates it is for internal use in that object, class or function. Though this object is not stable, it should be up to date enough for us to use it, and we aren't modifying it directly.
Tested with SocketIO version 2.2.0 on Chrome.
I didn't find a way to do it like socket.io, but using a simple js function to transform message into json it's doing the same job. Here you can try this:
function formatMessage(packetType, data) {
var message = {'packetType': packetType, 'data': data}
return JSON.stringify(message)
}
With:
socket.on('message', function(packet){
packet = JSON.parse(packet)
switch (packet.packetType) {
case 'init':
...
and
socket.send(formatMessage('init', {message}));
I would do so, of course it is the abstract code ... you would have to implement all the listeners and the logic to get the ids of the users to work
Client
var currentUser = {
id: ? // The id of current user
};
var socketMessage = {
idFrom: currentUser.id,
idTo: ?, // Some user id value
message: 'Hello there'
};
socket.emit('message', socketMessage);
socket.on('emitedMessage' + currentUser.id, function(message) {
// TODO: handle message
});
Server
io.on('connection', function (socket) {
// Handle emit messages
socket.on('message', function(socketMessage) {
// With this line you send the message to a specific user
socket.emit('emitedMessage-' + socketMessage.idTo, {
from: socketMessage.idFrom,
message: socketMessage.message
});
});
});
More info: http://socket.io/docs/