What do these numbers mean in socket.io payload? - websocket

When I am using native websocket API I can see just a payload in my chrome console for sockts:
But when I use socket.io with their emit event, I can see some strange numbers before my actual payload. I do understand that colors mean that you either send or received the data, but what does the numbers like 42, 3, 2, 430, 420, 5 mean.
Is there a place I can get a full list of these numbers with descriptions?
The code which generates it is kind of big, so I just post small snippets.
Client side always look like this:
socket.emit('joinC', room, function(color){ ... });
Server side looks like this:
io.sockets.in(room).emit('moveS', {...});

I know you asked a while ago, but the information remains for those who are researching.
I did an analysis with reverse engineering in version 2.3.0 (socket.io) and 3.4.2 (engine.io) and got the following:
The first number is the type of communication for engine.io, using the enumerator:
Key
Value
0
"open"
1
"close"
2
"ping"
3
"pong"
4
"message"
5
"upgrade"
6
"noop"
The second number is the type of action for socket.io, using the enumerator
Key
Value
0
"CONNECT"
1
"DISCONNECT"
2
"EVENT"
3
"ACK"
4
"ERROR"
5
"BINARY_EVENT"
6
"BINARY_ACK"
There are other optional information that can be passed on, such as namespace and ID, but I will not go into that part.
After these codes he expects a Json Array, where index 0 is the name of the event and index 1 is the argument.
So the instruction 42["moveS",{"from":"g1", "to", "f3"}] is a message for engine.io (4), is an event for socket.io (2), which will emit the "moveS" action passing JSON {"from": "g1", "to", "f3"} as a parameter(Actually JSON.Parse({"from": "g1", "to", "f3"})).
Hope this helps. =D

Websockets allow you to send data back and forth over a full-duplex communication channel.
Socket.IO on the other hand is a realtime application framework that uses websockets as transport adding features like namespacing connections, rooms, fallback to other transports etc. To build all those features, the messages exchanged back and forward must cary some semantics so that Socket.IO knows what they mean (i.e. what kind of message is it, event, error etc) and what to do with them. For that it uses a protocol that frames the message with some codes that identify it's semantic. That's what you are seeing with those numbers.
Unfortunately the Socket.IO documentation is very terse and it's hard to understand exactly how those codes are combined and parsed. To get their exact meaning I think one needs to look at the Socket.IO source code.
EDIT from a socket.io Github issue:
This is handled in socket.io-parser and engine.io-parser, which are implementations of socket.io-protocol and engine.io-protocol respectively. You can find the protocol description for socket.io here and for engine.io here.
The encoding sections in these documents are of interest when looking at the actual data that is sent through the transports. The socket.io-protocol handles encoding of metadata, like namespaes to an engine.io-protocol handleable format.

Related

lighttpd/mod_wstunnel concatenates JSON messages

I'm trying to use lighttpd (v1.4.49) with mod_wstunnel.
$HTTP["url"] =~ "^/websocket" {
wstunnel.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "50007" ) ) )
wstunnel.frame-type = "text"
wstunnel.ping-interval = 30
}
The backend TCP-Server sends single line JSON-Messages that should be received by the WebSocket-Clients onmessage handler.
However, sometimes two successive messages are concatenated by mod_wstunnel and received (and passed to onmessage) as one message.
Is there any "end-of-message" token I could send to explicitly "tell" mod_wstunnel that the message is complete?
Thanks,
Sam
You should probably fix your application if your application depends on framing at the websockets layer. See https://www.rfc-editor.org/rfc/rfc6455#section-5.4
Unless specified otherwise by an extension, frames have no semantic
meaning. An intermediary might coalesce and/or split frames, if no
extensions were negotiated by the client and the server or if some
extensions were negotiated, but the intermediary understood all the
extensions negotiated and knows how to coalesce and/or split frames
in the presence of these extensions. One implication of this is that
in absence of extensions, senders and receivers must not depend on
the presence of specific frame boundaries.
Your backend is sending JSON and knows nothing about websockets, and therefore can not specify how mod_wstunnel should send websocket frames. Your client app should not depend on the websocket framing, but if you wanted to try to mitigate this on the server-side, your backend could pause between sending each JSON message. It would be better to fix your app to process complete JSON messages, one at a time.

Message formatting of OSC for MIDI messages

I'm using the github.com/hypebeast/go-osc/osc package to send OSC messages to an OSC server. For this I'm using OSCulator so that I can route the data as MIDI to Abelton Live.
The problem I'm having is I can not find any information on message formatting for things like note on, note off, duration etc. I found a guide on the OSCulator website that's a little helpful, but it doesn't go into much detail on messaging: http://s3.amazonaws.com/osculator/doc/OSCulator+2.12+Manual.pdf
For example, the following function works just fine, but I have no idea what the message is really doing:
func note(pitch float32 , velocity float32) {
// TODO: Pass client into function. Find out it's type.
client := osc.NewClient("localhost", 8765)
noteMsg := osc.NewMessage("/4/toggle2")
client.Send(noteMsg)
msg := osc.NewMessage("/4/xy")
msg.Append(pitch)
msg.Append(velocity)
client.Send(msg)
}
I mean, what purpose does the 4 play in this, and what is xy? Also, what other messages are available apart from toggle2? I thought there would be some sort of documentation online that has all the different types of messages available for MIDI type applications.
Your question seems to be more related to OSC itself.
OSC works like this:
You send a message to a server. A message is composed by an adress and some values.
In this case, /4/xy is the address. The 4 and the slashes you define what will be. When you receive it on the other side you will know what you want to receive, which means the address you're sending. So you will configure the server or the receiver to do something when it receives a message from a specific adress.
In the same way, you are appending values to the message. The quantity of values you already know, so you just have to do what you want with them when you receive.
Basically, if you decide to have a keyboard sending notes, you would use something like /keyboard/note as adress and send one value at a time, so you would read this value and do something with it.

Am I doing something wrong, or does echo.websocket.org not echo back empty payloads?

According to the spec for websockets protocol 13 (RFC 6455), the payload length for any given frame can be 0.
frame-payload-data ; n*8 bits in
; length, where
; n >= 0
I am building a websocket client to this spec, but when I sent echo.websocket.org a frame with an empty payload, I get nothing back. I experience the same using their GUI:
This is troublesome for me, since the way I'm building my client somewhat requires me to send empty frames when I FIN a multi-frame message.
Is this merely a bug in the Echo Test server? Do a substantial number of server implementations drop frames with empty payloads?
And if this is a bug in Echo Test, does anyone know how I might get in touch with them? The KAAZING site only has tech support contact info for their own products.
If you send a data-frame with no payload, there is nothing to echo back. This behaviour is fully correct. However, it might be standard-conform, too, to send back a dataframe with 0 payload, too. The main question is, if the application layer is informed at all, when a dataframe with no payload is received. This is probably not the case in most implementations.
With TCP this is similar: A TCP-Keepalive is a datagram with 0 payload. It is ack'd by the remote TCP-stack, but the application layer is not informed about it (i.e. select() does not return or a read()-syscall remains blocking), which is the expected behaviour.
An application-layer protocol should not rely on the datagrams to structure the data, but should merely expect a stream of bytes without taking regard on how these are transported.
I just tried the echo test on websocket.org with empty payloads and it seems to work fine using Chrome, Safari and Firefox (latest versions of each). Which browser are you using?
Btw, that demo program doesn't abide by any "echo protocol" (afaik), so there's no formal specification that dictates what to do on empty data in a WebSocket set of frames.
If you need help using WebSocket, there are Kaazing forums: http://developer.kaazing.com/forums.

Pubnub chat application with storage

I'm looking to develop a chat application with Pubnub where I want to make sure all the chat messages that are send is been stored in the database and also want to send messages in chat.
I found out that I can use the Parse with pubnub to provide storage options, But I'm not sure how to setup those two in a way where the messages and images send in the chat are been stored in the database.
Anyone have done this before with pubnub and parse? Are there any other easy options available to use with pubnub instead of using parse?
Sutha,
What you are seeking is not a trivial solution unless you are talking about a limited number of end users. So I wouldn't say there are no "easy" solutions, but there are solutions.
The reason is your server would need to listen (subscribe) to every chat channel that is active and store the messages being sent into your database. Imagine your app scaling to 1 million users (doesn't even need to get that big, but that number should help you realize how this can get tricky to scale where several server instances are listening to channels in a non-overlapping manner or with overlap but using a server queue implementation and de-duping messages).
That said, yes, there are PubNub customers that have implemented such a solution - Parse not being the key to making this happen, by the way.
You have three basic options for implementing this:
Implement a solution that will allow many instances of your server to subscribe to all of the channels as they become active and store the messages as they come in. There are a lot of details to making this happen so if you are not up to this then this is not likely where you want to go.
There is a way to monitor all channels that become active or inactive with PubNub Presence webhooks (enable Presence on your keys). You would use this to keep a list of all channels that your server would use to pull history (enable Storage & Playback on your keys) from in an on-demand (not completely realtime) fashion.
For every channel that goes active or inactive, your server will receive these events via the REST call (and endpoint that you implement on your server - your Parse server in this case):
channel active: record "start chat" timetoken in your Parse db
channel inactive: record "end chat" timetoken in your Parse db
the inactive event is the kickoff for a process that uses start/end timetokens that you recorded for that channel to get history from for channel from PubNub: pubnub.history({channel: channelName, start:startTT, end:endTT})
you will need to iterate on this history call until you receive < 100 messages (100 is the max number of messages you can retrieve at a time)
as you retrieve these messages you will save them to your Parse db
New Presence Webhooks have been added:
We now have webhooks for all presence events: join, leave, timeout, state-change.
Finally, you could just save each message to Parse db on success of every pubnub.publish call. I am not a Parse expert and barely know all of its capabilities but I believe they have some sort or store local then sync to cloud db option (like StackMob when that was a product), but even if not, you will save msg to Parse cloud db directly.
The code would look something like this (not complete, likely errors, figure it out or ask PubNub support for details) in your JavaScript client (on the browser).
var pubnub = PUBNUB({
publish_key : your_pub_key,
subscribe_key : your_sub_key
});
var msg = ... // get the message form your UI text box or whatever
pubnub.publish({
// this is some variable you set up when you enter a chat room
channel: chat_channel,
message: msg
callback: function(event){
// DISCLAIMER: code pulled from [Parse example][4]
// but there are some object creation details
// left out here and msg object is not
// fully fleshed out in this sample code
var ChatMessage = Parse.Object.extend("ChatMessage");
var chatMsg = new ChatMessage();
chatMsg.set("message", msg);
chatMsg.set("user", uuid);
chatMsg.set("channel", chat_channel);
chatMsg.set("timetoken", event[2]);
// this ChatMessage object can be
// whatever you want it to be
chatMsg.save();
}
error: function (error) {
// Handle error here, like retry until success, for example
console.log(JSON.stringify(error));
}
});
You might even just store the entire set of publishes (on both ends of the conversation) based on time interval, number of publishes or size of total data but be careful because either user could exit the chat and the browser without notice and you will fail to save. So the per publish save is probably best practice if a bit noisy.
I hope you find one of these techniques as a means to get started in the right direction. There are details left out so I expect you will have follow up questions.
Just some other links that might be helpful:
http://blog.parse.com/learn/building-a-killer-webrtc-video-chat-app-using-pubnub-parse/
http://www.pubnub.com/blog/realtime-collaboration-sync-parse-api-pubnub/
https://www.pubnub.com/knowledge-base/discussion/293/how-do-i-publish-a-message-from-parse
And we have a PubNub Parse SDK, too. :)

How can I know what message I've received while using Protocol Buffers library?

It seems I don't understand something simple about Protocol Buffers, but this is very important question for me and for my real use-case.
While reading documentation about Protocol Buffers I don't understand how one know which message you should decode from the stream? All examples about some defined Message, but if you have defined several completelly different messages and you want to send those messages between 2 processes -- how do you know which message you have just received?
Or maybe Protocol Buffers do not try to address this problem and leave this question for another abstraction level?
Or maybe I should pack the message into structure like that:
message wrapper {
required string message_name = 1;
string packed_message = 2;
}
And then I should decode message in 2 stages: get the message_name at first, and then decode real packed message at second stage, should not I?
Look at self describing messages section

Resources