I'm currently learning node.js by developing a small chat application over TCP (classic case).
What I would like to do now is to extend this application to use static users:
All logic around messaging should be based on users rather than
sessions.
Each session therefore needs to be saved along with a
reference to the user (so that the application can send messages in
the correct session based on user name).
I have looked into Redis to store this data but not been successful so far.
tcpServer.on 'connection', (tcpSocket) -> tcpSocket.write "Welcome"
redis.hmset("sessions", userid, tcpSocket)
tcpSocket.on "data", (data) ->
redis.hvals "sessions", (err, repl) ->
repl.forEach (reply, y) ->
reply.Write("Data is written to clients!")
tcpServer.listen 7000
Current error message: TypeError: Object [object Object] has no method 'write'
This indicates that I cannot store a tcpSocket as I'm trying to do right now. Can someone tell me if I can make minor adjustments to make this right or should I rethinkt this solution?
Redis supports very few data types: string, lists, sets, sorted lists and hash tables. That's it. Some other types, like Date, can be easily converted to/from string, so it can be used too. But socket is a complex internal object, so you most probably can't convert it into string and therefore can't store in Redis.
Since socket objects are to complex to store in Redis it seems like the way to go is to add identifier to the object according to link below.
This identifier could then of course be saved in a database to create realtions between different objects etc.
How to uniquely identify a socket with Node.js
Related
I don't know how can I write a smart contract in Solana that after executing the logic, returns an array of integers, strings, ... to the client, and how can I fetch it using Web3?
There's a syscall available to on-chain programs called set_return_data, which puts data into a buffer that can be read by the higher-level programs using get_return_data. This all mediated through opaque byte buffers, so you'll need to know how to decode the response.
If you want to fetch the data from the client side, you can simulate the transaction and read the data back from the return_data field in the response: https://edge.docs.solana.com/developing/clients/jsonrpc-api#results-50
The RPC support in simulated transactions is very new in version 1.11, but the return data is available in earlier versions.
Source code for set_return_data at https://github.com/solana-labs/solana/blob/658752cda710cb358d7ccbbc2cee06bf8009c2d4/sdk/program/src/program.rs#L102
Source code for get_return_data at https://github.com/solana-labs/solana/blob/658752cda710cb358d7ccbbc2cee06bf8009c2d4/sdk/program/src/program.rs#L117
So, programs do not return data (other than success or failure).
However; most programs write data to a program owned account's data field and this could be read from client apps (Rust, Python, TS/JS, etc.).
If using the Solana web3 library, you can call getAccountInfo on the Connection object. This will return the byte array of the account. You will then need to deserialize that data. You have to know how the program serializes the data to reverse it successfully.
Check the Solana Cookbook for overview using borsh https://solanacookbook.com/guides/serialization.html#how-to-deserialize-account-data-on-the-client
I want to store all the blockchain data in offchain database.
rpc has a function called EXPERIMENTAL_changes, I was told that I can do that by http polling of this method but I am unable to find out how to use it.
http post https://rpc.testnet.near.org jsonrpc=2.0 id=dontcare method=EXPERIMENTAL_changes \ params:='{ "changes_type": "data_changes", "account_ids": ["guest-book.testnet"], "key_prefix_base64": "", "block_id": 19450732 }'
For example here the results give:
"change": { "account_id": "guest-book.testnet", "key_base64": "bTo6Mzk=", "value_base64": "eyJwcmVtaXVtIjpmYWxzZSwic2VuZGVyIjoiZmhyLnRlc3RuZXQiLCJ0ZXh0IjoiSGkifQ==" }
What is key_base64?
Decoding it to string gives m::39
What is m::39?
For example, I have the following state data in the rust structure.
pub struct Demo {
user_profile_map: TreeMap<u128, User>,
user_products_map: TreeMap<u128, UnorderedSet<u128>>, // (user_id, set<product_id>)
product_reviews_map: TreeMap<u128, UnorderedSet<u128>>, // (product_id, set<review_id>)
product_check_bounty: LookupMap<u128, Vector<u64>>
}
How to know anything gets changed in these variables?
Will I have to check every block id for the point the contract is deployed, to know where there is the change?
I want to store all the blockchain data in offchain database.
If so, I recommend you take a look at the Indexer Framework, which allows you to get a stream of blocks and handle them. We use it to build Indexer for Wallet (keeps track of every added and deleted access key, and stores those into Postgres) and Indexer for Explorer (keeps track of every block, chunk, transaction, receipt, execution outcome, state changes, accounts, and access keys, and stores all of that in Postgres)
What is m::39?
Contracts in NEAR Protocol have access to the key-value storage (state), so at the lowest-level, you operate with key-value operations (NEAR SDK for AssemblyScript defines Storage class with get and set operations, and NEAR SDK for Rust has storage_read and storage_write calls to preserve data).
Guest Book example uses a high-level abstraction called PersistentVector, which automatically reads and writes its records from/to NEAR key-value storage (state). As you can see:
export const messages = new PersistentVector<PostedMessage>("m");
Guest Book defines the messages to be stored in the storage with m prefix, hense you see m::39, which basically means it is messages[39] stored in the key-value storage.
What is key_base64?
As key-value storage implies, the data is stored and accessed by keys, and the key can be binary, so base64 encoding is used to enable JSON-RPC API users with a way to query those binary keys as well (there is no way you can pass a raw binary blob in JSON).
How to know anything gets changed in these variables? Will I have to check every block id for the point the contract is deployed, to know where there is the change?
Correct, you need to follow every block, and check the changes. That is why we have built the Indexer Framework in order to enable community building services on top of that (we chose to build applications Indexer for Wallet and Indexer for Explorer, but others may decide to build GraphQL service like TheGraph)
If I have multiple processes and am using socket.io-redis, when I do io.to(room).emit(namespace, message); is this handled seamlessly and efficiently? Or am I misunderstanding socket.io-redis's role?
Hi in short as far as I know about this is-
io.to('room').emit('namespace', 'message');
Means, sending message named 'namespace' with value 'message' to all clients in 'room' channel, including sender.
Detail info (found in here)-
// send to current request socket client
socket.emit('message', "this is a test");// Hasn't changed
// sending to all clients, include sender
io.sockets.emit('message', "this is a test"); // Old way, still compatible
io.emit('message', 'this is a test');// New way, works only in 1.x
// sending to all clients except sender
socket.broadcast.emit('message', "this is a test");// Hasn't changed
// sending to all clients in 'game' room(channel) except sender
socket.broadcast.to('game').emit('message', 'nice game');// Hasn't changed
// sending to all clients in 'game' room(channel), include sender
io.sockets.in('game').emit('message', 'cool game');// Old way, DOES NOT WORK ANYMORE
io.in('game').emit('message', 'cool game');// New way
io.to('game').emit('message', 'cool game');// New way, "in" or "to" are the exact same: "And then simply use to or in (they are the same) when broadcasting or emitting:" from http://socket.io/docs/rooms-and-namespaces/
// sending to individual socketid, socketid is like a room
io.sockets.socket(socketid).emit('message', 'for your eyes only');// Old way, DOES NOT WORK ANYMORE
socket.broadcast.to(socketid).emit('message', 'for your eyes only');// New way
Even more can be found here.
Basic-
Actually the thing is your question is so sort that it is very difficult for others to understand what u exactly need. So, I assume u need to know basic concepts behind this also. So I am adding this part also for your kind info.
The concept here with socket.io with Redis is u should manage connection with socket and store the data in redis as DB.
Redis normally used for applying a layer upon DB (or caching database) so that some data can be stored for a time interval. So between that time, if any query is needed, data will come from Redis, not from DB query.
This system is applied for performance tuning so that your system can handle a huge load at the same time.
In your case, u can cache data for a short time interval for sending the messages through socket.io.
More can be found here-
http://notjoshmiller.com/socket-io-rooms-and-redis/
http://goldfirestudios.com/blog/136/Horizontally-Scaling-Node.js-and-WebSockets-with-Redis
https://github.com/socketio/socket.io-redis/issues/98
Think this answer will surely help u.
I'm looking to develop a chat application with Pubnub where I want to make sure all the chat messages that are send is been stored in the database and also want to send messages in chat.
I found out that I can use the Parse with pubnub to provide storage options, But I'm not sure how to setup those two in a way where the messages and images send in the chat are been stored in the database.
Anyone have done this before with pubnub and parse? Are there any other easy options available to use with pubnub instead of using parse?
Sutha,
What you are seeking is not a trivial solution unless you are talking about a limited number of end users. So I wouldn't say there are no "easy" solutions, but there are solutions.
The reason is your server would need to listen (subscribe) to every chat channel that is active and store the messages being sent into your database. Imagine your app scaling to 1 million users (doesn't even need to get that big, but that number should help you realize how this can get tricky to scale where several server instances are listening to channels in a non-overlapping manner or with overlap but using a server queue implementation and de-duping messages).
That said, yes, there are PubNub customers that have implemented such a solution - Parse not being the key to making this happen, by the way.
You have three basic options for implementing this:
Implement a solution that will allow many instances of your server to subscribe to all of the channels as they become active and store the messages as they come in. There are a lot of details to making this happen so if you are not up to this then this is not likely where you want to go.
There is a way to monitor all channels that become active or inactive with PubNub Presence webhooks (enable Presence on your keys). You would use this to keep a list of all channels that your server would use to pull history (enable Storage & Playback on your keys) from in an on-demand (not completely realtime) fashion.
For every channel that goes active or inactive, your server will receive these events via the REST call (and endpoint that you implement on your server - your Parse server in this case):
channel active: record "start chat" timetoken in your Parse db
channel inactive: record "end chat" timetoken in your Parse db
the inactive event is the kickoff for a process that uses start/end timetokens that you recorded for that channel to get history from for channel from PubNub: pubnub.history({channel: channelName, start:startTT, end:endTT})
you will need to iterate on this history call until you receive < 100 messages (100 is the max number of messages you can retrieve at a time)
as you retrieve these messages you will save them to your Parse db
New Presence Webhooks have been added:
We now have webhooks for all presence events: join, leave, timeout, state-change.
Finally, you could just save each message to Parse db on success of every pubnub.publish call. I am not a Parse expert and barely know all of its capabilities but I believe they have some sort or store local then sync to cloud db option (like StackMob when that was a product), but even if not, you will save msg to Parse cloud db directly.
The code would look something like this (not complete, likely errors, figure it out or ask PubNub support for details) in your JavaScript client (on the browser).
var pubnub = PUBNUB({
publish_key : your_pub_key,
subscribe_key : your_sub_key
});
var msg = ... // get the message form your UI text box or whatever
pubnub.publish({
// this is some variable you set up when you enter a chat room
channel: chat_channel,
message: msg
callback: function(event){
// DISCLAIMER: code pulled from [Parse example][4]
// but there are some object creation details
// left out here and msg object is not
// fully fleshed out in this sample code
var ChatMessage = Parse.Object.extend("ChatMessage");
var chatMsg = new ChatMessage();
chatMsg.set("message", msg);
chatMsg.set("user", uuid);
chatMsg.set("channel", chat_channel);
chatMsg.set("timetoken", event[2]);
// this ChatMessage object can be
// whatever you want it to be
chatMsg.save();
}
error: function (error) {
// Handle error here, like retry until success, for example
console.log(JSON.stringify(error));
}
});
You might even just store the entire set of publishes (on both ends of the conversation) based on time interval, number of publishes or size of total data but be careful because either user could exit the chat and the browser without notice and you will fail to save. So the per publish save is probably best practice if a bit noisy.
I hope you find one of these techniques as a means to get started in the right direction. There are details left out so I expect you will have follow up questions.
Just some other links that might be helpful:
http://blog.parse.com/learn/building-a-killer-webrtc-video-chat-app-using-pubnub-parse/
http://www.pubnub.com/blog/realtime-collaboration-sync-parse-api-pubnub/
https://www.pubnub.com/knowledge-base/discussion/293/how-do-i-publish-a-message-from-parse
And we have a PubNub Parse SDK, too. :)
Is there a way to send events from the server to all or some clients without using collections.
I want to send events with some custom data to clients. While meteor is very good in doing this with collections, in this case the added complexity and storage its not needed.
On the server there is no need for Mongo storage or local collections.
The client only needs to be alerted that it received an event from the server and act accordingly to the data.
I know this is fairly easy with sockjs but its very difficult to access sockjs from the server.
Meteor.Error does something similar to this.
The package is now deprecated and do not work for versions >0.9
You can use the following package which is originally aim to broadcast messages from clients-server-clients
http://arunoda.github.io/meteor-streams/
No collection, no mongodb behind, usage is as follow (not tested):
stream = new Meteor.Stream('streamName'); // defined on client and server side
if(Meteor.isClient) {
stream.on("channelName", function(message) {
console.log("message:"+message);
});
}
if(Meteor.isServer) {
setInterval(function() {
stream.emit("channelName", 'This is my message!');
}, 1000);
}
You should use Collections.
The "added complexity and storage" isn't a factor if all you do is create a collection, add a single property to it and update that.
Collections are just a shape for data communication between server and client, and they happen to build on mongo, which is really nice if you want to use them like a database. But at their most basic, they're just a way of saying "I want to store some information known as X", which hooks into the publish/subscribe architecture that you should want to take advantage of.
In the future, other databases will be exposed in addition to Mongo. I could see there being a smart package at some stage that strips Collections down to their most basic functionality like you're proposing. Maybe you could write it!
I feel for #Rui and the fact of using a Collection just to send a message feel cumbersome.
At the same time, once you have several of such message to send around is convenient to have a Collection named something like settings or similar where you keep these.
Best package I have found is Streamy. It allows you to send to everybody, or just one specific user
https://github.com/YuukanOO/streamy
meteor add yuukan:streamy
Send message to everybody:
Streamy.broadcast('ddpEvent', { data: 'something happened for all' });
Listen for message on client:
// Attach an handler for a specific message
Streamy.on('ddpEvent', function(d, s) {
console.log(d.data);
});
Send message to one user (by id)
var socket = Streamy.socketsForUsers(["nJyQvECmkBSXDZEN2"])._sockets[0]
Streamy.emit('ddpEvent', { data: 'something happened for you' }, socket);