I have a Ruby TCPServer that does not recieve TCP packets intermittently. Note the below is run inside another threads << Thread.new do as I have multiple TCP post listeners.
server = TCPServer.new(7207)
Thread.start(server.accept) do |client|
# process packet, send to AWS SQS
raw = ""
while (line = client.gets)
raw += line
end
sender = client.peeraddr
text = raw.unpack1("H*")
message_body = { payload: text, rx_at: Time.current, sender:}
puts "#{Time.current} : --- New uplink #{message_body}"
# send message to AWS SQS
client.close
end
I see the packets in tcpdump / wireshark but not on my TCPServer. I have a pcap file available:
https://www.dropbox.com/s/7m3hr1b7065tenx/tcp.pcap?dl=0
Example lost packets occurred on:
ip 10.0.225.43 at 27/07/2022 20:56:57 and 27/07/2022 20:39:31
Apologies after checking Wireshark this looks like a network problem not a Ruby / dev problem. The device was not sending FIN frames so the network stack was not publishing the packet up to my app.
Related
I want to use a NodeMCU device (Lua based top level) to act as a websocket server to 1 or more browser clients.
Luckily, there is code to do this here: NodeMCU Websocket Server
(courtesy of #creationix and/or #moononournation)
This works as described and I am able to send a message from the client to the NodeMCU server, which then responds based on the received message. Great.
My questions are:
How can I send messages to the client without it having to be sent as a response to a client request (standalone sending of data)? When I try to call socket.send() socket is not found as a variable, which I understand, but cannot work out how to do it! :(
Why does the decode() function output the extra variable? What is this for? I'm assuming it will be for packet overflow, but I can never seem to make it return anything, regardless of my message length.
In the listen method, why has the author added a queuing system? is this essential or for applications that perhaps may receive multiple simultaneous messages? Ideally, I'd like to remove it.
I have simplified the code as below:
(excluding the decode() and encode() functions - please see the link above for the full script)
net.createServer(net.TCP):listen(80, function(conn)
local buffer = false
local socket = {}
local queue = {}
local waiting = false
local function onSend()
if queue[1] then
local data = table.remove(queue, 1)
return conn:send(data, onSend)
end
waiting = false
end
function socket.send(...)
local data = encode(...)
if not waiting then
waiting = true
conn:send(data, onSend)
else
queue[#queue + 1] = data
end
end
conn:on("receive", function(_, chunk)
if buffer then
buffer = buffer .. chunk
while true do
local extra, payload, opcode = decode(buffer)
if opcode==8 then
print("Websocket client disconnected")
end
--print(type(extra), payload, opcode)
if not extra then return end
buffer = extra
socket.onmessage(payload, opcode)
end
end
local _, e, method = string.find(chunk, "([A-Z]+) /[^\r]* HTTP/%d%.%d\r\n")
local key, name, value
for name, value in string.gmatch(chunk, "([^ ]+): *([^\r]+)\r\n") do
if string.lower(name) == "sec-websocket-key" then
key = value
break
end
end
if method == "GET" and key then
acceptkey=crypto.toBase64(crypto.hash("sha1", key.."258EAFA5-E914-47DA-95CA-C5AB0DC85B11"))
conn:send(
"HTTP/1.1 101 Switching Protocols\r\n"..
"Upgrade: websocket\r\nConnection: Upgrade\r\n"..
"Sec-WebSocket-Accept: "..acceptkey.."\r\n\r\n",
function ()
print("New websocket client connected")
function socket.onmessage(payload,opcode)
socket.send("GOT YOUR DATA", 1)
print("PAYLOAD = "..payload)
--print("OPCODE = "..opcode)
end
end)
buffer = ""
else
conn:send(
"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 12\r\n\r\nHello World!",
conn.close)
end
end)
end)
I can only answer 1 question, the others may be better suited for the library author. Besides, SO is a format where you ask 1 question normally.
How can I send messages to the client without it having to be sent as a response to a client request (standalone sending of data)?
You can't. Without the client contacting the server first and establishing a socket connection the server wouldn't know where to send the messages to. Even with SSE (server-sent events) it's the client that first initiates a connection to the server.
I need help with Crystal Lang websockets, I want to know how to upgrade my connection on websocket. I want make simple websocket server
hope this help
require "http/server"
SOCKETS = [] of HTTP::WebSocket
ws_handler = HTTP::WebSocketHandler.new do |socket|
puts "Socket opened"
SOCKETS << socket
socket.on_message do |message|
SOCKETS.each { |socket| socket.send "Echo back from server: #{message}" }
end
socket.on_close do
puts "Socket closed"
end
end
server = HTTP::Server.new([ws_handler])
address = server.bind_tcp "0.0.0.0", 3000
puts "Listening on http://#{address}"
server.listen
https://medium.com/#muhammadtriwibowo/simple-websocket-using-crystal-13b6f67eba61
if you are looking for ready to use something, then you can use Shivneri framework created by me - which provides a javascript library and MVC based approach to create a socket server.
How to create a web socket end point
class ChatController < Shivneri::WebSocketController
#[On("message")]
def receive_message(data : String)
# send message to caller
clients.current.emit("message", "Received message is #{data}")
# send message to all clients
clients.emit("message", "Someone sent message #{data}")
end
end
How to connect to web socket end point using javascript
Shivneri framework provides a javascript library shivneri-ws-client-javascript to help you create a real time web application
var socket = new shivneriWsClient.Instance();
socket.on("message", function(data){
console.log("data", data);
});
await socket.init(`<web-socket-url>`);
// emit event to server
socket.emit("message","Successfully connected")
It provides many functionalities like grouping of clients, events when client is connected & disconnected etc.
For more information, take a look at shivneri websocket doc - https://shivneriforcrystal.com/tutorial/websocket/
I am trying to gather information from a TCP connection made to my web server to assist in our troubleshooting efforts, similar to http://speedguide.net/analyzer.php tool.
We have a simple PHP server script test page that users connect that returns their private IP to an AJAX call waiting for the response.
I would like to either build on that or prefer using Ruby. I played with the PacketFu lib and get all the information I think I need, however, I'm having trouble with the recipe to combine it all:
listen on port x,
parse the packet
respond back to client.
Using Ruby's TCPServer I can easily handle 1 and 3. With Packetfu, 2.
I've coded with PHP in the past but only HTML-based, no sockets. And I'm not really all that familiar with Ruby sockets either.
Though the packet stream and client.accept don't seem to play nice. The packets aren't always IP or TCP meeting the Packetfu::Packet.is_ip? or is_tcp?.
Could someone point me in the right direction or give me some practical example of how I might combine the two, or adjust my thinking on how I would accomplish this task?
This is the Playground code:
require 'socket'
require 'json'
require 'packetfu'
iface = ARGV[0] || "eno1"
server = TCPServer.open(31337)
cap = PacketFu::Capture.new(:iface => iface, :start => true, :promisc => true)
loop {
cap.stream.each do |p|
pkt = PacketFu::Packet.parse(p)
if pkt.is_ip? || pkt.is_tcp?
if pkt.tcp_dport == 31337
print "Source Addr: #{pkt.ip_saddr}\n"
print "Source Port: #{pkt.tcp_src}\n"
print "Destination Addr: #{pkt.ip_daddr}\n"
print "Destination Port: #{pkt.tcp_dport}\n"
print "TCP Options: #{pkt.tcp_options.inspect}\n"
print "TCP Win: #{pkt.tcp_win}\n"
print "TCP SYN?: #{pkt.tcp_flags.syn}\n"
print "TCP ACK?: #{pkt.tcp_flags.ack}\n"
print "TCP FLAGS ALL: #{pkt.tcp_flags.inspect}\n"
print "TTL: #{pkt.ip_ttl}\n"
print "IP FRAG: #{pkt.ip_frag}\n"
end
end
client = server.accept # Wait for a client to connect
h = { ipaddress: client.peeraddr[2] }
client.puts h.to_json
client.close
end
}
This is the output:
Source Addr: 172.20.0.15
Source Port: 41165
Destination Addr: 172.20.0.10
Destination Port: 31337
TCP Options: "NOP,NOP,TS:216432150;57946250"
TCP Win: 229
TCP SYN?: 0
TCP ACK?: 1
TCP FLAGS ALL: #<struct PacketFu::TcpFlags urg=0, ack=1, psh=0, rst=0, syn=0, fin=0>
TTL: 61
IP FRAG: 16384
This is the browser response:
{"ipaddress":"172.20.0.15"}
I have a UDP server that binds to all addresses on a system, I would like to know what ip address the message was addressed to. Any ideas how to do this?
Here is my example code:
sock = Socket.new(Socket::AF_INET, Socket::SOCK_DGRAM, 0)
sock.bind(Addrinfo.udp('', 2400))
while(true)
sockset = IO.select([sock])
sockset[0].each do |sock|
data = sock.recvfrom(1024)
puts "data: " + data.inspect
end
end
sock.close
This will produce something like:
data: ["test message\n", #<Addrinfo: 172.16.5.110:41949 UDP>]
Am I able to set a socket option, or something, to return the local IP?
Just a note, this needs to work for IPv6 too. Thanks in advance, Dave.
UNIX Network Programming has this to say about this very subject:
With a UDP socket, however, the destination IP address can only be obtained by setting the IP_RECVDSTADDR socket option for IPv4 or the IPV6_PKTINFO socket option for IPv6 and then calling recvmsg instead of recvfrom.
Ruby’s socket library has recvmsg which is a bit easier to use than the underlying C function, but still needs a bit of work to get the info needed. The destination IP address is included in the array of ancillary data returned from recvmsg. Here’s a version of your code adapted to use recvmsg and get the destination address for IPv4:
require 'socket'
require 'ipaddr'
sock = Socket.new(Socket::AF_INET, Socket::SOCK_DGRAM, 0)
# Set the required socket option:
sock.setsockopt :IPPROTO_IP, :IP_RECVDSTADDR, true
sock.bind(Addrinfo.udp('0.0.0.0', 2400))
while(true)
sockset = IO.select([sock])
sockset[0].each do |sock|
mesg, sender, _, *anc_data = sock.recvmsg
# Find the relevant data and extract it into a string
dest = IPAddr.ntop(anc_data.find {|d| d.cmsg_is?(:IP, :RECVDSTADDR)}.data)
puts "Data: #{mesg}, Sender: #{sender.ip_address}, Destination: #{dest}"
end
end
And here is a version for IPv6. There is also a RECVPKTINFO socket option, which I think may have superseded PKTINFO – depending on your system you may need to use that instead.
require 'socket'
sock = Socket.new(Socket::AF_INET6, Socket::SOCK_DGRAM, 0)
# Set the socket option for IP6:
sock.setsockopt :IPPROTO_IPV6, :IPV6_PKTINFO, true
sock.bind(Addrinfo.udp('0::0', 2400))
while(true)
sockset = IO.select([sock])
sockset[0].each do |sock|
mesg, sender, _, *anc_data = sock.recvmsg
# Find and extract the destination address
dest = anc_data.find {|d| d.cmsg_is?(:IPV6, :PKTINFO)}.ipv6_pktinfo_addr
puts "Data: #{mesg}, Sender: #{sender.ip_address}, Destination: #{dest.ip_address}"
end
end
Ruby also provides a Socket.udp_server_loop method, which yields the message and a UDPSource object to the block you provide, and this source object has a local_address field. Looking at the source this appears to check the PKTINFO data like I do above to get the destination address for IPv6 requests, but not for IPv4. This method binds to all available IP addresses individually, and just uses the address of the incoming interface for IP4 requests, which may not be accurate for a weak end system model. However it might be simpler for you to use Socket.udp_server_loop.
Is there a way I can listen to more than one port on UDPServer in Ruby?
I need to listen to some devices that send me data over multiple ports, and I need to receive them and use the same operations on the data.
I'm right now using:
udp_socket = UDPSocket.new
udp_socket.bind( "", 8500 )
while($running) do
payload, host = udp_socket.recvfrom(1500)
process payload
end
but the code stops on:
payload, host = udp_socket.recvfrom(1500)
and I cannot run more code until it receives data.
You could launch multiple threads.
require 'thread'
ports = [1500, 1501, 1502]
# Create threads to listen to each port
threads = ports.map do |port|
Thread.new do
while($running) do
payload, host = udp_socket.recvfrom(port)
process payload
end
end
end
# Wait for all threads to complete
threads.map &:join