How to decode JeroMQ byte array in PyZMQ? - zeromq

I am trying to hook up a JeroMQ publisher to a PyZMQ subscriber. It works well but I don't know how to decode/deserialize the data I am getting on the Python side.
For example, here is a byte array that I am sending from Java: [10, 10, 7, 55, 79]. My goal would be to recover the same array of integers in Python. In practice, I am getting this here b"\n\n\xf97O" on the Python end. I was hoping that a bytes.decode("utf-8") would maybe get me something like 101075579, but apparently utf-8 is the wrong codec. Do you know what kind of object b"\n\n\xf97O" is?
Here is the code I am using:
Java side
ZContext context = new ZContext();
ZMQ.Socket broadcastSocket = context.createSocket(ZMQ.PUB);
broadcastSocket.bind("tcp://*:55555");
byte[] payload = new byte[] {10, 10, 7, 55, 79};
broadcastSocket.send(payload);
Python side
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket = setsockopt_string(zmq.SUBSCRIBE, "")
socket.connect("tcp://127.0.0.1:55555")
while 1:
message = socket.recv()
print(message) # outputs b"\n\n\xf97O"
Do you have an idea of how to solve this problem? Note that ZMQ.Socket.sendMore(String) sends objects that do get recognized by Python as strings of bytes but I am not sure how to properly parse them.
Thanks in advance.

it's a bytes object, also called a byte-string. If you just convert it to a list, you get a list of integers, one per byte:
>>> list(b"\n\n\xf97O")
[10, 10, 249, 55, 79]
and you can also subscript and iterate over it directly without even using list, e.g. message[4] will be 79.
(I'm not sure about the discrepancy between 7 and 249, but I'm guessing you miscopied something on your end, or used data from two different runs).

Related

vb6 - readfile hang on read serial port?

i use readfile to read data from serial port by api instead of WinComm. it works okay but hang on data receive when other side does not sent data, here are some codes:
'----------on port connect section---------------
With cTimeOuts
.ReadIntervalTimeout = 20
.ReadTotalTimeoutMultiplier = 20
.ReadTotalTimeoutConstant = 20
.WriteTotalTimeoutMultiplier = 20
.WriteTotalTimeoutConstant = 20
End With
rtLng = SetCommTimeouts(hPort, cTimeOuts)
'----------on data receive section---------------
With dOverlapped
.Internal = 0
.InternalHigh = 0
.offset = 0
.OffsetHigh = 0
.hEvent = 0
End With
call ReadFile(hPort, BytesBuffer(0), UBound(BytesBuffer) + 1, dwBytesRead, dOverlapped)
i tried:
a. set ReadIntervalTimeout to MAXDWORD and ReadTotalTimeoutMultiplier & ReadTotalTimeoutConstant both to 0, it can avoid hang on receive data, but the receive data will incomplete for random occurred, so this is not my option.
b. some example code on internet using
Call ReadFile(hPort, BytesBuffer(0), UBound(BytesBuffer) + 1, dwBytesRead, 0) using 0 instead of Overlapped setting, but the program crashed.
so i don't know using what to fix this, another thread? or Overlapped? some actual vb6 code is better cause i'm not familiar with this section, thank you!

How to send an RTS 802.11 packet using Scapy (and get a CTS response)

I'm quite new to Scapy, and I'm trying to craft an RTS packet and send it to an AP, in order to get a CTS response. However, I'm having a really hard time figuring out the proper way to do it (being a beginner in networking and 802.11 packets doesn't help either).
This is the code I have for now:
bytes = struct.pack("<H", 123) # 123 microseconds
timeval = struct.unpack(">H", bytes)[0]
pkt = RadioTap()/Dot11(addr1 = target_addr, addr2 = my_addr, type = 1, subtype = 11, ID = timeval)
I know that type must be equal to 1 since it's a Control packet, and that subtype must be equal to 11 because it's an RTS packet. However, when I send the packet with either sr() or srp() or sr1() I either get no response back (Scapy waits for a response but nothing gets back so it just continues waiting) or I get the exact message I sent.
This question mentions adding a Dot11Elt() layer at the end, however that changes nothing in my case.
This is the type of response I get back:
And if I open the 0th element of the response tuple with Wireshark, I get:
I've hidden the MAC addresses, but they are the sameas those I put in the packet I sent to the AP (target_addr and my_addr). I'm expecting to get back a CTS with my_addr as "destination address".
What am I doing wrong?

pyzmq recv_json can't decode message sent by send_json

Here is my code with the extraneous stuff stripped out:
coordinator.py
context = zmq.Context()
socket = context.socket(zmq.ROUTER)
port = socket.bind_to_random_port(ZMQ_ADDRESS)
poller = zmq.Poller()
poller.register(socket, zmq.POLLIN)
while True:
event = poller.poll(1)
if not event:
continue
process_id, val = socket.recv_json()
worker.py
context = zmq.Context()
socket = context.socket(zmq.DEALER)
socket.connect('%s:%s' % (ZMQ_ADDRESS, kwargs['zmq_port']))
socket.send_json(
(os.getpid(), True)
)
what happens when I run it:
process_id, val = socket.recv_json()
File "/Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/zmq/sugar/socket.py", line 380, in recv_json
return jsonapi.loads(msg)
File "/Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/zmq/utils/jsonapi.py", line 71, in loads
return jsonmod.loads(s, **kwargs)
File "/Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/simplejson/__init__.py", line 451, in loads
return _default_decoder.decode(s)
File "/Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/simplejson/decoder.py", line 406, in decode
obj, end = self.raw_decode(s)
File "/Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/simplejson/decoder.py", line 426, in raw_decode
raise JSONDecodeError("No JSON object could be decoded", s, idx)
JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)
and if I dig in with ipdb:
> /Users/anentropic/.virtualenvs/myproj/lib/python2.7/site-packages/zmq/sugar/socket.py(380)recv_json()
379 msg = self.recv(flags)
--> 380 return jsonapi.loads(msg)
381
ipdb> p msg
'\x00\x9f\xd9\x06\xa2'
hmm, that doesn't look like JSON... is this a bug in pyzmq? am I using it wrong?
Hmm, ok, found the answer.
There is an annoying asymmetry in the ØMQ interface, so you have to be aware of the type of socket you are using.
In this case my use of ROUTER/DEALER architecture means that the JSON message sent from the DEALER socket, when I do send_json, gets wrapped in multipart message envelope. The first part is a client id (I guess this is the '\x00\x9f\xd9\x06\xa2' that I got above) and the second part is the JSON string we are interested in.
So in the last line of my coordinator.py I need to do this instead:
id_, msg = socket.recv_multipart()
process_id, val = json.loads(msg)
IMHO this is bad design on the part of ØMQ/pyzmq, the library should abstract this away and have just send and recv methods, that just work.
I got the clue from this question How can I use send_json with pyzmq PUB SUB so it looks like PUB/SUB architecture has the same issue, and no doubt others too.
This is described in the docs but it's not very clear
http://zguide.zeromq.org/page:all#The-Asynchronous-Client-Server-Pattern
Update
In fact, I found in my case I could simplify the code further, by making use of the 'client id' part of the message envelope directly. So the worker just does:
context = zmq.Context()
socket = context.socket(zmq.DEALER)
socket.identity = str(os.getpid()) # or I could omit this and use ØMQ client id
socket.connect('%s:%s' % (ZMQ_ADDRESS, kwargs['zmq_port']))
socket.send_json(True)
It's also worth noting that when you want to send a message the other direction, from the ROUTER, you have to send it as multipart, specifying which client it is destined for, eg:
coordinator.py
context = zmq.Context()
socket = context.socket(zmq.ROUTER)
port = socket.bind_to_random_port(ZMQ_ADDRESS)
poller = zmq.Poller()
poller.register(socket, zmq.POLLIN)
pids = set()
while True:
event = poller.poll(1)
if not event:
continue
process_id, val = socket.recv_json()
pids.add(process_id)
# need some code in here to decide when to stop listening
# and break the loop
for pid in pids:
socket.send_multipart([pid, 'a string message'])
# ^ do your own json encoding if required
I guess there is probably some ØMQ way of doing a broadcast message rather than sending to each client in a loop as I do above. I wish the docs just had a clear description of each available socket type and how to use them.

Porting TurboPower Blowfish to .Net

I have an application that was originally written in Borland C++ and used a Blowfish algorithm implemented in the TurboPower LockBox component .
This application has now been ported to C#. Currently I call a Borland C++ dll that uses this algorithm. However, when running the application on a 64-bit OS, I get errors whenever attempting to use this dll. If I compile the application as 32-bit, everything works, but we want to have this application work as a 64-bit app. As far as I can tell, that means I need a .Net Blowfish algorithm that works like the C++ one.
I found Blowfish.Net and it looks promising. However, when I use the same key and text the encrypted results do not match. I did find out the C++ dll uses the BlowfishECB algorithm. It also converts the result to Base 64, which I have also done.
Any help with this would be appreciated. Here is some test code in C#.
//Convert the key to a byte array. In C++ the key was 16 bytes long
byte[] _key = new byte[16];
Array.Clear(_key, 0, _key.Length);
var pwdBytes = System.Text.Encoding.Default.GetBytes(LicEncryptKey);
int max = Math.Min(16, pwdBytes.Length);
Array.Copy(pwdBytes, _key, max);
//Convert the string to a byte[] and pad it to to the 8 byte block size
var decrypted = System.Text.Encoding.ASCII.GetBytes(originalString);
var blowfish = new BlowfishECB();
blowfish.Initialize(_key, 0, _key.Length);
int arraySize = decrypted.Length;
int diff = arraySize%BlowfishECB.BLOCK_SIZE;
if (diff != 0)
{
arraySize += (BlowfishECB.BLOCK_SIZE - diff);
}
var decryptedBytes = new Byte[arraySize];
Array.Clear(decryptedBytes, 0, decryptedBytes.Length);
Array.Copy(decrypted, decryptedBytes, decrypted.Length);
//Prepare the byte array for the encrypted string
var encryptedBytes = new Byte[decryptedBytes.Length];
Array.Clear(encryptedBytes, 0, encryptedBytes.Length);
blowfish.Encrypt(decryptedBytes, 0, encryptedBytes, 0, decryptedBytes.Length);
//Convert to Base64
string result = Convert.ToBase64String(encryptedBytes);
It won't compatible with your TurboPower LockBox data.
I'd suggest that you provide a utility to do the data migration by decoding using LockBox in C++ (32-bit), outputting to temp files/tables and re-encoding using Blowfish.Net and C# (64-bit).
This data migration is done once before any upgrade to the .NET version, then it's all compatible with it.
Since you're changing the format: you could also change the format and omit the Base64 conversion by storing binary files/BLOBs, other ideas may also be useful like applying multiple encryptions, or replacing Blowfish by something else.

Writing Byte Arrays to serial with Ruby

I'm not clear on how to write simple byte code arrays with ruby, more-so I'm absolutely stumped on how to use the Ruby SerialPort library, well to be honest I have it working pretty well however I have only been successful in sending ASCII over the serial port.
For example it's really simple to write ASCII:
#sp = SerialPort.new "/dev/tty.usbserial-A6004cNN", 19200
#sp.write "test"
Which obviously writes test to that serial device. This works fine and I've been able to get all the expected results sent to a micro-controller (arduino) in this case. The issue is that I need to write output which the serial device will read like so:
{0x01,0x09,0x04,0x00, 'f',0xff,0xcc,0x33}
I've tried using str.unpack but am still unable to produce the desired hex values output as bytes as above.
In Java it is simple using it's serial library:
byte[] cmd = { 0x01,0x09,0x04,0x00, 'f',(byte)0xff,(byte)0xcc,(byte)0x33 };
serialPort.write( cmd );
How can I output the proper bytecode to my serial device with Ruby?
#sp.write [32.chr, 7.chr, 8.chr, 65.chr].to_s
#sp.write ["\x01\x09\x04\x00", 'f', "\xff\xcc\x33"].to_s
But we can have more fun than that (muhahaha...)
class Array
def chr
self.map { |e| e.chr }
end
end
So then:
>> [1,2,3,65,66,67].chr
=> ["\001", "\002", "\003", "A", "B", "C"]
>> [1,2,3,65,66,67].chr.to_s
=> "\001\002\003ABC"

Resources