I'm programming with sockets in Mac 10.6.8. Whenever I receive a packet, it starts with the IP header. I've been using Wireshark to analyze incoming packets, and I noticed that my machine's socket implementation will consistently change the "total length" field in the IP header. Specifically, it will subtract the IP header length and reverse the bytes (from network to host order).
For example, here's the beginning of an IP header as reported by Wireshark:
45 c0 00 38 ...
That breaks down as follows:
4 bits (0x4): IP version: 4
4 bits (0x5): IP header length: 5 words (20 bytes)
8 bits (0xc0): differentiated services flags
16 bits (0x0038): total length: 56 bytes
However, when I print the contents of the buffer filled by recvfrom for the same packet, I get a different lede:
ssize_t recvbytes = recvfrom(sock->fd, buffer, size, /*flags=*/0,
(struct sockaddr*)src, &src_len);
returns
45 c0 24 00 ...
4 bits (0x4): IP version: 4
4 bits (0x5): IP header length: 5 words (20 bytes)
8 bits (0xc0): differentiated services flags
16 bits (0x2400): total length: 9216 bytes (?!)
I figured out that before I get access to the buffer, the socket implementation is reading the total length, subtracting the IP header length, and then writing it back in host order (little endian on my machine) rather than network order (big endian). In this example, that means:
read the total length: 0x0038 = 56
subtract the header length: 56 - 20 = 36
write back in host order: 36 = 0x0024 (big endian) = 0x2400 (little endian = host order on my machine)
The problem gets worse. It won't just change the total length of the outermost IP header. It will also change the total length fields of internal IP headers, e.g., the one buried in an ICMP "time exceeded" message (which must include the original IP header of the dropped packet). Funnier still, it won't subtract the IP header length from the internal headers; it just reverses the byte order.
Is this happening to anyone else? Is it part of a standard I'm unaware of? Is there a way to fix my machine's socket implementation to stop tampering with packets? How is Wireshark able to get around this problem?
Thanks in advance for your consideration.
EDIT: My code and Makefile are available on GitHub. I wrote a fixip_osx function to allow verifying IP checksums:
https://github.com/thejohnfreeman/netutils/blob/master/lib/ip.c
void fixip_osx(struct ip* ip) {
/* Something on my Mac subtracts the header length from `ip_len` and stores
* it in host order (little endian). */
u16_t ip_hdrlen = ip->ip_hl << 2;
u16_t ip_totlen = ip->ip_len + ip_hdrlen;
ip->ip_len = htons(ip_totlen);
}
However, it's still a problem for verifying ICMP checksums when the payload contains another IP header.
The problem exists whether I compile with Clang 3.2 (built from trunk) or GCC 4.7 (MacPorts port), so I assume the problem lies in either the sockets implementation (packaged with Mac OS) or in Mac OS X itself.
The BSD suite of platforms (excluding OpenBSD) present the IP offset and length in host byte order. All other platforms present in the received network byte order. This is a "feature", and is referenced in the man page for IP(4) - Internet Protocol (FreeBSD, OS X).
The ip_len and ip_off fields must be provided in host byte order .
All other fields must be provided in network byte order.
IP length can equal packet length - IP header length in FreeBSD/NetBSD.
Reference: Stevens/Fenner/Rudolph, Unix Network Programming Vol.1, p.739
I have to deal with these anomalies with a user space implementation of the PGM network protocol, specific code:
https://code.google.com/p/openpgm/source/browse/trunk/openpgm/pgm/packet_parse.c#76
It's actually quite annoying to detect for AutoConf, I think all packages have this hard coded on a per-platform basis. I've seen a bug report (header byte order config options detected incorrectly) raised this week on this very issue.
It is very unlikely that Mac itself is doing that. That would fundamentally break the IP protocol if it were. More likely whatever is capturing the packets and delivering them to recvfrom() (presumably you are doing a promiscuous network capture, right?) is what is transforming the data after Mac is done processing it. ireshark operates on a lower level and has access to the actual network data.
Related
I have one machine with closed protocol and another device "gateway Modbus" from the same manufacter. This gateway convert this protocol to RS-485 Modbus.
When I send a command packet (modbus function 16) to gateway, gateway send (converted) specific packet to the machine and when I inject this packet over simple UART communication, machine can understand and change values too. I create a list with some cloned commands, but I need to know how CRC/checksum/etc is calculed (I think) to create custom packets.
I already used RevEng tool (https://reveng.sourceforge.io/) and CRCcalculator (https://crccalc.com/) trying to find some common crc algorithm with cloned packets, but none worked.
Some cloned packets, where 2 last bytes is CRC/etc. In this packets I changed the temperature value from 0x11 to 0x15 and last 2 bytes changed too (maybe crc/checksum/etc):
9A56F1FE0EB9001100000100641C
9A56F1FE0EB90012000001006720
9A56F1FE0EB90013000001006620
9A56F1FE0EB9001400000100611C
9A56F1FE0EB9001500000100601C
RevEng output:
./reveng -w 16 -l -s 9A56F1FE0EB9001100000100641C 9A56F1FE0EB90012000001006720 9A56F1FE0EB90013000001006620 9A56F1FE0EB9001400000100611C 9A56F1FE0EB9001500000100601C
./reveng: no models found
Someone can help me?
It's not a CRC. The second-to-last byte is the exclusive-or sum of the preceding bytes. I'm not sure what the last byte is, but since it is only taking on two different values in your example, it does not appear to be part of a check value. Or if it is, it's a rather ineffective check value algorithm.
I have an industrial sensor which is connected to a Interface Module that provides information to the end user via telnet.
Telnet Configuration
Port: 10001
Data Format: Little-Endian
Data payload
Preamble - 32 bit
Article number - 32 bit
Serial number - 32 bit
Channels - 64 bit
Status - 32 bit
Frame number M / - 16 bit/ 16 bit
bytes per frame
Measuring value counter - 32 bit
Measuring value frame 1 - N * 32 bit
..
..
This should be available after the TCP headers (if I understand correctly) i.e. Payload.
I am trying to learn golang with more practical examples and I am thinking in the following direction:
Create a TCP server for listening to the sensor's Interface module IP and port number
Extract the payload and just print it out as seperate parameters like preamble, article number etc. as a beginners task
I have gone through some blog-posts about TCP server/client creations but they only address string in their payload. How can I extract such a byte orientated custom payload as mentioned above?
P.S.
I can understand that this can be achieved in python probably via struct.pack and struct.unpack hence some correlation as understanding would be appreciated
In section 6.2 of your linked pdf document above there is a table listing the data you seem to list in the question
I would convert that table into a struct and then read into it with the encoding/binary module. See https://golang.org/pkg/encoding/binary/#example_Read for a relevant example
"All measuring values are transmitted as Int32, Uint32 or Float depending on the connected sensor" probably easiest to get them as uint32 and then convert them later
I've found what code parses IP (v4) packets in the kernel source tree. This function, ip_rcv, can to a high degree of certainty detect whether a packet is correct or not, as is outlined in one of the comments:
Length at least the size of an ip header
Version of 4
Checksums correctly. [Speed optimisation for later, skip loopback checksums]
Doesn't have a bogus length
Malformed packets are simply dropped. This function seems to get a bunch of bytes that should resemble an IP packet, but what if some malicious actor would sneak an extra byte on the line? If not handled correctly, all the chunks of bytes that ip_rcv receives from now on will start 1 byte off and no correct IP packet can be reconstructed anymore. I assume the kernel does something smarter than to try all different byte offsets at which to start parsing an IP packet. What exactly, I'm unable to find. Could anyone shed some light on this?
I haven't taken the time to look at the kernel code but most protocol stacks are going to work by parsing data immediately following the previous stack location and not by searching for data.
In the case of Ethernet, an Ethernet frame header is typically 14 bytes in size. It can vary but the header itself indicates the different length in the etherType field when necessary. In this example, the NIC (Network Interface Card) will receive an Ethernet frame. If the frame is destined for this NIC then the data passed from the NIC driver to the IP stack will be an Ethernet frame containing this 14-byte header followed immediately by the IP header (first nibble will be 4 if it is a version 4 IP header for instance).
Again, I didn't look at the network stack code but there are two common cases here:
1) The IP stack is told this is an Ethernet frame and only needs to parse the Ethernet frame header for its length and the very next byte must be an IP header or the data is deemed not an IP frame.
2) The IP stack is given a pointer to the beginning of the data immediately following the Ethernet frame header and the IP stack then starts parsing at that location.
Recently I have been trying to save list of hostnames from captured DHCP packets. I have found out, every DHCP hostname (option 12) should have form defined in RFC 1035. So if I understand it correctly, hostname should be encoded in 7-bit ASCII and have other restrictions like:
- name should not start with digit and should omit some forbidden characters.
Almost every device I have encountered in packets fulfill this constraint, but not Windows devices (Vendor ID MSFT 5.0). IMHO Windows DHCP client takes computer (mobile) name and fill it in hostname option.
Problem occurs, when computer name is set for example to "Lukáš-PC". Wireshark display this hostname as Luk\240\347-PC. (240 and 347 are numbers in octal). To see for myself I have printed values in packets with printf("%hhu", c) (C language).
á = 160
š = 231
IMHO I think this is simple char variable overflow. I tried deduce original value from overflow value, but I haven't found any relation between character and known encodings. So my questions are:
Is there any way to convert these values back to original?
If yes, what was original character encoding, when overflow happened?
Thanks.
Default char is usually signed, and extends to int when passed to a variadic function. To ensure that it is printed unsigned, use printf("%hhu", c) or printf("%d", (unsigned char)c);.
The correct encoding is impossible to know because it depends on each system's settings.
Note that any compliant systems MUST encode names according to RFC 3490, but Windows seems to enjoy violating standards.
The characters á and š that you are seing are encoded using code page 852 (Latin-2 - Central European languages).
Unfortunately there is no simple way how you can figure out the encoding used only by looking at DHCP requests. In principle the DHCP client can use any code page it wants. If you are working in a private/controlled network, then it is probably safe to assume the all clients are using the same code page and explicitly encode the strings using that particular code page.
I am debugging some snmp code for an integer overflow problem. Basically we use an integer to store disk/raid capacity in KB. However when a disk/raid of more than 2TB is used, it'll overflow.
I read from some internet forums that snmp v2c support integer64 or unsigned64. In my test it'll still just send the lower 32 bits even though I have set the type to integer64 or unsigned64.
Here is how I did it:
a standalone program will obtain the capacity and write the data to a file. example lines for raid capacity
my-sub-oid
Counter64
7813857280
/etc/snmp/snmpd.conf has a clause to pass thru the oids:
pass_persist mymiboid /path/to/snmpagent
in the mysnmpagent source, read the oidmap into oid/type/value structure from the file, and print to stdout.
printf("%s\n", it->first.c_str());
printf("%s\n", it->second.type.c_str());
printf("%s\n", it->second.value.c_str());
fflush(stdout);
use snmpget to get the sub-oid, and it returns:
mysuboid = Counter32: 3518889984
I use tcpdump and the last segment of the value portion is:
41 0500 d1be 0000
41 should be the tag, 05 should be the length, and the value is only carrying the lower 32-bit of the capacity. (note 7813857280 is 0x1.d1.be.00.00)
I do find that using string type would send correct value (in octetstring format). But I want to know if there is a way to use 64-bit integer in snmp v2c.
I am running NET-SNMP 5.4.2.1 though.
thanks a lot.
Update:
Found the following from snmpd.conf regarding pass (and probably also pass_persist) in net-snmp doc page. I guess it's forcing the Counter64 to Counter32.
Note:
The SMIv2 type counter64 and SNMPv2 noSuchObject exception are not supported.
You are supposed to use two Unsigned32 for lower and upper bytes of your large number.
Counter64 is not meant to be used for large numbers this way.
For reference : 17 Common MIB Design Errors (last one)
SNMP SMIv2 defines a new type Counter64,
https://www.rfc-editor.org/rfc/rfc2578#page-24
which is in fact unsigned 64 bit integer. So if your data fall into the range, using Counter64 is proper.
"In my test it'll still just send the lower 32 bits even though I have set the type to integer64 or unsigned64" sounds like a problem, but unless you show more details (like showing some code) on how you tested it out and received the result, nobody might help further.