Win32 Raw Ethernet Programming / NPCap - winapi

I need a good Win32 VC++ API for Raw Ethernet Programming.
I have written an application in VC++ win32 to communicate to a microcontroller having Gigabit Ethernet. Both windows side and microcontroller side software is written by me. The application is a CNC controller, so I need a high-speed, low-latency communication interface at a distance of 10 metres with superior noise immunity capabilities, and Ethernet just fits. Just to be clear, there is no TCP/IP layer, and the communication works on raw Ethernet II frames for low latency constraints.
Currently, I am using NPCap, and everything works OK, but there are some problems. First to use it from windows, NPCap drivers need to be installed to make it work on every PC. Second, the NPCap API seems underpowered for my needs. I don't know if I am using it correctly or not, but the API itself does not respond to high-performance IO requests. after writing some benchmarking tests. In a transmit test, the NPCap API managed a modest 3.2 Mbps with a packet size of 76 bytes. The receive test was not so accurate but was in the ballpark of 56 kbps speed, which just seems so wrong. Having said that, the latency is very good. When I wrote such a receive and transmit test on the microcontroller side, it went to a whooping 700 Mbps range, and my Windows task manager just lit up like Christmas.
The WireShark application, which I think is built on NPCap, does not feel so slow and seems to be able to capture all frames no matter what speed one throws at it. I don't know if I'm doing something wrong with NPCap, but here is the code for reference:
#include <pcap.h>
#pragma comment (lib, "Packet.lib")
#pragma comment (lib, "wpcap.lib")
#include <tchar.h>
BOOL LoadNpcapDlls()
{
_TCHAR npcap_dir[512];
UINT len;
len = GetSystemDirectory(npcap_dir, 480);
if (!len) {
fprintf(stderr, "Error in GetSystemDirectory: %x", GetLastError());
return FALSE;
}
_tcscat_s(npcap_dir, 512, _T("\\Npcap"));
if (SetDllDirectory(npcap_dir) == 0) {
fprintf(stderr, "Error in SetDllDirectory: %x", GetLastError());
return FALSE;
}
return TRUE;
}
struct Ethernet
{
pcap_t* handle;
int8 Open()
{
char errbuf[PCAP_ERRBUF_SIZE];
pcap_if_t* networkdevice;
if (pcap_findalldevs_ex(PCAP_SRC_IF_STRING, NULL, &networkdevice, errbuf) == -1)
return -1;
while (networkdevice != NULL)
{
String networkdevicedescription = networkdevice->description;
String networkdevicetargettplink = "Network adapter 'TP-LINK Gigabit Ethernet USB Adapter' on local host";
String networkdevicetargerealtek = "Network adapter 'Realtek Gaming GbE Family Controller' on local host";
if (networkdevicedescription == networkdevicetargettplink)
{
if (LoadNpcapDlls() == 0)
return -3;
handle = pcap_open_live(networkdevice->name, 65535, PCAP_OPENFLAG_PROMISCUOUS, 1, errbuf);
if (handle == 0)
return -4;
pcap_freealldevs(networkdevice);
return 0;
}if (networkdevicedescription == networkdevicetargerealtek)
{
if (LoadNpcapDlls() == 0)
return -3;
handle = pcap_open_live(networkdevice->name, 65535, PCAP_OPENFLAG_PROMISCUOUS, 1, errbuf);
if (handle == 0)
return -4;
pcap_freealldevs(networkdevice);
return 0;
}
networkdevice = networkdevice->next;
}
return -2;
}
char* Send(void* _data, uint32 lenght)
{
if (pcap_sendpacket(handle, (const uint8*)_data, lenght))
return pcap_geterr(handle);
return 0;
}
void* Recieve()
{
struct pcap_pkthdr* header = 0;
const u_char* data = 0;
pcap_next_ex(handle, &header, &data);
return (void*)data;
}
void* Recieve(uint32 *bytes)
{
struct pcap_pkthdr* header = 0;
const u_char* data = 0;
pcap_next_ex(handle, &header, &data);
*bytes = header->len;
return (void*)data;
}
Ethernet()
{
ZeroMemory(this, sizeof(Ethernet));
}
~Ethernet()
{
ZeroMemory(this, sizeof(Ethernet));
}
};
Honestly I dont like this NPCap thing. Isn't there a native Win32 API like the Win32 socket programming API for raw Ethernet frames?

Related

ArduinoHTTPClient websocket undocumented max message size?

I have an Arduino MKRNB 1500 (with LTE-M network capability).
My code uses a websocket to upload messages to a server. The messages are uploaded every second or so of around 800-1000 bytes. My websocket server accepts these messages (i have tried with a browser client). But the ArduinoHTTPClient library WebSocketClient refuses to send messages that are over 128 bytes. The Arduino just hangs from that point onwards.
Because of the network latency, this means i cannot send more than around 600 bytes a second.
This limitation seems arbitrary, and is not documented as far as i have seen. It can be easily reproduced using the following code. Sending smaller messages more frequently is not an option because the network latency of LTE-M is around 150ms.
How can i send larger messages?
#include <ArduinoHttpClient.h>
#include <MKRNB.h> //For LTE-M or NB-IOT connections
#include "arduino_secrets.h"
// initialize the LTE-M library instance
NBClient nbClient;
GPRS gprs;
NB nbAccess;
char server[] = "echo.websocket.org"; // server address
const char PINNUMBER[] = "0000"; // = SIM SECRET_PINNUMBER;
int port = 80; // port 80 is the default for HTTP
WebSocketClient client = WebSocketClient(nbClient, server, port);
int count = 120;
void setup() {
Serial.begin(9600);
// LTE-M connection
Serial.println(F("Connecting to LTE-M network"));
boolean connected = false;
while (!connected) {
if ((nbAccess.begin(PINNUMBER) == NB_READY) &&
(gprs.attachGPRS() == GPRS_READY)) {
connected = true;
} else {
Serial.println("Not connected");
delay(1000);
}
}
}
void loop() {
Serial.println("starting WebSocket client");
client.begin();
while (client.connected() and count <= 1000) {
Serial.print("Sending hello ");
Serial.println(count);
// send a hello #
client.beginMessage(TYPE_TEXT);
client.print(count);
client.print(": ");
int i = 0;
while (i<= count){
client.print("X");
i++;
}
client.endMessage();
// increment count for next message
count++;
// check if a message is available to be received
int messageSize = client.parseMessage();
if (messageSize > 0) {
Serial.println("Received a message:");
Serial.println(client.readString());
}
delay(1000);
}
Serial.println("disconnected");
}
After examining the ArduinoHttpClient library, it turns out the WebSocketClient.h file has a limited buffer defined on line 89:
uint8_t iTxBuffer[128];
i changed that to
uint8_t iTxBuffer[4096];
And the problem was resolved.
custom size must be supported soon.
https://github.com/arduino-libraries/ArduinoHttpClient/issues/68

NKE. Can't handle file uploads and other high load connections

I got a problem when using a kernel extension that filters network traffic.
My code was written according to Apple's tcplognke example.
Everything goes OK but when I attempt to upload a file bigger than 500 kb - connection drops.
Here is simplified kext code:
errno_t tl_data_fn(void *cookie, socket_t so, const struct sockaddr *addr, mbuf_t *data, mbuf_t *control, sflt_data_flag_t flags, FilterSocketDataDirection direction) {
errno_t result = 0;
if (check_tag(data, gidtag, FILTER_TAG_TYPE, direction == FilterSocketDataDirectionIn ? IN_DONE : OUT_DONE)) {
return result;
}
if (!cookie) return result;
filter_cookie *f_cookie = get_filter_cookie(cookie);
uint32_t data_size = (uint32_t)mbuf_pkthdr_len(*data);
uint32_t offset = 0;
printf("tl_data_ft: %d", data_size);
while (offset < data_size) {
FilterNotification notification;
if (direction == FilterSocketDataDirectionIn) {
notification.event = FilterEventDataIn;
} else {
notification.event = FilterEventDataOut;
}
notification.socketId = (uint64_t)so;
notification.inputoutput.dataSize = min(data_size - offset, sizeof(notification.inputoutput.data));
mbuf_copydata(*data, offset, notification.inputoutput.dataSize, notification.inputoutput.data);
offset += notification.inputoutput.dataSize;
send_notification(f_cookie, &notification);
}
result = EJUSTRETURN;
if (result == EJUSTRETURN) {
mbuf_freem(*data);
if (control != NULL && *control != NULL)
mbuf_freem(*control);
}
return result;
}
errno_t tl_data_in_fn(void *cookie, socket_t so, const struct sockaddr *from, mbuf_t *data, mbuf_t *control, sflt_data_flag_t flags) {
return tl_data_fn(cookie, so, from, data, control, flags, FilterSocketDataDirectionIn);
}
errno_t tl_data_out_fn(void *cookie, socket_t so, const struct sockaddr *to, mbuf_t *data, mbuf_t *control, sflt_data_flag_t flags) {
return tl_data_fn(cookie, so, to, data, control, flags, FilterSocketDataDirectionOut);
}
And the user space code:
int s = socket(PF_SYSTEM, SOCK_DGRAM, SYSPROTO_CONTROL);
//connect to driver
FilterNotification notification;
while (recv(s, &notification, sizeof(FilterNotification), 0) == sizeof(FilterNotification)) {
FilterClientResponse response;
response.socketId = notification.socketId;
response.direction = (notification.event == FilterEventDataIn) ? FilterSocketDataDirectionIn : FilterSocketDataDirectionOut;
response.dataSize = notification.inputoutput.dataSize;
memcpy(response.data, notification.inputoutput.data, notification.inputoutput.dataSize);
send(s, &response, sizeof(response), 0);
}
When I asked on apple develper forum, the developer said "I don’t see any attempt to handle send-side flow control here. Without that a file upload can easily eat up all of the available mbufs, and things will go badly from there" but there is not examples at all. Can someone help me? thanks.
The problem was in socket buffer. When I inject a lot of data very quickly, the buffer becomes full and inject_data_in/inject_data_out functions returns error.
The workaround is to store pending packets in kernel space (You can use TAILQ for example) and then, when socket becomes available for writing (To get this event you can use kqueue on OS X) continue injection

How can I to detect if keyboard is attached?

I have a MFC Application that sometime work on touch screen
I want to open my virtual keyboard when focused edit box if keyboard isn't attached
Here's a quick attempt at using GetRawInputDeviceList to check for presence of a RIM_TYPEKEYBOARD device. I've skimped on the error handling though but hopefully it's obvious where it needs it. This does detect me removing my USB keyboard.
bool detectKeyboard() {
bool bHasKeyboard = false;
UINT nDevices;
UINT ret = GetRawInputDeviceList(NULL, &nDevices, sizeof(RAWINPUTDEVICELIST));
if (ret == 0) {
PRAWINPUTDEVICELIST pRawInputDeviceList = new RAWINPUTDEVICELIST[nDevices];
ret = GetRawInputDeviceList(pRawInputDeviceList, &nDevices,
sizeof(RAWINPUTDEVICELIST));
if (ret != (UINT)-1) {
for (UINT i = 0; i < nDevices; ++i) {
if (pRawInputDeviceList[i].dwType == RIM_TYPEKEYBOARD) {
bHasKeyboard = true;
break;
}
}
}
// else error calling GetRawInputDeviceList to fetch the devices list
delete[] pRawInputDeviceList;
}
// else error calling GetRawInputDeviceList to return number of devices
return bHasKeyboard;
}
(And it's been a while since I wrote C++ or MFC so apologies for any poor style.)

send and receive via Serial port Windows

I have a application running on windows, which will send data over serial port.
Here is the code:
m_hCommPort= ::CreateFile(L"\\\\.\\COM3",
GENERIC_READ|GENERIC_WRITE, 0, 0, OPEN_EXISTING,0,0);
if(m_hCommPort == INVALID_HANDLE_VALUE)
{
printf("COM error: %d\n", GetLastError());
}
config.DCBlength = sizeof(config);
if((GetCommState(m_hCommPort, &config) == 0))
{
CloseHandle(m_hCommPort);
printf("Get configuration port has a problem.\n");
return FALSE;
}
config.BaudRate = 9600;
config.StopBits = ONESTOPBIT;
config.Parity = PARITY_NONE;
config.ByteSize = DATABITS_8;
config.fDtrControl = 0;
config.fRtsControl = 0;
if (!SetCommState(m_hCommPort, &config))
{
CloseHandle(m_hCommPort);
printf( "Failed to Set Comm State Reason: %d\n",GetLastError());
return E_FAIL;
}
Here is the code for Send only (Working) (continuously sending )
while(1)
{
Sleep(5000);
int isWritten = WriteFile(m_hCommPort, txData, 9/*(DWORD)sizeof(txData)*/, &dwBytesWritten, NULL);
printf("isWritten: %d, dwBytesWritten: %d \n", isWritten, dwBytesWritten);
}
After this I have added code for Receive data too, then send is NOT WORKING. I mean not able to send data over UART. WriteFile() seems not executed, its stuck.
Here I have added a thread to receive data, is thread causing the problem ? or do I need to do something else ?
void ReceiverThread(void *param)
{
DWORD dwRead=0;
BOOL fWaitingOnRead = FALSE;
OVERLAPPED osReader = {0};
osReader.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
if (osReader.hEvent == NULL)
printf("Error creating overlapped event; abort.\n");
while(1)
{
if (!ReadFile(m_hCommPort, &Byte, 1, &dwRead, &osReader)) {
if (GetLastError() != ERROR_IO_PENDING) // read not delayed?
printf("Error in communications; report it.\n");
else
fWaitingOnRead = TRUE;
}
else {
rxData[rxHeadIndex++]= Byte;
rxHeadIndex = (rxHeadIndex) & QUEUE_MASK;
}
}
}
SetCommMask (m_hCommPort, EV_RXCHAR/ | EV_ERR); //receive character event
_beginthread(ReceiverThread,0,NULL);
while(1)
{
Sleep(5000);
int isWritten = WriteFile(m_hCommPort, txData, 9/*(DWORD)sizeof(txData)*/, &dwBytesWritten, NULL);
printf("isWritten: %d, dwBytesWritten: %d \n", isWritten, dwBytesWritten);
}
Thanks in advance.
Ashok
I've encountered a similar problem a while ago.
I found out that WriteFile(..) blocks if a ReadFile(..) is currently in progress for a serial port. So that's a problem if ReadFile(..) blocks if there is no data to be read.
I've solved the problem by checking if there is data available within the serial buffer to be read by using the function ClearCommError(..). This ensures that ReadFile(..) can read something immediately and does not unnecessarily block the device. Try changing your ReceiverThread into something like this:
void ReceiverThread(void *param)
{
DWORD dwRead=0;
COMSTAT comStat;
char buffer[1024];
while (m_hCommPort != INVALID_HANDLE_VALUE)
{
if ((ClearCommError(m_hCommPort, NULL, &comStat) != FALSE) && (comStat.cbInQue > 0))
{
/* todo: ensure buffer is big enough to contain comStat.cbInQue bytes! */
if (ReadFile(m_hCommPort, buffer, comStat.cbInQue, &dwRead, NULL) != FALSE)
{
/* do something with data in buffer */
}
}
/* avoid busy-wait */
if (comStat.cbInQue == 0)
{
SleepEx(1, FALSE);
}
}
}
This way ReadFile(..) is only called if data is available and meanwhile WriteFile(..) can send data without being blocked.
Unfortunately I've not been able to make ClearCommError(..) blocking so I used the SleepEx(1, FALSE); work-around to avoid a busy-wait and therefore prefenting the ReceiverThread to eat up the CPU.
config.fDtrControl = 0;
config.fRtsControl = 0;
These settings turn the DTR and RTS handshake lines off. Most serial devices pay attention to these signals. They won't send anything when your DTR signal is off, assuming that the machine is not powered up. And won't send anything when your RTS signal is off, assuming that the machine is not ready to receive any data.
So what you observed is entirely normal.
Since the device appears to be "normal" and does pay attention to the handshake lines, you'll want to configure the DCB to let the device driver automatically control these signals. Fix:
config.fDtrControl = DTR_CONTROL_ENABLE;
config.fRtsControl = RTS_CONTROL_HANDSHAKE;
Also the default for terminal emulators like Putty and HyperTerminal. Use such a program first to ensure that the wiring and device are functional. If you can't get any device data to show up in such a program then it won't work with your program either. If this all checks out then also set the fDsrSensitivity, fOutxCtsFlow and fOutxDsrFlow properties to TRUE so that you will, in turn, pay attention to the handshake signals of the device.

Linux to WinXP over UDP lag

I'm having a problem with a UDPSocket wrapper I've written. I have a high bandwidth low latency local network over which I'm sending UDP packets back and forth. I don't have a concern too much for reliability of packet arrival, but it is incredibly important that the packets that do arrive do so quickly. Here's the relevant code for setting up a socket:
bool UDPSocket::create() {
int on = 1;
#ifdef WIN32
if(WSAStartup(MAKEWORD(1,1), &SocketInfo) != 0) {
MessageBox(NULL, "Cannot initialize WinSock", "WSAStartup", MB_OK);
}
#endif
m_sock = socket(PF_INET, SOCK_DGRAM, 0);
#ifdef WIN32
if(setsockopt(m_sock, SOL_SOCKET, SO_REUSEADDR, (const char*)&on, sizeof(on)) == SOCKET_ERROR)
return false;
#else
if(setsockopt(m_sock, SOL_SOCKET, SO_REUSEADDR, (const char*)&on, sizeof(on)) == -1)
return false;
#endif
addrLen = sizeof(struct sockaddr);
return true;
}
bool UDPSocket::bind(const int port) {
if(!is_valid())
return false;
m_addr.sin_family = AF_INET;
m_addr.sin_addr.s_addr = htonl(INADDR_ANY);
m_addr.sin_port = htons(port);
if(::bind(m_sock, (struct sockaddr*)&m_addr, sizeof(m_addr))<0) {
std::cout << "UDPSocket: error on bind" << std::endl;
return false;
}
return true;
}
bool UDPSocket::send(const std::string s) const {
const char* buf = s.c_str();
::sendto(m_sock, buf, strlen(buf), 0, (const sockaddr*)&clientAddr, addrLen);
return true;
}
bool UDPSocket::setDestination(const std::string ip, const int port) {
memset(&clientAddr, 0, sizeof(clientAddr));
clientAddr.sin_family = AF_INET;
clientAddr.sin_addr.s_addr = inet_addr(ip.c_str());
clientAddr.sin_port = htons(port);
return true;
}
int UDPSocket::recv(std::string& s) const {
char buffer[MAXRECV + 1];
struct timeval tv;
fd_set fdset;
int rc, nread;
memset(&buffer, 0, sizeof(buffer));
FD_ZERO(&fdset);
FD_SET(m_sock, &fdset);
tv.tv_sec = 0;
tv.tv_usec = m_timeout;
rc = select(m_sock + 1, &fdset, (fd_set *) 0, (fd_set *) 0, &tv);
if(FD_ISSET(m_sock, &fdset)) {
#ifdef WIN32
nread = ::recvfrom(m_sock, buffer, MAXRECV, 0, (sockaddr*)&clientAddr, const_cast< int * __w64 >(&addrLen));
#else
nread = ::recvfrom(m_sock, buffer, MAXRECV, 0, (sockaddr*)&clientAddr, (socklen_t*)&addrLen);
#endif
if(nread < 0) {
return -1;
} else if(nread == 0) {
return 0;
}
s = std::string(buffer);
return nread;
} else {
return 0;
}
}
void UDPSocket::set_non_blocking(const bool b) {
mNonBlocking = b;
#ifdef WIN32
u_long argp = b ? 1 : 0;
ioctlsocket(m_sock, FIONBIO, &argp);
#else
int opts = fcntl(m_sock, F_GETFL);
if(opts < 0) return;
if(b)
opts |= O_NONBLOCK;
else
opts &= ~O_NONBLOCK;
fcntl(m_sock, F_SETFL, opts);
#endif
}
My user code, on both ends, create a "sending" and "receiving" UDPSocket and bind them to their respective ports, then use send() to send data and recv() to receive. On one hand, the linux side seems to receive practically immediately, but the Windows side has a delay up to 1 second before receiving data. However, ::recv() never returns 0 in this time. Am I missing something obvious?
have you tried all four combinations (linux->linux, win->linux, linux->win, win->win)? which have delays and which not?
also, use a packet sniffer (like tcpdump or wireshark) to see if the delay is before or after hitting the wire.
It's not a direct answer. May be you can try tools like ttcp (http://www.pcausa.com/Utilities/pcattcp.htm), which can both test tcp and udp performance. And, you may also check their source code, which is in public domain.
If you've got the time to experiment, try an IOCP version of the code.
I never trusted the win select implementation, with its 64 socket limit.

Resources