VB6 TCP IP Communication without Doevents or subclassing - vb6

What I need to happen with a VB6 application I maintain is the following.
Establish a connection to a known address and port over a Ethernet
network.
Send a request
Wait for a response.
I tried using WinSock and Winsock replacement but they all rely in one form or another the messaging loop inherent in various Windows application. I don't know enough about the Winsock API how to implement the above algorithm in VB6 (or in any other language.
My application is a VB6 CAD/CAM software that controls metal cutting machines over a dedicated Ethernet network. The software has been maintain for 20 years and we developed several driver for different types of motion controllers. To date the API for these motion controllers consist of
Opening a connection to the hardware
Sending a request to the hardware (for example Position of an axis)
Waiting for a response (usually occurs in milliseconds).
Some of these controller work over a Ethernet Network but until now I never had to directly interact with the ports. I used the company supplied libraries to handle things. And they work in the way I mentioned above and throw a timeout error if a response doesn't occur in some defined time.
The problem with Winsock is that I have to insert DoEvents to get it a respond. This causes havoc with how we handle multi-tasking in our VB6 application. The replacement like CSocketMaster use subclassing which also cause havoc with our multi-tasking.
So any help on how to use the Winsock API or a third party dll that can do what I need to do as outlined above. I wouldn't be asking if I haven't seen other motion control do what I want to do.

Check out VbAsyncSocket repo on github for pure VB6 asynchronous sockets implementation (using WSAAsyncSelect API for sockets to post event notifications).
Contrary to it's name the class does support SyncSendArray and SyncReceiveArray methods for synchronous operations -- without DoEvents but with Timeouts.
In the same repo there is a handy cWinSockRequest contributed class, that is very similar to WinHttpRequest object as baked into the OS. This helper class will be very familliar to you if you have experience with JSON/XML (generally RESTful services over http/https) for accessing services/devices over plain tcp/udp sockets.
Another option would be to use cTlsClient contributed class, that can connect to host/device over tcp (no udp here) and provides ReadText/WriteText and ReadArray/WriteArray (synchronous) methods. The added benefit here is that the class supports both plain unencrypted sockets and SSL encrypted channels if need be.
We are using these classes to (synchronously) access ESP/POS printers from our LOB applications. Most POS printers also provide serial (USB-to-COM) links too, so we are abstracting our access w/ connector classes -- SyncWaitForEvent over async sockets and WaitForMultipleObjects on overlapped ReadFile/WriteFile APIs (oh, the irony)

I think it is rare for it to be appropriate to do networking
synchronously, however this isn't networking in the traditional sense.
This is a wire from a PC to a controller. This is like a string
between two cans. In this case with a large old program, the most
appropriate approach is the one that works the best and is the easiest
to maintenance. < /end2cents >
If VB6 + Winsock isn't working out for you, writing this in .NET and building it into a COM visible DLL for your VB6 program will fit the bill.
The example below will get you started. If you do more than the occasional call to this, it will be slow as it opens and closes the connection on each call. It should be easy to expand it to allow for reusing an open connection for back and forth communication between the PC and controller. Just be careful that you don't create a memory leak!
/// <summary>
/// Sends a message to the specified host:port, and waits for a response
/// </summary>
public string SendAndReceive(string host, int port, string messageToSend, int millisecondTimeout)
{
try
{
using (var client = new TcpClient())
{
client.SendTimeout = client.ReceiveTimeout = millisecondTimeout;
// Perform connection
client.Connect(host, port);
if (client.Connected)
{
using (var stream = client.GetStream())
{
// Convert the message to a byte array
var toSend = Encoding.ASCII.GetBytes(messageToSend);
// Send the message
stream.Write(toSend, 0, toSend.Length);
// Get a response
var response = new byte[client.ReceiveBufferSize];
stream.Read(response, 0, client.ReceiveBufferSize);
return Encoding.ASCII.GetString(retVal);
}
}
else
{
return null;
}
}
}
catch
{
return null;
}
}

As it turns out the answer involved implementing Allen's Suggestion.
The specific problem was communicating between two device involved in the control of a piece of machinery. The device that did the motion control is acting as a server, while the PC providing the motion data was a client.
The Ethernet network was being used in lieu of a proprietary bus interface or a RS-232/422 serial interface. So many of the considerations involved in serving up data over a widespread internet were not a factor. The network consisted of known devices residing at fixed IPs listening to specific ports.
After talking to people who make other motion control. The logic for the client turned out to be surprisingly simple.
Send the Data
Wait in a Loop for the response breaking out if it taking too long.
Handle any errors in the connection.
On the server side, we were lucky in that we had control over the software running on the motion controller. So there the communication loop was designed to be as fast as possible. One key point was to keep all data below 512 bytes so it is all contained in a single packet. They also took great care in priortizing the communication handler and the data structure so it can send out a response in tens of microseconds.
Another point was that in this application of dedicated clients and server that UDP is preferred over TCP as Operation Systems particularly Windows were in the habit of shutting down idle TCP Connections unexpectedly.
Because the client software is slowly transition over to the .NET framework that was another factor for implementing Allen's idea. The library discussed by #wqw worked as well.

Your problem is that you are using the Winsock control incorrectly. That probably stems from a flaw in your interaction model.
Don't "send and wait" because blocking like that is your big mistake. There is no "waiting" anyway unless you think sitting in a buzz loop is waiting.
Instead send your request and exit from that event handler. All of your code is contained in event handlers, that's how it works. Then as DataArrival events are raised you append the new arrived fragment to a stream buffer then scan the assembled stream for a complete response. Then go ahead and process the response.
Handle timeouts using a Timer control that you enable after sending. When you have assembled a completed response disable the Timer. If the interval elapses and Timer event is raised do your error processing there.
You seem to have a particularly chatty protocol so you shouldn't have to do anything else. For example you can just clear your stream buffer after processing a complete response, since there can't be anything else left in there anyway.
Forget "multitasking" and avoid DoEvents() calls like the plague they are.
This is very simple stuff.

Related

Resolve Windows socket error WSAENOBUFS (10055)

Our application has a feature to actively connect to the customers' internal factory network and send a message when inspection events occur. The customer enters the IP address and port number of their machine and application into our software.
I'm using a TClientSocket in blocking mode and have provided callback functions for the OnConnect and OnError events. Assuming the abovementioned feature has been activated, when the application starts I call the following code in a separate thread:
// Attempt active connection
try
m_socketClient.Active := True;
except
end;
// Later...
// If `OnConnect` and socket is connected...send some data!
// If `OnError`...call `m_socketClient.Active := True;` again
When IP + port are valid, the feature works well. But if not, after several thousand errors (and many hours or even days) eventually Windows socket error 10055 (WSAENOBUFS) occurs and the application crashes.
Various articles such as this one from ServerFramework and this one from Microsoft talk about exhausting the Windows non-paged pool and mention (1) actively managing the number outstanding asynchronous send operations and (2) releasing the data buffers that were used for the I/O operations.
My question is how to achieve this and is three-fold:
A) Am I doing something wrong that memory is being leaked? For example, is there some missing cleanup code in the OnError handler?
B) How do you monitor I/O buffers to see if they are being exhausted? I've used Process Explorer to confirm my application is the cause of the leak, but ideally I'd need some programmatic way of measuring this.
C) Apart from restarting the application, is there a way to ask Windows to clear out or release I/O operation data buffers?
Code samples in Delphi, C/C++, C# fine.
A) The cause of the resource leak was a programming error. When the OnError event occurs, Socket.Close() should be called to release low-level resources associated with the socket.
B) The memory leak does not show up in the standard Working Set memory use of the process. Open handles belonging to your process need to be monitored which is possible with GetProcessHandleCount. See this answer in Delphi which was tested and works well. This answer in C++ was not tested but the answer is accepted so should work. Of course, you should be able to use GetProcessHandleCount directly in C++.
C) After much research, I must conclude that just like a normal memory leak, you cannot just ask Windows to "clean up" after you! The handle resource has been leaked by your application and you must find and fix the cause (see A and B above).

How to improve RPC data throughput over high latency network

I am working on client-server software using Microsoft RPC (over TCP) as the communication method. We sometimes transfer files from the client to the server. This works fine in local networks. Unfortunately, when we have a high latency, even a very wide bandwidth does not give a decent transfer speed.
Based on a WireShark log, the RPC layer sends a bunch of fragments, then waits for an ACK from the server before sending more and this causes the latency to dominate the transfer time. I am looking for a way to tell RPC to send more packets before pausing.
The issue seems to be essentially the same as with a too small TCP window, but there might be an RPC specific fragment window at work here, since Wireshark does not show the TCP-level window being full. iPerf connection tests with a small window do give those warnings, and a speed similar to the RPC transfer. With larger windows sizes, the iPerf transfer is three times faster than the RPC, even with a reasonable (40ms) latency.
I did find some mentions of an RPC fragment window at microsoft's site (https://msdn.microsoft.com/en-us/library/gg604601.aspx) and in an RPC document (http://pubs.opengroup.org/onlinepubs/9629399/chap12.htm search for window_size), but these seem to concern only connectionless (UDP) RPC. Additionally, they mention an RPC "fack" message and I observed only regular TCP level ACK:s in the log.
My conclusion is that either the RPC layer is using a stupidly low TCP window, or it is limiting the number of fragment packages it sends at a time by some internal logic. Either way, I need to make it send more between ACKs. Is there some way to do this?
I could of course just transfer the file over multiple simultaneous connections, but that seems more like a work-around than a solution.
PS. I know RPC is not really designed for file transfer, but this is a legacy application and the RPC pipe deals with authentication and whatnot, so keeping the file transfer there would be best, at least for now.
PPS. I guess that if the answer to this question is a configuration option, this would be better suited for SuperUser, but an API setting would be ideal, which is why I posted this here.
I finally found a way to control this. This Microsoft documentation page: Configuring Computers for RPC over HTTP contains registry settings that set the windows RPC uses, at least when used in conjunction with RPC over HTTP.
The two most relevant settings were:
HKLM\Software\Microsoft\Rpc\ClientReceiveWindow: DWORD
Making this higher (some MB:s, in bytes) on the client machine made the download to the client much faster.
HKLM\Software\Microsoft\Rpc\InProxyReceiveWindow: DWORD
Making this higher on the server machine made the upload faster.
The downside of these options is that they are global. The first one will affect all RPC clients on the client machine and the latter will affect all RPC over HTTP proxying on the server. This may have serious caveats, but a tenfold speed increase is nothing to be scoffed at, either.
Still, setting these on a per-connection basis would be much better.

Two-way communication between kernel-mode driver and user-mode application?

I need a two-way communication between a kernel-mode WFP driver and a user-mode application. The driver initiates the communication by passing a URL to the application which then does a categorization of that URL (Entertainment, News, Adult, etc.) and passes that category back to the driver. The driver needs to know the category in the filter function because it may block certain web pages based on that information. I had a thread in the application that was making an I/O request that the driver would complete with the URL and a GUID, and then the application would write the category into the registry under that GUID where the driver would pick it up. Unfortunately, as the driver verifier pointed out, this is unstable because the Zw registry functions have to run at PASSIVE_LEVEL. I was thinking about trying the same thing with mapped memory buffers, but I’m not sure what the interrupt requirements are for that. Also, I thought about lowering the interrupt level before the registry function calls, but I don't know what the side effects of that are.
You just need to have two different kinds of I/O request.
If you're using DeviceIoControl to retrieve the URLs (I think this would be the most suitable method) this is as simple as adding a second I/O control code.
If you're using ReadFile or equivalent, things would normally get a bit messier, but as it happens in this specific case you only have two kinds of operations, one of which is a read (driver->application) and the other of which is a write (application->driver). So you could just use WriteFile to send the reply, including of course the GUID so that the driver can match up your reply to the right query.
Another approach (more similar to your original one) would be to use a shared memory buffer. See this answer for more details. The problem with that idea is that you would either need to use a spinlock (at the cost of system performance and power consumption, and of course not being able to work on a single-core system) or to poll (which is both inefficient and not really suitable for time-sensitive operations).
There is nothing unstable about PASSIVE_LEVEL. Access to registry must be at PASSIVE_LEVEL so it's not possible directly if driver is running at higher IRQL. You can do it by offloading to work item, though. Lowering the IRQL is usually not recommended as it contradicts the OS intentions.
Your protocol indeed sounds somewhat cumbersome and doing a direct app-driver communication is probably preferable. You can find useful information about this here: http://msdn.microsoft.com/en-us/library/windows/hardware/ff554436(v=vs.85).aspx
Since the callouts are at DISPATCH, your processing has to be done either in a worker thread or a DPC, which will allow you to use ZwXXX. You should into inverted callbacks for communication purposes, there's a good document on OSR.
I've just started poking around WFP but it looks like even in the samples that they provide, Microsoft reinject the packets. I haven't looked into it that closely but it seems that they drop the packet and re-inject whenever processed. That would be enough for your use mode engine to make the decision. You should also limit the packet capture to a specific port (80 in your case) so that you don't do extra processing that you don't need.

Best way to communicate from KEXT to Daemon and block until result is returned from Daemon

In KEXT, I am listening for file close via vnode or file scope listener. For certain (very few) files, I need to send file path to my system daemon which does some processing (this has to happen in daemon) and returns the result back to KEXT. The file close call needs to be blocked until I get response from daemon. Based on result I need to some operation in close call and return close call successfully. There is lot of discussion on KEXT communication related topic on the forum. But they are not conclusive and appears be very old (year 2002 around). This requirement can be handled by FtlSendMessage(...) Win32 API. I am looking for equivalent of that on Mac
Here is what I have looked at and want to summarize my understanding:
Mach message: Provides very good way of bidirectional communication using sender and reply ports with queueing mechansim. However, the mach message APIs (e.g. mach_msg, mach_port_allocate, bootstrap_look_up) don't appear to be KPIs. The mach API mach_msg_send_from_kernel can be used, but that alone will not help in bidirectional communication. Is my understanding right?
IOUserClient: This appears be more to do with communicating from User space to KEXT and then having some callbacks from KEXT. I did not find a way to initiate communication from KEXT to daemon and then wait for result from daemon. Am I missing something?
Sockets: This could be last option since I would have to implement entire bidirectional communication channel from KEXT to Daemon.
ioctl/sysctl: I don't know much about them. From what I have read, its not recommended option especially for bidirectional communication
RPC-Mig: Again I don't know much about them. Looks complicated from what I have seen. Not sure if this is recommended way.
KUNCUserNotification: This appears to be just providing notification to the user from KEXT. It does not meet my requirement.
Supported platform is (10.5 onwards). So looking at the requirement, can someone suggest and provide some pointers on this topic?
Thanks in advance.
The pattern I've used for that process is to have the user-space process initiate a socket connection to the KEXT; the KEXT creates a new thread to handle messages over that socket and sleeps the thread. When the KEXT detects an event it needs to respond to, it wakes the messaging thread and uses the existing socket to send data to the daemon. On receiving a response, control is passed back to the requesting thread to decide whether to veto the operation.
I don't know of any single resource that will describe that whole pattern completely, but the relevant KPIs are discussed in Mac OS X Internals (which seems old, but the KPIs haven't changed much since it was written) and OS X and iOS Kernel Programming (which I was a tech reviewer on).
For what it's worth, autofs uses what I assume you mean by "RPC-Mig", so it's not too complicated (MIG is used to describe the RPC calls, and the stub code it generates handles calling the appropriate Mach-message sending and receiving code; there are special options to generate kernel-mode stubs).
However, it doesn't need to do any lookups, as automountd (the user-mode daemon to which the autofs kext sends messages) has a "host special port" assigned to it. Doing the lookups to find an arbitrary service would be harder.
If you want to use the socket established with ctl_register() on the KExt side, then beware: The communication from kext to user space (via ctl_enqueuedata()) works OK. However opposite direction is buggy on 10.5.x and 10.6.x.
After about 70.000 or 80.000 send() calls with SOCK_DGRAM in the PF_SYSTEM domain complete net stack breaks with disastrous consequences for complete system (hard turning off is the only way out). This has been fixed in 10.7.0. I workaround by using setsockopt() in our project for the direction from user space to kext as we only send very small data (just to allow/disallow some operation).

High-Performance In-Browser Networking

(Similar in spirit to but different in practice from this question.)
Is there any cross-browser-compatible, in-browser technology that allows a high-performance perstistent network connection between a server application and a client written in, say, Javascript? Think XMLHttpRequest on caffeine. I am working on a visualisation system that's restricted to at most a few users at once, and the server is pretty robust, so it can handle as much as it needs to. I would like to allow the client to have access to video streamed from the server at a minimum of about 20 frames per second, regardless of what their graphics hardware capabilities are.
Simply put: is this doable without resorting to Flash or Java?
I'm not sure what you mean by XMLHttpRequest on caffeine...the performance of a remote polling object like that are subject to the performance of the client and the server, not of the language constructs themselves. Granted, there is HTTP overhead in AJAX, but the only viable alternative is to use HTTP long polling (which basically keeps the server connection open longer and passes chunks of data down bit by bit in the background. It's literally the same as AJAX, except the connection stays open until something happens (thus moving the HTTP overhead to idle time).
If I recall correctly, Opera had some kind of sockets implementation a while back, but nobody uses Opera.

Resources