I am writing a cross-platform library which, among other things, provides a socket interface, and while running my unit-test suite, I noticed something strange with regard to timeouts set via setsockopt(): On Windows, a blocking recv() call seems to consistently return about half a second (500 ms) later than specified via the SO_RCVTIMEO option.
Is there any explanation for this in the docs I missed? Searching the web, I was only able to find a single other reference to the problem – could somebody who owns »Windows Sockets
Network Programming« by Bob Quinn and Dave Shute look up page 466 for me? Unfortunately, I can only run my test Windows Server 2008 R2 right now, does the same strange behavior exist on other Windows versions as well?
From Networking Programming for Microsoft Windows by Jones and Ohlund:
SO_RCVTIMEO optval
Type: int
Get/Set: Both
Winsock Version: 1+
Description : Gets or sets the timeout value (in milliseconds)
associated with receiving data on the
socket
The SO_RCVTIMEO option sets the
receive timeout value on a blocking
socket. The timeout value is an
integer in milliseconds that indicates
how long a Winsock receive function
should block when attempting to
receive data. If you need to use the
SO_RCVTIMEO option and you use the
WSASocket function to create the
socket, you must specify
WSA_FLAG_OVERLAPPED as part of
WSASocket's dwFlags parameter.
Subsequent calls to any Winsock
receive function (such as recv,
recvfrom, WSARecv, or WSARecvFrom)
block only for the amount of time
specified. If no data arrives within
that time, the call fails with the
error 10060 (WSAETIMEDOUT). If the
receiver operation does time out the
socket is in an indeterminate state
and should not be used.
For performance reasons, this option
was disabled in Windows CE 2.1. If you
attempt to set this option, it is
silently ignored and no failure
returns. Previous versions of Windows
CE do implement this option.
I'd think the crucial information in this is:
If you need to use the SO_RCVTIMEO option and you use the WSASocket
function to create the socket, you
must specify WSA_FLAG_OVERLAPPED as
part of WSASocket's dwFlags parameter
I hope this is still useful :)
I am having the same problem. Going to use
patchedTimeout = max ( unpatchedTimepit - 500, 1 )
Tested this with the unpatchedTimepit == 850
your problem is not in rcv function timeout!
if your application have a while loop to check and receive just put an if statement to check the receive buffer last index for '\0' char to check is the receiving string is ended or not.
typically if rcv function is still receiving return value is the size of received data. size can be used as last index of buffer array.
do{
result = rcv(s,buf,len,0);
if(buf[result] == '\0'){
break;
}
}
while(result > 0);
Related
Based on the following code, I built a version of an echo server, but with a threaded delay. This was built because I've noticed that upon initial connection, my first send is sent back to the client, but the client does not receive it until a second send. My real-world use case is that I need to send messages to the server, do a lot of processing, and then send the result back... say 10-30 seconds later (could be hours in some cases).
http://www.wangafu.net/~nickm/libevent-book/Ref8_listener.html
So here is my code. For brevity's sake, I have only included the libevent-related code; not the threading code or other stuff. When debugging, a new connection is set up, the string buffer is filled properly, and debugging reveals that the writes go successfully.
http://pastebin.com/g02S2RTi
But I only receive the echo from the send-before-last. I send from the client numbers to validate this and when I send a 1 from the client, I receive nothing from the server via echo... even though the server is definitely writing to the buffer using evbuffer_add ( I have also tried this using bufferevent_write_buffer).
From the client when I send a 2, I then receive the 1 from the previous send. It's like my writes are being cached.... I have turned off nagle.
So, my question is: Does libevent cache sends using the following method?
evbuffer_add( outputBuffer, buffer, length );
Is there a way to flush this cache? Is there some other method to mark the cache as finished or complete? Can I force a send? It never sends on it's own... I have even put in delays. Replacing evbuffer_add with "send" works perfectly every time.
Most likely you are affected by Nagle algorithm - basically it buffers outgoing data, before sending it to the network. Take a look at this article: TCP/IP options for high-performance data transmission.
Here is an example how to disable buffering:
int flag = 1;
int result = setsockopt(sock, /* socket affected */
IPPROTO_TCP, /* set option at TCP level */
TCP_NODELAY, /* name of option */
(char *) &flag, /* the cast is historical
cruft */
sizeof(int)); /* length of option value */
This is more of a observation and also a suggestion for whats the best way to handle this scenario.
I have two threads one just pumps in data and another receives the data and does lot of work before sending it another socket. Both the threads are connected via a Domain socket. The protocol used here is UDP. I did not want to use TCP as it is stream based, which means if there is little space in the queue my data is split and sent. This is bad as Iam sending data that should not be split. Hence I used DGRAM. Interestingly when the send thread overwhelms the recv thread by pumping so much data, at some point the Domain socket buffer gets filled up and sendto() returns ENOBUFS. I was of the opinion that should this happen, sendto() would block until the buffer is available. This would be my desired behaviour. However this does not seem to be the case. I solve this problem in a rather weird way.
CPU Yield method
If I get ENOBUFS, I do a sched_yield(); as there is no pthread_yield() in OSX. After that I try to resend again. If that fails I keep doing the same until it is taken. This is bad as Iam wasting cpu cycles just doing something useless. I would love if sendto() blocked.
Sleep method
I tried to solve the same issue using sleep(1) instead of sched_yield() but this of no use as sleep() would put my process to sleep instead of just that send thread.
Both of them does not seem to work for me and Iam running out of options. Can someone suggest what is the best way to handle this issue? Is there some clever tricks Iam not aware of that can reduce unnecessary cpu cycles? btw, what the man page says about sentto() is wrong, based on this discussion http://lists.freebsd.org/pipermail/freebsd-hackers/2004-January/005385.html
The Upd code in kernel:
The udp_output function in /sys/netinet/udp_usrreq.c, seems clear:
/*
* Calculate data length and get a mbuf
* for UDP and IP headers.
*/
M_PREPEND(m, sizeof(struct udpiphdr), M_DONTWAIT);
if (m == 0) {
error = ENOBUFS;
if (addr)
splx(s);
goto release;
}
I'm not sure why sendto() isn't blocking for you... but you might try calling this function before you each call to sendto():
#include <stdio.h>
#include <sys/select.h>
// Won't return until there is space available on the socket for writing
void WaitUntilSocketIsReadyForWrite(int socketFD)
{
fd_set writeSet;
FD_ZERO(&writeSet);
FD_SET(socketFD, &writeSet);
if (select(socketFD+1, NULL, &writeSet, NULL, NULL) < 0) perror("select");
}
Btw how big are the packets that you are trying to send?
sendto() on OS X is really nonblocking (that is M_DONTWAIT flag for).
I suggest you to use stream based connection and just receive the whole data on the other side by using MSG_WAITALL flag of the recv function. If your data has strict structure than it would be simple, just pass the correct size to the recv. If not than just send some fixed-size control packet first with the size of the next chunk of data and then the data itself. On the receiver side you would be wait for control packet of fixed size and than the data of size from control packet.
Making an asynchronous request with Wininet, when the status callback function is called with INTERNET_STATUS_REQUEST_COMPLETE, I get the http status code.
result = HttpQueryInfo(
this->requestHandle,
HTTP_QUERY_STATUS_CODE | HTTP_QUERY_FLAG_NUMBER,
&value,
&sizeofDword,
&index);
The status code returned is 200. After that, I call InternetReadFile().
result = InternetReadFile(
this->requestHandle,
((char*)(this->buffer)) + this->totalBytesReceived,
this->bufferSize - this->totalBytesReceived,
&bytesRead);
this->totalBytesReceived += bytesRead;
It returns true and sets lpNumberOfBytesRead to zero. GetLastError() returns ERROR_IO_PENDING, then I wait the callback function is called again with INTERNET_STATUS_REQUEST_COMPLETE.
When that occurs, InternetReadFile() returns true and again sets lpNumberOfBytesRead to zero.
If I debug the application, I can see after the first InternetReadFile() that the response data is already on the lpBuffer. Moreover, if I call Sleep() for one second before InternetReadFile(), InternetReadFile() works correctly.
Sleep(1000);
result = InternetReadFile( ...
Am I missing any step?
I've encountered similar problem, and still the same even using Sleep(1000). I was connecting to a 3rd party camera stream, I've tried it on debug version, which worked perfectly. But when I went back to release version, it just wouldn't work.
I fixed it by changing HttpOpenRequest's lplpszAcceptTypes to NULL, then everything started working to work.
Seems like when WinINet dealing with internet connection, with different type of mobile phone / os /debug or release binary version/ have some kind of different operation which we can not possibly know.
I have written a server and a client that are using an overlapped named pipe. My problem is mainly with Readfile() and GetOverlappedResult().
Note that this program is a test code. It will be integrated later in a framework (I'm porting linux code to unix that uses AF_UNIX adress family for socket connections)
I describe the server part. I have 2 threads :
1) the main thread opens an overlapped named pipe, then loop over WaitForMultipleObjects(). WaitForMultipleObjects() waits for 3 events: the 1st one waits for a client to connect. The 2nd allows me to cleanly quit the program. The 3rd is signaled when an operation is pending in ReadFile().
2) The second thread is launched when the client is connected. It loops over ReadFile().
Here is the server code:
http://pastebin.com/5rka7dK7
I mainly used MSDN doc (named pipe server using overlapped I/O, named pipe client), the SDK, and other doc on internet, to write that code. Look in [1] for the client code. The client code needs some love, but for now, I focus on making the server working perfectly.
There are 4 functions in the server code (i forget the function that display error messages):
a) svr_new: it creates the overlapped named pipe and the 3 events, and calls ConnectNamedPipe()
b) svr_del frees all the resources
c) _read_data_cb: the thread that calls ReadFile()
d) the main() function (the main thread), which loops over WaitForMultipleObjects()
My aim is to detect in _read_data_cb() when the client disconnects (ReadFile() fails and GetLastError() returns ERROR_BROKEN_PIPE) and when data comes from the client.
What I don't understand:
Should I call GetOverlappedResult() ?
If yes, where ? When ReadFile() fails and GetLastError() returns ERROR_IO_PENDING (line 50 of the paste) ? When WaitForMultipleObjects() returns (line 303 of the paste, I commented the code there) ? Somewhere else ?
I do a ResetEvent of the event of ReadFile() when WaitForMultipleObjects() returns (line 302 of the paste). Is it the correct place to call it ?
With the code I pasted, here is the result if the client sends these 24 bytes (the ReadFile() buffer is of size 5 bytes. I intentionnaly set that value to test what to do if a client sends some data larger than the ReadFile() buffer)
message : "salut, c'est le client !"
output:
$ ./server.exe
waiting for client...
WaitForMultipleObjects : 0
client connected (1)
WaitForMultipleObjects : 2
* ReadFile : 5
WaitForMultipleObjects : 2
* ReadFile : 5
WaitForMultipleObjects : 2
* ReadFile : 5
WaitForMultipleObjects : 2
* ReadFile : 5
WaitForMultipleObjects : 2
* ReadFile : 4
Note: WaitForMultipleObjects() can be called less than that, it seems random.
So, in my code, I do not call getOverlappedResult(), ReadFile() succeeds (il reads 5*4 + 4 = 24 bytes), but I don't know when the read operation has finished.
Note: I I add a printf() when ReadFile() fails with ERROR_IO_PENDING, that printf() is called indefinitely.
In addition, the client sends 2 messages. The one above, and another one 3seconds later. The 2nd message is never read and ReadFile() fails with the error ERROR_SUCCESS... (so to be precise, ReadFile() returns FALSE and GetLastError() returns ERROR_SUCCESS)
So, I'm completely lost. I have searched hours on Internet, in MSDN, in the SDK code (Server32.c and Client32.c). I still do not know what to do in my specific case.
So, ca someone explain me how to use GetOverlappedResult() (if I have to use it) to know how to check if the read operation finished, and where ? And even, if someone can fix my code :-) I gave the code so that everyone can test it (i find a lot of doc on internet, but it is almost always not precise at all :-/ )
thank you
[1] http://pastebin.com/fbCH2By8
Take a look at I/O Completion Ports. In my opinion it's the most efficient way to receive and handle notifications about overlapped operations in Windows. So basically you will need to use GetQueuedCompletionStatus and GetQueuedCompletionStatusEx in blocking and non-blocking mode when you're ready to process new completion events, instead of calling GetOverlappedResult from time to time. As a matter of fact, you can even get rid of WaitForMultipleObjects completely.
Also, which flavor of Unix are you targeting? In Solaris there's a very similar abstraction. Check out man port_create.
Unfortunately, there's nothing similar in Linux. Signals (including real-time) can be used to some extent as waitable completion objects, but they are not as comprehensive as the ports in Windows and Solaris.
Using MS Visual Studio 2008 C++ for Windows 32 (XP brand), I try to construct a POP3 client managed from a modeless dialog box.
Te first step is create a persistent object -say pop3- with all that Boost.asio stuff to do asynchronous connections, in the WM_INITDIALOG message of the dialog-box-procedure. Some like:
case WM_INITDIALOG:
return (iniPop3Dlg (hDlg, lParam));
Here we assume that iniPop3Dlg() create the pop3 heap object -say pointed out by pop3p-. Then connect with the remote server, and a session is initiated with the client’s id and password (USER and PASS commands). Here we assume that the server is in TRANSACTION state.
Then, in response to some user input, the dialog-box-procedure, call the appropriate function. Say:
case IDS_TOTAL: // get how many emails in the server
total (pop3p);
return FALSE;
case IDS_DETAIL: // get date, sender and subject for each email in the server
detail (pop3p);
return FALSE;
Note that total() uses the POP3’s STAT command to get how many emails in the server, while detail() uses two commands consecutively; first STAT to get the total and then a loop with the GET command to retrieve the content of each message.
As an aside: detail() and total() share the same subroutines -the STAT handle routine-, and when finished, both leaves the session as-is. That is, without closing the connection; the socket remains opened an the server in TRANSACTION state.
When any option is selected by the first time, the things run as expected, obtaining the desired results. But when making the second chance, the connection hangs.
A closer inspection show that the first time that the statement
socket_.get_io_service().run();
Is used, never ends.
Note that all asynchronous write and read routines uses the same io_service, and each routine uses socket_.get_io_service().reset() prior to any run()
Not also that all R/W operations also uses the same timer, who is reseted to zero wait after each operation is completed:
dTimer_.expires_from_now (boost::posix_time::seconds(0));
I suspect that the problem is in the io_service or in the timer, and the fact that subsequent executions occurs in a different load of the routine.
As a first approach to my problem, I hope that someone would bring some light in it, prior to a more detailed exposition of the -very few and simple- routines involved.
Have you looked at the asio examples and studied them? There are several asynchronous examples that should help you understand the basic control flow. Pay particular importance to the main event loop started by invoking io_service::run, it's important to understand control is not expected to return to the caller until the io_service has no more remaining work to do.