Can I use GetOverlappedResult to return the number of bytes read if ReadFile with OVERLAPPED IO returned true (meaning the result was never pending?)
yes. you can call GetOverlappedResult in this case.
in your concrete case, if ReadFile return true, mean that io operation already completed. the GetOverlappedResult from self side first of all check IO_STATUS_BLOCK at begin of OVERLAPPED . more concrete checked Internal (NTSTATUS Status) for STATUS_PENDING
The status code for the I/O request. When the request is issued, the
system sets this member to STATUS_PENDING to indicate that the
operation has not yet started. When the request is completed, the
system sets this member to the status code for the completed request.
if here not STATUS_PENDING (Status != STATUS_PENDING) GetOverlappedResult understand that io already completed. if it error status (Status < 0) convert it to win32 error and set last error, otherwise read InternalHigh (The number of bytes transferred for the I/O request. The system sets this member if the request is completed without errors.) and copy it to *lpNumberOfBytesTransferred.
so in your case - Status != STATUS_PENDING and Status >= 0 (because ReadFile return true) - and GetOverlappedResult just return true to you and set *lpNumberOfBytesTransferred = (DWORD)lpOverlapped->InternalHigh
If the operation completes synchronously you cannot use use GetOverlappedResult to query the result or the bytes read.
If, on the other hand, an operation is completed immediately, then &NumberOfBytesRead passed into ReadFile is valid for the number of bytes read. In this case, ignore the OVERLAPPED structure passed into ReadFile; do not use it with GetOverlappedResult or WaitForSingleObject.
See https://support.microsoft.com/en-gb/help/156932/asynchronous-disk-i-o-appears-as-synchronous-on-windows for details.
Related
When the code executes the ReadFile the call never returns. Previously the way I saved the Handle and passed it from function to function was not properly done. I changed the way the handle was stored and it works with all of the other calls in the program except the read. I have looked and compared all of the variables used and they check out.
Here is the code. instance->Master has the handle from the create. I added a GetCommState call before the read and it executes correctly.
'''
ATCA_STATUS swi_silab_receive_byte(ATCASWIMaster_t* instance, uint8_t* data)
{
uint8_t retries = 3;
DWORD NoBytesRead = 0; // Bytes read by ReadFile()
uint8_t SerialBuffer; //Buffer to send and receive data
DWORD Byte_count = (DWORD)sizeof(SerialBuffer);
ATCA_STATUS status;
while ((retries > 0) && (NoBytesRead < 1))
{
//Read data and store in a buffer
status = GetCommState(instance->hMaster, &instance->dcbMaster);
if (status == 0)
printf(" init GetcommState failed\n");
status = ReadFile(instance->hMaster, &SerialBuffer, Byte_count, &NoBytesRead, NULL);
retries--;
}
if (status == FALSE)
{
printf_s("\nError! in ReadFile()\n\n");
return ATCA_TIMEOUT;
}
else
{
printf("Read Success Serial Buffer = %x\n", &SerialBuffer);
*data = SerialBuffer;
//printf("Read Success Data = %x\n", *data);
return ATCA_SUCCESS;
}
}
'''
I am happy to state there is no issue in the code. Thanks to Zhu Song who made a comment about reading the ReadFile remarks. If there is no data to read then readfile will just wait. A check with the logic analyzer showed the write executes but doesn't actually write, hence the read is not actually able to read.
Thanks to everyone who commented
According to ReadFile:
The ReadFile function returns when one of the following conditions occur:
The number of bytes requested is read.
A write operation completes on the write end of the pipe.
An asynchronous handle is being used and the read is occurring asynchronously.
An error occurs.
To cancel all pending asynchronous I/O operations, use either:
CancelIo—this function only cancels operations issued by the calling thread for the specified file handle.
CancelIoEx—this function cancels all operations issued by the threads for the specified file handle.
The docs for FILE_FLAG_OVERLAPPED on WriteFile() say you must provide OVERLAP and recommend NULL for lpNumberOfBytesWritten because value is misleading. However docs for GetOverlappedResult() say to only call if WriteFile() returned FALSE with ERROR_IO_PENDING. So that leaves the case where ReadFile() / WriteFile() complete in the API call itself. How are you supposed to get the number of bytes read/written? Do you presume it's the number requested? But WriteFile() says "When writing to a non-blocking, byte-mode pipe handle with insufficient buffer space, WriteFile returns TRUE with * lpNumberOfBytesWritten < nNumberOfBytesToWrite".
TIA!!
If hFile was opened with FILE_FLAG_OVERLAPPED The
lpNumberOfBytesWritten parameter should be set to NULL.
this is not true (mistake or lie). the lpNumberOfBytesWritten can be set to NULL, but not should. if I/O request complete synchronous with success in *lpNumberOfBytesWritten will be valid number of bytes.
note also that lpNumberOfBytes must not point to location which will be valid until operation is complete (like lpOverlapped) - it can point to local variable in function for example and you can exit from function before I/O is complete - and this is will be ok too. system simply copy InternalHigh from OVERLAPPED to *lpNumberOfBytes. so in pseudo code :
if (lpNumberOfBytes) *lpNumberOfBytes = (ULONG)lpOverlapped->InternalHigh;
obvivous correct value in *lpNumberOfBytes will be only if I/O already completed with success. so and can use it only in this case. system doesn't remember value of lpNumberOfBytes - because this it must be valid only during [Write|Read]File call but not during I/O active (which can be longer in case asynchronous I/O)
GetOverlappedResult we can call if I/O request complete synchronous with success (in case ReadFile or WriteFile, if thay return TRUE) of if pending returned. this api can not be called only in case I/O request just fail (ReadFile or WriteFile return FALSE and GetLastError() != ERROR_IO_PENDING)
so the best always pass not 0 lpNumberOfBytes to api and use it, if api complete just with success. otherwise use GetOverlappedResult or if say you use BindIoCompletionCallback - you direct got dwNumberOfBytesTransfered in callback.
so in conceptually can use next code:
inline ULONG BOOL_TO_ERROR(BOOL f)
{
return f ? NOERROR : GetLastError();
}
HANDLE hFile = CreateFile(*, FILE_GENERIC_READ, FILE_SHARE_READ, 0,
OPEN_EXISTING, FILE_FLAG_OVERLAPPED, 0);
if (hFile != INVALID_HANDLE_VALUE)
{
UCHAR buf[0x200];
OVERLAPPED ov = {};
ULONG NumberOfBytesRead;
ULONG dwError = BOOL_TO_ERROR(ReadFile(hFile, buf, sizeof(buf), &NumberOfBytesRead, &ov));
switch (dwError)
{
case ERROR_IO_PENDING:
dwError = BOOL_TO_ERROR(GetOverlappedResult(hFile, &ov, &NumberOfBytesRead, TRUE));
if (dwError != NOERROR) goto __default;
[[fallthrough]];
case NOERROR:
DbgPrint("NumberOfBytesRead=%x\n", NumberOfBytesRead);
// use GetOverlappedResult(hFile, &ov, &NumberOfBytesRead, TRUE) here also possible
break;
__default:
default:
DbgPrint("dwError = %u\n", dwError);
}
CloseHandle(hFile);
}
I am calling ReadFile() WinAPI to copy the file contents to a char array, inside my VC++ code. Have placed GetLastError() immediately after ReadFile().
for( read some n no: of files)
{
FileRead(fp,destCharArray,ByesToRead,NoOfBytesRead,NULL);
int ret = GetLastError();
}
GetLastError() is returning 183 only when 1st file is read. For all
other file reads its returning 183. But eventhough 183 is returned the
contents of file are copied to charArray. And the problem is that the
file read does not happen for some 28th file (here too return status
is 183). Irrespective of successful or unsuccessful file read, 183 is
returned!
According to http://msdn.microsoft.com/en-us/library/windows/desktop/ms681382(v=vs.85).aspx
error code 183 means "ERROR_ALREADY_EXISTS".
What does the above error status signify in ReadFile() context.?
Can anyone kindly help me in figuring out why?
Your code is incorrectly calling GetLastError. You should only call GetLastError if the immediately prior Win32 API call failed, and that API returns status information through GetLastError.
Here the API in question is ReadFile. The documentation says:
Return value
If the function succeeds, the return value is nonzero (TRUE).
If the function fails, or is completing asynchronously, the return value is zero (FALSE). To get extended error information, call the GetLastError function.
In other words you must only call it if ReadFile returns FALSE.
Your code should look something like this:
if (!ReadFile(fp,destCharArray,ByesToRead,NoOfBytesRead,NULL))
{
DWORD err = GetLastError();
// handle error probably by raising exception
}
Your code is returning the error code for an earlier failure that is unrelated to the call to ReadFile.
The last error might result from calling CreateFile prior. This function sets the last error value to ERROR_ALREADY_EXISTS if you specify CREATE_ALWAYS or CREATE_NEW for dwCreationDisposition.
It is important to know that the last error can be set by any function. You should always check the return value of the function which indicates if the function failed.
I am building a Visual C++ WinSock TCP server using BindIoCompletionCallback, it works fine receiving and sending data, but I can't find a good way to detect timeout: SetSockOpt/SO_RCVTIMEO/SO_SNDTIMEO has no effect on nonblocking sockets, if the peer is not sending any data, the CompletionRoutine is not called at all.
I am thinking about using RegisterWaitForSingleObject with the hEvent field of OVERLAPPED, that might work but then CompletionRoutine is not needed at all, am I still using IOCP ? is there a performance concern if I use only RegisterWaitForSingleObject and not using BindIoCompletionCallback ?
Update: Code Sample:
My first try:
bool CServer::Startup() {
SOCKET ServerSocket = WSASocket(AF_INET, SOCK_STREAM, 0, NULL, 0, WSA_FLAG_OVERLAPPED);
WSAEVENT ServerEvent = WSACreateEvent();
WSAEventSelect(ServerSocket, ServerEvent, FD_ACCEPT);
......
bind(ServerSocket......);
listen(ServerSocket......);
_beginthread(ListeningThread, 128 * 1024, (void*) this);
......
......
}
void __cdecl CServer::ListeningThread( void* param ) // static
{
CServer* server = (CServer*) param;
while (true) {
if (WSAWaitForMultipleEvents(1, &server->ServerEvent, FALSE, 100, FALSE) == WSA_WAIT_EVENT_0) {
WSANETWORKEVENTS events = {};
if (WSAEnumNetworkEvents(server->ServerSocket, server->ServerEvent, &events) != SOCKET_ERROR) {
if ((events.lNetworkEvents & FD_ACCEPT) && (events.iErrorCode[FD_ACCEPT_BIT] == 0)) {
SOCKET socket = accept(server->ServerSocket, NULL, NULL);
if (socket != SOCKET_ERROR) {
BindIoCompletionCallback((HANDLE) socket, CompletionRoutine, 0);
......
}
}
}
}
}
}
VOID CALLBACK CServer::CompletionRoutine( __in DWORD dwErrorCode, __in DWORD dwNumberOfBytesTransfered, __in LPOVERLAPPED lpOverlapped ) // static
{
......
BOOL res = GetOverlappedResult(......, TRUE);
......
}
class CIoOperation {
public:
OVERLAPPED Overlapped;
......
......
};
bool CServer::Receive(SOCKET socket, PBYTE buffer, DWORD length, void* context)
{
if (connection != NULL) {
CIoOperation* io = new CIoOperation();
WSABUF buf = {length, (PCHAR) buffer};
DWORD flags = 0;
if ((WSARecv(Socket, &buf, 1, NULL, &flags, &io->Overlapped, NULL) != 0) && (GetLastError() != WSA_IO_PENDING)) {
delete io;
return false;
} else return true;
}
return false;
}
As I said, it works fine if the client is actually sending data to me, 'Receive' is not blocking, CompletionRoutine got called, data received, but here is one gotcha, if the client is not sending any data to me, how can I give up after a timeout ?
Since SetSockOpt/SO_RCVTIMEO/SO_SNDTIMEO wont help here, I think I should use the hEvent field in the OVERLAPPED stucture which will be signaled when the IO completes, but a WaitForSingleObject / WSAWaitForMultipleEvents on that will block the Receive call, and I want the Receive to always return immediately, so I used RegisterWaitForSingleObject and WAITORTIMERCALLBACK. it worked, the callback got called after the timeout, or, the IO completes, but now I have two callbacks for any single IO operation, the CompletionRoutine, and the WaitOrTimerCallback:
if the IO completed, they will be called simutaneously, if the IO is not completed, WaitOrTimerCallback will be called, then I call CancelIoEx, this caused the CompletionRoutine to be called with some ABORTED error, but here is a race condition, maybe the IO will be completed right before I cancel it, then ... blahblah, all in all its quite complicated.
Then I realized I dont actually need BindIoCompletionCallback and CompletionRoutine at all, and do everything from the WaitOrTimerCallback, it may work, but here is the interesting question, I wanted to build an IOCP-based Winsock server in the first place, and thought BindIoCompletionCallback is the easiest way to do that, using the threadpool provied by Windows itself, now I endup with a server without IOCP code at all ? is it still IOCP ? or should I forget BindIoCompletionCallback and build my own IOCP threadpool implementation ? why ?
What I did was to force the timeout/completion notifications to enter a critical section in the socket object. Once in, the winner can set a socket state variable and perform its action, whatever that might be. If the I/O completion gets in first, the I/O buffer array is processed in the normal way and any timeout is directed to restart by the state-machine. Similarly if the timeout gets in first, the I/O gets CancelIOEx'd and any later queued completion notification is discarded by the state-engine. Because of these possible 'late' notifications, I put released sockets onto a timeout queue and only recycle them onto the socket object pool after five minutes, in a similar way to how the TCP stack itself puts its sockets into 'TIME_WAIT'.
To do the timeouts, I have one thread that operates on FIFO delta-queues of timing-out objects, one queue for each timeout limit. The thread waits on an input queue for new objects with a timeout calculated from the smallest timeout-expiry-time of the objects at the head of the queues.
There were only a few timeouts used in the server, so I used queues fixed at compile-time. It would be fairly easy to add new queues or modify the timeout by sending appropriate 'command' messages to the thread input queue, mixed-in with the new sockets, but I didn't get that far.
Upon timeout, the thread called an event in the object which, in case of a socket, would enter the socket object CS-protected state-machine, (these was a TimeoutObject class which the socket descended from, amongst other things).
More:
I wait on the semaphore that controls the timeout thread input queue. If it's signaled, I get the new TimeoutObject from the input queue and add it to the end of whatever timeout queue it asks for. If the semaphore wait times out, I check the items at the heads of the timeout FIFO queues and recalculate their remaining interval by sutracting the current time from their timeout time. If the interval is 0 or negative, the timeout event gets called. While iterating the queues and their heads, I keep in a local the minimum remaining interval before the next timeout. Hwn all the head items in all the queues have non-zero remaining interval, I go back to waiting on the queue semaphore using the minimum remaining interval I have accumulated.
The event call returns an enumeration. This enumeration instructs the timeout thread how to handle an object whose event it's just fired. One option is to restart the timeout by recalcuating the timeout-time and pushing the object back onto its timeout queue at the end.
I did not use RegisterWaitForSingleObject() because it needed .NET and my Delphi server was all unmanaged, (I wrote my server a long time ago!).
That, and because, IIRC, it has a limit of 64 handles, like WaitForMultipleObjects(). My server had upwards of 23000 clients timing out. I found the single timeout thread and multiple FIFO queues to be more flexible - any old object could be timed out on it as long as it was descended from TimeoutObject - no extra OS calls/handles needed.
The basic idea is that, since you're using asynchronous I/O with the system thread pool, you shouldn't need to check for timeouts via events because you're not blocking any threads.
The recommended way to check for stale connections is to call getsockopt with the SO_CONNECT_TIME option. This returns the number of seconds that the socket has been connected. I know that's a poll operation, but if you're smart about how and when you query this value, it's actually a pretty good mechanism for managing connections. I explain below how this is done.
Typically I'll call getsockopt in two places: one is during my completion callback (so that I have a timestamp for the last time that an I/O completion occurred on that socket), and one is in my accept thread.
The accept thread monitors my socket backlog via WSAEventSelect and the FD_ACCEPT parameter. This means that the accept thread only executes when Windows determines that there are incoming connections that require accepting. At this time I enumerate my accepted sockets and query SO_CONNECT_TIME again for each socket. I subtract the timestamp of the connection's last I/O completion from this value, and if the difference is above a specified threshold my code deems the connection as having timed out.
This code is for an HTTPS server using blocking sockets:
request := '';
start := gettickcount;
repeat
if SSL_pending(ssl) > 0 then
begin
bytesin := SSL_read(ssl, buffer, sizeof(buffer)-1);
if bytesin > 0 then
begin
buffer[bytesin] := #0;
request := request + buffer;
end
else break; // read failed
end; // pending
until (gettickcount - start) > LARGETIMEOUT;
// "request" is ready, though possibly empty
SSL_pending() always returns zero and the SSL_read() is never reached. If the SSL_pending() call is removed, SSL_read() is executed. Why doesn't SSL_pending() indicate how many bytes are available?
Note that if you call SSL_read() and the number of bytes returned is less than your buffer size, you've read everything and are done.
If the incoming data is larger than your buffer size, the first SSL_read() call fills the buffer, and you can repeat calling SSL_read() until you can't fill the buffer.
BUT if the incoming data is an exact multiple of your buffer size, the last chunk of data fills the buffer. If you attempt another SSL_read() thinking there might be more data on a blocking socket, it hangs indefinitely. Hence the desire to check SSL_pending() first. Yet that doesn't appear to work.
How do you avoid hanging on a final SSL_read()? (I can't imagine the answer is to go non-blocking, since that means you could never use SSL_read with blocking.)
UPDATE: The following works. Apparently SSL_pending() doesn't work until after the first SSL_read():
request := '';
repeat
bytesin := SSL_read(ssl, buffer, sizeof(buffer)-1);
if bytesin > 0 then
begin
buffer[bytesin] := #0;
request := request + buffer;
end
else break; // read failed
until SSL_pending(ssl) <= 0;
// "request" is ready, though possibly empty
You are using SSL_pending() the completely wrong way. OpenSSL uses a state machine, where SSL_pending() indicates if the state machine has any pending bytes that have been buffered and are awaiting processing. Since you are never calling SSL_read(), you are never buffering any data or advancing the state machine.
If the SSL_pending function returns a return code of 0, it does not necessarily mean that there is no data immediately available for reading on the SSL session. A return code of 0 indicates that there is no more data in the current SSL data record. However, more SSL data records may have been received from the network already. If the SSL_pending function returns a return code of 0, issue the select function, passing the file descriptor of the socket to check if the socket is readable. Readable means more data has been received from the network on the socket.