I am using async_write to write a series of bytes to a serial port slave fd and x number of bytes are reported being transferred by the async_write_completion_handler. In a second thread, I am running a tight loop, doing a read() on the serial port master fd; the serial port is being created via pseudo-tty. If the number of bytes being transferred by the async_write exceeds 793, I see multiple reads occuring before the completion_handler is fired. The total number of bytes seen by the system read is always 25 bytes larger that the number of bytes pass to async_write. AND, an extra 25 bytes is added for each time 793 bytes are read: if the buffer is 2000 bytes, the total bytes from the read() will be 2050. In these extra-bytes situation, it appears that the bytes from the previous write are still in the buffer returned to the read().
Any help would be appreciated.
Related
In Go Should the "Sys" stat or any other stat/combination reported by runtime.ReadMemStats approximately equal the resident memory set reported by ps aux?
Alternatively, assuming some memory may be swapped out, should the Sys stat be approximately greater than or equal to the RSS?
We have a long-running web service that deals with a high frequency of requests and we are finding that the RSS quickly climbs up to consume virtually all of the 64GB memory on our servers. When it hits ~85% we begin to experience considerable degradation in our response times and in how many concurrent requests we can handle. The run I've listed below is after about 20 hours of execution, and is already at 51% memory usage.
I'm trying to determine if the likely cause is a memory leak (we make some calls to CGO). The data seems to indicate that it is, but before I go down that rabbit hole I want to rule out a fundamental misunderstanding of the statistics I'm using to make that call.
This is an amd64 build targeting linux and executing on CentOS.
Reported by runtime.ReadMemStats:
Alloc: 1294777080 bytes (1234.80MB) // bytes allocated and not yet freed
Sys: 3686471104 bytes (3515.69MB) // bytes obtained from system (sum of XxxSys below)
HeapAlloc: 1294777080 bytes (1234.80MB) // bytes allocated and not yet freed (same as Alloc above)
HeapSys: 3104931840 bytes (2961.09MB) // bytes obtained from system
HeapIdle: 1672339456 bytes (1594.87MB) // bytes in idle spans
HeapInuse: 1432592384 bytes (1366.23MB) // bytes in non-idle span
Reported by ps aux:
%CPU %MEM VSZ RSS
1362 51.3 306936436 33742120
We want to take the output of a 16-bit Analog to Digital Converter, which is coming at a rate of 10 million samples per second and SAVE the sequence of 16 output bits in a computer memory. How to save this 16-bit binary voltage signal (0V, 5V) in a computer memory?
If a FPGA is to be used, please elaborate the method.
Sample Data and feed to fifo
Take data from fifo and prepare UDP frames and send data over ethernet
Received UDP packets on PC side and put in memory
From what i've read around about winsock, recv has a tendency to not receive all the data from a sender in a single call. When people say this do they mean, for example, i send 300 bytes from a client and i call recv on the server, it's possible that it could only receive 200 some bytes on it's first call and the buffer will be filled with those 200 bytes? What happens to the last 100 bytes?
also, let's say a buffer is too small, like 512 bytes or something and the client sends 600. will the first recv call fill the buffer to capacity, and then just drop the last 88 bytes? if i call recv again with a different buffer, will the first 88 bytes of that buffer be the rest of the data?
and thirdly, if one and two are true, and i receive a whole packet of data in separate buffers, will i have to splice them together into one buffer to start parsing my data out of it?
I'm assuming TCP here.
is it possible that it could only receive 200 some bytes on it's first call and the buffer will be filled with those 200 bytes?
Yes.
What happens to the last 100 bytes?
You'll receive them next time.
also, let's say a buffer is too small, like 512 bytes or something and the client sends 600. will the first recv call fill the buffer to capacity, and then just drop the last 88 bytes?
No.
if i call recv again with a different buffer, will the first 88 bytes be the rest of the data?
Yes.
and thirdly, if one and two are true, and i receive a whole packet of data in separate buffers, will i have to splice them together into one buffer to start parsing my data out of it?
That's up to you. Do you need the data in one contiguous buffer? If so, yes.
What is the maximum number of bytes that can be read in a single read on an x86?
Should it be limited to size of MDR register,or the size of data bus,or the the design of physical memory (I.e how many cells are read in a single access).
I'm designing a system that will write time series data to a file. The data is blocks of 8 bytes divided into two 4 bytes parts, time and payload.
According to MSDN the WriteFile function is atomic ( http://msdn.microsoft.com/en-us/library/aa365747(VS.85).aspx ), if the data written is less than a sector in size.
Since the file will only contain these blocks (there is no "structure" of the file so it's not possible to reconstruct a damaged file), added one after each other, it's vital that the whole block, or nothing is written to the file at all times.
So the question is, have I understood it correctly that a writefile less than a sector in size is alway written completely to disk or not written at all, no matter what happens during the actual call to writefile ?
WriteFile is atomic as long as the write does not cross a sector boundary in the file. So if the sector size is 512 bytes, writing 20 bytes starting at file offset 0 will be atomic, but the same data written at file offset 500 will not be atomic. In your case the writes should be atomic, since the sector size should be a multiple of 8.
This MSDN blog has more information on how to do an atomic multi-sector write without using transacted NTFS.