I set my report size to 64 bytes and want to stream single reports (say 2 for now) to the host. My understanding is that there is a ReadFile buffer where these reports can sit. At the host, I have a 64 byte buffer that I use to read single reports. If I send one report from the device, the host reads it fine. If I use two ReadFiles in a loop, the second ReadFile times out. The device is sending two reports. I don't know if they're getting on the ReadFile buffer at the same time, so when the host reads the end point for the first report, the buffer gets purged and I lose the second report? If there are indeed 2 reports on the ReadFile buffer, do I need to read them both at once? How would I know how many reports are on the buffer?
ReadFile reads as many reports as there are in the HID driver's ring buffer up to the numberOfBytesToRead parameter.
The respective HID driver will implement everything as needed. You need not worry about whether those packets arrive "simultaneously". They won't.
The first packet should tell you the length of the report (i.e. a collection of packets), which in turn should allow you to figure out whether you have the full report, yet.
Of course you will have to keep an internal representation of the data from the report, because the packet buffers can only be at most 64 byte in size according to the specification. So to collect a full report you will have to handle that yourself or use the Hid_* routines described in the WDK.
Related
We're trying to write a driver/API for a custom data acquisition device, which captures several "channels" of data. For the sake of discussion, let's assume this is a several-channel video capture device. The device is connected to the system via an 8xPCIe Gen-1 link, which has a theoretical throughput of 16Gbps. Our actual data rate will be around 2.8Gbps (~350MB/sec).
Because of the data rate requirement, we think we have to be careful about the driver/API architecture. We've already implemented a descriptor based DMA mechanism and the associated driver. For example, we can start a DMA transaction for 256KB from the device and it completes successfully. However, in this implementation we're only capturing the data in the kernel driver, and then dropping it and we aren't streaming the data to the user-space at all. Essentially, this is just a small DMA test implementation.
We think we have to separate the problem into three sections: 1. Kernel driver 2. Userspace API 3. User Code
The acquisition device has a register in the PCIe address space which indicates whether there is data to read for any channel from the device. So, our kernel driver must poll for this bit-vector. When the kernel driver sees this bit set, it starts a DMA transaction. The user application however does not need to know about all these DMA transactions and data, until an entire chunk of data is ready (For example, assume that the device provides us with 16 lines of video data per transaction, but we need to notify the user only when the entire video frame is ready). We need to only transfer entire frames to the user application.
Here was our first attempt:
Our user-side API allows a user application to register a function callback for a "channel".
The user-side API has a "start" function, which can be called by the user application, which uses ioctl to send a start message to the kernel driver.
In the kernel driver, upon receiving the start message, we started a kernel thread, which continuously monitors the "data ready" bit-vector, and when it sees new data, copies it over to a driver-allocated (kmalloc) buffer. It keeps doing this until the size of the collected data reaches the "frame size".
At this point a custom linux SIGNAL (similar to SIGINT, SIGHUP, etc) is sent to the process which is running the driver. Our API catches this signal and then calls back the appropriate user callback function.
The user callback function calls a function in the API (transfer_data), which uses an ioctl call to send a userspace buffer address to the kernel, and the kernel completes the data transfer by doing a copy_to_user of the channel frame data to userspace.
All of the above is working OK, except that the performance is abysmal. We can only achieve about 2MB/sec of transfer rate. We need to completely re-write this and we're open to any suggestions or pointers to examples.
Other notes:
Unfortunately, we can not change anything in the hardware device. So we must poll for the "data-ready" bit and start DMA based on that bit.
Some people suggested to look at Infiniband drivers as a reference, but we're completely lost in that code.
You're probably way past this now, but if not here's my 2p.
It's hard to believe that your card can't generate interrupts when
it has transferred data. It's got a DMA engine, and it can handle
'descriptors', which are presumably elements of a scatter-gather
list. I'll assume that it can generate a PCIe 'interrupt'; YMMV.
Don't bother trawling the kernel for existing similar drivers. You
might get lucky, but I suspect not.
You need to write a blocking read, which you supply a large memory buffer to. The driver read op (a) gets gets a list of user pages for your user buffer and locks them in memory (get_user_pages); (b) creates a scatter list with pci_map_sg; (c) iterates through the list (for_each_sg); (d) for each entry writes the corresponding physical bus address and data length to the DMA controller as what I presume you're calling a 'descriptor'.
The card now has a list of descriptors which correspond to the physical bus addresses of your large user buffer. When data arrives at the card, it writes it directly into user space, into your user buffer, while your user-level read is still blocked. When it has finished the descriptor list, the card has to be able to interrupt, or it's useless. The driver responds to the interrupt and unblocks your user-level read.
And that's it. The details are nasty, of course, and poorly documented, but that should be the basic architecture. If you really haven't got interrupts you can set up a timer in the kernel to poll for completion of transfer, but if it is really a custom card you should get your money back.
I am using a FT232RL chip with FTD2XX_NET.dll I've made a program which writes and reads data to/from AVR atmega32 mcu. First writes data, then reads data as answer.
Now, i want to make an event which indicated me if there's available unreaded data, only when AVR sends data to FTDI buffer and ONLY then. Whithout forcing my program to making loops for checking available data. For my purpose, i want to do the mcu to sends data only when he wants, and the PC must to knows when there's new data in FTDI buffer's chip.
I know that It's impossible for the pc to know when AVR sending data to the FTDI. But this which I mean it's that I need some way for my program to know if FTDI have New unreaded data to it's own buffer.
I don't won't to running read operator over and over in an infinity loop as I do now.
You should create a read thread which does your reading in the background. Then from that thread you can signal an even to notify another part of your application when you have data. I'm not sure what language you are using but you should easily be able to find an example of threading and event notification with a Google search.
I am using EVK1105 development board with AVR Studio 5 as development IDE for my AVR project.
I am using FreeRTOS in it. I have 3 USART ports on this board. One external module is connected to my AVR32 board via USART-RS232 mode. It sends me continuous serial data to my board on USART0 with 19230 baudrate, 7-databits, odd parity, stopbit-1 and normal-channel mode. I created a new task for this purpose. After each 9 data bytes it sends '\n' and '\r'. So in my task I keep on collecting the 9 databytes in a string buffer and then transmit it on USART1. I am using polling method to collect data from USAR0 which is receiving port. But I am facing problem in receiving data. I don't know if its timing issue or something or the scheduler switches the task while collects the data. But I don't get the required data.
Following are things I have already checked as troubleshooting
1. Connected my external module to my PC hyper-terminal which gives me perfect result.
2. Implemented the same thing of using receiving from USART0 and whatever received is transmitted to USART1 as without FreeRTOS. Its works fine.
Please suggest some idea what may be wrong. I am using a queue to communicate between Tx and Rx task to pass the string buffer from USART0 to USART1. Is it problem in handling queue? How can I troubleshoot the queue?
I am using a delay of 50ms in my infinite task loop in Rx Task. Can it create a problem? If I don't use any delay the OS crashes. Please suggest some good practices to create a new task in FreeRTOS so that I will not get any timing issue.
For such a use case, I would not use a polling method with 50ms delay to retrieve data from UART peripheral. You can easily lose received data depending on the system load and UART reception buffer size.
At least use an interrupt on UART data reception that copies every received byte into a local buffer that will be read by your TX thread.
You can have an even better solution using a DMA channel to receive your data frame and be notified when 9 bytes have been received. I don't know if your AVR device has a DMA peripheral or not.
Are you still working on this? The statement of your problem is vague, but there I have several suggestions/leading questions.
1) You may want some documents to see what the registers are
Get the giant datasheet pdfs at
http://www.atmel.com/dyn/products/product_docs.asp?category_id=163&family_id=607&subfamily_id=2138&part_id=4117
2) In this and an earlier post you state that you have, in some cases, been able to RX data. You will need to find the USART HW initialization code from those example projects and get them into the freeRTOS example project. In particular calls to
gpio_enable_module() with {AVR32_USART0_RXD_0_0_PIN, AVR32_USART0_RXD_0_0_FUNCTION}
To connect to USART to CPU
and i believe
InitRs232()
Just doing this requires poking around a lot of code - there's alot of dependencies.
2) What function are you calling to retrieve data from USART0? 19kbaud is approximately 2000bytes/sec or 1 byte/0.5ms, so 50ms polling is not nearly enough. I'd suggest that your RX task poll continuously (never sleep explicitly) but at a lower priority than the TX task.
3) Concentrate on debugging the RX task at the call to retrieve data. Use the debugger to look at the hardware registers for the usart. In particular
USART0 cr register AVR32_USART_CR_RXEN_MASK should be set to enable RX
USART0 csr register AVR32_USART_CSR_RXRDY_MASK will indicate if there is new data there
You can also check the overlow flag to see if you have missed some data.
When the read of USART0 rhr occurs it should be a byte that you sent.
If you are still working on this I can look into this a bit more.
I'm processing Midi on the iPad and everything is working fine and I can log everything that comes in and all works as expected. However, in trying to recieve long messages (ie Sysex), I can only get one packet with a maximum of 256 bytes and nothing afterwards.
Using the code provided by Apple:
MIDIPacket *packet = &packetList->packet[0];
for (int i = 0; i > packetList->numPackets; ++i) {
// ...
packet = MIDIPacketNext (packet);
}
packetList->numPackets is always 1. After I get that first message, no other callback methods are called until a 'new' sysex message is sent. I don't think that my MIDI processing method would be called with the full packetList (which could potentially be any size). I would have thought I would recieve the data as a stream. Is this correct?
After digging around the only thing I could find was this: http://lists.apple.com/archives/coreaudio-api/2010/May/msg00189.html, which mentions the exact same thing but was not much help. I understand I probably need to implement buffering, but I can't even see anything past the first 256 bytes so I'm not sure where to even start with it.
My gut feeling here is that the system is either cramming the entire sysex message into one packet, or breaking it up into multiple packets. According to the CoreMidi documentation, the data field of the MIDIPacket structure has some interesting properties:
A variable-length stream of MIDI messages. Running status is not allowed. In the case of system-exclusive messages, a packet may only contain a single message, or portion of one, with no other MIDI events.
The MIDI messages in the packet must always be complete, except for system-exclusive.
(This is declared to be 256 bytes in length so clients don't have to create custom data structures in simple situations.)
So basically, you should look at the declared length field of the MIDIPacket and see if it is larger than 256. According to the spec, 256 bytes is just the standard allocation, but that array can hold more if necessary. You might find that the entire message has been crammed into that array.
Otherwise, it seems that the system is breaking the sysex messages up into multiple packets. Since the spec says that running status is not allowed, then it would have to send multiple packets, each with a leading 0xF0 byte. You would then need to create your own internal buffer to store the contents of these messages, stripping away the status bytes or header as necessary, and appending the data to your buffer until you read a 0xF7 byte which denotes the end of the sequence.
I had a similar issue on iOS. You are right MIDI packets number is always 1.
In my case, when receiving multiple MIDI events with the same timestamp (MIDI events received at the same time), iOS does not split those multiple MIDI events in multiple packets, as expected.
But, fortunately nothing is lost ! Indeed instead of receiving multiple packets with their correct number of bytes, you will receive a single packet with multiple events in it and the number of bytes will be increased accordingly.
So here what you have to do is:
In your MIDI IN callback, parse all packets received (always 1 for iOS), then for each packet received you must check the length of the packet as well as the MIDI status, then loop into that packet to retrieve all MIDI events in the current packet.
For instance, if the packet contains 9 bytes, and the MIDI status is a note ON (3 bytes message), that means your current packet contains more than a single note ON, you must then parse the first Note ON (bytes 0 to 2) then check the following MIDI status from byte 3 and so on ..
Hope this helps ...
Jerome
There is a good reference of how to walk through a MIDI packet in this file of a GitHub project : https://github.com/krevis/MIDIApps/blob/master/Frameworks/SnoizeMIDI/SMMessageParser.m
(Not mine, but it helped me solve the problems that got me to this thread)
A Win32 application (the "server") is sending a continuous stream of data over a named pipe. GetNamedPipeInfo() tells me that input and output buffer sizes are automatically allocated as needed. The pipe is operating in byte mode (although it is sending data units that are bigger than 1 byte (doubles, to be precise)).
Now, my question is this: Can I somehow verify that my application (the "client") is not missing any data when reading from the pipe? I know that those read/write operations are buffered, but I suppose the buffers will not grow indefinitely if the client doesn't fetch the data quickly enough. How do I know if I missed something? Does the server (or the pipe?) silently discard data that is not read in time by the client?
BTW, can I rely on proper alignment of the data the client reads using ReadFile()? As far as I understood, ReadFile() may return with less bytes read than specified, i.e. NumberOfBytesRead <= NumberOfBytesToRead. Do I have to check every time that NumberOfBytesRead is a multiple of sizeof(double)?
The write operation will block if there is no more room in the pipe's buffers. This is from my (old) copy of the SDK manual:
When an application uses the WriteFile
function to write to a pipe, the write
operation may not finish if the pipe
buffer is full. The write operation is
completed when a read operation (using
the ReadFile function) makes more
buffer space available.
Sorry, didn't find out how to comment on your post, Neil.
The write operation will block if there is no more room in the pipe's buffers.
I just discovered that Sysinternals' FileMon can also monitor pipe operations. For testing purposes I connected the client to the named pipe and did no read operations, just waiting. The server writes a few hundred kB to the pipe every 4--5 seconds, even though nobody is fetching the data from the pipe on the client side. No blocking write operation ... And so far no limits in buffer-size seem to have been reached.
This is either a very big buffer ... or the server does some magic additional to just using WriteFile() and waiting for the client to read.