I need to set up USB communication between a Windows 7 host and a Linux device for data transfer. I was able to compile the Linux kernel on the device to include the Gadget Zero driver in the kernel (not as a loadable module - Linux version 3.0.15). My project has some requirements, which also explains why I chose Gadget Zero:
1) On the Windows 7 host, a kernel mode driver must be used to communicate over the USB connection for sending and receiving bulk data. (speed is not important, not a lot of data at once).
2) On the linux device, no requirements on USB side except send and receive data easily over USB link. The data received will eventually be "unmarshalled" to call functions in another kernel module (and those responses packaged and sent back to the host).
3) Multiple linux devices will be connected to the host, so need easy way to enumerate connected devices and communicate with them.
So due to the requirements, I decided against the Gadget Serial. I'm having serious issues sending and receiving data over the virtual COM port in kernel mode (KMDF) in Win 7 host. WinUSB does not seem to want to open my attached device (I'm using KMDF windows USB driver from template in VS2012) Also, the gadget serial driver on the linux side, I cannot find the functions where the data is received and sent. Plus, any received data on the linux device seems to be echoed back to the host for some reason. (and to test this, I wrote a simple user-mode app in Windows, which is a no-no for my project).
Gadget Zero, it appears much simpler on the linux side. I can plug the USB cable to the Win7 host, and I can get the device to appear in the device manager. However, again I am having problems with getting communication going over the link. Gadget Zero has 2 bulk endpoints, so this shouldn't be an issue. Surely, someone has made data communication possible between a Windows host and a linux device using Gadget Zero? With Gadget Zero, it should be easy to enumerate the connected linux devices and communicate with them.
The trick is to keep the Windows side communication in kernel mode. Can someone point me in the right direction perhaps with Gadget Zero, Windows 7 KMDF, and some sample source code? I have a hard time believing no one has done this before because my internet searches don't turn up much. (and mostly user-mode solutions with Gadget Serial).
Thanks!
So you're writing a Win32 driver in which you want to communicate with your linuxed usb? I haven't written much win32 kernel code, but I believe I've seen a huge section in the doc, saying something like "This is how you make usb drivers"... That'd be it. In other words, when in kernel mode you have access to the full kernel usb layer. You don't need an existing driver or whatnot.
On the linux side you can use the serial gadget, in a different run mode. Only the default run mode, registers it self as VCP. There exist a more basic mode:
modprobe g_serial use_acm=0
Give it your own vendor id and you'll be able to attach your very own custom win32 driver. The 'multiple linux devices' will be handled by Windows. (Multiple instances of your driver, will be initiated.)
The echo you're seeing btw, is most likely a terminal feature. (The terminal mode on uarts will echo.) You have to disable it, when connecting. And now that you're at it, you also have to disable the xon/xoff, esc chars etc. (Standard legacy rubbish.)
And another thing. I'm not sure the gadget zero actually sends the data onto the line. It's meant for testing the gadget framework. (I could be mistaken though.)
Anyway, you've prolly solved this issue years ago. I'd be nice to know what you came up with.
Related
I wonder if anyone can help at all, a bit of a specialist problem this.
I have an application that needs to read and analyse a number of USB devices (not simultaneously, they are each run in seperate tests and could in theory be run on different machines).
Each of the USB devices is based on the USB HID class, and are manufactured by different companies, none of these USB devices are designed to be run on PC, but are meant for a different platform, however for the purposes of testing the devices the client has requested that the test application is run from a PC.
Some of the devices will start up, be recognised by windows which will initialise and start them correctly using the generic HID class driver built into windows, the devices will then start sending correct data packets of the data to be tested.
Some of the devices will start up, be recognised by windows which will try to start them but fail to fully to initialise them leaving them in a half initialised state. This is fine, as I can use my beagle protocol analyser to capture the initialisation packets from the genuine platform and then use the LibUSBDotNet library to replicate the remaining packets in the initialisation sequence and get them to start sending the packets correctly.
The problem I have is with one particular device (though there are some more I haven't tested yet so it's quite possible one of those may also exhibit the same problem). The issue is the the Windows HID class driver recognises the device and trys to initialise and start it, this works after a fashion and the device starts sending data.
The problem is that the data being sent is different to that which is sent to the genuine platform (containing only a subset of the full data). It's as though windows has initialised the device into a different mode.
When I capture the initialisation packets from both the PC and the genuine platform using my USB protocol analyser I see that Windows is sending some slightly different initialisation packets. Using LibUSBDotNet to resend the correct packets once Windows has already started the device seems to have no effect.
My problem is that I need to stop windows from trying to initialise the device using the standard HID class driver, I've tried removing the driver in Device Manager but it still initialises it (and the driver is magically reassigned in device manager). I've done some investigation and there are possible alternatives:
Create a specific driver which windows will assign to the particular VID/PID of the device but that does nothing, then I can use LibUSBDotNet to send the correct initialisation sequence to the device from within my own code.
Use something like WinUSB to create a proper driver for the device (or possibly to create a "dead" driver like 1.
Will a driver with a specific VID/PID defined be used by windows in preference to it's inbuilt USB HID class driver? If not then I would be wasting my time going down this route?
Note, my mac initialises the problem device correctly, and I've asked the question of the client whether the application can be developed for Mac and their answer was frustrating Windows only.
I've no experience in writing proper Windows drivers, though I have experience in talking to USB at a relatively low level (so that part doesn't worry too much). Can anyone suggest a good course of action (before I potentially waste weeks investigating how to write drivers for the PC only to find my selected course of action can't deliver what I required).
Any help or suggest much appreciated.
Thanks,
Rich
Added after trying suggestions below:
I tried using the LibUsbDotNet inf wizard to create the necessary files and install them and this appeared to work - certainly the device was now appearing in Device Manager as a libusb-win32 device - not HID device and the associated driver was libusb driver. Even after doing this the device still seems to become initialised and start sending the wrong type of data packets although now those packets are no longer handled by the class driver and are just lost.
I also came across Zadig which has a similar inf creation wizard for WinUSB and this had exactly the same result.
A colleague has suggested that it might not be windows itself that is switching the device into this mode, rather the device identifying that it is connected to a windows machine and switching itself into this mode. I suspect this is the case, in which case I am stuck - time to have another conversation with the client.
Many thanks for the help.
You're using libusb-win32 as a filter driver; that is, the HidUsb device driver is assigned and loaded for your device, but then the libusb-win32 driver is loaded on top and gives you unobstructed access to the hardware.
If you don't want a HidUsb (or any other class driver) to perform any communication "on your behalf", simply associate libusb-win32 as a device driver with your hardware. For this, you'd have to create an .INF file associating it with the VID/PID/Revision of each USB device. If I recall correctly, libusb-win32 even comes with a utility to generate such .INF files.
If you install this .INF file e.g. with PnpUtil.exe (available on Vista or higher), you might still run into issues where, although you're a better match than the generic HID driver, the HID driver is still selected.
The generic HID driver matches devices by their Compatible IDs (i.e. by a USB interface class) while you'd be matching by Hardware IDs (which have higher priority). However, Windows might give priority to other aspects, such as your driver being unsigned. Read: How Windows Selects Drivers
Luckily, even in that scenario, signing drivers with a self-generated certificate (use CertUtil.exe, MakeCat.exe and SignTool.exe) is not too difficult.
I have an Arduino application talking over USB to an application on Windows 8 using the MAVLINK protocol. The connection appears as COM3.
Is there a Windows application that can spy on this connection and display the traffic going in both directions? Raw bytes are fine, I don't need the protocol decoded.
You could log serial port activity using Portmon. (Edit: You need to first connect to the local computer via the Computer menu, and you must start capture on the port before a program opens it.)
You may not want to log USB traffic. Such a log would include a lot of extra information relating to the USB to serial adapter which is providing COM3. Portmon would only give you the bytes transferred over COM3, and the Mavlink protocol is entirely contained within that data stream. If you're sure you want to log all USB traffic to and from that device, then I recommend SnoopyPro. In Windows 7, you need to run it as administrator.
If you can use Windows XP in your environment, USB sniff should work for you. If you need something more powerful (and are willing to pay a fee for it) then USBLyzer might be a viable option.
The answer is SnoopyPro, and you can download it at:
SnoopyPro Sourceforge
This tool allows you to get USB information and also USB communication data. I used it in the past to know how a USB device worked in order to do its driver on Linux. I used this tool as a sniffer.
Basically, SnoopyPro allows you to intercept, display, record and analyze the USB protocol and all transferred data between any USB device connected to your PC and applications. It can be successfully used in application development, USB device driver or hardware development and offers the powerful platform for effective coding, testing and optimization.
I'm developing a project that has a number of hardware sensors connecting to the deployment machine through RS232 serial ports.
But ... I'm developing on a machine without an physical RS232 serial ports, but I would like to make fake serial ports that I can connect to and output data from with the aim of faking input from hardware sensors.
Does anyone know of a way to create a fake serial port and control it on Windows XP?
If you are developing for Windows, the com0com project might be, what you are looking for.
It provides pairs of virtual COM ports that are linked via a nullmodem connetion. You can then use your favorite terminal application or whatever you like to send data to one COM port and recieve from the other one.
EDIT:
As Thomas pointed out the project lacks of a signed driver, which is especially problematic on certain Windows version (e.g. Windows 7 x64).
There are a couple of unofficial com0com versions around that do contain a signed driver. One recent verion (3.0.0.0) can be downloaded e.g. from here.
I know this is an old post, but in case someone else happens upon this question, one good option is Virtual Serial Port Emulator (VSPE) from Eterlogic
It provides an API for creating kernel mode virtual comport devices, i.e. connectors, mappers, splitters etc.
However, some of the advertised capabilities were really not capabilities at all.
EDIT
A much better choice, Eltima. This product is fully baked. Good developer tech support. The product did all it claimed to do. Product options include both desktop applications, as well as software development kits with APIs.
Neither of these products are open source, or free. However, as other posts here have pointed out, there are other options. Here is a list of various serial utilities:
com0com (current)
com0com - With Signed Driver (old version)
Yet another place for com0com with Signed Driver (Pete's Blog)
Tactical Software
Termite
COM Port Serial Emulator
Kermit (obsolete, but still downloadable)
HWVSP3
HHD Software (free edition)
I use com0com - With Signed Driver, on windows 7 x64 to emulate COM3 AND COM4 as a pair.
Then i use COM Dataport Emulator to recieve from COM4.
Then i open COM3 with the app im developping (c#) and send data to COM3.
The data sent thru COM3 is received by COM4 and shown by 'COM Dataport Emulator' who can also send back a response (not automated).
So with this 2 great programs i managed to emulate Serial RS-232 comunication.
Hope it helps.
Both programs are free!!!!!
There's always the hardware route. Purchase two USB to serial converters, and connect them via a NULL modem.
Pro tips:
1) Windows may assign new COM ports to the adapters after every device sleep or reboot.
2) The market leaders in chips for USB to serial are Prolific and FTDI. Both companies are battling knockoffs, and may be blocked in future official Windows drivers. The Linux drivers however work fine with the clones.
Another alternative, even though the OP did not ask for it:
There exist usb-to-serial adapters.
Depending on the type of adapter, you may also need a nullmodem cable, too.
They are extremely easy to use under linux, work under windows, too, if you have got working drivers installed.
That way you can work directly with the sensors, and you do not have to try and emulate data.
That way you are maybe even save from building an anemic system.
(Due to your emulated data inputs not covering all cases, leading you to a brittle system.)
Its often better to work with the real stuff.
i used eltima make virtual serial port for my modbus application debug work. it is really very good application at development stage to check serial port program without connecting hardware.
Do you know a way to use the Windows XP API to reset the USB bus? In other words, I'd like the OS to kick out any USB devices that are currently connected, and then auto-detect everything anew.
I'm aware of devcon, and I suppose I could do system calls out to it, but I'm hoping for a direct call into the API.
From kernel mode: You can force a specific USB device to be re-connected, as if it was unplugged and replugged again, by sending an IOCTL_INTERNAL_USB_CYCLE_PORT to its PDO. (This can only be done from a kernel mode, e.g. through a helper driver.) This 'cycle' operation will cause a USB reset to occur, after which the device would be re-enumerated. For example, if the device comes back with a different USB device descriptor, a different driver may be matched for it.
From user mode: You can do this by ejecting the device through the CfgMgr API. For example, to go over all USB hubs and eject all devices:
Find all devices having device interface GUID_DEVINTERFACE_USB_HUB with SetupDiGetClassDevs(... DIGCF_DEVICEINTERFACE).
Enumerate over the returned device information set (SetupDiEnumDeviceInfo).
For each device, get the DevInst member:
Invoke CM_Get_Child(DevInst) and then CM_Get_Sibling repeatedly to go over all child nodes of the hub (i.e. the USB devices).
For each child node, call CM_Request_Device_Eject.
Well, use can use the Setup API (SetupDiXXX functions) to enumerate the USB devices in the system, and then call WinUsb_ResetPipe on each one, but I'm not sure if that's what you're looking for. It's been a while since I worked with USB devices, but as I recall, there is no standard way to reset a device (i.e. simulate a power off/power on cycle). If it's possible for a particular device, you'd have to send an appropriate IOCTL (using DeviceIOControl) to the driver. The IOCTL would vary from manufacturer to manufacturer.
It's possible to cycle the parent port on the USB hub the device is attached to, as well. This will result in, among other things, apparrent unplug/replug actions, as you will see a balloon popup when this occurs.
Much of this is poorly documented, and honestly, I've gotten the impression there are only a handful of people at Microsoft who really understand it well. The design decision I've made for future devices I design is that I intend to include watchdog functionality on both sides, as well as a device-side full reset function. That way, if the device figures out it is confused, it can just cut its own power for a second and fully reset, if the host can't communicate with it, it could do the same thing, and if the device thinks everything is fine but the host knows better, the host could order it to reset.
There are at least three APIs worth looking into for this problem: the Setup API, the Config Manager API, and various WMI extensions. However, be cautious about diving into WMI if you intend to use an Embedded XP target, as you will have to include a lot of other things in your OS image you might otherwise not need.
As far as I know, there is no way to do this - you can issue a command to have PnP rescan the bus for new devices, but that isn't the same as issuing a bus reset.
Furthermore, just because from a hardware perspective you issued a bus reset doesn't mean that Windows will remove the PDOs that represent the children of the hub and redetect them; the USB bus driver can (and does) do just what I describe (i.e. issue hardware bus resets without disturbing the device tree), and only after the device doesn't respond does it issue the surprise removal and yank it from the tree.
From time to time, I need to dump USB traffic under Windows, mostly to support hardware under Linux, so my primary goal is to produce dump files for protocol analysis.
For USB traffic, it seems that SniffUsb is the clear winner... It works under Windows XP (but not later) and has a much nicer GUI than earlier versions. It produces huge dump files, but everything is there.
However, my device is in fact a USB serial device, so I turned to Portmon which can sniff serial port traffic without the USB overhead.
After five years waiting, now it's possible to sniff usb packets on windows
See http://desowin.org/usbpcap/tour.html for a quick tour. It works pretty well
Since people don't seem to realize it, Wireshark does monitor USB traffic and has a parser for it; but the catch is it only works under Linux. Wireshark on Windows will not do this.
It may be possible to plug the USB device you want to monitor, along with a Linux machine (with Wireshark running) and your Windows machine and just use the USB device under Windows.
Problem with the above? I don't know how the Linux machine or the Windows machine will detect each other.
Busdog, an open source project hosted on github, has worked well for me. It has a driver it installs to allow it to monitor USB communications. The config window allows you to reinstall or remove the device at any time.
You can select the USB device you want from an enumerated list. A nice feature is to have it automatically trace a new device that is plugged in:
Data communications to and from an SWR analyzer I was reverse engineering were captured flawlessly:
USBSnoop works too - and is free.
Or, you could buy a USB to Ethernet converter and use whatever network sniffer you prefer to see the data.
Personally, I'd use QEMU or KVM and instrument their USB passthrough code, and then use libusb to prototype the replacement driver in user space (this latter bit I've done before; writing USB device drivers in Python is fun!).
Microsoft Message Analyzer was able to capture USB traffic, with Device and Log File parser from MS: link
Update: as mentioned by #facetus, MS Message Analyzer has been retired on November 25 2019.