Device misdetected as serial mouse - windows

Question

I'm working on a device which communicates with a PC through a (virtual) serial port. The problem is that the data we are sending occasionally gets incorrectly identified by Windows as a bus mouse, after which the "Microsoft Serial Ballpoint" driver is loaded and the mouse pointer starts jumping around on the screen and randomly clicking on things.
A bit of Googling reveals that is an old and well-known problem with serial devices where the usual work-around is a bit of registry hacking to disable the offending driver. That it is a lot to demand from our users however and I'd rather not have our application messing around with the user's registry. Especially not when the fix is dependent on the Windows version and the user may well be using a bus mouse.
Instead I'd like to avoid the problem by changing our protocol to not send any data which may get us misidentified as a mouse. The only problem is that I'm not quite certain what patterns to avoid.
Apparently Microsoft's Mouse protocol consists of packets of four bytes where the MSB of the first is set and that of the last three is clear.
Would sending only 7-bit ASCII suffice? Are there any other devices I need to worry about being detected as?

Answer 1

I just encountered this problem myself on Windows 7 Professional x64, and a solution that worked for me was to go into the registry and edit the following value:
Location: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\sermouse
Key: Start
Value: 3
Change Value to 4 and it will stop this problem occurring.
Here is a list of all valid Start values:
0 Boot (loaded by kernel loader). Components of the driver stack for the boot (startup) volume must be loaded by the kernel loader.
1 System (loaded by I/O subsystem). Specifies that the driver is loaded at kernel initialization.
2 Automatic (loaded by Service Control Manager). Specifies that the service is loaded or started automatically.
3 Manual. Specifies that the service does not start until the user starts it manually, such as by using Device Manager.
4 Disabled. Specifies that the service should not be started.
A reg edit command would be as follows:
REG ADD "HKLM\SYSTEM\CurrentControlSet\Services\sermouse" /V Start /T REG_DWORD /F /D 4
You then need to restart the computer, which should now start correctly and not attempt to discover a serial mouse.
good luck.

Answer 2

It turns out that mouse detection in Windows is normally handled by the serenum.sys filter driver. This driver implements support for legacy serial mice along with serial plug-and-play. Microsoft has even provided the sourcecode as a WDK sample.
During detection the ports switches to 1200-7-N-1 mode while asserting DTR+RTS to which a response is expected within 200 ms, with a couple of retries in case of failure. Unfortunately for a legacy mouse a single M or B character suffices as identification.
In our case the protocol was reworked to avoid these characters and now appears not to be misidentified anymore.
However we were using a virtual USB serial port and for a traditional serial port this approach may be somewhat difficult as anything sent at a different baud rate is liable to look like line noise. In this case I suppose the easiest workaround is probably, as has already been suggested, to avoid making any unsolicited transmissions.
Alternatively with the serial control signals actually hooked up, or intercepted by a USB CDC device, processing the DTR or RTS signals and holding off on output. Actually implementing the plug-and-play protocol would be an even niftier option. Supposedly there are cheap RS232 cables around without a full complement of control signals though so this approach might still fail.

Answer 3

I also encountered this problem, fixed it by disabling "serial enumerator" in the advanced properties of the FTDI driver (properties of COM ports in Device Manager). This is described in http://www.ftdichip.com/Support/Documents/AppNotes/AN_107_AdvancedDriverOptions_AN_000073.pdf.

Answer 4

I have encountered this Windows bug myself. Here is my own research on the topic:
Microsoft acknowledges this bug: http://support.microsoft.com/kb/819036
Start with downloading their tool and see if it solves the issue.
Download & install their program.
Run it from the command prompt from C:\program\Microsoft comdisable\
Write comdisable /list when executing the program.
All ports on the computer will be shown.
Write comdisable /disable COMx where x is the port number.
Do this for all ports on the computer.
Reboot.
This should hopefully work as an universal solution.
Alternatively, you can hack in boot.ini but I don't believe this works in Vista/Win 7. I have some app note from Cisco systems describing how to do this. If the above doesn't solve your problem, please let me know.

Answer 5

In my development environment, I've simply disabled Microsoft Serial Mouse from the Device Manager.
This seems to solve the culprit of my problem. Before doing so, the CH340G chip I've used in my design used to lower the DTR five times before initiating the connection, effectively rebooting my Arduino-based board and render it useless.

Answer 6

Maybe this helps:
We had the same problem with FTDI FT232RL.We found out, that it was a hardware issue of our PCB.
FTDI-Datasheet says about #RESET-Pin: Active low reset pin. This can be used by an external device to reset the FT232R. If not required can be left unconnected, or pulled up to VCC.
RESET-Pin was not required in our application, so we connected it to Vcc via 1k Pull-Up.
It seemed that the pull-up of #RESET-Pin caused an undefined start-up of the FT232RL, at least every second converter, we connected to a USB-socket caused a serial-ball-point in the devive manager. We removed the pull-up-resistor at #RESET-Pin, therewith the #RESET-Pin is unconnected. Since then every interface worked proberly and didn't any longer create serial-ball-points in the Windows device manager.

Answer 7

If you have a "true" serial port, or an USB dongle (RS-232, RS-485, it does not matter) this problem can be worked around by first opening the serial port in question with a terminal, or whatever application you want to monitor it with, and only then plugging the device in. For your own sake, you should also pay attention to remove the device before terminating the connection.
With FTDI chips soldered on the device itself, you are busted. It took a few rounds for me to explain the management that a device communicating on it's own paired with an FTDI chip soldered on the PCB meeting Windows computers won't likely pass for user-friendliness, no matter how slick an USB socket may look like on the cabinet... (Thankfully, all these conditions coming together are quite rare and unusual)

Answer 8

I had this problem since 2010 with serial scale heads connected to the pc. Usb to serial converter or not.. I use onkly SILABS device's CP2102 or the like.. I worked around it by simply allowing the driver to install and then in device manager look for the ballpoint driver under mouse/HIDA and then simply DISABLE the driver, DO NOT UNINSTALL IT simply disable it. Then when you reboot even with the driver instaled it seems windows ignores the comport as serial mouse and uses the data from the input. You will also find that if the ballpoint driver is active then that COMport is in use and sometimes returns a COM PORT not accessible... hope this helps some one out there :) Tx Ben

Answer 9

Code tot stop GPS from being detected as serial mouse.
Below is the code for a subroutine in C#.net. It checks if the registry key is set to 4 and if not it issues the configuration command to disable sermouse.
Embed this subroutine in a program which runs at startup and it will correct the setting after a windows update.
Maybe useful if you get annoyed when this problem happens time and again
private void Stop_sermouse()
{
string k =
"HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Services\\sermouse";
object v = Microsoft.Win32.Registry.GetValue(k, "Start", null);
if (v==null)
{
MessageBox.Show("No Registry Key for sermouse");
}
else
{
string sr = v.ToString();
if (sr == "4")
{; }
else
{
DialogResult mbox = MessageBox.Show("disable sermouse ? " + v.ToString(), "Found sermouse enabled! ", MessageBoxButtons.YesNo);
if (mbox == DialogResult.Yes)
{
// prepare a small job to issue confuguration command
ProcessStartInfo s = new ProcessStartInfo("cmd.exe", "/c sc config sermouse start=disabled");
Process p = new Process();
s.Verb = "runas"; // Must run as administrator
s.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
p.StartInfo = s;
// and run the command
p.Start();
//check if the registry is modified indeed
v = Microsoft.Win32.Registry.GetValue(k, "Start", null);
sr = v.ToString();
if (sr == "4")
{
MessageBox.Show("finished ''sc config sermouse start=disabled'' but not succesfull in registry!");
}
else
{
MessageBox.Show("sermouse is disabled");
}
}
}
}
}

Answer 10

Turns out there is a setting to stop windows trying to enmumerate devices that connect as a COM port.
Make sure that "Serial Enumerator" is unchecked under "Miscellaneous Options"