Debugging Visual Basic Runtime Error 5 - vb6

I have a Visual Basic 4 32 bit application which results in a
Run-time error '5':
Invalid procedure call
I start WinDbg and do
sxe *
to break on all exceptions. However, it displays the message without breaking into the debugger before.
If I break at the time of the message, the callstack is not very useful. Is there a trick to break-in on Visual Basic 4 applications at the time the problem occurs?
Problem FAQ:
Can you run it in the IDE?
Yes I can and I can probably fix the bug when I do that. However, I'd like to get some more information at the time the end user runs the application so that I don't need to be able to reproduce it.
What do you want to do in case you find the cause of the error?
Well, it depends on the type of the error. Maybe I can fix it, maybe I can't. Usually I get some very helpful information out of WinDbg.
When does the error occur?
It occurs reproducibly when the user presses a close button on a form.
What does the application do?
Oh, that's a good question. I don't exactly know. It uses RS232 communication but it's not yet involved.
Do you know anything about the data the user can enter?
I almost know nothing about it. In this particular case, it's not necessary to enter data.

The error is from the VB virtual machine not from an API.
I can't see how WinDbg would be useful on VB4 which is interpreted and is in P-Code.
Visual Basic run-time error 5 Indicates one of the following:
An invalid collection or property name was entered.
An out parameter was NULL.
The value is not one of the supported values or falls outside the supported range.
The property is read-only.
The property cannot be changed after the object is created.
An invalid index was entered.
And in case it's an API call VB uses LoadLibrary and GetProcAddress.
Some controls have lost default values over time and this can cause the first.
The third can happen if modern technology exceeds the program's expection, e.g. disk space.
Also see (sorry no links, they are on my hard disk)
Q131007 HOWTO: Use Windows NT WinDbg.exe with Visual Basic (it's for NT 3.5)
Q166275 HOWTO: Debug a Native Code Visual Basic Component in VC++ (talks about what you can see in VB in a debugger)
VB4 can't make symbols

As the error occurs when the users presses the close button i assume there is a (very tight) loop running which doesn't receive the expected data (probably a nullstring)
I expect that that loop is continuously reading data from the RS232 device and the RS232 is on the form which is being closed, even though it is not involved the connection might already be open and polling
What you should do is exit that loop and close the connection with the RS232 device before closing the form. You might be able to do that in the Form_Unload() event, but it might also have to take place before that, so you might have to do it in the code of the close button ... make sure though that the user won't be using the X to close the form though
Personally i would run the program from within the IDE, this will point you to the errornous part of the code directly and gives you the opportunity to watch the contents of the variables involved and control states ... that is much faster than taking a guess based on the data of WinDbg

Related

The thread '<No Name>' (0xb24) has exited with code 0 (0x0)

whenever i try to run program for example,
if i have to run "frmphonebook" so in
Application.Run(new frmphonebook());
I typed but when i run it it run another form, and it happens to each and every form and it is displaying output as
The thread 'vshost.RunParkingWindow' (0x63c) has exited with code 0 (0x0).
The thread '<No Name>' (0xb24) has exited with code 0 (0x0).
how to solve this ?
You can give your threads a name it would also help you in your debugging...
But in many apps threads are created implicitly and you have no control over the name.
So that is not an error message. Code 0 means everything went according to plan. Any non-zero code usually indicates an error.
edit: You can also disable the display of these messages, when debugging, do right click on output, and choose what do you want see.
If a thread has exited with code 0 it ran successfully. On Codeproject is a Beginners-Guide-to-Threading
This article on threading might also be helpfull. This question on so could also be of use. A list of System Error Codes
One of the things you will learn about using the Debugger is that you will see what we might call "the soft white underbelly" (an allusion to alligators' anatomy) of the system: all kinds of DLLs being loaded and unloaded, the somewhat complex arrangement of "helper" threads being started and stopped... etc.
It can be distracting to a less experienced user, to see all of these messages. However, over time, you will come to understand that the Debugger is simply being truthful and verbose. The details it is displaying for you might not really be relevant to your debugging process, but it cannot "know" that; it is only displaying factual information, and you have to sort out what is relevant and what is not.
As for Windows Forms applications, I have myself noticed that there seem to be several "helper" threads, typically with no name, or (as is frequently seen by me when debugging), they are named things like "vshost.RunParkingWindow". Typically, you have to trust that the system is creating threads on your behalf, in addition to any threads you might create yourself. As others have suggested, give your own threads meaningful names so you can tell them apart from the system's threads.
You can get further insight into the multithreaded structure of your Windows Forms app by putting a breakpoint somewhere in your UI update code, and when it hits, use Debug/Windows/Threads to bring up a view of all the threads running in your process space. You'll be quite surprised, I think, by how many there are! Try creating and .Show()-ing several forms in your app, one by one. I think you'll see that each .Show() operation creates a new window, and with it, several supporting threads for that window.
You may also see messages in the debug window such as the following: "A first chance exception of type 'System.ObjectDisposedException' occurred in System.Windows.Forms.dll". Many times there are system exception handlers that perform a reasonable default action on your behalf. This message appearing without a break in the debugger indicates that some default handler took care of this exception for you.
The system support for something like a Windows forms application is somewhat complicated, to make YOUR implementation easier and simpler. When you run the debugger, you get to see some of these details. Over time, you will learn what is "usual" and what is indicative of a problem.
Check to see if there are some files in your web app that have been rendered inaccessible. For my case my chart control created a text file which was read only and it threw an exception. Deleted the file and the folders and voila
i found your solution i think....i the visual studio go to project >properties >linker >system look for the Subsystem line and click the down arrow and change to Console(....words....).
it worked for me !! ENJOY"

Troubleshooting VB6 App Crash after XP to Win7 Upgrade

I have a VB6 application that I provide support for. This application works on both Windows XP and Windows 7. Some users were migrated from Windows XP to Windows 7 using the User State Migration tool. These users now receive a generic "Application has crashed" Windows error message when they open certain screens (forms) in the application. My assumption is that there is a missing dll/ocx reference, but I'm having trouble tracking it down.
I've tried many/varied troubleshooting techniques:
Full uninstall and reinstall of my application
Manually re-registering all dll's and ocx's that I know are used
Running Process Monitor on a broken computer and a working computer to compare what dll's and ocx's are accessed. The answer might be here but even after filtering out most of the background noise the amount of data is overwhelming. At a minimum I reviewed all of the calls right before it crashes and all of the calls that were not successful. All of the non-successful calls match between working and non-working.
Installed the Windows Debugger Tools and captured a crash dump. Analyzed the crash dump with DebugDiag. DebugDiag says the exception is in msvbvm60.dll. I tried building a PDB file for my exe and loading it in DebugDiag to get more detail about where the exception is occuring but DebugDiag doesn't want to accept the PDB (might be doing something wrong here, but it just seems to ignore it. This same PDB file works fine when I do remote debugging, however.)
I recompiled my VB6 program without any optimizations in PCode. I've read online that sometimes building in PCode, while bad for performance, will tell you the real exception.
Used the above created PDB file to remote debug the VB6 application. The debugger says that the application crashes after the new window has been created, on a line that sets MousePointer = vbHourGlass... To me it seems unlikely that this is the real cause of the error. There are at least 20 other locations in the program where this same line is called and all work fine.
(Forgot about this one)
Used Dependency Walker and profiled the application on both a working and non-working computer. All errors found by dependency walker were the same between the two computers. There were no additional dependencies found on a working computer, and all missing dependencies on the non-working computer were also missing on the working one.
None of these actions changed my error message or showed me what the error is (unless it really is the mouse cursor issue)... There are no entries in the Windows Event Log related to the app crash.
The non-working and working computers all have the same base Windows 7 image, the only difference is whatever is being changed by USMT, which further convinces me that this is some kind of quirky configuration change or a missing dll/ocx or perhaps an unregistered dll/ocx.
Any ideas or thoughts on how I can track down the root cause of the issue would be greatly appreciated.
Update 1 - Response to questions
#MarkHall I have tried running it as admin, though not with UAC off. The application runs fine on a Windows 7 box as a non-admin with full UAC. Windows XP was 32-bit, Windows 7 is 64-bit, but again it works just fine on a like for like box where the user was not migrated from Windows XP.
#Beaner It's possible that it stores settings somewhere that have been corrupted, but the remote debugging leads me to think that it's more likely something else since it seems to die on a step related to the UI, which then makes me think it's probably a missing dll/ocx reference.
#Bob77 The application is installed into Program Files (x86). While many of the libraries do reside in the same folder, they are all registered.
Peter, often I've noticed that the debugger will indicate a line of code that is actually incorrect, depending on WHERE in the actual assembly language the fault occurs. You should look REAL close around your statement that sets the cursor to vbHourGlass. Your exception is PROBABLY happening BEFORE that line of code, but that line is what the debugger thinks is the actual faulted line of code.
Since you said it happens when a window OPENS, I'd look real close at any ocx's you may have referenced on the form, but perhaps NOT actually being used, or called. You might have one there that you don't intend to be there, that could be causing security issues, or something on Win7? Edit the .frm file by hand if you have to, and look at all the GUIDs the form references.
It is possible that one machine is using PER-USER registration, and the other is using PER-SYSTEM registration?? I don't know...
I would take a much closer look at the form that you are trying to open, and be VERY cautious of everything you are doing in the form load events, and so on. This sounds like it could be something as stupid as Windows Aero being enabled on one system, and not another, or some other sort of "Theme" setting that is throwing the VB Form Rendering routine into a hissyfit... Perhaps even something as stupid as a transparent color index in the icon you selected for that from?
If you are still developing this app, (or at least maintaining it), create an entirely NEW form, and re-create all the controls, etc, on the form (resist the temptation to copy/paste them from the old one...), and then see if THAT does the trick. Then, copy all the event code to the new form one event at a time, with at LEAST enough event code to make the form function, even if it's just a "dead form", that loads no data, or whatever the form is supposed to do. Check and debug after each change, and you WILL find it eventually. Of course, make sure you isolate one of the defunct systems to have a platform that you can duplicate the issue on, or then it's just guessing. I find that using something like Acronis w/ Universal Restore is a great option to then take the image file into a good HV, like VirtualBox, and then restore that image as a VM, so you can debug without interfering with your actual users. This sounds like a lot of work, but then again, so is re-writing an application that already exists, right? :)
Failing THAT... /* and */ are your friends!! (Well, we're dealing with VB, so ' would be your best friend! heh... But I'd start commenting out all the code on the form until that sucker opens. Then once it opens, start putting one line back at a time, and re-running it... That's called "VooDoo Debugging", but sometimes, you gotta do what you gotta do...
THANKS A LOT PETER! :) Now you got ME so involved in this, I feel like I'M the one debugging this sucker! Like if it was MY code I was trying to fix! :)
Let me know if any of this helps... I am actually quite interested in what you discover.

Windows 7 and VB6: Event Error ID 1000

I have a completely random error popping up on a particular piece of software out in the field. The application is a game written in VB6 and is running on Windows 7 64-bit. Every once in a while, the app crashes, with a generic "program.exe has stopped responding" message box. This game can run fine for days on end until this message appears, or within a matter of hours. No exception is being thrown.
We run this app in Windows 2000 compatibility mode (this was its original OS), with visual themes disabled, and as an administrator. The app itself is purposely simple in terms of using external components and API calls.
References:
Visual Basic for Applications
Visual Basic runtime objects and procedures
Visual Basic objects and procedures
OLE Automation
Microsoft DAO 3.51 Object Library
Microsoft Data Formatting Object Library
Components:
Microsoft Comm Control 6.0
Microsoft Windows Common Controls 6.0 (SP6)
Resizer XT
As you can see, these are pretty straightforward, Microsoft-standard tools, for the most part. The database components exist to interact with an Access database used for bookkeeping, and the Resizer XT was inserted to move this game more easily from its original 800x600 resolution to 1920x1080.
There is no networking enabled on the kiosks; no network drivers, and hence no connections to remote databases. Everything is encapsulated in a single box.
In the Windows Application event log, when this happens, there is an Event ID 1000 faulting a seemingly random module -- so far, either ntdll.dll or lpk.dll. In terms of API calls, I don't see any from ntdll.dll. We are using kernel32, user32, and winmm, for various file system and sound functions. I can't reproduce as it is completely random, so I don't even know where to start troubleshooting. Any ideas?
EDIT: A little more info. I've tried several different versions of Dependency Walker, at the suggestion of some other developers, and the latest version shows that I am missing IESHIMS.dll and GRPSVC.dll (these two seems to be well-known bugs in Depends.exe), and that I have missing symbols in COMCTRL32.dll and IEFRAME.dll. Any clues there?
The message from the application event log isn't that useful - what you need is a post mortem process dump from your process - so you can see where in your code things started going wrong.
Every time I've seen one of these problems it generally comes down to a bad API parameter rather than something more exotic, this may be caused by bad data coming in, but usually it's a good ol fashioned bug that causes the problem.
As you've probably figured already this isn't going to be easy to debug; ideally you'd have a repeatable failure case to debug, instead of relying on capturing dump files from a remote machine, but until you can make it repeatable remote dumps are the only way forwards.
Dr Watson used to do this, but is no longer shipped, so the alternatives are:
How to use the Userdump.exe tool to create a dump file
Sysinternals ProcDump
Collecting User-Mode dumps
What you need to get is a minidump, these contain the important parts of the process space, excluding standard modules (e.g. Kernel32.dll) - and replacing the dump with a version number.
There are instructions for Automatically Capturing a Dump When a Process Crashes - which uses cdb.exe shipped with the debugging tools, however the crucial item is the registry key \\HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\AeDebug
You can change your code to add better error handling - especially useful if you can narrow down the cause to a few procedures and use the techniques described in Using symbolic debug information to locate a program crash. to directly process the map files.
Once you've got a minidump and the symbol files WinDbg is the tool of choice for digging into these dumps - however it can be a bit of a voyage to discover what the cause is.
The only other thing I'd consider, and this depends on your application structure, is to attempt to capture all input events for replay.
Another option is to find a copy of VMWare 7.1 which has replay debugging and use that as the first step in capturing a reproducible set of steps.
Right click your executable object and let it be WINXP compatible pending
when you discover source of the problem to finally solve it

How to prevent error pop-up message box for failed program (.exe) when running batch file

I'm running a test script from batch file.
Because it is test, the programs are expected to fail once in a while. It is file as long as error code is returned so I can continue and mark specific test as failed.
However there is very annoying behavior of executable files under Microsoft Windows - if something fails it pop-ups window like:
This application has failed to start because foo.dll was not found, Re-installing the application may fix the problem
<OK>
Or even better:
The instruction at "..." referenced to memory at "..." ..
Click on OK to terminate the program
Click on CANCEL to debug the program
The result is known - the script execution blocks till somebody presses "Ok" button. And when we talk about automatic scripts that may run automatically at night in some headless virtual machine, it may be very problematic.
Is there a simple way to prevent such behavior and just make an application to exit with failure code - without changing the code of the program itself?
Is this possible at all?
The answer is following: You need to disable WER.
Simplest description for this I found at http://www.noktec.be/archives/259
Simply (ON XP): Right Click on My Computer > Advanced > Error Reporting > Disable
Voila - programs crash silently!
This does not solves problem when DLL is missing, but this is much rare case and this is good enough for me.
You can suppress AV's and such from showing a dialog box by running your application, or the script (the script engine, like cscript.exe), under a debugger.
Use Gflags.exe, or modify the registry directly, and set Image File Execution Options for the image in question. See this article for details on how to use the appropriate registry keys. You can set it up using a debugger commandline like "C:\Debuggers\ntsd.exe -g -G -c'command'", where you can pass commands to ignore certain types of exceptions in the -c"commmand" argument. This will effectively give you a tool to suppress interactive dialogs as a result of exceptions like AV, and will let the process continue (presumably to immediate end after the exception has occured).
This article explains the commands you can use to control exceptions and events from withing the debugger.
The -g and -G flags make sure that the process won't break into the debugger automatically during process start and end respectively. You'll have to play with the various exception suppression options to make sure that you 'eat' all possible first and second chance exceptiosn that might cause the process to break into the debugger.
Also, if you can tolerate a process being broken into the debugger (as against being stuck showing a dialog box), perhaps that would be a better option overall. You can evaluate each debug break in batch mode at a later time and decide which bugs you care to fix.
It is possible. We used to use IBM's Rational Robot product which could monitor the screen for specific items and, if found, send keystrokes to windows and other sorts of things.
We actually used it for fully automated unit and system testing, much like you're trying to do.
Now I thought that Robot has been through quite a few name changes so it may be hard to find but there it is, right on IBM's web page and with a free downloadable trial for you. It's not cheap, clocking in at a smidgeon under USD5,000 but it was worth it for us.
There's also TestComplete where you could get a licence for just unedr USD1,000 - it touts "Black-box testing - Functional testing of any Windows application" as one of its features and also has a downloadable demo to see if it's suitable before purchase.
However, you may be able to find another product to do the same sort of thing.
I initially thought of Expect but the ActiveState one seems to concentrate on console applications which leads me to believe it may not do graphics well.
The only other option I can suggest is to write your own program in VBScript. I've done this before to automate the starting of many processes (log on to work VPN, start mail, log in and so on) so I could be fully set up with one mouseclick instead of having to start everything manually.
You can use AppActivate to bring a window to the foreground and SendKeys to send arbitrary keypresses to it after that. It's possible you may be able to cobble together something from that if you want a cheaper solution.

What is the "Cannot set allocations" error, who emits it and what can I do about it?

We've been plagued for several years by occasional reports from customers about a non-descript error message "Cannot set allocations" that appears on startup of our app. We have never been able to reproduce the problem in our own test environments so far. I have now run out of ideas for attempting to track this down. Here's a collection of observations that have accumulated over time:
Error message text reads "Cannot set allocations" (note absence of punctuation).
The window title simply reads "Error" (or the localized equivalent).
The "Cannot set allocations" text is always in English regardless of OS locale.
I have so far not been able to locate the DLL or EXE containing the message text.
Google is chock full of reports of this error for a variety of different products - but no solutions.
The only unifying aspect between the affected products I could make out so far was that they all appear to come in the form of DLLs that load into third-party processes (such as addins for Visual Studio or Windows Explorer shell extensions).
Our app is actually a shareware COM-addin for MS Outlook, written in Delphi (i.e. native code - no .NET).
The prime suspect in our case is the third-party licensing wrapper that we're using which decrypts and uncompresses our DLL into memory on the fly. Obviously I couldn't simply give an unprotected version of our app to the affected customers to verify this suspicion. Maybe the other vendors that this has been reported against are using similar products.
Debug versions of the protection wrapper supplied to us by the licensing vendor yielded no results: The log files looked exactly the same as from sessions where the error did not occur. Apparently the "inner" DLL gets decrypted and uncompressed all right but for some reason still fails to get loaded by the host process.
By creating an unprotected "loader" DLL we have been able to pinpoint the occurrence of the error somewhere behind the LoadLibrary call that is supposed to load our DLL into memory.
Extensive logging and global exception hooks in our own code (both the unprotected loader and the protected "core"-DLL) yielded no results at all. The error is obviously raised somewhere else.
The problem described in this earlier question of mine was very probably prompted by the same issue. This was before we created the unprotected loader stub.
The error only occurs at about 1-2% of our customers - whereas typically all installations at any affected customer's site are affected the same.
Sometimes the error goes away after we release a new version but often it will come back again after a couple of weeks or months.
Once the error has started to occur on a machine it does so consistently.
The error never occurs while connected to the affected machine via remote access (e.g. VNC, RDP, TeamViewer, etc.) and none of the affected customers are within travel distance from us so all we have to go by is log files and "eye-witness reports".
One customer reported that the error message dialog apparently was non-modal, i.e. he was able to simply move the dialog box to the side and continue working with the application (minus the functionality that our DLL would have provided). Not sure whether this is universally true in all other occurrences, too.
In some cases customers have been able to permanently rid themselves of the error by disabling or uninstalling other addins from other vendors that were sharing the host application with our own product.
The error has so far been observed on Windows XP, Vista and 7.
During the last few weeks we had a surge of reports from Outlook 2003 / Windows 7 users. Could the situation have been made worse by a recent Windows/Office-update?
Does anyone have any experience with this error at all?
Or any more ideas for investigating this?
I have only recently had this happen, which prompted my search and I ended up here. I can tell you that with me for sure it is in windows 7 home premium BUT ONLY WITH IE9 (which I hate by the way) it reduces the user back to the dummy stage and assumes that we have to be repeated flagged about everything.It will keep asking you if you want to disable add ons to speed up load times but usually on things that aren't really the things slowing the browser down in the first place,it is there is too much garbage loading in the first place.But back to the "Cannot set allocations", I for one have never expirianced it in any other browser which is not to say it doesn't happen with them.
This is going to be pure guess-work, but it sounds like maybe your third-party licensing software is trying to load your DLL at a particular location in memory, which - on these failing systems - happens to already be occupied by something else, perhaps a global hook DLL.
If you have a customer who is willing to work with you a bit, it might shed some light on the situation to get a crash dump (e.g., with ADPlus or maybe simpler with Sysinternals' ProcDump) when the error message is showing. That would show what modules are loaded and possibly the callstack (if it is from a message box at the time of the error as opposed to one that is catching an exception after the problem).
I also have experienced the "Cannot set allocations" issue. Royal pain. I had disabled Java, since I did not seem to need it, I used add/remove programs to remove Java from my system. Then I stated to get those errors. I have reinstalled, but disabled Java in IE explorer. Now I do not get the error anymore. Not a programmer, don't know why this happened. Maybe a clue for someone.
Win 7 - 64bit OS IE Explorer 10. Hope this helps someone figure this out. John
I've seen this happen. In my case a global hook dll created a large memory file mapping, perhaps to the memory the licensing dll was counting on.
I see "Cannot set Allocations" when I open google chrome only. Also after that, chrome closes with a msg saying "Whoa chrome has crashed..."
Still no solution :(
Also not a programmer, but it always happens when I open Chrome. It opens second window with error message 'cannot set allocation'. I usually close it and go on my way. if i don't it usually causes a crash. doesnt happen on any other browsers.

Resources