I am new to kernel mode windows driver development and having issues getting debug information out of the WFP callout samples. I am looking for either more options or tips of what I am doing wrong with the options I have been trying.
To start with I downloaded the WFP sampler and followed the instructions in it's description. I am using a Windows 8.1 x64 host and a Windows 7 x64 target connected via a simulated serial cable. They are both VirtualBox VMs.
The primary command I am debugging with is
WFPSampler.Exe -s PROXY -l FWPM_LAYER_ALE_BIND_REDIRECT_V4 -aaid "C:\Program Files (x86)\Internet Explorer\iexplore.exe" -pla 10.0.2.15 -v
Where 10.0.2.15 is the IP address of a diffrent network interface than the routing table is endign traffic to. I also used the following inspect command as recommended by the instructions:
WFPSampler.exe -s BASIC_PACKET_EXAMINATION -l FWPM_LAYER_INBOUND_IPPACKET_V4 -v
"netsh wfp show state" shows the callout and filter associated with the expected layer
However I never got any messages from traceview as per the last steps of those instructions. Tracelog had similar lack of output. This was true with or without WPP tracing enabled on the driver project. I also tried higher verbosity, all to no effect.
Additionally the remote debugger in visual studio either stopped at "Driver Post Install Actions (x64) (possible reboot): Pass" or "Waiting to reconnect..." rebooting the target did not cause more output in the Debugger Immediate Window (the Output window never got any output). Rebooting the target VM did not get any unusual prompts (some things I read implied it should). The reconnect checkbox when setting up the debugger sometimes caused it to get past the "Waiting to reconnect..." prompt, other times it was not needed. I set a number of breakpoints that should have been hit including one at the top of every classify function and none were ever hit.
I tried to debug with WinDbg, but well, frankly I can't find the documentation about how to use this tool. I start it on the target machine and chose kernel debug, local. then I get a prompt that tells me the symbol search path is invalid and no clue what I am supposed to set it to. Any documentation on how to use this tool as opposed to just installing it may be helpful if it lets me debug these callout drivers.
Finally of course I tried just debugging it based on symptoms and I find that the examination callout does nothing as far as I can tell, while the proxy callout just eats all traffic from the targeted application, with one caveat. That caveat is that when I target firefox or chrome with the proxy callout and try to launch that app it fails to launch a UI and the partially started process cannot be killed from task manager.
I assumed that behavior may be the result of the sample failing to check FwpsQueryPacketInjectionState but modifying as best I can figure out that should be used does not seem to change the behavior.
So in short I am stuck and need direction please?
I started experimenting with the WFPSampler and also discovered that tracing didn't work. Oddly enough, there was a lot of additional code to make WPP tracing work on multiple OS's, but the sample doesn’t capture any trace events. So I debugged it and found the call to the nt!WmiTraceMessage always passed 2 as the TRACE_HANDLE. This seemed bad. Safe it to say, this wasn't exactly straight forward to unscramble WPP macros and then finally find the origin in the .vcxprog file. The definition of WPP_COMPID_LEVEL_LOGGER(COMPID,LEVEL)=2 is incorrect. While I was there, I also converted the DbgPrintEx Levels to match Tace_Level by adding 2 so Error/Warning match the model.
Complete the following steps for SysLib/WFPSampler and Sys/WFPSamplerCalloutDriver
Open the Project
Right Click on WFPSamplerCalloutDriver
Unload Project
Right Cick on WFPSamplerCalloutDriver
Edit WFPSamplerCalloutDriver.vcxproj
Type: WppPreprpocessorDefinitions
Change this definition in both places in the file:
To:
<WppTraceFunction>DbgPrintEx(COMPID,LEVEL,MSG,...)</WppTraceFunction
<WppPreprocessorDefinitions>WPP_COMPID_LEVEL_LOGGER(COMPID,lvl)=(WPP_CONTROL(0).Logger),;WPP_COMPID_LEVEL_ENABLED(COMPID,lvl)=(WPP_CONTROL(0).Level >= lvl+2)</WppPreprocessorDefinitions>
Save the File
Right Cick on WFPSamplerCalloutDriver
Click Reload Project
Rebuild
I believe if you adopt this model of using DbgPrintEx as your trace function, you can switch to WPP without editing the entire project. However, I still think it's better to just convert in your program.
Cheers,
John
Not exactly an answer to this question, I never got a debugger working, but an explanation of why the wfpsampler sample didn't work for proxying by ALE layers. The end goal of this project.
In the Proxy callout there were these lines (i think before my refactor they were in the PerformProxySocketRedirection method):
if (ipProtocol == IPPROTO_TCP)
pBindRequest1->portReservationToken = (pRedirectData)->pProxyData->tcpPortReservationToken;
else if (ipProtocol == IPPROTO_UDP)
pBindRequest1->portReservationToken = (pRedirectData)->pProxyData->udpPortReservationToken;
udpPortReservationToken or tcpPortReservationToken should have been initialized by a call to CreatePersistentUdpPortReservation or CreatePersistentTcpPortReservation and this is handled in a helper function named HlprWinSockCreatePortReservation, but said helper function is never called, from anywhere in the entire solution. Those fields are never set, anywhere, ever.
Additionaly, that value only needed set if the local port was being changed, but that block of code executed regardless.
remove these lines, and so long as you don't change the local port, the ale redirect sample works as you would expect including changing the outgoing interface if you change the local IP. If you do need to change the local port, you will have to use HlprWinSockCreatePortReservation to initialize the appropriate port reservation token variable in the PROXY_DATA. This must be done in user mode, so you need to do it in the service when setting up the filter.
Disclaimer: I think that is the only important change, but if you are here for a solution to this problem know that while tracking this down without a working debugger i changed a ridiculously large amount of code so there may have been more to it I forgot over the course of the last few weeks. But I did go back and verify that that block was always called and those fields were never set in the original.
Related
I have a Visual Basic 4 32 bit application which results in a
Run-time error '5':
Invalid procedure call
I start WinDbg and do
sxe *
to break on all exceptions. However, it displays the message without breaking into the debugger before.
If I break at the time of the message, the callstack is not very useful. Is there a trick to break-in on Visual Basic 4 applications at the time the problem occurs?
Problem FAQ:
Can you run it in the IDE?
Yes I can and I can probably fix the bug when I do that. However, I'd like to get some more information at the time the end user runs the application so that I don't need to be able to reproduce it.
What do you want to do in case you find the cause of the error?
Well, it depends on the type of the error. Maybe I can fix it, maybe I can't. Usually I get some very helpful information out of WinDbg.
When does the error occur?
It occurs reproducibly when the user presses a close button on a form.
What does the application do?
Oh, that's a good question. I don't exactly know. It uses RS232 communication but it's not yet involved.
Do you know anything about the data the user can enter?
I almost know nothing about it. In this particular case, it's not necessary to enter data.
The error is from the VB virtual machine not from an API.
I can't see how WinDbg would be useful on VB4 which is interpreted and is in P-Code.
Visual Basic run-time error 5 Indicates one of the following:
An invalid collection or property name was entered.
An out parameter was NULL.
The value is not one of the supported values or falls outside the supported range.
The property is read-only.
The property cannot be changed after the object is created.
An invalid index was entered.
And in case it's an API call VB uses LoadLibrary and GetProcAddress.
Some controls have lost default values over time and this can cause the first.
The third can happen if modern technology exceeds the program's expection, e.g. disk space.
Also see (sorry no links, they are on my hard disk)
Q131007 HOWTO: Use Windows NT WinDbg.exe with Visual Basic (it's for NT 3.5)
Q166275 HOWTO: Debug a Native Code Visual Basic Component in VC++ (talks about what you can see in VB in a debugger)
VB4 can't make symbols
As the error occurs when the users presses the close button i assume there is a (very tight) loop running which doesn't receive the expected data (probably a nullstring)
I expect that that loop is continuously reading data from the RS232 device and the RS232 is on the form which is being closed, even though it is not involved the connection might already be open and polling
What you should do is exit that loop and close the connection with the RS232 device before closing the form. You might be able to do that in the Form_Unload() event, but it might also have to take place before that, so you might have to do it in the code of the close button ... make sure though that the user won't be using the X to close the form though
Personally i would run the program from within the IDE, this will point you to the errornous part of the code directly and gives you the opportunity to watch the contents of the variables involved and control states ... that is much faster than taking a guess based on the data of WinDbg
I have a VB6 application that I provide support for. This application works on both Windows XP and Windows 7. Some users were migrated from Windows XP to Windows 7 using the User State Migration tool. These users now receive a generic "Application has crashed" Windows error message when they open certain screens (forms) in the application. My assumption is that there is a missing dll/ocx reference, but I'm having trouble tracking it down.
I've tried many/varied troubleshooting techniques:
Full uninstall and reinstall of my application
Manually re-registering all dll's and ocx's that I know are used
Running Process Monitor on a broken computer and a working computer to compare what dll's and ocx's are accessed. The answer might be here but even after filtering out most of the background noise the amount of data is overwhelming. At a minimum I reviewed all of the calls right before it crashes and all of the calls that were not successful. All of the non-successful calls match between working and non-working.
Installed the Windows Debugger Tools and captured a crash dump. Analyzed the crash dump with DebugDiag. DebugDiag says the exception is in msvbvm60.dll. I tried building a PDB file for my exe and loading it in DebugDiag to get more detail about where the exception is occuring but DebugDiag doesn't want to accept the PDB (might be doing something wrong here, but it just seems to ignore it. This same PDB file works fine when I do remote debugging, however.)
I recompiled my VB6 program without any optimizations in PCode. I've read online that sometimes building in PCode, while bad for performance, will tell you the real exception.
Used the above created PDB file to remote debug the VB6 application. The debugger says that the application crashes after the new window has been created, on a line that sets MousePointer = vbHourGlass... To me it seems unlikely that this is the real cause of the error. There are at least 20 other locations in the program where this same line is called and all work fine.
(Forgot about this one)
Used Dependency Walker and profiled the application on both a working and non-working computer. All errors found by dependency walker were the same between the two computers. There were no additional dependencies found on a working computer, and all missing dependencies on the non-working computer were also missing on the working one.
None of these actions changed my error message or showed me what the error is (unless it really is the mouse cursor issue)... There are no entries in the Windows Event Log related to the app crash.
The non-working and working computers all have the same base Windows 7 image, the only difference is whatever is being changed by USMT, which further convinces me that this is some kind of quirky configuration change or a missing dll/ocx or perhaps an unregistered dll/ocx.
Any ideas or thoughts on how I can track down the root cause of the issue would be greatly appreciated.
Update 1 - Response to questions
#MarkHall I have tried running it as admin, though not with UAC off. The application runs fine on a Windows 7 box as a non-admin with full UAC. Windows XP was 32-bit, Windows 7 is 64-bit, but again it works just fine on a like for like box where the user was not migrated from Windows XP.
#Beaner It's possible that it stores settings somewhere that have been corrupted, but the remote debugging leads me to think that it's more likely something else since it seems to die on a step related to the UI, which then makes me think it's probably a missing dll/ocx reference.
#Bob77 The application is installed into Program Files (x86). While many of the libraries do reside in the same folder, they are all registered.
Peter, often I've noticed that the debugger will indicate a line of code that is actually incorrect, depending on WHERE in the actual assembly language the fault occurs. You should look REAL close around your statement that sets the cursor to vbHourGlass. Your exception is PROBABLY happening BEFORE that line of code, but that line is what the debugger thinks is the actual faulted line of code.
Since you said it happens when a window OPENS, I'd look real close at any ocx's you may have referenced on the form, but perhaps NOT actually being used, or called. You might have one there that you don't intend to be there, that could be causing security issues, or something on Win7? Edit the .frm file by hand if you have to, and look at all the GUIDs the form references.
It is possible that one machine is using PER-USER registration, and the other is using PER-SYSTEM registration?? I don't know...
I would take a much closer look at the form that you are trying to open, and be VERY cautious of everything you are doing in the form load events, and so on. This sounds like it could be something as stupid as Windows Aero being enabled on one system, and not another, or some other sort of "Theme" setting that is throwing the VB Form Rendering routine into a hissyfit... Perhaps even something as stupid as a transparent color index in the icon you selected for that from?
If you are still developing this app, (or at least maintaining it), create an entirely NEW form, and re-create all the controls, etc, on the form (resist the temptation to copy/paste them from the old one...), and then see if THAT does the trick. Then, copy all the event code to the new form one event at a time, with at LEAST enough event code to make the form function, even if it's just a "dead form", that loads no data, or whatever the form is supposed to do. Check and debug after each change, and you WILL find it eventually. Of course, make sure you isolate one of the defunct systems to have a platform that you can duplicate the issue on, or then it's just guessing. I find that using something like Acronis w/ Universal Restore is a great option to then take the image file into a good HV, like VirtualBox, and then restore that image as a VM, so you can debug without interfering with your actual users. This sounds like a lot of work, but then again, so is re-writing an application that already exists, right? :)
Failing THAT... /* and */ are your friends!! (Well, we're dealing with VB, so ' would be your best friend! heh... But I'd start commenting out all the code on the form until that sucker opens. Then once it opens, start putting one line back at a time, and re-running it... That's called "VooDoo Debugging", but sometimes, you gotta do what you gotta do...
THANKS A LOT PETER! :) Now you got ME so involved in this, I feel like I'M the one debugging this sucker! Like if it was MY code I was trying to fix! :)
Let me know if any of this helps... I am actually quite interested in what you discover.
I'm trying to figure out how to set up a printer language monitor without using WIN32 API's AddMonitor() (i.e. by editing the Registry or using an INF file), but I keep running into problems, and I'm not seeing any good examples of INF files for this purpose.
For testing purposes, I edited ntprint.inf to point PJL_LANGUAGE_MONITOR to my DLL instead of pjlmon.dll, and changed the appropriate Registry keys to also do the same. Every time I try to reload the print spooler I can see that it seems to successfully load my language monitor, based on debug output, but when I print I keep getting debug messages that say "UpdateNow not set" with a PID of whatever editor I'm using.
My question is, do you think this "UpdateNow not set" message is related to the problem, or is it just some inconsequential symptom?
Employer moved on to another project, moot point now.
I have a program that I run on multiple network PCs. When I compiled the most recent version, it runs extremely slowly on 2 PCs on the network, but runs fine for everyone else.
This used to happen with my old dev PC when I had an additional 2gb RAM installed. When I would remove the additional 2gb and recompile, it would then work fine for everyone.
Now, I am on a completely new machine and am having the same issue. I've tried to rebuild the project after rebooting, but still have the same issue.
For all other PCs, the program loads in about 3-5 seconds. On these 2 PCs, it takes anywhere from 45 seconds to 1.5 mins to load...
One of the PCs is an older Dell Dimension 8200, but the other is a newer OptiPlex that is identical to several other PCs on the network, so this is what is really making it so confusing.
For now, I've had to revert to the old version so it will run correctly for everyone.
Does anyone have any idea of anything to try?
Thanks in advance!!!
Edit:
Ok, it was an exhausting day yesterday trying various things to solve this issue. Here is what I tried and where the problem begins:
Using the new program
Went back to old versions of all updated components, but still had the same issue
Using the old program
I decided to go back to the drawing board and start from the old version of the application and incrementally add the new features a small piece at a time.
Recompiled the old version using the old components - program works fine
Updated to new DevExpress components - program works fine
Updated to new ESBPCS components - program works fine
Updated to new DeepSoftware components - program works fine
Ok, so now we know there is nothing with the component sets I've updated...
Added 1 image to each of 2 image lists - program works fine
Added new database table - program works fine
Added code to open and close the new table - program works fine
Added new action to action list and added a menu item and toolbar button to new action (action does nothing at this point) - program works fine
Added a new BLANK form to the application and added code to open the new form - BAM!!!
So, adding just one form to the application is what's causing the issue! I removed all the code for the opening of the form, commented out the uses clauses and removed the uses entry from the project source and everything is back to normal!
Anybody have any idea about this?
Thanks!
Edit 2:
For #Warren P - here is my .DPR source:
program Scheduler;
uses
ExceptionLog,
Forms,
SchedulerMainUnit in 'SchedulerMainUnit.pas' {FrmMain},
SchedulerDBInfoUnit in 'SchedulerDBInfoUnit.pas' {FrmDBInfo},
SchedulerHistoryUnit in 'SchedulerHistoryUnit.pas' {FrmHistory},
SchedulerOptionsUnit in 'SchedulerOptionsUnit.pas' {FrmOptions},
SchedulerExtVersionUnit in 'SchedulerExtVersionUnit.pas' {FrmExtVersion},
SchedulerSplashUnit in 'SchedulerSplashUnit.pas' {FrmSplash},
SchedulerInfoUnit in 'SchedulerInfoUnit.pas' {FrmInfo},
SchedulerShippedUnit in 'SchedulerShippedUnit.pas' {FrmShipped}; {<-- This is the new form with the issue}
{$R *.res}
begin
Application.Initialize;
Application.Title := 'SmartWool WIP Scheduling Assistant';
Application.CreateForm(TFrmMain, FrmMain);
Application.CreateForm(TFrmDBInfo, FrmDBInfo);
Application.CreateForm(TFrmHistory, FrmHistory);
Application.CreateForm(TFrmOptions, FrmOptions);
Application.CreateForm(TFrmExtVersion, FrmExtVersion);
Application.Run;
end.
And here is the intialization section of the main form to create the splash:
initialization
FrmSplash:=TFrmSplash.Create(Application);
FrmSplash.Show;
FrmSplash.Refresh;
Edit 3:
Anybody??? Please?
It could be that the program is waiting for timeouts when trying to access resources that are not available on that machine such as network drives or Internet hosts.
Try running Process Monitor when starting up your program and look for file open calls. Filter the output so it only shows your process.
http://technet.microsoft.com/en-us/sysinternals/bb896645
Performance problems initially can seem very daunting at first.
I have been on many teams where people have tried to guess at a reason for performance problems. This sometimes works, but is far less effective than actually measuring the code.
When reproducible on a development machine, I would recommend a profiler.
There was a previous question that asked about
Delphi Profiling tools which has several possible tools you could use.
When you can't reproduce the problem on your development machine, then it becomes a bit more difficult, but not impossible. Typically I have found that problems are related to an application dependency that is different, and not performing well. Understanding the external influences on your application can help pinpoint the problem.
Specifically common external problems in some of my applications.
Network
Database
Application Servers
Installation or Data File Location (i.e. Disk Performance)
Virus and Malware Scanners
Other application interring with yours such as a virus.
To monitor for items related to the network (i.e. Database, web services, etc...)
I typically use Wireshark which allows me to see if resources are responding in expected times. My most common problem is poor performing DNS and can found using Wireshark.
You can use the AutoRuns program to determine everything that starts up when your computer does, it's useful in determine differences between machines.
But most of all I have logging that can be turned on in my applications and this allows me to isolate the problem to a specific area of code. This narrowing down to a specific section of code reduces the guessing, and allows you to focus on a few possible problems.
I created a log function for this that I call at specific places (in your case especially during startup). It adds a timestamp to each log text and stores them in a TMemo that is regularly saved. Not only very helpful when debugging, but may also shed some light on your problem.
Are you using code signing - ie Microsoft Authenticode? If so, then outdated certificate authorities on the computers can cause significant delays to startup.
First, I would try to defragment the hard disk. If still slow, I would check the power supply. Maybe your hard disk are getting insufficient energy.
Check if there is the same antivirus software on those 2 problematic computers. If so, then your Delphi application may match byte pattern used in some virus made in Delphi. Update virus definitions to solve it, or report false alarm to antivirus company, or change antivirus software.
Check if there isn't any printer installed on those 2 problematic computers. If it is so, then add any printer and try again.
Idea 1:
One reason I have seen for very slow application load time, is when printing or reporting system components like Developer Express Express Print, are in your application.
The problem I saw when using Developer Express Printing components, is that I had an offline or non-responsive network printer in my list of printers (check the control panel printer icon) that was not responding. Some of those Developer Express components seem to read some information from each printer you have installed, and the solution was to go to those clients, and delete old printers from their control panel, that were no longer being used. Each not-responding network printer added up to 60 seconds for a TCP Timeout, to the startup time of my application.
Update - Idea 2:
Download MS DebugView and install it on the machine that runs slowly. Now go back to your main development PC, open the IDE, open your main project file (right click on the project, view project source in project viewer), this will show you the contents of your main project source file (.dpr). go to the main begin....end. block. Now set a breakpoint on the main begin statement, and single step INTO (not OVER) and you will see all the module initialization sections. In each one add this: OutputDebugString('ModuleName').
Now when you run this inside the Delphi Ide you will see messages, and see how far apart they come in, and understand what is taking a long time to initialize. Instead of installing the delphi ide onto the machine that runs slowly, Debug View (which is less than 400kb single executable) will be run, and it will show you these debug messages, along with a nice time display (##.# seconds) for each message.
MS Debug view is here.
Are you allowing the forms to be constructed on initialization within the DPR source? If so, you may do well to consider whether or not you want those forms sucking up memory the entire time, more-over if you want those forms to be wasting the application's time on load.
A rule of thumb: If the form is used a LOT during the application's execution, allow it to be constructed when the application loads (this will work out faster over-all than constructing the instance "on-demand").
If the form is not used very often at all (for example, a Dialog or an About Box), delete the "Application.CreateForm" line from the DPR source, and instead construct your instance on request...
var
LForm: TfrmAbout;
begin
Application.CreateForm(LForm, TfrmAbout);
try
LForm.ShowModal;
finally
LForm.Free;
end;
end;
Now that form (which may not even be displayed during the program's execution) is not sucking up system resources, and will not slow down the application's load time.
It may not solve your problem 100%, but it should certainly help!
I'm running a test script from batch file.
Because it is test, the programs are expected to fail once in a while. It is file as long as error code is returned so I can continue and mark specific test as failed.
However there is very annoying behavior of executable files under Microsoft Windows - if something fails it pop-ups window like:
This application has failed to start because foo.dll was not found, Re-installing the application may fix the problem
<OK>
Or even better:
The instruction at "..." referenced to memory at "..." ..
Click on OK to terminate the program
Click on CANCEL to debug the program
The result is known - the script execution blocks till somebody presses "Ok" button. And when we talk about automatic scripts that may run automatically at night in some headless virtual machine, it may be very problematic.
Is there a simple way to prevent such behavior and just make an application to exit with failure code - without changing the code of the program itself?
Is this possible at all?
The answer is following: You need to disable WER.
Simplest description for this I found at http://www.noktec.be/archives/259
Simply (ON XP): Right Click on My Computer > Advanced > Error Reporting > Disable
Voila - programs crash silently!
This does not solves problem when DLL is missing, but this is much rare case and this is good enough for me.
You can suppress AV's and such from showing a dialog box by running your application, or the script (the script engine, like cscript.exe), under a debugger.
Use Gflags.exe, or modify the registry directly, and set Image File Execution Options for the image in question. See this article for details on how to use the appropriate registry keys. You can set it up using a debugger commandline like "C:\Debuggers\ntsd.exe -g -G -c'command'", where you can pass commands to ignore certain types of exceptions in the -c"commmand" argument. This will effectively give you a tool to suppress interactive dialogs as a result of exceptions like AV, and will let the process continue (presumably to immediate end after the exception has occured).
This article explains the commands you can use to control exceptions and events from withing the debugger.
The -g and -G flags make sure that the process won't break into the debugger automatically during process start and end respectively. You'll have to play with the various exception suppression options to make sure that you 'eat' all possible first and second chance exceptiosn that might cause the process to break into the debugger.
Also, if you can tolerate a process being broken into the debugger (as against being stuck showing a dialog box), perhaps that would be a better option overall. You can evaluate each debug break in batch mode at a later time and decide which bugs you care to fix.
It is possible. We used to use IBM's Rational Robot product which could monitor the screen for specific items and, if found, send keystrokes to windows and other sorts of things.
We actually used it for fully automated unit and system testing, much like you're trying to do.
Now I thought that Robot has been through quite a few name changes so it may be hard to find but there it is, right on IBM's web page and with a free downloadable trial for you. It's not cheap, clocking in at a smidgeon under USD5,000 but it was worth it for us.
There's also TestComplete where you could get a licence for just unedr USD1,000 - it touts "Black-box testing - Functional testing of any Windows application" as one of its features and also has a downloadable demo to see if it's suitable before purchase.
However, you may be able to find another product to do the same sort of thing.
I initially thought of Expect but the ActiveState one seems to concentrate on console applications which leads me to believe it may not do graphics well.
The only other option I can suggest is to write your own program in VBScript. I've done this before to automate the starting of many processes (log on to work VPN, start mail, log in and so on) so I could be fully set up with one mouseclick instead of having to start everything manually.
You can use AppActivate to bring a window to the foreground and SendKeys to send arbitrary keypresses to it after that. It's possible you may be able to cobble together something from that if you want a cheaper solution.