I've been playing with VirtualBox implementations of Xubuntu. I'm learning provisioning test boxes with content using both VBoxManage and Vagrant. Unfortunately my boxes work very erratically. I'd like to enable the debug mode in VirtualBox to better understand why the boxes sometimes freeze up.
Current top menu: VirtualBox VM / Machine / View / Input / Devices / Window / Help. I'd like to be able to get to the / Debug / top menu with its selection of Statistics / Command Line / Logging / Show Log as displayed in the VirtualBox Graphical User Input (GUI) tool, [select Virtual Machine --> Settings --> User Interface.]
As I understand it, I'm looking for the Built In Debugger. I've found the instructions, but they don't make sense to me.
The debugger can be enabled in three ways:
Start the VM directly using VirtualBox --startvm, with an additional
--dbg, --debug, or --debug-command-line argument.
Set the VBOX_GUI_DBG_ENABLED or VBOX_GUI_DBG_AUTO_SHOW environment
variable to true before launching the VirtualBox process. Setting
these variables (only their presence is checked) is effective even
when the first VirtualBox process is the VM selector window. VMs
subsequently launched from the selector will have the debugger
enabled.
Set the GUI/Dbg/Enabled extra data item to true before launching the
VM. This can be set globally or on a per VM basis.
I know how to start my virtual boxes in two ways. 1) Open the Virtual Box GUI, click on the machine of interest, then start it up. 2) Start the box up from the folder that contains my custom Vagrantfile, then $ vagrant up.
Its not clear to me from the above link on Virtualbox, how to set up vagrant or VBoxManage or VirtualBox GUI to start up a box with Debug mode enabled (or better yet, enable Debug mode when building a custom box...). I can't believe someone built up a nice GUI then omit the chance to implement Debug into the tool.
Note: I'm using MacOS for my host; I've had best luck using bstoots/xubuntu-16.04-desktop-amd64 as the base for my guest virtual machine box. Anybody been here before? Tips and hints as to how to start a box with debug enabled? Many thanks.
So I tried few more things (Virtual Box 5.2.20 r125813 on 64-bit Win7 Enterprise).
Method 3'' (per machine ExtraDataItem: does NOT do anything, even does not add 'Debug' menu to the VM window)
Method 3' (global ExtraDataItem): adds 'Debug' menu to the VM windows but does not break at the VM start (VM is just running, you can open the debug console and stop it but then of course we are deep into the boot process, or after...). But it could be useful ... no harm of having 'Debug' as a default.
Method 1: Works BUT not as described, even in the VBox own User manual is confused, page 261 describes the options WRONG. However the chapter 8 gives some ideas, here we go:
you can add the environment variables to the command line:
C:\Program Files\Oracle\VirtualBox>vboxmanage startvm "SomeVM" -E VBOX_GUI_DBG_AUTO_SHOW=true -E VBOX_GUI_DBG_ENABLED=true
will show the 'Debug' menu, open the debug window and load the VM halted at the reset vector
VBOX_GUI_DBG_ENABLED=true
alone will just add 'Debug' to the VM's window (the VM will run)
VBOX_GUI_DBG_AUTO_SHOW=true
alone will load VM halted but no 'Debug' menu so really nothing to do ... (however this can be paired with the global setting!)
The remark in the manual (and online) that the variables have to be only defined is NOT true, unless set to "true" they do not have any discernible effect.
BTW: the ExtraDataItem line is:
<ExtraDataItem name="GUI/Dbg/Enabled" value="true"/>
I decided to set it up: this way all the VMs have the 'Debug' menu enabled but start as usual, if I want to debug one from the start then I use the command line with
-E VBOX_GUI_DBG_AUTO_SHOW=true
https://reactos.org/wiki/VirtualBox#Built-in_VirtualBox_.28low-level.29_debugger explains things a little better, however only for Windows (I presume 7 ...). There are 3 methods:
1 - Start VBox from the command line window adding the options as listed (see the VBox User manual for the command line method of running). The advantage that it is per machine, disadvantage: the command line is long and best issued from a shell script (in case of Windogs: BAT or CMD).
2 - Declare the environment variables:
I tried some under Windows 7 (64-bit Enterprise), here I declared User environment variables VBOX_GUI_DBG_ENABLED and VBOX_GUI_DBG_AUTO_SHOW (Computer / Properties / Advanced System Settings / Environment Variables) and after restarting Virtual Box GUI when I started the VM it would come to the debug console. My VM was x86_64 and like many other kernel debuggers this one was quite useless for stepping through ROM BIOS (at least the initial portion). I am too lazy to see if I can set the break-point either after the ROM BIOS is RAM-ed (relocated to RAM) or in the boot-loader (1st I would have to find if and how this VM relocates BIOS and then where would be the best place to break into the boot-loader, which is custom); but I did things like that in the past with similar debugger (this seems to borrow stuff from the oled Compusoft's CodeView x86 kernel debugger ..).
3' - Modify VBox global or per-machine configuration files, I have not tried that but I tracked the global file:
%homedisk%:\Users\%username%.VirtualBox\VirtualBox.xml
it is pretty straightforward, I assume adding the listed item shoudl work (%homedisk% is usually C, substitute %username% for the login name).
3'' - Modify the indiviual VM's file (*.vbx). They VM's are in %homedisk%:\Users\%username%\VirtualBox VMs
The *.vbox files are XML format and find where to add the data is also simple. This method has advantage of being perVM, disadvantage of possibility of screwing the VM up (so make a backup)
I have VirtualBox on my home iMac but I have not tried this yet. I did not dig into VirtualBox on Mac file structures, but I would be surprised if they were not somewhat analogous to Windows. The command line should be pretty similar (paths of course would be different, and the shell script too), one possible annoyance might be that you may have to use sudo ...
Regardless of the host, debugging the initial boot sequence of x86_64 is not trivial because usually in the beginning we time-travel to 80s and pretend that we are running 386 with two cascaded PICs and the extended address lines controller by a keyboard controller ... lots of fun! (or not ...)
Related
I have a few VMs in Parallels 11, running several different versions of OS X, for automated software builds.
I recently copied those VMs to a different host, so I'd have a second set to fall back on in case of a hardware failure.
The problem: the copied VMs have the same Hardware UUIDs as the corresponding originals, and that causes a problem backing up the VMs with Time Machine, since Time Machine links backup images to machines by their Hardware UUIDs. If you have two VMs with the same Hardware UUID, Time Machine running in those two VMs will try to write to the same sparsebundle file on the backup server.
I googled the issue and found this: http://kb.parallels.com/en/11197 -- but those instructions don't work for Parallels 11: when you double-click the pvm, it is imported immediately, without asking whether it was moved or copied. When I first imported the copies, Parallels asked me whether the VMs were moved or copied, and I answered that they were copied. The KB article makes it sound like that should have made Parallels assign new Hardware UUIDs to the VMs, but that didn't happen (it did assign new MAC addresses, though).
I'm going to try finding the Hardware UUID in the PVM using a hex search; if I can find it, maybe I can change it manually, using a hex editor. It seems like a very yucky procedure, though, for something you'd think Parallels should handle itself. Is there any better way of doing this? Is this a bug in Parallels 11, or did they deliberately remove the Hardware UUID reset functionality (and if so, why)?
I was able to change it by:
Shutting down the virtual machine
Find the virtual machine bundle located on disk in Finder, right click on it and "Show package contents" on the actual .pvm file.
Then find the config.pvs file inside and open it with a text editor.
Change an arbitrary digit in the <SourceVmUuid>
value
Start the virtual machine
You can then go to:
About this Mac -> System Report -> Hardware
and inspect that the Hardware UUID has changed.
I've recently started seeing this line in my Visual Studio 2005 output window when launching my application:
FTH: (7156): *** Fault tolerant heap shim applied to current process. This is usually due to previous crashes. ***
I've tried turning off the fault tolerant heap using the instructions here:
http://msdn.microsoft.com/en-us/library/dd744764(VS.85).aspx
I'm running Windows 7 64-bit edition, so I have made the changes to both the 32-bit and 64-bit registries, and run the "Rundll32.exe fthsvc.dll,FthSysprepSpecialize" command using both the 32-bit and 64-bit versions of Rundll32.exe.
However, after rebooting I am still getting the fault tolerant heap when trying to debug my application!
This is a real problem since it masks the bug I am trying to reproduce, and it also kills performance.
Does anyone have any other suggestions how to disable the fault tolerant heap?
To disable it for a single application
Go to the HKEY_LOCAL_MACHINE and HKEY_CURRENT_USER versions of
Software\Microsoft\Windows
NT\CurrentVersion\AppCompatFlags\Layers\your_application.exe and
delete the FaultTolerantHeap entry.
From here (actually here)
Set this registry value to 0:
HKEY_LOCAL_MACHINE\Software\Microsoft\FTH\Enabled
You can add the name of your executable to the ExclusionList.
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\FTH\ExclusionList
Works for me.
You can edit the application manifest to excluding your program from PCA
see also:How to reset Program Compatibility Assistant for testing
you can clear the list of applications tracked by FTH without stopping this service by following these steps:
Click the Start menu.
Right-click Computer and click Manage.
Click Event Viewer -> Applications and Services Logs -> Microsoft ->
Windows -> Fault-Tolerant-Heap.
View FTH Events.
you will find file named operational by right click and choose clear log,
then you can run you program again and warning message will disappear,
it worked with me without restarting operating system.
On Windows 10 the registry location is:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\FTH
You can remove you executable from the list in:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\FTH\State
or you can run this command from an elevated command prompt
Rundll32.exe fthsvc.dll,FthSysprepSpecialize
You may need to reboot your machine
"Rundll32.exe fthsvc.dll,FthSysprepSpecialize" looks to only clear the list of currently flagged applications. if your application still causes oddities, the FTH should still step in and take over.
as already mentioned:
Set this registry value to 0: HKEY_LOCAL_MACHINE\Software\Microsoft\FTH\Enabled
this should disable FTH for the whole system.
I had to rename the file as well because the registry entries associated with this key were empty of applicable data. I expect that they populate if you have a misbehaving application. But in my case I was debugging my own application within Visual Studio. So in that case, it was my process that was somehow loading the FTH whether the FTH Service was running or not. And in fact I had no applications listed that were previously tagged as misbehaving.
But I had to follow these instructions:
http://billroper.livejournal.com/960825.html
because it wouldn't let me rename the file until I took ownership and made sure I had full control.
I had similar issue when running a Unit test using (Microsoft::VisualStudio::CppUnitTestFramework).
Somehow I had violated some heap allocation, and next time I tried to debug I received the message : "Fault tolerant heap shim applied to current process. This is usually due to previous crashes. " and the debug environment froze.
To get it to work again, I had to remove test case, recompile and add it again and recompile, then I could set breakpoint and step into the test.
Also ran into this. Renaming/deleting AcXtrnal.dll inside Windows\AppPatch seems to work for me. I like how this Microsoft recommended action (which I did first) does nothing.
At present we are developing a graphic app on a linux box using opengl. We have had a bit of trouble getting a decent debugger working. At present we use GDB via a ssh connection, but that is a tad painful. Next we have tried running DDD on the target platform, but we run out of space on screen. Next we have tried running an xserver via cygwin on windows and doing an "export DISPLAY=RemoteHost:0.0" on the linux box. In this situation DDD will run on the windows box, but when we boot the app it also wants to pipe its output to the windows box.
What we really want to do is have DDD output on our windows box, and the output of the main app on the linux box.
I am not a x11 guru and I would love some help in this area.
Cheers
James
While logged in into the linux box:
DISPLAY=windows-box:0 ddd /path/to/app
DDD should now pop up on your windows box. If you run the app at that point, it will attempt to display on windows box as well, since the DISPLAY environment variable is inherited. You need to reset it:
(gdb) set env DISPLAY :0
(gdb) run
At that point, the app should display on the linux box, as desired.
Many applications accept -display command line argument. If yours does, an alternative is to do this:
(gdb) run -display :0
instead of resetting the DISPLAY environment variable.
I have a driver that directly avoids the BSoD and turns the screen of Windows Vista into black with few colourful dots and stripes. I am finding a memory dump on the system afterwards and DbgView is wonderfully showing me a stack trace (stating that this might be a garbled stack and some parts might be incorrect). Pity is, the commands found in the stack are nowhere to be seen in the part of the code that is obviously breaking down the whole thing. (I can leave our the whole routine, but not parts of it.)
Does someone know a way to get debug messages either into the memory dump or out of the serial port to read them in an external debugger? (My testsystem stall if the debugger is connected, but this might be me not knowing enough on the how and why of remotely connected kernel debugging.)
I would like to get to know the point where my code fails, because browsing through code seems fine to me is getting a little futile and I could need a hint.
Description from MSDN on how to setup your debugging session.
The setup procedure is :
Setup 2 PCs, the first with your develop environment and the second which is the debug target. (Vista+Vista64 worked best for me)
Setup a debugger connection by either using a Firewire cable or a Null-Modem cable. (Null-Modem works almost always, but Firewire is better from a performance standpoint. USB is expensive and didn't work out of the box for me ...)
Setup the target computer to boot in 'debug' mode MSDN
Having WinDbg running and waiting for a kernel connection, boot the target computer in debug mode.
While boot up, WinDbg should print messages about the target system. Here you will see DbgPrint messages etc. (All DbgPrint is disabled per default in Vista (not in XP), and you must enable it link)
You can set breakpoints in modules in WinDbg by defining breakpoints.
PS: bcdedit sometimes fails to setup the debug mode. I have no clue why. But there seems a certain order in which you must tell it the debug parameters. Trying different combination's does work wonders ... .
You can use commands in WinDbg when you break into the process. A couple of interesting ones :
lm displays all modules currently loaded
lm m pattern displays all modules satisfying the search condition (e.g. "lm m kernel32*"
x module_name!function_name_pattern lists the symbols in the module (e.g. "kernel32!Create*")
bl lists all currently set breakpoints
bp module_name!function_name sets a breakpoint at the start of the function specified.
bc * deletes all breakpoints
.hh bp displays the help for "bp"
.reload /u x.sys reloads the x.sys module debug settings. (this is helpful when you want to rebuild your project, and get a 'could not create x.pdb ...' message).
Enable all DbgPrint output under Vista :
enable_dbg_print_vista.reg
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Debug Print Filter]
"DEFAULT"=dword:0000000f
Does anyone know ways of partially or fully automating driver test installation?
I am new to driver development and am used to more of a test-driven approach in higher level languages, so moving to the kind of environment where I can't easily test as I go has been a step up for me. I am using Virtual PC for my test environment and currently have to reset it, open device manager, choose the device, click through a bunch of "Are you really sure you wouldn't rather install one of these system drivers" type dialogs, then finally reset the test environment while restarting WinDbg in the host machine just as the test environment is booting up... argh.
After repeating this process many, many times already, surely there has to be a be a better way of doing this? What tools/methods/tricks do commercial driver developers use to run up their driver in a test environment?
Note, this isn't about unit testing drivers, I haven't got to that stage yet or know if it is even possible. This is just about firing up a test environment with WinDbg attached to make sure that some small change I may have done is doing what I expect.
It seems to me that a virtualization software + a "mock objects" (layering) approach (as suggested by Aaron Digulla) + scripts (as suggested by Sergius) can simplify device driver development.
But if you use Visual Studio to develop user-level applications, you can use it for kernel device driver development too with VisualDDK (+ VirtualKD to debug over a named pipe, which is faster than over a virtual COM port), which addresses specifically the annoyances that you mention; from its home page:
... This project brings the simplicity and
convenience of Windows application
development to the driver development
world. No more manual creation of
build scripts, copying of driver
files, installing drivers from INFs,
switching between WinDbg and the
source editor or waiting for seconds
after each step due to the extra-slow
virtual COM port. Just create a driver
project using a convenient Driver
Wizard, select a virtual machine, and
enjoy debugging your driver directly
from Visual Studio. Want to test a
change? Just normally press Shift-F5,
modify your driver, rebuild it and
launch again. VisualDDK will unload
the old driver, install the new one
and load it automatically and quickly.
Bored with WinDbg loading symbol files
for minutes and looking up symbols for
seconds? Just let VisualDDK optimize
this for you using its own DIA-based
symbol engine. Using C++/STLPort in
your drivers? VisualDDK will natively
visualize all STL containers and
strings, as good as Visual Studio does
for user-mode applications. ...
You can write some shell scripts (using sc.exe and devcon.exe) to automate deployment tasks (no opening device manager, clicking on buttons, etc). And make snapshot of the system ready to debug (needn't wait for system boot).
Don't forget to check your driver with DriverVerifier!
Example of my own script :)
sc create FsFilter type= filesys binPath= c:\FSFilterDrv.sys
sc start FsFilter
pause
sc stop FsFilter
sc delete FsFilter
Follow the advice I gave here. Basically, test as little as possible with the real system.
In your case, I've got another tip: Virtual PC is using a virtual hard disk (that's probably a file on your real hard disk).
You don't need to install your driver, you can simply replace the new files in the virtual hard disk. This is often not possible in the running system but in a virtual system, you can open the virtual disk file and change it (since Windows isn't locking the files in it).
I'm not sure about Virtual PC but other emulators come with tools to work with virtual disk images. If VPC can't do it, check out VirtualBox.
It all depends a little on what kind of driver you are writing. But in many cases, writing an appropriate makefile (or something similar) that handles driver installation, start/stop, and launching of a test harness can already be good enough.
I also configure all of my test machines to automatically logon (AutoAdminLogon), map net drives, and launch an appropriate command prompt after startup. Running a specific test is then a matter of typing in a single command only.
One word concerning VirtualPC: VirtualPC is very handy for kernel mode development, but do not forget that it emulates a uniprocessor machine only -- so be sure to regularly test the code on a multiprocessor machine as well. That said, the VHD trick may seem handy, but it somewhat ties you to Virtual PC -- writing appropriate scripts that equally work on VirtualPC as on a real machine therefore seems a better approach to me.
Finally, consider it a shameless plug, but if you are looking for a unit testing framework for Windows kernel mode code, I have written one: cfix.
I think the DevCon utility (described in this OSR Online article) will help you. You should be able to setup batch files that do the job on one click.
It's free to sign up with osronline.com, and you'll probably have to sign up to get to that article. And if you are writing drivers, you WANT to sign up. These guys have been doing this for a long time, and there's a LOT of really good info on that web site.