With Intel Advisor, I ran the following on Linux in the cloud and downloaded the result folders to my local WIndows machine:
advixe-cl --collect=survey ./My_Program.exe
advixe-cl --collect=map ./My_Program .exe
advixe-cl --collect=dependencies ./My_Program .exe
Then I opened my Windows GUI for Advisor. I clicked the "Open Result" icon and opened the advixeproj file. It correctly shows sub-folders for survey, map and dependencies. When I open any of them, all I see in each of the window panes is:
No Data
To collect data about your application's performance, compile your application in Release Build settings and run Survey analysis.
My application is a C program (My_Program.exe) that calls a shared object written in assembly language (NASM). I assume Advisor can work with assembly language programs and shared objects because VTune does, so I don't think that's the problem.
Next to the Application field I click "Browse" and browse to the My_Program.exe, but Advisor says the file "is not an executable binary." Maybe that's because this is Windows and the binary is for Linux.
My question is: how do I view results from a Linux cloud server downloaded to my local Windows machine for analysis with the Windows GUI? I do that regularly with VTune without any problems.
Thanks.
In short - the method described in the question is generally correct, but for Advisor it is also important to specify --project-dir (keep it the same accross all analysis types).
1) [on linux] advixe-cl --collect=survey --project-dir ./my_project_dir ./My_Program.exe
2) [on linux] advixe-cl --collect=tripcounts --project-dir ./my_project_dir ./My_Program.exe
etc..
3) copy my_project_dir folder from Linux to Windows
4) [on windows in Advisor GUI or with advixe-gui - open] ./my_project_dir and use "Show My Result" button.
This is covered on Intel forum too, as noted in "comments" to the original question.
In addition There are 2 other different methods : using --snapshot command or just exchaning interactive HTML GUIs (available for Roofline and Offload features). They are described at e.g. given nice article: https://software.intel.com/content/www/us/en/develop/documentation/advisor-cookbook/top/analyze-performance-remotely-and-visualize-results-on-macos.html
Related
Intel VTune Amplifier has the possibility to profile a parallel application executed on a remote machine.
Intel Advisor doesn't have such an option. According to this document, you have to use the command-line version of Intel Advisor:
This makes it possible to automate many tasks as well as analyze an
application running on remote hosts
However, the GUI version has many features not offered by the cl version (like suggestions about how to solve vectorization/multi-thread inefficiency etc).
I tried to run advixe-cl on the remote machine and then copy locally the project (and produced results). It works, but some features are lost. As last chance I tried to ssh -X the remote machine and the use advixe-gui, but it seems that the main core of my Xeon Phi KNL is too weak to ruun properly such a graphic application.
What is the correct/best use of Intel Advisor in such a scenario?
The recommended way is described by you here: "run advixe-cl on the remote machine and then copy locally the project".
But you mentioned that "some features were lost". What did you loose exactly?
The key defficiency of given command-line+GUI approach is that you may not see your source code in "Source View" tabs initially. To overcome this limitation, you have to adjust Project Properties of your local project copy and specify "Source Search" and sometimes to "Binaries/Symbol Search" specifying directories providing path to the location where original source code and sometimes executable binarry plus DWARF/pdb debug info files are located.
In case you used "-no-auto-finalize" option in command line (which is more advanced scenario), you may also need to use Re-Finalize feature (available only starting from 2017 Update 2 new release) or (for older versions) make sure that you provide Binary/Symbol/Source Search after opening local project copy, but before "Show My Result" upload data action.
My scenario is I'm supporting a VB6 app at the place I work and in the last few weeks it has started crashing more often than it ever used to. It uses both a local Access MDB database and a remote SQL Server DB for different types of storage. The good news is we are writing a replacement app, the band news I need to support this one in the meantime and the vendor is long gone from this world.
What are some ways I could try and diagnose what is causing the crash? For example so far I've tried ODBC tracing (For the MDB component), SQL Profiler tracing and ProcMon on a client PC.
Is there anything else I could try to discover what the app was trying to do at the time of the crash?
You can also start in a debugger.
windbg or ntsd (ntsd is a console program and maybe installed). Both are also from Debugging Tools For Windows.
Download and install Debugging Tools for Windows
http://msdn.microsoft.com/en-us/windows/hardware/hh852363
Install the Windows SDK but just choose the debugging tools.
Create a folder called Symbols in C:\
Start Windbg. File menu - Symbol File Path and enter
srv*C:\symbols*http://msdl.microsoft.com/download/symbols
then
windbg -o -g -G c:\windows\system32\cmd.exe /k batfile.bat
You can press F12 to stop it and kb will show the call stack (g continues the program). If there's errors it will also stop and show them.
Type lm to list loaded modules, x *!* to list the symbols and bp symbolname to set a breakpoint
Use db address (as in db 01244 to see what's at that memory.
If programming in VB6 then this environmental variable link=/pdb:none stores the symbols in the dll rather than seperate files. Make sure you compile the program with No Optimisations and tick the box for Create Symbolic Debug Info. Both on the Compile tab in the Project's Properties.
I tried to learn "Lena's reversing for newbies", when some trouble arise.
I start Pixtopian Book with ollyDbg, then try to have MessageBox with message about uregistered version.
Then i switch to OllyDbg, stop program executing and press "Alt+F9" for "Back to user mode" which stop the program after it exit from DLL.
But after this program does not work, it's frozen and does not respond to my actions.
If i turn off "Back to user mode" program normally work.
What's the problem? Can i try to use "Back to user mode" in IDA (uses WinDbg) or some other debugger and How i can do this? Can i repair it function in OllyDbg?
P.S. It's like the program stopped and didn't run after use "ALT+F9".
/Sorry for my English, i'm just learning ;-)/
First of all, Ollydbg is meant for 32 bit platform. It means that it will run only on a 32 bit OS and can only debug 32 bit apps.
In x64 Operating Systems (specifically Windows) there is a feature called compatibility mode that lets you run 32 bit apps. The 32 bit code is either run by emulation or natively (when the hardware itself implements the instruction set and then it is called x86-64).
So, when you try to run Ollydbg in a x64 environment it will run, but you will experience problems like the one you are facing. It occurs because Ollydbg is run in an emulation mode. Being a debugger it needs access to the registers and other system structures, which it is denied. What it can see is a virtual image of the system.
So the solution to the problem is using a Virtual Machine.
You would install a 32 bit OS in it and debug the app using Ollydbg. As far as Virtual Machines are concerned, I would recommend VMWare. You can use either the workstation or player version. The latter is free but does not support snapshots.
Other solutions are Virtual Box, Parallels Workstation and Microsoft Virtual PC.
The disadvantages of them are that Virtual Box does not support hardware breakpoints, Parallels Workstation is no longer supported as of 2014 and moreover there you would get a BSOD if you try to single step through FPU instructions. I have not tested Virtual PC though.
Note : Ollydbg does not supports x64 but its author is working on a x64 version.
I just learned how to update changes to the exe file for ollydby v 2.01e.
suppose I wished to change a jl command to a jmp; do this by clicking the executable modules button, choose the file and right click to view the file. then record the changes and save file. The saved file also has a backup in case something goes wrong.
I just did a thorough learning of the pixtopian file. When I downloaded
that tutorial I get the file pixtopian107.exe only. Since it didn't agree with the tutorial I investigated further. That file is an installation file. It produces a regular pixtopian.exe file which is the file you wish to play with.
I also noticed that in running the file it never enters the main module.
this is because of the TLS callback. right now I am trying to learn how
to overcome this which is how i came to this site in the first place.
I am using ollydbg vs2.01e very effectively.
Another thing, in vs 2.01e I am still trying to save changed data. Until I succeed I am recording the changes in the exe files using hex editor.
From my experience I can tell that this functionality won't work on Windows 7.
On Windows 7 64bit --> Won't work at all.
On Windows 7 32bit --> Will work partially, but only when using option "File>Attach" in OllyDbg.
For me, best solution was to use Windows XP 32bit, then it worked fine.
A few of my users have problems starting up my Java application on Windows XP and 7. I cannot reproduce the bug on my systems.
On Mac, the next step would be to ask them to send their system profile to me by going to Apple Menu > About This Mac > More Info and then saving the system profile.
Is there an equivalent on Windows? I would love to see:
Version numbers for the system
Hardware configuration
Shell paths: $HOME, $APPDATA, $LOCALAPPDATA
Drive layout
Programs running on startup
Loaded drivers and DLLs
Possible system logs
The built-in "System Information" tool in Windows provides most of what you want, and a lot of other useful information. No System logs though.
Have your users run "MSINFO32" from the start menu. Then File -> Export if you want a text output, or File -> Save... if you want a file that you can open with MSINFO32 on your end.
Whenever I'm debugging on a webserver (Windows) DebugDiag is good for memory/dll issues. You can load the crash dumps right into WinDBG.
For the system information, you will probably have to write your own script to collect this. WMI should let you collect the remainder of the system settings.
Here are a couple of possibilities that get part way there. The PsTools suite from SysInternals has a number of command line utilities that might be useful for this. In particular psinfo (system information) and pslist (running processes). Also, Process Explorer could be useful too since it has an option to save information to a file.
Does anyone know ways of partially or fully automating driver test installation?
I am new to driver development and am used to more of a test-driven approach in higher level languages, so moving to the kind of environment where I can't easily test as I go has been a step up for me. I am using Virtual PC for my test environment and currently have to reset it, open device manager, choose the device, click through a bunch of "Are you really sure you wouldn't rather install one of these system drivers" type dialogs, then finally reset the test environment while restarting WinDbg in the host machine just as the test environment is booting up... argh.
After repeating this process many, many times already, surely there has to be a be a better way of doing this? What tools/methods/tricks do commercial driver developers use to run up their driver in a test environment?
Note, this isn't about unit testing drivers, I haven't got to that stage yet or know if it is even possible. This is just about firing up a test environment with WinDbg attached to make sure that some small change I may have done is doing what I expect.
It seems to me that a virtualization software + a "mock objects" (layering) approach (as suggested by Aaron Digulla) + scripts (as suggested by Sergius) can simplify device driver development.
But if you use Visual Studio to develop user-level applications, you can use it for kernel device driver development too with VisualDDK (+ VirtualKD to debug over a named pipe, which is faster than over a virtual COM port), which addresses specifically the annoyances that you mention; from its home page:
... This project brings the simplicity and
convenience of Windows application
development to the driver development
world. No more manual creation of
build scripts, copying of driver
files, installing drivers from INFs,
switching between WinDbg and the
source editor or waiting for seconds
after each step due to the extra-slow
virtual COM port. Just create a driver
project using a convenient Driver
Wizard, select a virtual machine, and
enjoy debugging your driver directly
from Visual Studio. Want to test a
change? Just normally press Shift-F5,
modify your driver, rebuild it and
launch again. VisualDDK will unload
the old driver, install the new one
and load it automatically and quickly.
Bored with WinDbg loading symbol files
for minutes and looking up symbols for
seconds? Just let VisualDDK optimize
this for you using its own DIA-based
symbol engine. Using C++/STLPort in
your drivers? VisualDDK will natively
visualize all STL containers and
strings, as good as Visual Studio does
for user-mode applications. ...
You can write some shell scripts (using sc.exe and devcon.exe) to automate deployment tasks (no opening device manager, clicking on buttons, etc). And make snapshot of the system ready to debug (needn't wait for system boot).
Don't forget to check your driver with DriverVerifier!
Example of my own script :)
sc create FsFilter type= filesys binPath= c:\FSFilterDrv.sys
sc start FsFilter
pause
sc stop FsFilter
sc delete FsFilter
Follow the advice I gave here. Basically, test as little as possible with the real system.
In your case, I've got another tip: Virtual PC is using a virtual hard disk (that's probably a file on your real hard disk).
You don't need to install your driver, you can simply replace the new files in the virtual hard disk. This is often not possible in the running system but in a virtual system, you can open the virtual disk file and change it (since Windows isn't locking the files in it).
I'm not sure about Virtual PC but other emulators come with tools to work with virtual disk images. If VPC can't do it, check out VirtualBox.
It all depends a little on what kind of driver you are writing. But in many cases, writing an appropriate makefile (or something similar) that handles driver installation, start/stop, and launching of a test harness can already be good enough.
I also configure all of my test machines to automatically logon (AutoAdminLogon), map net drives, and launch an appropriate command prompt after startup. Running a specific test is then a matter of typing in a single command only.
One word concerning VirtualPC: VirtualPC is very handy for kernel mode development, but do not forget that it emulates a uniprocessor machine only -- so be sure to regularly test the code on a multiprocessor machine as well. That said, the VHD trick may seem handy, but it somewhat ties you to Virtual PC -- writing appropriate scripts that equally work on VirtualPC as on a real machine therefore seems a better approach to me.
Finally, consider it a shameless plug, but if you are looking for a unit testing framework for Windows kernel mode code, I have written one: cfix.
I think the DevCon utility (described in this OSR Online article) will help you. You should be able to setup batch files that do the job on one click.
It's free to sign up with osronline.com, and you'll probably have to sign up to get to that article. And if you are writing drivers, you WANT to sign up. These guys have been doing this for a long time, and there's a LOT of really good info on that web site.