Instrumenting a Rust executable for profiling - windows

I'm trying to analyze the performance of a Rust executable on Windows, but cannot seem to instrument the executable image using the VSInstr.exe tool.
To get started I set up a binary cargo package (cargo new --bin sample), and enabled generation of a PDB by adding
[profile.release]
debug = true
to the standard Cargo.toml file. Executing cargo build --release produces both sample.exe and sample.pdb. Navigating to target/release and executing vsinstr sample.exe produces the following diagnostic (VS2017 and VS2019, respectively):
Error VSP1011: Unable to obtain debug information. Link with the /Profile linker switch.
Unable to obtain debug information. Link with the /PROFILE linker switch.
Fair enough, so let's add the following .cargo/config file:
[build]
rustflags = ["-C", "link-args=/PROFILE"]
to pass the required flag to the linker. Running cargo build --release again produces the same artifacts, sample.exe and sample.pdb. Attempting to instrument the image using vsinstr sample.exe now produces the following diagnostic (VS2017 and VS2019, respectively):
Error VSP1033: The file '<path>\sample.exe' does not contain a recognized executable image.
The file does not contain a recognized executable image.
I'm not sure how to proceed from here, or how to further diagnose the core issue.
Is there something wrong with what I'm doing?
Am I running into a limitation of rustc/Clang?
Most importantly, how do I fix the issue?
Notes:
The /PROFILE linker option is documented to be "available only in Enterprise (team development) versions". That information seems outdated. Using the option on a C++ application in Visual Studio 2019 Community had the desired effect.
I've verified, that the /PROFILE flag gets passed to the linker by adding the -Z print-link-args flag to the rustflags in .cargo/config to see the actual linker command line.
Profiling any given Rust application in sample mode (CPU Usage) from the Visual Studio IDE works as long as the respective PDB is available. That's helpful, but I'd really like to get accurate call graphs, so instrumentation is required.
Alongside VSInstr.exe, Visual Studio also ships a binary called vsinstr.legacy.exe, that apparently dumps out more information. Running this against the Rust executable produces a list of warnings that have the following shape:
Warning VSP2005: Internal instrumentation warning: Block start at 1400016D9 is inside instruction at 1400016D8. Removed.
The addresses in all messages differ exactly by 1, which looks an awful lot like an off-by-one error, presumably in the code that produces the PDBs.

Related

Building Tensorflow with LTCG

I'm trying to build Tensorflow 1.14 on Windows using VS 2017 with LTCG (link time code generation) enabled. I'm hitting this crash partway through the build:
external/bazel_tools/tools/def_parser/def_parser.exe bazel-out/x64_windows-opt/bin/tensorflow/contrib/layers/python/ops/_sparse_feature_cross_op.so.gen.def _sparse_feature_cross_op.so #bazel-out/x64_windows-opt/bin/tensorflow/contrib/layers/python/ops/_sparse_feature_cross_op.so.gen.def-0.params
ERROR: E:/tensorflow/tensorflow/contrib/layers/BUILD:22:1: DefParser tensorflow/contrib/layers/python/ops/_sparse_feature_cross_op.so.gen.def failed (Exit -1073741819): def_parser.exe failed: error executing command
My environment is:
Tensorflow version: 1.14 (no source edits). Retrieved from https://github.com/tensorflow/tensorflow.git, branch r1.14
Visual Studio version: VS 2017
Bazel version: 0.25.2
Steps:
set BAZEL_VC=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC
set BAZEL_VC_FULL_VERSION=14.16.27023
set BAZEL_VS=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise
python .\configure.py
<Use all of the defaults>
bazel build -s --config=opt --copt=/GL --linkopt=/LTCG //tensorflow/tools/pip_package:build_pip_package
I've tried various versions of bazel (0.21, 0.26, 0.27, 0.28) and also tried with VS 2019 while changing the BAZEL_* environment variables, but I'm still hitting the same error. I've run the external/bazel_tools/tools/def_parser/def_parser.exe bazel-out/x64_windows-opt/bin/tensorflow/contrib/layers/python/ops/_sparse_feature_cross_op.so.gen.def _sparse_feature_cross_op.so #bazel-out/x64_windows-opt/bin/tensorflow/contrib/layers/python/ops/_sparse_feature_cross_op.so.gen.def-0.params command locally and it does crash with the -1073741819 error code.
Has anyone had experience building Tensorflow with LTCG, or using Bazel with LTCG?
TF 1.14 requires Bazel 0.24.1, AFAIK it doesn't work with newer Bazel versions (>= 0.25).
I don't know what the problem could be, but I can tell you how to debug it.
You'll need to get Bazel 0.24.1's sources, add debug logging to the DEF parser, build Bazel from source, and use the resulting binary to build TensorFlow.
To do so:
download the Bazel 0.24.1 release
download and extract the 0.24.1 sources OR git clone Bazel's GitHub tree and check out the 0.24.1 tag
add debug logging / printf calls to third_party/def_parser/* as you see fit
with the 0.24.1 release binary, run bazel build //src:bazel.exe in the patched source tree
use the resulting bazel-bin\src\bazel.exe to build TensorFlow
if you need to add more debug logging, repeat steps 3..5
I just tried to build TF 2.2 with MSVC 2019 v142 toolset (exact version 14.25.28610) with /GL and /LTCG options and I got the same error, but in a slightly different place. Here is my cmd line:
set BAZEL_VC=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC
bazel build --config=opt --config=windows --compilation_mode=opt --strip=always --copt="/MT" --copt="/Oy" --copt="/fp:fast" --copt="/GL" --linkopt="/DEBUG:NONE" --linkopt="/LTCG" --linkopt="/NODEFAULTLIB:msvcrt.lib" --linkopt="/NODEFAULTLIB:vcruntime.lib" --local_ram_resources=6512 --subcommands //tensorflow/tools/lib_package:libtensorflow > out.log 2>&1
I had to specify the MSVC tools folder directly, because Bazel kept trying to use the older version, details in this issue.
This is where it fails:
SUBCOMMAND: # //tensorflow:tf_custom_op_library_additional_deps.dll [action 'DefParser tensorflow/tf_custom_op_library_additional_deps.dll.gen.def', configuration: e5dbf2de175ef0b99efae20c93576efaae21f61b49e23200be8ee726f25b19c6]
cd C:/users/roman.kruglov/_bazel_roman.kruglov/e5u6xdzn/execroot/org_tensorflow
SET PATH=C:\Program Files\Git\bin;C:\Program Files\Git\usr\bin;C:\WINDOWS;C:\WINDOWS\System32;C:\WINDOWS\System32\WindowsPowerShell\v1.0
SET PYTHON_BIN_PATH=C:/Users/roman.kruglov/AppData/Local/Programs/Python/Python37/python.exe
SET PYTHON_LIB_PATH=C:/Users/roman.kruglov/AppData/Local/Programs/Python/Python37/lib/site-packages
SET RUNFILES_MANIFEST_ONLY=1
SET TF2_BEHAVIOR=1
SET TF_CONFIGURE_IOS=0
SET TF_ENABLE_XLA=1
external/bazel_tools/tools/def_parser/def_parser.exe bazel-out/x64_windows-opt/bin/tensorflow/tf_custom_op_library_additional_deps.dll.gen.def tf_custom_op_library_additional_deps.dll #bazel-out/x64_windows-opt/bin/tensorflow/tf_custom_op_library_additional_deps.dll.gen.def-0.params
ERROR: C:/data_d/git/test/tensorflow/tensorflow/BUILD:750:1: DefParser tensorflow/tf_custom_op_library_additional_deps.dll.gen.def failed (Exit -1073741819)
My conjecture currently is - it happens because with /GL enabled cl.exe produces a different format of output. As specified here, .obj files produced with /GL will not be available to such linker utilities as EDITBIN and DUMPBIN. I guess this DefParser tool just can't read that output. I'll try to rebuild without Global Optimization on and share my findings.
I guess it's just not feasible.
P.S. Just a heads up. I tried to build with /GL and stuff numerous times with no luck. I managed to build without /GL and stuff numerous times. There are several posts in the internet with similar attempts all failing with roughly the same symptoms.
Thus I conclude that my conjecture was true and it's not possible to build TF with global optimizations or link time code generation, etc. I guess the same stands true for Linux, because using lto there changes output object files format as well.

Shared library under Windows and CMake: DLL not found before installation

The library mylib consists of the library proper, in directory lib/, and a test suite, in directory test/. It is completely under CMake control:
mylib/CMakeLists.txt:
...
add_subdirectory(lib)
add_subdirectory(test)
...
mylib/lib/CMakeLists.txt:
...
add_library(my_lib ${src_files})
...
mylib/test/CMakeLists.txt:
...
add_executable(mytest mytest.c)
target_link_libraries(mytest mylib)
Build steps are:
mkdir build
cd build
cmake ..
make
ctest # or make test
make install
Works under Linux, stable since many years. Under Windows10 though, a message window pops up, entitled "mytest.exe - System error": "The code execution cannot proceed because mylib.dll was not found. Reinstalling the program may fix this problem."
No, installing (rather than reinstalling) would not be a good solution: I need to first test the library before I install it (btw: this excludes most solutions proposed in response to somewhat similar questions).
Isn't CMake supposed to work cross-platform? What is the minimally invasive adjustment to make the above build steps work under Windows?
The right way of doing this on Windows is to populate the PATH environment variable for the test run:
set_tests_properties(your_test_name
PROPERTIES
ENVIRONMENT PATH="path-containing-your-dll")
I believe you can use generator expression if path-containing-your-dll is a function of an artifact that you generate in your build.
Cherry on top: since cmake 3.13, the variable VS_DEBUGGER_ENVIRONMENT can also be set on the target for having a nice debugging behaviour inside Visual Studio (eg. being able to debug the application directly from Visual instead of going through ctest).

Boost installation error: No toolsets were configured

I've been trying this for over 5 days and I have no idea how to get this to work. I had successfully installed boost once, then I got my computer re-imaged and now it's just not happening. I have Windows 7 Enterprise, and 64-bit operating system.
I downloaded boost from here sourceforge
unzipped it into program files.
I then went to the VS 2013 Native Command prompt. Changed directory to boost tools/build
Ran bootstrap.bat
I then ran ./b2 address-model=64
but it did not give me directories for the compiler and the linker like last time.
I then ran ./b2 --prefix=C:\ProgramFiles\boost_1_58_0
but again nothing happens. I get the following errors:
Warning: No Toolsets were configured.
Warning: Configuring default toolset ""msvc"
Warning: If the default is wrong, your build may not work correctly
warning: Use the "toolset=xxxxx" option to overrride out guess
warning: for more configuration please consult
I have no idea why this worked the first time I had done this and why this isn't working now. Can someone please help me out. I know nothing about Unix but I need to install this so I can use the libraries.
I compile boost with both mingw (64 bit) and msvc 2013 pro. I have never in my life used the vs command prompt to build boost with msvc. Here are my commands to build 64 bit binaries on both toolchains.
First go into the boost folder and just double click bootstrap.bat. This should run and build bjam/b2. Nothing special required and doesn't matter what compiler this gets built with.
Then simply run, in a normal command prompt:
bjam.exe -a -j8 --toolset=msvc --layout=system optimization=speed link=shared threading=multi address-model=64 --stagedir=stage\MSVC-X64 release stage
Where -a forces rebuild all, -j8 means for the build to use 8 cores (adjust this based on your processor capabilities), toolset is obvious, layout means how to structure the naming of the output files, address-model is obvious, stagedir is where to output the built binaries (either relative or absolute path) and release stage is the type of build the system. See more here.
Same thing but using 64 bit mingw.
bjam.exe -j8 --toolset=gcc --layout=system optimization=speed link=shared threading=multi address-model=64 --stagedir=stage\x64 release stage
You can even go on to build additional libraries with a second pass tweaking your arguments. For example, after I've built the core libs with the above command, I run the following command to build in zlib and gzip support.
bjam.exe -a -j8 --toolset=msvc --layout=system optimization=speed link=shared threading=multi address-model=64 --stagedir=stage\MSVC-X64 --with-iostreams -s BZIP2_SOURCE=C:\dev\libraries\cpp\bzip2-1.0.6 -s ZLIB_SOURCE=C:\dev\libraries\cpp\zlib-1.2.8 release stage
Anyway that's just as an example. I linked to the full docs for the build system. Try building using these commands NOT inside a vs command prompt. If you still have problems then please post specific errors.
A note about MSVC
So, msvc compiler + boost supports a type of linking where it will automatically include additional libraries that it knows it needs. I believe boost does this by using the #pragma directive in headers. For example you might use boost::asio configured in such a way that it needs boost::system (for error codes and such). Even if you don't explicitly add boost::system library to the linker options, msvc will "figure out" that it needs this library and automatically try to link against it.
Now an issue arises here because you're using layout=system. The libraries are named simply "boost_" + the lib name, like "boost_system.dll". However, unless you specify a preprocessor define, this auto linking will try and link to a name like "boost_system_msvc_mt_1_58.lib". That's not exact as I can't recall the exact name, but you get the idea. You specifically told boost to name your libraries with the system layout (boost_thread, boost_system) etc instead of the default, which includes the boost version, "MT" if multithreaded, the compiler version, and lots of other stuff in the name. So that's why the auto linking feature goes looking for such a crazy weird name and fails. To fix this, add "BOOST_AUTO_LINK_NOMANGLE" in the Preprocessor section of your C++ settings in visual studio.
More on that here. Note that the answer here gives you a different preprocessor definition to solve this problem. I believe I ended up using BOOST_AUTO_LINK_NOMANGLE instead (which ended up working for me) because the ALL_DYN_LINK macro turned out to be designed as an "internal" define that boost controls and sets itself. Not sure as it's been some time since I had this issue, but the define I provide seems to solve the same root issue anyway.

Heat.exe: 64-bit .dll fails to be converted to a 64-bit .msi

I tried to generate a 64-bit FireBreath MSI installer using heat.exe but got the following output:
heat.exe : warning HEAT1108 : The command line switch 't:' is deprecated.
Please use 't' instead.
heat.exe : warning HEAT5150 : Could not harvest data from a file that was
expected to be a SelfReg DLL:
C:\Users\firebreath-master\buildPlugin\bin\Release\myPlugin64.dll.
If this file does not support SelfReg you can ignore this warning.
Otherwise, this error detail may be helpful to diagnose the failure:
Unable to load file:
C:\Users\firebreath- master\buildPlugin\bin\Release\myPlugin64.dll, error: 193
In order to generate the 64-bit .msi I copied the MyPlugin_auto.wxs that was generated from the 32-bit .dll, and build again using the 64-bit .dll and the MyPlugin_auto.wxs that was copied previously. 64-bit msi was generated without errors, I installed it, test it and it works, but I would like to ask if it is a better way to get over the heat.exe bug.
Heat is telling you that it saw a Dll with the DllRegisterServer() function exposed and that it failed to harvest any COM information. This typically means that the DLL is missing dependencies and that DllRegisterServer() failed to run correctly during the harvesting.
I'd take a look at the DLL using Depends and look for missing dependencies. Put those DLL's into the directory and try harvesting again.

How to compile Qt for 64-bit Windows from a 32-bit environment with Visual C++ 2010 Express?

I am trying to compile the Qt library (I don't need the demos or examples) for 64-bit Windows. There are instructions here but I run into the error described in the comment below it. There doesn't seem to be a reference anywhere for how one might go about doing this process.
I am targetting Microsoft Visual C++ 2010 Express. It looks like I need Perl and the Windows SDK as well - how do I go about this process?
This process is quite tedious and time-consuming - but I will explain each step in detail here for the benefit of others who try to compile Qt in the future.
The first step is to install all of the prerequisites.
ActivePerl, which is used during the configuration process. You will need to restart after installing Perl since it modifies environment variables.
The Windows SDK 7.1 (formerly called the Platform SDK). Be sure to include the x64 libraries when you select the components to install.
Download the Qt source archive from the Qt Downloads page.
Extract the contents of the archive to an easy-to-remember location (like C:\). You need to remember this location later since we will be using it to set some environment variables.
Now open the Windows SDK 7.1 Command Prompt. Begin by setting the environment to 32-bit release mode (we need to build some of the tools as 32-bit applications):
setenv /release /x86
Set the following environment variables (example below assumes you extracted to C:\):
set QTDIR=C:\qt-everywhere-opensource-src-4.8.0
set PATH=%PATH%;%QTDIR%\bin
Now run cd %QTDIR% and specify the configuration options - example is included below:
configure -release -opensource -qt-zlib -qt-libpng -qt-libmng -qt-libtiff
-qt-libjpeg -qt-style-windowsxp -qt-style-windowsvista -platform
win32-msvc2010
Once the configuration process is complete, cd to the src directory and run:
qmake
nmake
This process may take a considerable amount of time, so now would be a good time to take a break and answer some questions here on Stack Overflow :)
The tools are now built and you need to compile Qt as a 64-bit library. Enter the following command:
setenv /x64
You will need to set the environment variables from step 5 again. Enter those commands now.
Run cd %QTDIR% and then rerun the configure command being sure to specify one additional option:
configure -release -opensource -qt-zlib -qt-libpng -qt-libmng -qt-libtiff
-qt-libjpeg -qt-style-windowsxp -qt-style-windowsvista -platform
win32-msvc2010 -no-qmake
The -no-qmake option is very important - it indicates that we want to skip the compilation of the qmake.exe program because we want to keep the 32-bit version.
Now things get really complicated here because of some dependency problems. The tools (like moc) that Qt needs to build the core library and some of the other components are listed as dependencies in the src.pro file. This means that the compiler will attempt to build them as 64-bit applications and then try to run them - which will of course fail on a 32-bit system. So what we need to do is edit src.pro and remove those dependencies ourselves. Scroll down near line 85 and look for a line that begins with:
!wince*:!ordered:!symbian-abld:!symbian-sbsv2 {
Each subsequent line in the section lists a sub-target and its dependencies. What you want to do now is remove all dependencies that begin with src_tools_. For example:
src_gui.depends = src_corelib src_tools_uic
Becomes:
src_gui.depends = src_corelib
There might be a better way of doing this, but I haven't figured it out yet :)
Now we cd into the src directory once again and run the following command
nmake sub-winmain sub-corelib sub-xml sub-network sub-sql sub-testlib
sub-gui sub-qt3support sub-activeqt sub-opengl sub-xmlpatterns sub-phonon
sub-multimedia sub-svg sub-script sub-declarative sub-webkit
sub-scripttools sub-plugins sub-imports
This builds only the Qt libraries and skips the tool dependencies. Note that this too may take a considerable amount of time.
You should now have 64-bit libraries in the lib folder that you can link against in your 64-bit Qt applications.
Edit: it turns out that even this wasn't enough since I still ran into some problems when linking the QtWebKit4.dll library (something about unresolved symbols). It turns out that someone else has already found the solution and you need to change QMAKE_HOST.arch to QMAKE_TARGET.arch in WebCore.pro.
Also, the above options will build QNetwork4.dll without OpenSSL support (you won't be able to access sites over HTTPS - even in a QWebView). This, thankfully isn't too hard to fix. Download and build OpenSSL for Win64 and append the options below to the command in step #9:
-openssl -I C:\OpenSSL\inc32 -L C:\OpenSSL\out32dll
(You'll have to change the paths if you installed OpenSSL somewhere other than C:\OpenSSL.)
Further edit: to save the trouble of doing this yourself, I have uploaded the compiled libraries here:
http://www.box.com/s/9710cbb278ef4890a7b5
As I mentioned in the comments to George Edison's answer, there is a bug in the Microsoft Visual C++ compiler that comes with the Windows SDK 7.1. For more information on this, see QTBUG-11445 and QTBUG-19175.
I have compiled the Qt 4.8.2 64-bit binaries following George's instructions, including the OpenSSH library. In addition, I applied Microsoft's hotfix to fix the compiler bug.
For your convenience, I have made the resulting 64-bit libraries available for download from here: https://www.box.com/s/8948c60c3cdd743ef83b

Resources