This is on Windows 7 Pro 64 bit with CUDA 6.5 and Intel Composer 2015, in Visual Studio 2013.
I have a project that contains a mix of native c++ and CUDA. I'm trying to switch from the Microsoft compiler to the Intel compiler, for better vectorization and performance. However, I can't get the project to build after installing Intel Composer and changing the project setting to use the Intel compiler.
Whenever I try to build, I get the error:
nvcc fatal : Compiler 'cl.exe' in PATH different than the one specified with -ccbin
I tried changing the compiler to the Intel compiler located in C:\Program Files (x86)\Intel\Composer XE 2015\bin\intel64, by adding
-ccbin C:\Program Files (x86)\Intel\Composer XE 2015\bin\intel64\icl.exe
but that errors with
nvcc fatal : redefinition of argument 'compiler-bindir'
How can I configure CUDA to use the Intel compiler instead of the Microsoft one?
Only the microsoft cl.exe compiler is supported on Windows. The supported platforms are indicated in the windows getting started document.
The intel compiler is supported as the host compiler on linux however.
Related
I am using the Visual Studio 2019 product for Windows development on a 64 bit Windows and if I investigate the differences between the compilation and linking of 32 or 64 bit programs I only notice a difference in the linking stage. 32 bit linking specifies /MACHINE:X86 whilst 64 bit linking specifies /MACHINE:X64
However, this Microsoft link:
https://learn.microsoft.com/en-us/cpp/build/reference/machine-specify-target-platform?view=msvc-170
has this comment about the /MACHINE option : Usually, you don't have to specify the /MACHINE option. LINK infers the machine type from the .obj files.
So I am confused about how the Microsoft compiler directs either 32 or 64 bit output.
Is it at the compilation stage that object files have a platform designation, ie 32 or 64 bit?
Or is the target platform only specified at the linker stage?
Or is the target platform somehow controlled via the location of cl.exe which is run to generate the executable? For example, on my PC, cl.exe is located in 2 separate places:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64
and
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX64\x86
Each cl.exe only supports one architecture. Just running cl.exe should tell you its target or you can look at the options for /arch.
Object files contain CPU specific code. .obj files are in COFF format and contain information about the CPU.
I'm trying to cross-compile for a 64-bit ARMv8 / Raspbian (DietPi actually), from Windows, but I'm getting a series of issues with 3rd-party crates.
What I installed
rust toolchain 1.61.0
ARMv8 gcc toolchain (from here)
MS Visual Studio 2019 C++ build tools (from here)
(IntelliJ IDEA UE and the IntelliJ plugin - FYI but not really relevant to the question)
From there it is possible to add the required target - note that it's dependent on the ARM gcc toolchain (for example the 32-bit version is armv7-unknown-linux-gnueabihf):
rustup target add aarch64-unknown-linux-gnu
Then I edited %USERPROFILE%\.cargo\config and added those lines:
[target.aarch64-unknown-linux-gnu]
linker = "aarch64-linux-gnu-gcc.exe"
And finally, I added those to the PATH:
%USERPROFILE%\.cargo\bin
c:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin (for cmake)
c:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin (for msbuild)
[ARMv8 gcc toolchain]\bin
How I cross-compiled
After making sure everything was compiling and running correctly for Windows, I tried to cross-compile:
cargo build -r --target=aarch64-unknown-linux-gnu
The problem I have
While this worked for simple applications, it quickly becomes clear that many crates fail to compile. For example, freetype-sys, which is a dependency of plotters that I'm using:
error: failed to run custom build command for `freetype-sys v0.13.1`
Caused by:
process didn't exit successfully: `D:\projects\rust\humidity\rh\target\release\build\freetype-sys-4feef64f7ae6c484\build-script-build` (exit code: 101)
--- stdout
[...]
running: "cmake" "[...]\\freetype-sys-0.13.1\\freetype2" "-DWITH_BZip2=OFF" "-DWITH_HarfBuzz=OFF" "-DWITH_PNG=OFF" "-DWITH_ZLIB=OFF" "-DCMAKE_INSTALL_PREFIX=D:\\projects\\rust\\hum
idity\\rh\\target\\aarch64-unknown-linux-gnu\\release\\build\\freetype-sys-3464f88f9fbe3bc0\\out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_ASM_FLAGS=
-ffunction-sections -fdata-sections -fPIC" "-DCMAKE_BUILD_TYPE=Release"
-- Building for: Visual Studio 15 2017
-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.19043.
-- Configuring incomplete, errors occurred!
See also "D:/projects/rust/humidity/rh/target/aarch64-unknown-linux-gnu/release/build/freetype-sys-3464f88f9fbe3bc0/out/build/CMakeFiles/CMakeOutput.log".
--- stderr
CMake Error at CMakeLists.txt:119 (project):
Failed to run MSBuild command:
MSBuild.exe
to get the value of VCTargetsPath:
Microsoft (R) Build Engine version 16.0.462+g62fb89029d for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
Build started 11/06/2022 11:53:19.
Project "D:\projects\rust\humidity\rh\target\aarch64-unknown-linux-gnu\release\build\freetype-sys-3464f88f9fbe3bc0\out\build\CMakeFiles\3.13.19031502-MSVC_2\VCTargetsPath.vcxproj" on node 1 (default targets).
c:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Microsoft\VC\v160\Microsoft.CppBuild.targets(378,5): error MSB8020: The build tools for Visual Studio 2017 (Platform Toolset = 'v141') cannot be found. To build usin
g the v141 build tools, please install Visual Studio 2017 build tools. Alternatively, you may upgrade to the current Visual Studio tools by selecting the Project menu or right-click the solution, and then selecting "Retarget solution". [D:\p
rojects\rust\humidity\rh\target\aarch64-unknown-linux-gnu\release\build\freetype-sys-3464f88f9fbe3bc0\out\build\CMakeFiles\3.13.19031502-MSVC_2\VCTargetsPath.vcxproj]
Done Building Project "D:\projects\rust\humidity\rh\target\aarch64-unknown-linux-gnu\release\build\freetype-sys-3464f88f9fbe3bc0\out\build\CMakeFiles\3.13.19031502-MSVC_2\VCTargetsPath.vcxproj" (default targets) -- FAILED.
A previous crate required the path to 2019 MSBuild.exe, hence the extra PATH earlier which solved that problem.
This one seems to require MS VS 2017 build tools. This is getting desperate, so I think the problem is coming from something else.
What else I have tried
EDIT1:
I noticed that the Build Tools for Visual Studio 2017 (version 15.9) (here) include a cross-compiler to ARM64. So
I installed this version
launched the (somewhat hidden) vcvarsamd64_arm64.bat script to setup the environment
replaced the aarch64-linux-gnu-gcc.exe executable in %USERPROFILE%\.cargo\config with cl.exe which is the MS compiler/linker.
from the project directory, cargo clean
cargo build -r --target=aarch64-unknown-linux-gnu
It compiles much faster than the gcc toolchain, but it fails compiling the freetype crate:
Compiling freetype v0.7.0
error: could not find native static library `freetype`, perhaps an -L flag is missing?
Same result with the gcc toolchain and MS VC 2017.
Question: What exactly is required to cross-compile to this target? Am I missing something?
do I need to install several versions of VS build tools? I imagine they'll conflict if they're all in the PATH
do I need to install cmake separately, instead of using the one available in VS? (see PATH defined earlier with CMake)
is it simply not possible from Windows?
EDIT2: I'm starting to believe that the freetype create, which hasn't been updated for a few years and is still in version 0.7.0, cannot be cross-compiled for some reason.
UPDATE: I worked around the problem by replacing plotters with something else. It removed the freetype dependency (this module really has an issue) and allowed the cross-compilation to complete successfully.
I'm still interested by a solution to the problem, but it probably involves generating or finding the library for the target and finding a way to feed it to the compiler in the flow, so it may be somewhat convoluted.
In the project properties page of CUDA project in visual studio, there seems to be the option to custom the host compiler. But after I selected the intel C++ compiler, the -ccbin option still points to the cl.exe.
selecting intel C++ compiler as the platform toolset
the -ccbin option still points to cl.exe
I do understand that under windows environment, the default host compiler for CUDA project is cl.exe and the following posts confirmed that cl.exe was the only option on windwos. But these were quite some time ago, I'd like to ask again if this is still the case or we use a different host compiler now?
Intel C++ Composer and CUDA
Specify compiler NVCC uses to compile host-code
It's still the case. The only supported environments are listed in the installation guide for windows. The intel compiler is not listed.
By comparison, the corresponding section of the linux installation guide shows that a certain version of the intel compiler (ICC) is supported for the host compiler.
In the future, you should be able to refer to the corresponding documents published with newer CUDA toolkits to determine compiler support.
I often find myself using Intel C++ together with CUDA for fluid simulations, and I may be able to help if you are still interested in an answer.
You have not specified which version of Visual Studio you are referring to, but I imply that you are talking about Visual Studio Professional/Enterprise or any of that line of fully featured IDEs.
While I haven't extensively used any of those, there is a way of using Intel C++ (and by extension, any compiler) with CUDA, in Visual Studio Code. If you decide to go this route, and since you mentioned Windows, here is the procedure:
To make things simple, download the C++ tutorial build from here. If you want to make the build yourself, instructions are found on MSDN here.
After the download, you should have a build.bat file in the project directory. Open the file to edit it.
Replace everything inside the build.bat file as follows:
#echo off
call "Path\to\compilervars.bat" intel64
call "Path\to\vcvarsall.bat" x64
set compilerflags=/Zi /EHsc -I"%MKLROOT%"\include
set linkerflags= mkl_intel_ilp64.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib
icl.exe /Foobj\helloworld /Fdobj\ -c -I. -I"%CUDA_PATH%"\include %compilerflags% helloworld.cpp /link %linkerflags%
"Path\to\nvcc.exe" -gencode=arch=compute_61,code=\"sm_61,compute_61\" -IPath\to\CUDA\include -G -maxrregcount=0 --machine 64 --compile -cudart static -g -DWIN64 -DNDEBUG -D_CONSOLE -Xcompiler "/EHsc /W3 /nologo /Od /FS /Fdobj\ /Zi /RTC1 /MD /MP" -o obj\hello2.obj "hello.cu"
icl.exe -o bin\hello.exe /Fdobj\ obj\hello2.obj obj\helloworld.obj "%CUDA_PATH%"\lib\x64\cudart.lib %compilerflags% /link %linkerflags%
In other words, we tell the build process to:
Look for the compilervars.bat file (found in Intel C++ compiler /bin folder) and set Intel specific macros and variables for the build process
Look for the vcvarsall.bat file (found in Visual C++ compiler /VC folder) and set Visual C++ specific macros and variables for the build process
Set the Compiler and Linker Flags for the Intel C++ compiler.
IMPORTANT: Call the Intel C++ compiler to generate an Obj file from all the non-CUDA (i.e. .cpp files of the project - In this case, helloworld.cpp) and also include the CUDA headers if you need them. Make sure no CUDA kernel calls (functions with <<< >>>) happen here.
THEN: Call the CUDA compiler to generate an Obj file from all the CUDA files (i.e. the .cu extensions - In this case, hello.cu), in this case, I call the Obj file hello2.obj (I explicitly renamed it to show you how to do that).
FINALLY Call the C++ Compiler to generate the executable file by mentioning both Obj files as well as the cudart library for static linking.
Note that CUDA still needs VC++ so we allow it to access VC++ for only the part where it compiles CUDA files (which is why we had to initialize vcvarsall.bat), and stick to Intel C++ for the rest.
In the project properties page of CUDA project in visual studio, there seems to be the option to custom the host compiler. But after I selected the intel C++ compiler, the -ccbin option still points to the cl.exe.
selecting intel C++ compiler as the platform toolset
the -ccbin option still points to cl.exe
I do understand that under windows environment, the default host compiler for CUDA project is cl.exe and the following posts confirmed that cl.exe was the only option on windwos. But these were quite some time ago, I'd like to ask again if this is still the case or we use a different host compiler now?
Intel C++ Composer and CUDA
Specify compiler NVCC uses to compile host-code
It's still the case. The only supported environments are listed in the installation guide for windows. The intel compiler is not listed.
By comparison, the corresponding section of the linux installation guide shows that a certain version of the intel compiler (ICC) is supported for the host compiler.
In the future, you should be able to refer to the corresponding documents published with newer CUDA toolkits to determine compiler support.
I often find myself using Intel C++ together with CUDA for fluid simulations, and I may be able to help if you are still interested in an answer.
You have not specified which version of Visual Studio you are referring to, but I imply that you are talking about Visual Studio Professional/Enterprise or any of that line of fully featured IDEs.
While I haven't extensively used any of those, there is a way of using Intel C++ (and by extension, any compiler) with CUDA, in Visual Studio Code. If you decide to go this route, and since you mentioned Windows, here is the procedure:
To make things simple, download the C++ tutorial build from here. If you want to make the build yourself, instructions are found on MSDN here.
After the download, you should have a build.bat file in the project directory. Open the file to edit it.
Replace everything inside the build.bat file as follows:
#echo off
call "Path\to\compilervars.bat" intel64
call "Path\to\vcvarsall.bat" x64
set compilerflags=/Zi /EHsc -I"%MKLROOT%"\include
set linkerflags= mkl_intel_ilp64.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib
icl.exe /Foobj\helloworld /Fdobj\ -c -I. -I"%CUDA_PATH%"\include %compilerflags% helloworld.cpp /link %linkerflags%
"Path\to\nvcc.exe" -gencode=arch=compute_61,code=\"sm_61,compute_61\" -IPath\to\CUDA\include -G -maxrregcount=0 --machine 64 --compile -cudart static -g -DWIN64 -DNDEBUG -D_CONSOLE -Xcompiler "/EHsc /W3 /nologo /Od /FS /Fdobj\ /Zi /RTC1 /MD /MP" -o obj\hello2.obj "hello.cu"
icl.exe -o bin\hello.exe /Fdobj\ obj\hello2.obj obj\helloworld.obj "%CUDA_PATH%"\lib\x64\cudart.lib %compilerflags% /link %linkerflags%
In other words, we tell the build process to:
Look for the compilervars.bat file (found in Intel C++ compiler /bin folder) and set Intel specific macros and variables for the build process
Look for the vcvarsall.bat file (found in Visual C++ compiler /VC folder) and set Visual C++ specific macros and variables for the build process
Set the Compiler and Linker Flags for the Intel C++ compiler.
IMPORTANT: Call the Intel C++ compiler to generate an Obj file from all the non-CUDA (i.e. .cpp files of the project - In this case, helloworld.cpp) and also include the CUDA headers if you need them. Make sure no CUDA kernel calls (functions with <<< >>>) happen here.
THEN: Call the CUDA compiler to generate an Obj file from all the CUDA files (i.e. the .cu extensions - In this case, hello.cu), in this case, I call the Obj file hello2.obj (I explicitly renamed it to show you how to do that).
FINALLY Call the C++ Compiler to generate the executable file by mentioning both Obj files as well as the cudart library for static linking.
Note that CUDA still needs VC++ so we allow it to access VC++ for only the part where it compiles CUDA files (which is why we had to initialize vcvarsall.bat), and stick to Intel C++ for the rest.
I'm compiling Berkley's Caffe to a static library, MT, MSVC 12 (VS2013) and linking it to a dll.
It works fine.
Now I'm trying to compile it with Intel C++ 2015 compiler - the static lib build fine, but the dependent dll fails with the following linker error:
fatal error LNK1104: cannot open file
'libboost_thread-iw-mt-s-1_58.lib'
There isn't such library in my Boost build indeed, but I don't know where this dependency came from. Except for the compiler I didn't change any other setting, and I can't find that Boost lib in either Caffe of my dll's Linker->Input->Additional dependencies.
How can I fix this?
Thanks!
Found the solution here: https://software.intel.com/en-us/articles/intel-c-compiler-for-windows-fatal-link-error-lnk1104-when-using-intel-c-compiler-with-boost-libraries
Copied:
When building an application that uses the Boost libraries with the
Intel® C++ Compiler, you may get linker errors like the ones shown
below due to incorrect libraries being linked to the application:
fatal error LNK1104: cannot open file
'libboost_thread-iw-mt-1_33_1.lib' fatal error LNK1104: cannot open
file 'libboost_thread-iw-1_33_1.lib' ... The root cause is missing
Boost libraries for the Intel® C++ Compiler.
The preferred solution is to recompile all required Boost libraries
with the Intel® C++ Compiler (libraries with the infix "iw" are
created because of this). However, this is not mandatory. The
libraries provided for the different Microsoft Visual Studio* versions
are safe to use as well. Perform the following steps to use them
instead:
Open the Boost configuration file "auto_link.hpp". Search for 1
elif defined(__ICL) 2 3 4
// Intel C++, no version number: 5 6
# define BOOST_LIB_TOOLSET "iw" Change "iw" depending on which Microsoft Visual Studio version you're using:
"vc71": Microsoft Visual Studio .NET 2003 "vc80": Microsoft Visual
Studio 2005 "vc90": Microsoft Visual Studio 2008 "vc100": Microsoft
Visual Studio 2010 "vc110": Microsoft Visual Studio 2012 "vc120":
Microsoft Visual Studio 2013 "vc140": Microsoft Visual Studio 2015
Rebuild your application to resolve the linker errors.