Configuring openssl to use windows threads and files on another platform - windows

I am in the process of porting a library to Xbox Series X console, and openssl (and by extension libcrypto) is one of dependencies of the library.
Openssl project is generated using Visual Studio 2019 from Makefiles. It mostly works after some work, and compiles using Xbox toolchain, but i have unresolved external symbols during linking. Mostly CRYPTO_THREAD_xxx and InterlockedExchangeAdd ones. I have noticed that my VS project only has crypto/threads_none.c in solution explorer, but i have OPENSSL_THREADS defined. Therefore, these functions never get into the final object file, as threads_none.c defines them only if OPENSSL_THREADS is not defined.
How can i generate this project so that it has same files as windows library? Namely, i would love it to use threads_win.c, and there are possibly other differences as well. I have zero idea how platform configuration works in openssl, it seems to use some proprietary system which is dark and unknown magic to me, held together with perl scripts.

Related

Why does my application require Visual C++ Redistributable package

I'm writing a simple C++ application in Visual Studio. It also has a setup project.
It works well on my development machine, but when I'm installing this application on user's machine it requires Visual C++ Redistributable Package. I'm wondering why does my application require C++ Redistributable? Standard C++ runtime library is shipped with Windows, isn't it?
The only version of the C runtime library which is shipped by Microsoft with most of 32 bit Windows versions is msvcrt.dll. This library provides a typical set of library functions required by C and C++ programs. These include string manipulation, memory allocation, C-style input/output calls, etc.
Visual Studio 6.0's compiler links against this library, so if you are developing in VS 6.0 you shouldn't encounter any problems on most users' machines.
However, if you are developing in VS 2005, VS 2008, VS 2010, VS 2012, VS 2013 or VS 2015, you have to distribute additional C runtime libraries along with your application. This is because their compilers link against msvcrt80.dll, msvcrt90.dll, msvcrt100.dll, msvcrt110.dll, msvcrt120.dll and msvcrt140.dll respectively, which are not shipped with Windows.
Solutions:
Possible solution is to link statically with runtime library, but it may cause a lot of problems in case you have both .exe and .dll in your application. Don't do that.
To be more specific, I'll allow myself to quote a part of this answer:
Using /MT is risky if you create DLLs as well as an EXE. You'll end up
with multiple copies of the CRT in your program. This was especially a
problem with earlier versions of VS where each CRT would get its own
heap, not so much with VS2012. But you can still have ugly runtime
problems when you have more than one "errno" variable for example.
Using /MD is highly recommended to avoid such lossage.
Another possible solution is to require an appropriate Microsoft Visual C++ Redistributable package to be installed on the user's machine.
It may be done by specifying this requirement in prerequisites property in your setup project.
Also, you can distribute the runtime dll by including in your setup project the appropriate "merge module". In this case don't forget to add the appropriate "policy merge module" to avoid errors caused by incorrect runtime version.
Finally, you can just put required DLLs in the same folder in which your application is installed.
Further reading:
"Redistributing Visual C++ Files" - Official MSDN documentation
Even though some comments said that «link statically with runtime library, but it may cause a lot of problems when you have both .exe and .dll in your application.» this is NOT TRUE. First we DON'T statically link DLLs! We statically link OBJs and LIBs. LIBs are static libraries; DLLs are dynamic libraries, and you may choose to use LIBs (static) or DLLs (dynamic). It's entirely up to you to choose. The ONLY drawback (for the DLL fans) is that if you want to update one library, you need to compile and link again. I personally deploy ALL my software static linked and because of that I earn the bonus of don't even need installers. The software I develop is 100% portable (a feature that in the pre-installer era was general procedure), and the final user is free to simple COPY from one folder to another or even from the hard drive to flash drive (or vice-versa). The error message «DLL not found.» simply doesn't exist ... NEVER.
Some folks think of statically linking as toy software: WRONG! I can write a full featured application that connects to a DBMS (Oracle, SQL Server, ...) or any other kind of application.

How to use GCC with Microsoft Visual Studio?

I am creating a very large project (a few thousand lines) and so would rather not use Notepad++. An IDE would make it so much easier. I have experience with Microsoft Visual Studio and love it. Is there some easy way to use Cygwin's GCC from within Microsoft Visual Studio?
Alternately, are there any other good Windows IDEs for GCC besides NetBeans and Eclipse? (I hate both of them with a passion.)
There are several ways to go here:
Option 1: Create a Custom Build Tool
Visual Studio 2005 and newer will let you register custom build tools. They tell the IDE how to transform files of one form (e.g. a .cpp file) into another form (e.g. an .obj file).
So far as I know, no one has done this yet for GCC. And, doing it yourself requires writing COM code, which is probably too deep a pool to dive into just for a single project. You'd have to have a compelling reason to take this project on.
You then have to manually adjust each project to tell it to use the custom build tool instead of the default, since you're using a file name extension (.cpp, probably) that Visual C++ already knows about. You'll run into trouble if you try to mix the VC++ and g++ compilers for a single executable built from multiple modules.
On the plus side, if you were looking to start an open source project, this sounds like a good one to me. I expect you'd quickly gather a big user base.
Option 2: Makefile Project
Start Visual Studio and say File > New Project.
In the Visual C++ section, select Makefile Project
Fill out the Makefile Project Wizard:
Build command line: make
Clean commands: make clean
Rebuild command line: make clean all
You can leave the Output (for debugging) field alone if you've named your executable after the project name and it lands where Visual Studio expects to find it.
Leave the rest of the fields alone unless you know what they are and why you want to change them. As an example, you might choose to pass a -D flag on the Preprocessor definitions line to get separate debug and release outputs. If you know you want this, you know how to set it up, so I'm not going to make this long answer even longer in order to explain it.
You'll be asked the same set of questions for the Release build. If you want to bother with separate debug and release builds, you'd make any changes here.
Having done all this, you still have to create the Makefile, and add a make.exe to your PATH. As with the debug vs. release question, going into that level of detail would push this answer off topic.
As ugly as this looks, it's still easier than creating custom build tools. Plus, you say you need to port to Unix eventually, so you're going to need that Makefile anyway.
Option 3: Cross-Platform Development
You say you want to port this program to Unix at some point, but that doesn't mean you must use GCC on Windows now. It is quite possible to write your program so that it builds under Visual C++ on Windows and GCC/Makefiles on *ix systems.
There are several tools that make this easier. One very popular option is CMake, which is available as an installation time option in newer versions of Visual Studio. There are many alternatives such as SCons and Bakefile.
Clang
You can use the Clang compiler with Visual Studio to target Android, iOS, and Windows.
If you are targeting Android, you can use the Clang/LLVM compiler that ships with the Android NDK and toolchain to build your project. Likewise, Visual Studio can use Clang running on a Mac to build projects targeting iOS. Support for Android and iOS is included in the “Mobile Development with C++” workload. For more information about targeting Android or iOS check out our posts tagged with the keywords “Android” and “iOS”.
If you are targeting Windows, you have a few options:
Use Clang/LLVM; “Clang for Windows” includes instructions to install Clang/LLVM as a platform toolset in Visual Studio.
Use Clang to target Windows with Clang/C2 (Clang frontend with Microsoft Code Generation).
GCC
If your project targets Linux or Android, you can consider using GCC. Visual Studio’s C++ Android development natively supports building your projects with the GCC that ships with the Android NDK, just like it does for Clang. You can also target Linux – either remotely or locally with the Windows Subsystem for Linux – with GCC.
Check out our post on Visual C++ for Linux Development for much more info about how to use Visual Studio to target Linux with GCC. If you are specifically interested in targeting WSL locally, check out Targeting WSL from Visual Studio.
Source: https://devblogs.microsoft.com/cppblog/use-any-c-compiler-with-visual-studio/
I'm from the future.
I keep (poking at) a C/C++ toolchain using Visual Code on Win/Lin/Mac and MinGW installed from Choclatey.
(This was done for my sanity - install GDB and GCC however you want)
I've run it with GCC and GDB with IntelliSense using MS's own weird JSON makefiles.
Someday, someone (you?) will write a Gradle or Python script to generate these; for now the examples online in the docs seem to work.
It seems to require three types of JSON thing;
a single IntelliSense configuration for the whole workspace
a Debugging Configuration entry for each binary you want to debug
these can invoke the build tasks
a Build Task per-artifact
I don't think that there's a "require" or "dependency" thingie-mah-bob; sorry

Compiling a Credential Provider with MinGW

I found some examples from Microsoft, but I'm not sure how to get started.
I have what appears to be a VS project and a file with Registry entries. There are no makefiles included and not really any instructions on how to build.
I am trying to use the G++ compiler with MinGW. The use case is simple http authentication. I have this working on Linux with my pam-http project.
How do I go about compiling a simple Credential Provider?
Are there any tutorials that give build scripts/makefiles?
I would very much prefer to use FOSS where possible, hence MinGW and g++, and I have little experience with compiling on Windows (I used VS at a job several years ago). Ultimately I'd like to link in cURL, but I can figure that out once I get something built.
Note:
I found these, but I'm looking for build scripts using g++:
Building a custom credential provider for Windows 7
http://msdn.microsoft.com/en-us/magazine/cc163489.aspx
I share kberson's sentiments.
EDIT:
I found this on MinGW's website that says linking against MS VC created DLLs is possible.
I do not want to use Visual Studio. I'd prefer a command-line compile tool that isn't tied down to a specific build tool (like ANT or make).
To compile the example credential providers will be considerably easier if you use Visual Studio Express C++ (and then port to G++). You may also need to install the Windowns SDK
MSVC++ comes with command line building tools. So to build the custom credential provider for Windows 7 firstly extract the files into a directory. Then Install MSVC - use the shortcut on the start menu Visual Studio Command Prompt (2010) to open a command prompt and type
cd CredentialProviders\SampleCredentialProvider
msbuild
or to build the release configuration Win32
msbuild /p:Configuration=Release /p:Platform=Win32
Refer to MSBuild (Visual C++) Overview, MSBuild Command Line Reference and Building on the Command Line
These credential providers are built using COM - part of OLE2 - which is the Microsoft Component Object Model. It is possible to build interoperable components without using MSVC but more work. So to get started I'd develop using MSVC simply because all of the examples will work out of the box and then I'd port across to G++ as there will be issues and it is usually easier to start from a working system as it rules out problems in the COM bindings.
To understand COM it helps to read the Technical foundation of COM.

Why do 3ds Max plug-ins need to be built with a specific version of Visual C++?

The requirements listed in the 3ds Max SDK state that plug-ins for 3ds Max 2011 must be built with Visual C++ 9.0 (Visual Studio 2008).
If I create a DLL with a different version of Visual C++, won't the binary be identical? Is this simply a matter of choosing the right compiler settings?
What problems will I run into if I try to build a plug-in using Visual C++ 2010 (Visual Studio 2010)?
I don't know specifically for 3ds Max, but the usual reason is the C Runtime library. If the host application uses the DLL version of the CRT, then plugins will also need to use the same version.
Otherwise, imagine the case where your plugin creates some memory using malloc(), and passes it to the host application, which uses it and then calls free(). If your plugin and the host application are using different CRTs, the host's call to free() will fail or crash because it wasn't the host's CRT that malloc()ed that block of memory.
The binary won't be identical but the binary interfaces should be, which is what you need.
The reason you can't use VS2010 is because it is not yet production quality. They think you should wait for VS2010 SP1 at a minimum.
You think they are just being obstinate and stubborn, eh? Ruining all your fun. They have reasons. Good ones.
Because of bugs like this:
https://connect.microsoft.com/VisualStudio/feedback/details/565959
I use both visual studio 2008 and 2010 for plugin development.
Only difference I've seen is that the user need the vs c++ runtime version for 2010\2008.
But there might be pitfalls - but I have not encountered any problems with it yet.
C++ doesn't have a standardised binary interface. The compiler "mangles" each symbol name (i.e. each function or static data) to include information about namespaces and signature, so that namespaces, argument overloading, &c. can work. Each compiler is free to decide how to do this. Different MSVS compiler versions do name mangling in different ways, so in general you can't link a C++ library compiled with 2005 and a library compiled with 2008 together: this includes loading a 2008 DLL from a 2005 executable (for example). You can do this if the interface between the libraries is C, as long as the relevant functions are marked with extern "C" to prevent name mangling. And in practice the differences are not always that great: for example, I never had trouble using VS2005 SP1 to compile a library for 3ds Max 9, which supposedly requires VS2005 with no service pack.
Microsoft is trying to fix this incompatibility, so in VS2010 they introduced an option, so VS2010 can produce binaries compatible with VS2005 programs or VS2008 programs (maybe some earlier versions too, I forget). If you have to create a plugin to work with multiple 3ds Max versions, and you don't get caught out by any VS2010 bugs, this is probably a good option. Also, the Intel C++ compiler has a mode where it produces binaries that are compatible with an MSVS version of your choice, which might be a better option for you if it's for hobby use or you can afford the slightly expensive price tag. (They achieve this by copying the way MSVS does name mangling.)

Using MinGW/GCC built DLL in a Visual Studio 2010 C++/CLI project

I have a communication library built on top of Qt and Google Protocol Buffers. It's currently being built with MinGW/GCC on Windows. My goal is to use the same library in C# on .NET, with the help of a thin wrapper on top using C++/CLI (bridging the unmanaged code with managed code).
I tried using the MinGW produced DLL directly in my C++/CLI project, but I keep getting linker errors (cant remember the error codes right now, but something about missing tokens and functions/signatures).
First question is: Should I be able to use the MinGW-produced DLL with the Visual Studio compiler/linker? Or do I need to compile the library again, using only VS compiler for all projects?
If I should be able to use the MinGW-produced DLL directly, how do I reference it in Visual Studio 2010? In project settings it seems to look for *.lib files, but I can't find any .lib files in the output of MinGW/GCC. It does produce *.a files, but it seems like Visual Studio don't handle this kind of file..
I should also mention that both Qt and protobuf are also compiled with MinGW. But I can of course recompile all the parts in VS 2010 if necessary.. Would have been nice to save the recompile time though, since our buildserver already has a working setup using MinGW.
The easiest way to use it would be by recompiling it with Visual Studio. This is when I am assuming C++ types and classes used in the interface you intend to use.
In case you have a C interface to this library you could dynamically load the library via LoadLibrary and use GetProcAddress to access those functions.
However it depends completly on the way how you intend to use the library.

Resources