Bazel environment variables in build rules - windows

I want to refer to a DirectX SDK in the BUILD file. The problem is that (as far as I understand) Bazel supports passing environment variables only through --action_env=DXSDK_DIR argument for Bazel and it is meant to be used in actions, which must be defined in a plugin (.bzl file).
Is there any easier way to refer to the environment variable by using it as Make variable (includes = [ "$(DXSDK_DIR)/Include" ]) or do I need to write a plugin?

In principle you need a cc_library rule whose hdrs attribute globs the DirectX headers. For that you need to pretend that the DX SDK is part of your source tree. Bazel offers "repository rules" for that purpose.
1. Create a repository rule for the DirectX SDK
Depending on whether the SDK's location is known or needs to be discovered, you have two options.
a. Fixed SDK location
You can use this approach if you don't need to read any environment variables, run any binaries, or query the registry to find where the SDK is. This is the case if everyone who builds your rules will install the SDK to the same location.
Just add a new_local_repository rule to your WORKSPACE file, point the rule's path at the SDK's directory and write a simple build_file_content for it.
Example:
new_local_repository(
name = "directx_sdk",
path = "c:/program files/directx/sdk/includes",
build_file_contents = """
cc_library(
name = "sdk",
hdrs = glob(["**/*.h"]),
visibility = ["//visibility:public"],
)
""")
This rule creates the #directx_sdk repository with one rule in its root package, #directx_sdk//:sdk.
b. SDK discovery
You need to follow this approach if you need to read environment variables, run binaries, or query the registry to find where the SDK is.
Instead of using a new_local_repository rule, you need to implement your own. More info and examples are here.
Key points:
if your repository rule needs to read environment variables, add them to the list repository_rule(environ), e.g. repository_rule(..., environ = ["DXSDK_DIR"])
if you need to run some binaries that tell you where the SDK is, use repository_ctx.execute. You can use repository_ctx.which to find binaries on the PATH.
if you need to do registry queries, use repository_ctx.execute with reg.exe /query <args>
2. Depend on the SDK's cc_library
In your project, just depend on the SDK's library as if it was an ordinary cc_library:
cc_library(
name = "render",
...
deps = [
...
"#directx_sdk//:sdk",
],
)

Related

CMake: Use variables from existing Makefile of 3rdparty library

I'm facing the following scenario:
Existing project which uses cmake
External 3rdparty library which only comes with Makefiles
The difference of my situation compared to existing questions is that I don't need to have cmake to build the 3rdparty library via the Makefile. Instead, the 3rdparty library provides a library.mk Makefile which has variables like LIB_SRCS and LIB_INCS containing all source and header files required to compile the library.
My idea is to include the library.mk into the project's CMakeLists.txt and then adding those $(LIB_SRCS) and $(LIB_INCS) to target_sources().
My question: How can I include library.mk into the existing CMakeLists.txt to get access to the $(LIB_SRCS) and $(LIB_INCS) for adding them to target_sources()? I'm looking for something like this:
include("/path/to/library.mk") # Somehow include the library's `library.mk` to expose variables to cmake.
add_executable(my_app)
target_sources(
my_app
PRIVATE
main.c
$(LIB_SRCS) # Add 3rd-party library source files
$(LIB_INCS) # Add 3rd-party library header files
)
Using include() does not work as the library.mk is not a CMake list/file.
Since you can't be sure that your target system will even have Make on it, the only option is to parse the strings out of the .mk file, which might be easy if the variables are set directly as a list of filenames, or really hard if they are set with expansions of other variables, conditionals, etc. Do this with FILE(STRINGS) cmake doc.
Your plan will only work if the Makefiles are trivial, and do not set important compiler flags, define preprocessor variables, modify the include directory, etc. And if they really are trivial, skip the parsing, and just do something like aux_source_directory(<dir> <variable>) to collect all the sources from the library directory.
You might also consider building and maintaining a CMakeLists.txt for this third-party library. Do the conversion once, and store it as a branch off of the "vendor" main branch in your version control system. Whenever you update, update the vendor branch from upstream, and merge or rebase your modifications. Or just store it in your existing project, referring to the source directory of the 3rd-party stuff.

Include <headers.h> installed in non standard location

I'm currently working with a third party library, which has headers declared using angular brackets, like a standard library :
#include <header.h>
However, these headers are installed in a non standard place, something like /opt/company/software/version/part_software/include
With a more traditional builder like MAKE, I can just use CXXFLAGS to indicate to g++ to look in this folder too for libraries, which finally comes down to pass a -I/opt/company/software/version/part_software/include option to g++.
When trying to do the same thing in bazel, using copts = [ "-I/opt/company/software/version/part_software/include" ], I get a "path outside of the execution root" error.
It's my understanding that bazel don't like the place where the lib is installed because the build needs to be reproducible, and including a library located outside the execution root violate this constraint.
A ugly hack I've come with is to create symbolic link of the headers in /usr/local/include, and use copts = [ "-I/usr/local/include" ] in the bazel build. However, I find this approach very hacky, and I'd like to find a more bazely approach to the problem.
Note : I can't install the program during the bazel build, as it uses a closed installer on which I have no control over. This installer can't be run in the bazel's sandboxed environment, as it needs to write on certain paths not accessible within the environment.
So, it turns out that the bazelesque way of including a third part library is simply to create package encapsulating the library.
Thanks to this useful discussion, I've managed to create a package with my third party library.
First we need a BUILD file, here named package_name.BUILD
package(
default_visibility = ["//visibility:public"]
)
cc_library(
name = "third_party_lib_name", #name to reference the third party library in other BUILD files
srcs = [
"external/soft/lib/some_lib.so", #.so files to include in the lib
"software/lib/os/arch/lib_some_plugin.so",
],
hdrs = glob([ # the glob takes all the headers needed
"software/include/**/*.h",
"software/include/**/*.hpp",
]),
includes = ["software/include/"], # Specify which files are included when we use the library
)
Now we need to reference the lib a a submodule in the WORKSPACE file :
new_local_repository(
name = "package_name",
path = "opt/company/software/version",
# build_file: path to the BUILD file, here in the same directory that the main WORKSPACE one
build_file = __workspace_dir__ + "/package_name.BUILD",
)
Now, instead of using copt to references the needed headers, I'm just adding a line to the deps of the cc_rule when needed, e.g :
cc_library(
name="some_internal_lib",
srcs = ["some_internal_lib.cc"],
deps = [
"#package_name//:third_party_lib_name", #referencing the third party lib
],
)

For Gazelle, how can I make it use go_repository defined in the WORKSPACE?

In my WORKSPACE, I have defined the go_repostitory for importpath golang.org/x/net.
go_repository(
name = "org_golang_x_net",
commit = "5ccada7d0a7ba9aeb5d3aca8d3501b4c2a509fec",
importpath = "golang.org/x/net",
)
But when I run gazelle, the dependency added to BUILD.Bazel is not #org_golang_x_net//context:go_default_library. Instead, it is the following: //golang.org/x/net/context:go_default_library
Why does gazelle ignore the defined go_repository? Is there a way for it to consider deps defined in WORKSPACE?
I'm guessing from your previous question this is because you're using an empty prefix.
Gazelle resolves Go imports to Bazel labels in several steps:
If there is a go_library in your workspace with a matching importpath, Gazelle will use the name of that library.
If the import path is below the top-level prefix for your repository, the import will be considered "local", so you'll get a label like //golang.org/x/net/content:go_default_library.
Since you're using an empty prefix at the top level, this catch will catch every unknown import.
This should be improved. Gazelle's resolver should be aware of prefixes defined in different directories (currently, those are only used to determine the importpath attribute on generated rules). I've filed bazelbuild/bazel-gazelle#101 to fix that.
If the import is not considered local, Gazelle will generate a label for an external repository or for the vendor directory, depending on the external mode.
For now, you may want to add a directive to your top-level build file like:
# gazelle:prefix __do_not_match__
This will basically disable the second case, since no import will start with that. You'll still have an empty prefix in your src directory (set with # gazelle:prefix in src/BUILD.bazel), so your libraries will still have the correct importpath directives. You might want to set that to more specific subdirectories though.

Qt Installer project: how to generate package.xml and config.xml

I try to make my Qt Installer project more comfortable.
I need to centralize information about my application and components (config.xml and package.xml) in one file. I don't want to jump on different files with same name and search for changeable elements between xml tags.
My first thougt is doing it right in *.pro file of installer project. I place sections of variables in header of installer project file. But where I need to place the code for xml generating?
What is the better (native / comfortable / crossplatform) way to do this?
The answer is simple here: you cannot generate XML files for Qt Installer: you write them manually, as explained in the documentation.
This section describes the following tasks that you must accomplish to create the installer:
Create a package directory that will contain all the configuration files and installable packages.
Create a configuration file that contains information about how to build the installer binaries and online repositories.
Create a package information file that contains information about the installable components.
Create installer content and copy it to the package directory.
Use the binarycreator tool to create the installer.
However, if you look closer at the examples, you can still generate the installer in the *.pro file. Let's pick an example randomly, System Info:
TEMPLATE = aux
INSTALLER = installer
INPUT = $$PWD/config/config.xml $$PWD/packages
example.input = INPUT
example.output = $$INSTALLER
example.commands = ../../bin/binarycreator -c $$PWD/config/config.xml -p $$PWD/packages ${QMAKE_FILE_OUT}
example.CONFIG += target_predeps no_link combine
QMAKE_EXTRA_COMPILERS += example
OTHER_FILES = README
If you want to apply this to your project, I think you'll have to modify the ../../bin/binarycreator line and make it system aware, by changing your PATH. It might be possible to call an external script and parse XML files, and make the substitutions you would like to do, but you'd move the complexity to another place.
Instead of maintaining plain good old XML files, you would be creating something between XSLT and XML. Maybe you could just write XSLT (or XSL or XQUERY) and generate XML but I don't know anyone who is using it anymore. Last time I used it was when I was learning Computer Science a long time ago :)
This is possible using the QMAKE_SUBSTITUTES feature which will substitute qmake variables into the given input files and put the output in the build folder.
This runs at qmake time rather than at build time. If this is suitable then you just need to add a target to copy the generated files from the build dir to your source dir.
If you need it to run at build time then you can create a .pri file containing QMAKE_SUBSTITUTES and a target in the main .pro file that will run qmake on this file during the build process.
Main .pro file:
create_xml.commands += $(QMAKE) $$shell_quote($$PWD/config/generate_xml.pri) $$escape_expand(\n\t)
create_xml.commands += $(COPY) $$shell_quote($${OUT_PWD}/config.xml) $$shell_quote($$PWD/config) $$escape_expand(\n\t)
create_xml.commands += $(COPY) $$shell_quote($${OUT_PWD}/package.xml) $$shell_quote($$PWD/packages/my.app.id/meta) $$escape_expand(\n\t)
create_xml.depends = $$PWD/version.pri
QMAKE_EXTRA_TARGETS += create_xml
generate_xml.pri:
TEMPLATE = aux
message("Generating $$OUT_PWD/config.xml and $$OUT_PWD/package.xml")
# Get the version number
include(version.pri)
APP_VERSION = $$VERSION
QMAKE_SUBSTITUTES += package.xml.in config.xml.in
config.xml.in: Note that you need to escape the quotes.
<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<Installer>
<Name>MyApp</Name>
<Version>$$APP_VERSION</Version>
...

Linking with a Windows library outside the build folder

Is there a way to link with a library that's not in the current package path.
This link suggests placing everything under the local directory. Our packages are installed in some repository elsewhere. I just want to specify the libpath to it on windows.
authors = ["Me"]
links = "CDbax"
[target.x86_64-pc-windows-gnu.CDbax]
rustc-link-lib = ["CDbax"]
rustc-link-search = ["Z:/Somepath//CPP/CDbax/x64/Debug/"]
root = "Z:/Somepath//CPP/CDbax/x64/Debug/"
But trying cargo build -v gives me
package `hello v0.1.0 (file:///H:/Users/Mushfaque.Cradle/Documents/Rustc/hello)` specifies that it links to `CDbax` but does not have a custom build script
From the cargo build script support guide, it seems to suggest that this should work. But I can see that it hasn't added the path. Moving the lib into the local bin\x68_64-pc-windows-gnu\ path works however.
Update
Thanks to the answer below, I thought I'd update this to give the final results of what worked on my machine so others find it useful.
In the Cargo.toml add
links = "CDbax"
build = "build.rs"
Even though there is no build.rs file, it seems to require it (?) otherwise complains with
package `xxx v0.1.0` specifies that it links to `CDbax` but does not have a custom build script
Followed by Vaelden answer's create a 'config' file in .cargo
If this is a sub crate, you don't need to put the links= tag in the parent crate, even though it's a dll; even with a 'cargo run'. I assume it adds the dll path to the execution environment
I think the issue is that you are mistaking the manifest of your project with the cargo
configuration.
The manifest is the Cargo.toml file at the root of your project. It describes your project itself.
The cargo configuration describes particular settings for cargo, and allow for example to override dependencies, or in your case override build scripts. The cargo configuration files have a hierarchical structure:
Cargo allows to have local configuration for a particular project or
global configuration (like git). Cargo also extends this ability to a
hierarchical strategy. If, for example, cargo were invoked in
/home/foo/bar/baz, then the following configuration files would be
probed for:
/home/foo/bar/baz/.cargo/config
/home/foo/bar/.cargo/config
/home/foo/.cargo/config
/home/.cargo/config
/.cargo/config
With this structure you can specify local configuration per-project,
and even possibly check it into version control. You can also specify
personal default with a configuration file in your home directory.
So if you move the relevant part:
[target.x86_64-pc-windows-gnu.CDbax]
rustc-link-lib = ["CDbax"]
rustc-link-search = ["Z:/Somepath//CPP/CDbax/x64/Debug/"]
root = "Z:/Somepath//CPP/CDbax/x64/Debug/"
to any correct location for a cargo configuration file, it should work.

Resources