There is a protobuf that my project needs. The protobuf itself is in another repo altogether owned by a completely different team. However they do publish their artifacts on my company's internal Artifactory.
How can I use that (non Bazel) protobuf in my Bazel project?
This is the direction I was trying to go before I hit a deadend.
WORKSPACE file:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file")
http_archive(
name = "rules_proto",
sha256 = "80d3a4ec17354cccc898bfe32118edd934f851b03029d63ef3fc7c8663a7415c",
strip_prefix = "rules_proto-5.3.0-21.5",
urls = [
"https://github.com/bazelbuild/rules_proto/archive/refs/tags/5.3.0-21.5.tar.gz",
],
)
load("#rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()
http_file(
name = "some-service-protobuf",
url = "https://artifactory.xyz.com/artifactory/x/y/some-service-protobuf.dbgrel.zip",
)
BUILD file:
load("#rules_proto//proto:defs.bzl", "proto_library")
proto_library(
name = "SomeService.proto",
)
The resources I'm trying to use:
https://bazel.build/reference/be/java#java_proto_library
https://github.com/bazelbuild/rules_proto
Is fetching proto files from remote URL possible? How show I do it after copying those files in my repo.
You might try using http_archive instead:
https://bazel.build/rules/lib/repo/http#http_archive
And then use the build_file or build_file_content attributes to add the build file with the proto_library definition to the external workspace created from the zip file:
http_archive(
name = "some-service-protobuf",
url = "https://artifactory.xyz.com/artifactory/x/y/some-service-protobuf.dbgrel.zip",
build_file = "BUILD.some_service",
)
BUILD.some_service:
load("#rules_proto//proto:defs.bzl", "proto_library")
proto_library(
name = "some_service_proto",
srcs = ["SomeService.proto"],
)
Then in another build file you can do something like:
java_proto_library(
name = "some_service_java_protobuf",
deps = ["#some-service-protobuf//:some_service_proto"],
)
Related
I am trying to consume a Linux based binary of Webrtc (https://webrtc.org/) in a Conan package with another project and finding when the Visual Studio generator runs it tries to include Webrtc as webrtc.lib instead of libwebrtc.a.
Here's the conanfile.py recipe I am using to create the local Webrtc package...
class WebrtcConan(ConanFile):
name = "webrtc"
version = "0.1"
license = "<Put the package license here>"
author = "<Put your name here> <And your email here>"
url = "<Package recipe repository url here, for issues about the package>"
description = "<Description of Webrtc here>"
topics = ("<Put some tag here>", "<here>", "<and here>")
settings = {"os", "compiler", "build_type", "arch"}
options = {"shared": [True, False], "fPIC": [True, False]}
default_options = {"shared": False, "fPIC": True}
def package(self):
self.copy("*")
def package_info(self):
self.cpp_info.libs = self.collect_libs()
I'm exporting the package to the local cache like this...
conan export-pkg -f . webrtc/local
Then running the Visual Studio generator like this in the consuming project...
conan install . -g visual_studio
Reviewing the generated conanbuildinfo.props shows Webrtc included as webrtc.lib instead of libwebrtc.a.
I CAN get the correct library inclusion with the following changes to package and package_info...
class WebrtcConan(ConanFile):
name = "webrtc"
version = "0.1"
license = "<Put the package license here>"
author = "<Put your name here> <And your email here>"
url = "<Package recipe repository url here, for issues about the package>"
description = "<Description of Webrtc here>"
topics = ("<Put some tag here>", "<here>", "<and here>")
settings = {"os", "compiler", "build_type", "arch"}
options = {"shared": [True, False], "fPIC": [True, False]}
default_options = {"shared": False, "fPIC": True}
generators = "visual_studio"
def package(self):
self.copy("*.a", dst="lib", keep_path=False) <<< specific destination
def package_info(self):
self.cpp_info.libs = ["libwebrtc.a"] <<< specific library inclusion
Ideally I can use this default recipe (the first conanfile.py example) as I'm not the one responsible for creating the Webrtc Conan package ultimately. I am working with another team/developer responsible for this and I need to support several different platforms past Linux. Is there a setting I'm missing somewhere to control the library inclusion? Why does the Conan generator assume the library name is webrtc.lib for consumer inclusion when it is included within the package as libwebrtc.a?
I am using a package graphviz as part of a service and to use it I begin the bazel WORKSPACE file like this
new_local_repository(
name = "graphviz",
path = "/usr/local/Cellar/graphviz/2.49.1",
build_file_content = """
package(default_visibility = ["//visibility:public"])
cc_library(
name = "headers",
srcs = glob(["**/*.dylib"]),
hdrs = glob(["**/*.h"])
)
"""
)
...
The problem with it is its depends upon graphviz being downloaded, pre-installed and present in the path /usr/local/Cellar/graphviz/2.49.1. Is there a way to make it part of the bazel build process such that if its not present it will be fetched and put in the right place?
You can use http_archive to download one of graphviz's release archives:
https://docs.bazel.build/versions/main/repo/http.html#http_archive
From https://graphviz.org/download/source/ the 2.49.1 release is available at https://gitlab.com/api/v4/projects/4207231/packages/generic/graphviz-releases/2.49.1/graphviz-2.49.1.tar.gz
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "graphviz",
url = "https://gitlab.com/api/v4/projects/4207231/packages/generic/graphviz-releases/2.49.1/graphviz-2.49.1.tar.gz",
strip_prefix = "graphviz-2.49.1",
sha256 = "ba1aa7a209025cb3fc5aca1f2c0114e18ea3ad29c481d75e4d445ad44e0fb0f7",
build_file_content = """
package(default_visibility = ["//visibility:public"])
cc_library(
name = "headers",
srcs = glob(["**/*.dylib"]),
hdrs = glob(["**/*.h"])
)
""",
)
To answer the "if its not present" part of the question, I'm not aware of a straight forward way to accomplish automatically switching between something that's locally installed and downloading something. http_archive will always download the archive, and new_local_repository will always use something local.
There's the --override_repository flag, which replaces a repository with a local one, e.g. --override_repository=graphviz=/usr/local/Cellar/graphviz/2.49.1 would effectively replace the http_archive with a local_repository pointing to that path. However, bazel would then expect there to be a WORKSPACE file and BUILD file already at that location (i.e., there's no way to specify build_file_content)
You could specify both repository rules in the WORKSPACE file, and then use some indirection, a Starlark flag, and a select() to switch between the repositories using a command-line flag. It's a little involved though, and also not automatic. Something like this:
WORKSPACE:
http_archive(
name = "graphviz-download",
...,
)
new_local_repository(
name = "graphviz-installed",
...,
)
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "bazel_skylib",
urls = [
"https://github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz",
"https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz",
],
sha256 = "c6966ec828da198c5d9adbaa94c05e3a1c7f21bd012a0b29ba8ddbccb2c93b0d",
)
load("#bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")
bazel_skylib_workspace()
BUILD (e.g. in //third_party/graphviz):
load("#bazel_skylib//rules:common_settings.bzl", "bool_flag")
bool_flag(
name = "use-installed-graphviz",
build_setting_default = False,
)
config_setting(
name = "installed",
flag_values = {
":use-installed-graphviz": "True",
}
)
alias(
name = "headers",
actual = select({
":installed": "#graphviz-installed//:headers",
"//conditions:default": "#graphviz-download//:headers",
})
)
Then your code depends on //third_party/graphviz:headers, by default the alias will point to the downloaded version, and the flag --//third_party/graphviz:use-installed-graphviz will switch it to the installed version:
$ bazel cquery --output build //third_party/graphviz:headers
alias(
name = "headers",
actual = "#graphviz-download//:headers",
)
$ bazel cquery --output build //third_party/graphviz:headers --//third_party/graphviz:use-installed-graphviz
alias(
name = "headers",
actual = "#graphviz-installed//:headers",
)
Another option is to write (or find) a custom repository rule that combines the functionality of http_archive and local_repository, but that would probably be a fair bit of work.
Generally I think most people just use an http_archive and download the dependencies, and if there are specific needs for being offline or caching, there's --distdir for using already-downloaded artifacts for remote repository rules: https://docs.bazel.build/versions/main/guide.html#distribution-files-directories
Edit: An example of using rules_foreign_cc with graphviz:
WORKSPACE:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "rules_foreign_cc",
sha256 = "69023642d5781c68911beda769f91fcbc8ca48711db935a75da7f6536b65047f",
strip_prefix = "rules_foreign_cc-0.6.0",
url = "https://github.com/bazelbuild/rules_foreign_cc/archive/0.6.0.tar.gz",
)
load("#rules_foreign_cc//foreign_cc:repositories.bzl", "rules_foreign_cc_dependencies")
# This sets up some common toolchains for building targets. For more details, please see
# https://bazelbuild.github.io/rules_foreign_cc/0.6.0/flatten.html#rules_foreign_cc_dependencies
rules_foreign_cc_dependencies()
http_archive(
name = "graphviz",
url = "https://gitlab.com/api/v4/projects/4207231/packages/generic/graphviz-releases/2.49.1/graphviz-2.49.1.tar.gz",
strip_prefix = "graphviz-2.49.1",
sha256 = "ba1aa7a209025cb3fc5aca1f2c0114e18ea3ad29c481d75e4d445ad44e0fb0f7",
build_file_content = """\
filegroup(
name = "all_srcs",
srcs = glob(["**"]),
visibility = ["//visibility:public"],
)
""",
)
BUILD:
load("#rules_foreign_cc//foreign_cc:defs.bzl", "configure_make")
# see https://bazelbuild.github.io/rules_foreign_cc/0.6.0/configure_make.html
configure_make(
name = "graphviz",
lib_source = "#graphviz//:all_srcs",
out_shared_libs = ["libcgraph.so"], # or other graphviz libs
)
cc_binary(
name = "foo",
srcs = ["foo.c"],
deps = [":graphviz"],
)
foo.c:
#include "graphviz/cgraph.h"
int main() {
Agraph_t *g;
g = agopen("G", Agdirected, NULL);
agclose(g);
return 0;
}
Usage:
$ bazel build foo
INFO: Analyzed target //:foo (0 packages loaded, 2 targets configured).
INFO: Found 1 target...
Target //:foo up-to-date:
bazel-bin/foo
INFO: Elapsed time: 0.229s, Critical Path: 0.06s
INFO: 7 processes: 5 internal, 2 linux-sandbox.
INFO: Build completed successfully, 7 total actions
I am currently using bazel for my GoLang app.
container_image(
name = "my-golang-app",
base = "#ubuntu_base//image",
cmd = ["/bin/my-golang-app"],
directory = "/bin/",
files = [":my-golang-app"],
tags = [
"manual",
VERSION,
],
visibility = ["//visibility:public"],
)
Except my-golang-app folder I need to copy, while building this image, another folder named my-new-folder and everything in it.
How does one do that with bazel? I can't seem to find the solution in the bazel docs.
dockerfile {
run 'mkdir -p /usr/local/bin'
// add the lines below
add {
from 'docker/my-new-folder/'
into '/'
}
Use pkg_tar like the container_image docs reference. Something like this:
pkg_tar(
name = "my-files",
srcs = glob(["my-new-folder/**"]),
strip_prefix = ".",
)
container_image(
name = "my-golang-app",
files = [":my-golang-app"],
<everything else you already have>,
tars = [":my-files"],
)
pkg_tar has various options to control where your files end up, if you want something beyond just taking an entire directory. For more complicated arrangements, I find multiple pkg_tar rules for various directories linked together via deps a helpful pattern.
I'm new to bazel. I am trying out migrating some parts of my company's vast build to bazel. We have some protos zipped up in nexus. I need to grab several archives and compile all the contained proto files together. I have tried all sorts and just can't get it to work.
I'm using an http_archive with a filegroup to get the files.
http_archive(
name = "protos_1",
url = "..."
build_file_content = """
filegroup(
name = "files",
srcs = glob(["**/*.proto"]),
visibility = ["//visibility:public"]
)
""",
)
Now I figured I would just do something like:
proto_library(
name = "combined_protos",
srcs = [
"#protos_1//:files",
"#protos_2//:files",
],
)
but no dice. I've tried using the filegroups as deps and data and everything else I can think of. Any tips?
thanks.
What is the best way to build C++ code that uses the dlib library using Bazel? I.e., what would the BUILD rules look like?
I tried following the answer for OpenCV as follows, but had no luck:
cc_library(
name = "dlib",
srcs = glob(["build/dlib/*.so*"]),
hdrs = glob(["dlib/*.h"]),
includes = ["include"],
visibility = ["//visibility:public"],
linkstatic = 1,
)
I think I figured it out. Assuming dlib was unzipped to /opt/dlib-19.2 and built in /opt/dlib-19.2/build.
In your WORKSPACE file:
new_local_repository(
name = "dlib",
path = "/opt/dlib-19.2",
build_file = "dlib.BUILD",
)
In dlib.BUILD:
cc_library(
name = "dlib",
srcs = glob(["build/dlib/*.so*"]),
hdrs = glob(["dlib/**/*.h"]),
includes = ["."],
visibility = ["//visibility:public"],
linkstatic = 1,
)