I am building my C++ project using Bazel on macOS, and I added the gRPC as one of the deps
FYI, I successfully built with same settings on Linux, however, it fails for me on macOS.
WORKSPACE
git_repository(
name = "com_github_grpc_grpc",
remote = "https://github.com/grpc/grpc",
tag = "v1.20.1"
)
load("#com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
grpc_deps()
and the build file for my module
a.BUILD
cc_library(
name = "s12",
srcs = glob(
["**/*.cc"],
),
hdrs = glob(
["**/*.h",],
),
deps = [
"#com_github_grpc_grpc//:grpc++",
"#com_google_protobuf//:protobuf",
],
)
and in linking stage I got following error:
[libprotobuf ERROR external/com_github_sc_protobuf/src/google/protobuf/descriptor_database.cc:58] File already exists in database: google/protobuf/descriptor.proto
[libprotobuf FATAL external/com_github_sc_protobuf/src/google/protobuf/descriptor.cc:1370] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
external/bazel_tools/tools/test/test-setup.sh: line 310: 63268 Abort trap: 6 "${TEST_PATH}" "$#" 2>&1
Apparently, some file is included multiple times, however, I couldn't figure out why.
Related
I'm trying to build a hello_world cc_grpc_library using bazel 6.0.0 on Ubuntu 22.04, but I'm unable to do so.
Source tree:
WORKSPACE
MODULE.bazel
BUILD
helloworld.proto
.bazelrc
.bazelversion
My WORKSPACE file is empty. WORKSPACE.bzlmod does not exist.
MODULE.bazel contains:
module(name = "helloworld", version = "1.0")
bazel_dep(name = "grpc", version = "1.47.0", repo_name = "com_github_grpc_grpc")
BUILD contains:
load("#rules_proto//proto:defs.bzl", "proto_library")
load("#com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
load("#com_github_grpc_grpc//bazel:grpc_extra_deps.bzl", "grpc_extra_deps")
load("#com_github_grpc_grpc//bazel:cc_grpc_library.bzl", "cc_grpc_library")
grpc_deps()
grpc_extra_deps()
proto_library(
name = "helloworld_proto",
srcs = ["helloworld.proto"],
)
cc_proto_library(
name = "helloworld_cc_proto",
deps = [":helloworld_proto"],
)
cc_grpc_library(
name = "helloworld_cc_grpc",
srcs = [":helloworld_proto"],
grpc_only = True,
deps = [":helloworld_cc_proto"],
)
helloworld.proto is a copy of gRPC's helloworld.proto
When I type bazel build :helloworld_cc_grpc I get the following error and don't know how to solve it:
...external/grpc~1.47.0/bazel/grpc_deps.bzl", line 23, column 11, in grpc_bind_deps
native.bind(
Error: no native function or rule 'bind'
Available attributes: aar_import, action_listener, alias, android_binary, android_device, android_device_script_fixture, android_host_service_fixture, android_instrumentation_test, android_library, android_local_test, android_sdk, android_tools_defaults_jar, apple_cc_toolchain, available_xcodes, cc_binary, cc_host_toolchain_alias, cc_import, cc_libc_top_alias, cc_library, cc_proto_library, cc_shared_library, cc_shared_library_permissions, cc_test, cc_toolchain, cc_toolchain_alias, cc_toolchain_suite, config_feature_flag, config_setting, constraint_setting, constraint_value, environment, existing_rule, existing_rules, exports_files, extra_action, fdo_prefetch_hints, fdo_profile, filegroup, genquery, genrule, glob, j2objc_library, java_binary, java_import, java_library, java_lite_proto_library, java_package_configuration, java_plugin, java_plugins_flag_alias, java_proto_library, java_runtime, java_test, java_toolchain, label_flag, label_setting, objc_import, objc_library, package, package_group, package_name, platform, propeller_optimize, proto_lang_toolchain, proto_library, py_binary, py_library, py_runtime, py_test, repository_name, sh_binary, sh_library, sh_test, subpackages, test_suite, toolchain, toolchain_type, xcode_config, xcode_config_alias, xcode_version
bazel version output:
Bazelisk version: v1.15.0
Build label: 6.0.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Dec 19 15:52:35 2022 (1671465155)
Build timestamp: 1671465155
Build timestamp as int: 1671465155
I've also found this gRPC github issue but I'm not sure what to do with it.
How to build gRPC's hello world example using bzlmod to manage external dependencies?
I've tried building it with bazel's cc_grpc_library but that wraps actual dependency (gRPC) into yet another layer as it is visible from the comment on the provided link. Either way it wasn't building but I don't recall actual error.
gRPC doesn't yet support Bazel module so it can break but in your case, it looks weird to call grpc_deps() and grpc_extra_deps() functions in BUILD file because those are expected to be called in WORKSPACE file. If it turns out that module doesn't work yet, you may want to use it without module. Please take a look at this example.
I'm using bazel 4.1.0 to build to project.
Current set up:
go.mod file
...
github.com/mwitkow/go-proto-validators v0.3.2 // indirect
...
WORKSPACE file
load("#bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")
load("//:repositories.bzl", "go_repositories")
go_repositories()
in repositories.blz file
...
go_repository(
name = "com_github_mwitkow_go_proto_validators",
importpath = "github.com/mwitkow/go-proto-validators",
sum = "h1:qRlmpTzm2pstMKKzTdvwPCF5QfBNURSlAgN/R+qbKos=",
version = "v0.3.2",
)
....
in students/api.v1/api.proto file
import "google/api/annotations.proto";
import "github.com/mwitkow/go-proto-validators/validator.proto";
after running bazel run //:gazelle -- update-repos -from_file=go.mod -to_macro=repositories.bzl%go_repositories -prune=true and bazel run //:gazelle
it generate students/api.v1/BUILD.bazel file
proto_library(
name = "api_proto",
srcs = ["api.proto"],
visibility = ["//visibility:public"],
deps = [
"//github.com/mwitkow/go-proto-validators:validators_proto",
"#go_googleapis//google/api:annotations_proto",
],
)
go_proto_library(
name = "api_go_proto",
compilers = ["#io_bazel_rules_go//proto:go_grpc"],
importpath = "students/generated/api",
proto = ":api_proto",
visibility = ["//visibility:public"],
deps = [
"//github.com/mwitkow/go-proto-validators:validator_proto",
"#go_googleapis//google/api:annotations_go_proto",
],
)
When I run bazel build //..., it produces an error
ERROR: /my-app/students/api.v1/BUILD.bazel:15:17: no such package 'github.com/mwitkow/go-proto-validators': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
- /my-app/github.com/mwitkow/go-proto-validators and referenced by '/students/api.v1:api_go_proto'
ERROR: Analysis of target '//students/api.v1:api' failed; build aborted: Analysis failed
I would like to make the build work without error. Thank you very much for your help.
I think in BUILD.bazel file, //github.com/mwitkow/go-proto-validators:validators_proto should be #github.com/mwitkow/go-proto-validators:validators_proto, like #go_googleapis//google/api:annotations_proto but it's just a vague idea.
I like to assemble a Linux image for RaspberryPi with ROS2 and Turtlebot3 stuff. Yocto environment is already set up and worked fine until end of last year. Image file was successfully built and started as expected at target system.
My new application needs further dependencies, therefore I pulled latest changes from origin repositories few days ago. Unfortunately, this seem to broke functionality of build system. Own changes not yet applied.
Build process fails at the last stage when rootfs should be created. It claims several rpm would be missing. It turned out they're existing at different location.
Below example is about glibc-localedata-pa-pk-2.31-r0.cortexa7t2hf_neon_vfpv4.rpm but after several tries I noticed that a tons of packages are affected.
Expected: build/BUILD-poky-foxy-dunfell/deploy/rpm/cortexa7t2hf_neon_vfpv4/glibc-localedata-pa-pk-2.31-r0.cortexa7t2hf_neon_vfpv4.rpm
Existing: build/BUILD-poky-foxy-dunfell/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/glibc-locale/2.31-r0/deploy-rpms/cortexa7t2hf_neon_vfpv4/glibc-localedata-pa-pk-2.31-r0.cortexa7t2hf_neon_vfpv4.rpm
Another afftected package:
Expected:
build/BUILD-poky-foxy-dunfell/deploy/rpm/cortexa7t2hf_neon_vfpv4/libatomic1-9.3.0-r0.cortexa7t2hf_neon_vfpv4.rpm
Existing: build/BUILD-poky-foxy-dunfell/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/gcc-runtime/9.3.0-r0/deploy-rpms/cortexa7t2hf_neon_vfpv4/libatomic1-9.3.0-r0.cortexa7t2hf_neon_vfpv4.rpm
Basically rpm are expected to be at build/BUILD-poky-foxy-dunfell/deploy/rpm/cortexa7t2hf_neon_vfpv4/ but they are at build/BUILD-poky-foxy-dunfell/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/XXX/YYY/deploy-rpms/cortexa7t2hf_neon_vfpv4/
Were there any changes in buildsystem or configuration since end of 2020 or any known issues found? Any help would be appreaciated, many thanks in advance!
$ bitbake ros-image-turtlebot3-core -k
Build Configuration:
BB_VERSION = "1.46.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "arm-poky-linux-gnueabi"
MACHINE = "raspberrypi3"
DISTRO = "poky"
DISTRO_VERSION = "3.1.11"
TUNE_FEATURES = "arm vfp cortexa7 neon vfpv4 thumb callconvention-hard"
TARGET_FPU = "hard"
DISTRO_NAME = "Poky (Yocto Project Reference Distro)"
ROS_DISTRO = "foxy"
ROS_VERSION = "2"
ROS_PYTHON_VERSION = "3"
meta
meta-poky
meta-yocto-bsp = "dunfell:6a7335170ed70263c4b2c55d10fe741c237c2357"
meta-oe
meta-python
meta-perl
meta-networking = "dunfell:c56d6309efc11e9f0f4a549be45a783c73a49ba0"
meta-raspberrypi = "dunfell:59c2d6f7a8b1239bd7b587b9180c2a55f9c695a2"
meta-ros-common
meta-ros2
meta-ros2-foxy
meta-ros-backports-hardknott
meta-ros-backports-gatesgarth = "ros2foxy:638a8b93e7b328fab9ccf2321bea039a93a57dde"
meta-communication
meta-system
meta-usercode = "ros2foxy:886b2fed39384970dc7476a15009765785b52512"
Initialising tasks: 100% |##########################################################################################| Time: 0:00:16
Sstate summary: Wanted 367 Found 365 Missed 2 Current 2770 (99% match, 99% complete)
NOTE: Executing Tasks
ERROR: ros-image-turtlebot3-core-1.0-r0 do_rootfs: Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_rootfs(d)
0003:
File: '/home/user/ros_target/poky/meta/classes/image.bbclass', lineno: 247, function: do_rootfs
0243: progress_reporter.next_stage()
0244:
0245: # generate rootfs
0246: d.setVarFlag('REPRODUCIBLE_TIMESTAMP_ROOTFS', 'export', '1')
*** 0247: create_rootfs(d, progress_reporter=progress_reporter, logcatcher=logcatcher)
0248:
0249: progress_reporter.finish()
0250:}
0251:do_rootfs[dirs] = "${TOPDIR}"
File: '/home/user/ros_target/poky/meta/lib/oe/rootfs.py', lineno: 975, function: create_rootfs
0971: env_bkp = os.environ.copy()
0972:
0973: img_type = d.getVar('IMAGE_PKGTYPE')
0974: if img_type == "rpm":
*** 0975: RpmRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
0976: elif img_type == "ipk":
0977: OpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
0978: elif img_type == "deb":
0979: DpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
File: '/home/user/ros_target/poky/meta/lib/oe/rootfs.py', lineno: 366, function: __init__
0362: self.manifest = RpmManifest(d, manifest_dir)
0363:
0364: self.pm = RpmPM(d,
0365: d.getVar('IMAGE_ROOTFS'),
*** 0366: self.d.getVar('TARGET_VENDOR')
0367: )
0368:
0369: self.inc_rpm_image_gen = self.d.getVar('INC_RPM_IMAGE_GEN')
0370: if self.inc_rpm_image_gen != "1":
File: '/home/user/ros_target/poky/meta/lib/oe/package_manager.py', lineno: 765, function: __init__
0761: self.primary_arch = self.d.getVar('MACHINE_ARCH')
0762:
0763: if needfeed:
0764: self.rpm_repo_dir = oe.path.join(self.d.getVar('WORKDIR'), rpm_repo_workdir)
*** 0765: create_packages_dir(self.d, oe.path.join(self.rpm_repo_dir, "rpm"), d.getVar("DEPLOY_DIR_RPM"), "package_write_rpm", filterbydependencies)
0766:
0767: self.saved_packaging_data = self.d.expand('${T}/saved_packaging_data/%s' % self.task_name)
0768: if not os.path.exists(self.d.expand('${T}/saved_packaging_data')):
0769: bb.utils.mkdirhier(self.d.expand('${T}/saved_packaging_data'))
File: '/home/user/ros_target/poky/meta/lib/oe/package_manager.py', lineno: 733, function: create_packages_dir
0729: if destdir not in seendirs:
0730: bb.utils.mkdirhier(destdir)
0731: seendirs.add(destdir)
0732: try:
*** 0733: os.link(l, dest)
0734: except OSError as err:
0735: if err.errno == errno.EXDEV:
0736: bb.utils.copyfile(l, dest)
0737: else:
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/user/ros_target/build/BUILD-poky-foxy-dunfell/deploy/rpm/cortexa7t2hf_neon_vfpv4/glibc-localedata-pa-pk-2.31-r0.cortexa7t2hf_neon_vfpv4.rpm' -> '/home/user/ros_target/build/BUILD-poky-foxy-dunfell/work/raspberrypi3-poky-linux-gnueabi/ros-image-turtlebot3-core/1.0-r0/oe-rootfs-repo/rpm/cortexa7t2hf_neon_vfpv4/glibc-localedata-pa-pk-2.31-r0.cortexa7t2hf_neon_vfpv4.rpm'
ERROR: Logfile of failure stored in: /home/user/ros_target/build/BUILD-poky-foxy-dunfell/work/raspberrypi3-poky-linux-gnueabi/ros-image-turtlebot3-core/1.0-r0/temp/log.do_rootfs.2708
ERROR: Task (/home/user/ros_target/3rdParty/meta-ros/meta-ros-common/recipes-core/images/ros-image-turtlebot3-core.bb:do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 9034 tasks of which 9033 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/home/user/ros_target/3rdParty/meta-ros/meta-ros-common/recipes-core/images/ros-image-turtlebot3-core.bb:do_rootfs
Summary: There were 358 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
Edit:
I tried to make all missing RPM available by symlinks. Since really a lot (but still not all) packages were affected, I created short script for that:
#!/bin/bash
currDir=`pwd`
#find all existing RPM in WORK directory
find build/BUILD-poky-foxy-dunfell/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi -iname "*.rpm" > ${currDir}/tmp_allRpm.log
#find all RPM that are already deployed
cd build/BUILD-poky-foxy-dunfell/deploy/rpm/cortexa7t2hf_neon_vfpv4/
ls -1 *.rpm > ${currDir}/tmp_deployedRpm.log
cd $currDir
#create list of all RPM that are not yet deployed
grep -v -f ${currDir}/tmp_deployedRpm.log ${currDir}/tmp_allRpm.log > ${currDir}/tmp_toBeLinkedRpm.log
#create symlinks
xargs -i ln -s {} build/BUILD-poky-foxy-dunfell/deploy/rpm/cortexa7t2hf_neon_vfpv4/ <${currDir}/tmp_toBeLinkedRpm.log
This seemed to solved the origin issue but lead to further weired errors. So I decided to set up new Yocto/OpenEmbedded environment using my existing config files and recipes. It turned out to be a good decision - after several hours of compilation everything was done ;) Only downside of this is that I can't tell what went wrong and when it may happen again?!
I am attempting to build the gitlib-libgit2 Haskell package on macos via nix in roughly the following manner.
mkdir nix-repro && cd nix-repro
cabal2nix --no-check cabal://gitlib-libgit2 > gitlib-libgit2.nix
echo '(import <nixpkgs> {}).haskellPackages.callPackage ./gitlib-libgit2.nix {}' > default.nix
nix-build
I am seeing the following warnings and error:
ld: warning: /nix/store/spx9xz1jv3yhmqw8y3agki1fvdr2x2fv-libiconv-osx-10.11.6/lib/libiconv.dylib, ignoring unexpected dylib file
ld: warning: /nix/store/spx9xz1jv3yhmqw8y3agki1fvdr2x2fv-libiconv-osx-10.11.6/lib/libiconv.dylib, ignoring unexpected dylib file
/nix/store/s33984hx2gwcg2d4dgpcm4342md19qvh-cctools-binutils-darwin/bin/ranlib: object: dist/build/libHSgitlib-libgit2-3.1.2.1-1fYQZMedHRP3aXiBXJFDO2-ghc8.6.3.a(s2_meth.o) malformed object (unknown load command 1)
`ranlib' failed in phase `Ranlib'. (Exit code: 1)
builder for '/nix/store/yrran1p69pvdq1b34jqfq7dmd95j9fh9-gitlib-libgit2-3.1.2.1.drv' failed with exit code 1
error: build of '/nix/store/yrran1p69pvdq1b34jqfq7dmd95j9fh9-gitlib-libgit2-3.1.2.1.drv' failed
What does this mean, and how can I go about fixing it?
It only fails this way on macos, as far as I can tell. I'm on nixpkgs-unstable. I am able to build this package using other tools, so the failure must be something specific to the nix tooling I'm using to build it.
$ cat $HOME/.nix-defexpr/channels/nixpkgs/.git-revision
6e5caa3f8ac48750233ef82a94825be238940825
Here's the full gitlib-libgit2.nix expression if you don't want to run cabal2nix for yourself:
{ mkDerivation, base, bytestring, conduit, conduit-combinators
, containers, directory, exceptions, fast-logger, filepath, gitlib
, gitlib-test, hlibgit2, hspec, hspec-expectations, HUnit, mmorph
, monad-loops, mtl, resourcet, stdenv, stm, stm-conduit, tagged
, template-haskell, text, text-icu, time, transformers
, transformers-base, unliftio, unliftio-core
}:
mkDerivation {
pname = "gitlib-libgit2";
version = "3.1.2.1";
sha256 = "b90e0ad2e7e0f58379e02cbe60d2900c95f0a255c34bd3461f8ee5753a6aa23e";
libraryHaskellDepends = [
base bytestring conduit conduit-combinators containers directory
exceptions fast-logger filepath gitlib hlibgit2 mmorph monad-loops
mtl resourcet stm stm-conduit tagged template-haskell text text-icu
time transformers transformers-base unliftio unliftio-core
];
testHaskellDepends = [
base exceptions gitlib gitlib-test hspec hspec-expectations HUnit
transformers
];
doCheck = false;
description = "Libgit2 backend for gitlib";
license = stdenv.lib.licenses.mit;
}
I'm new to Bazel and learning its build work, currently I am trying to do with bazel build from a Maven project, please advise me how to make it working, thanks.
Here is the WORKSPACE file I am trying to define:
maven_jar(
name = "junit",
artifact = "junit:junit:3.8.1",
)
maven_jar(
name = "log4j1",
artifact = "org.apache.logging.log4j:log4j-core:2.6.2",
)
maven_jar(
name = "log4j2",
artifact = "org.apache.logging.log4j:log4j-api:2.6.2",
)
.....
Here is the BUILD file I am trying to define:
package(default_visibility = ["//visibility:public"])
java_binary(
name = "everything",
srcs = glob(["src/main/java/**/*.java"]),
resources = glob(["src/main/resources/**"]),
main_class = "src/main/java/com/test/test/test/App",
deps = [
"#junit//jar",
"#log4j1//jar",
"#log4j2//jar",
"#jackson//jar",
"#jsonsimple//jar",
"#commonsdbutils//jar",
"#commons//jar",
"#guava//jar",
"#poi//jar"],
)
Here is I got Bazel build results:
mbp:bazel_test me$ bazel build //:everything
INFO: Analysed target //:everything (1 packages loaded).
INFO: Found 1 target...
ERROR: /Users/me/git/test/test/BUILD:4:1: Building everything-class.jar (104 source files) failed (Exit 1)
src/main/java/com/test/test/test/Testapp.java:13: error: cannot find symbol
import org.apache.poi.ss.usermodel.Cell;
^
symbol: class Cell
location: program package org.apache.poi.ss.usermodel
.....
Target //:everything failed to build
I see two problems here:
The main_class attribute should use dots instead of slashes:
java_binary(
name = "everything",
srcs = glob(["src/main/java/**/*.java"]),
main_class = "src.main.java.com.test.test.test.App",
...
There's no org.apache.poi.ss.usermodel.Cell class in the poi jar.
Find the jar's name:
$ bazel query --output=build #poi//jar
...
java_import(
name = "jar",
visibility = ["//visibility:public"],
jars = ["#poi//jar:poi-ooxml-3.9.jar"],
srcjar = "#poi//jar:poi-ooxml-3.9-sources.jar",
)
The package's location is $(bazel info output_base)/external/poi, and now we know (from the srcs attribute in the bazel query output above) that the file is called poi-ooxml-3.9.jar:
$ unzip -l $(bazel info output_base)/external/poi/jar/poi-ooxml-3.9.jar | grep "org.apache.poi.ss.usermodel"
0 2012-11-26 17:14 org/apache/poi/ss/usermodel/
2970 2012-11-26 17:14 org/apache/poi/ss/usermodel/WorkbookFactory.class