Bazel & protobuf: how to choose a specific protoc version - protocol-buffers

I am working in a project using some proto sources that are already compiled using a specific version, I also need to compile some custom protos that are cohabiting in the same project, so the protoc needs to match the one that was used to generate the other ones.
I can see in the pre-generated ones:
#if PROTOBUF_VERSION < 3009000
#if 3009002 < PROTOBUF_MIN_PROTOC_VERSION
In mines:
#if PROTOBUF_VERSION < 3017000
#if 3017000 < PROTOBUF_MIN_PROTOC_VERSION
I don't quite understand which protoc is being used, the one installed on my system is 3.19.4.
Also this is my WORKSPACE:
http_archive(
name = "rules_proto",
sha256 = "66bfdf8782796239d3875d37e7de19b1d94301e8972b3cbd2446b332429b4df1",
strip_prefix = "rules_proto-4.0.0",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_proto/archive/refs/tags/4.0.0.tar.gz",
"https://github.com/bazelbuild/rules_proto/archive/refs/tags/4.0.0.tar.gz",
],
)
load("#rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()
http_archive(
name = "com_github_grpc_grpc",
urls = [
"https://github.com/grpc/grpc/archive/refs/tags/v1.44.0.tar.gz",
],
sha256 = "8c05641b9f91cbc92f51cc4a5b3a226788d7a63f20af4ca7aaca50d92cc94a0d",
strip_prefix = "grpc-1.44.0",
)
load("#com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
grpc_deps()
load("#com_github_grpc_grpc//bazel:grpc_extra_deps.bzl", "grpc_extra_deps")
grpc_extra_deps()
The error I am currently getting is:
In file included from cc/tensorflow/plugin_primeclient/grappler/grappler.cc:7:
bazel-out/aarch64-fastbuild/bin/cc/tensorflow/plugin/protos/graph.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
12 | #error This file was generated by a newer version of protoc which is
| ^~~~~
bazel-out/aarch64-fastbuild/bin/cc/tensorflow/plugin/protos/graph.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
13 | #error incompatible with your Protocol Buffer headers. Please update
| ^~~~~
bazel-out/aarch64-fastbuild/bin/cc/tensorflow/plugin/protos/graph.pb.h:14:2: error: #error your headers.
14 | #error your headers.
| ^~~~~

I'll try and describe the general process I take when tracking dependency problems down in Bazel, as it seems to be a regular problem that you'll probably run into again.
Before Bazel does anything to do with the build it's going to look in your WORKSPACE file to see if it needs to fetch any dependencies. It might not seem like an important detail, but Bazel handles WORKSPACE dependencies from top to bottom. We can use this behaviour to override the protobuf version used. Checkout the maybe macro if your interested in how this works.
So in your WORKSPACE file, the first dependency that you have is 'rules_proto' # version 4.0.0 with the http_archive. Then you are loading two macros from 'rules_proto' here;
load("#rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()
So first things first let's head over to the rules_proto releases page and find your specific release. Then click on the little hash on that page (circled in red).
Then click "browse files" in the top right;
This will allow you to browse the state of that repository at that specific version. Now as you loaded "repositories.bzl" you'll want to open that up and inspect it (ctrl-F search for protobuf). You'll find that it calls another private macro i.e.
protobuf_workspace(name = "com_google_protobuf")
If we look for where that macro was loaded you'll see that you'll need to follow that through to the 'proto/private/dependencies.bzl';
load("//proto/private:dependencies.bzl", "dependencies", "maven_dependencies", "protobuf_workspace")
After opening that up and searching again for protobuf you'll find the line that specifies the protobuf version;
"com_github_protocolbuffers_protobuf": {
#...
},
So by the looks of it you are using an older version of protobuf with Bazel than what is installed on your system. So in order to override the protobuf version in Bazel you need to simply add it as a dependency in your WORKSPACE before the rules_proto repository. e.g.
# file: //:WORKSPACE
http_archive(
name = "com_github_protocolbuffers_protobuf",
# TODO: Leave this empty and Bazel will tell you what to put here when you build.
sha256 = "",
# Note: Same version as your system deps.
strip_prefix = "protobuf-3.19.4",
urls = [
# Note: Same version as your system deps.
"https://github.com/protocolbuffers/protobuf/releases/download/v3.19.4/protobuf-all-3.19.4.tar.gz"
],
)
http_archive(
name = "rules_proto",
# The rest of the WORKSPACE...

Related

Using a particular version for golang dependency module

I'm trying to build postfix-exporter code from the github link.
It has a dependency on the go-systemd package as mentioned in go.mod file github.com/coreos/go-systemd/v22 v22.0.0.
I see in the go.mod file , the mentioned version for the package is v22.0.0.
But when I run go get -u for this path, it starts downloading the latest version ( v22.2.0) of go-systemd which has an issue in the latest commit and causing failure to compile .
The error coming in that is
github.com/coreos/go-systemd/v22#v22.2.0/sdjournal/journal.go:313:60: error: '_SD_ARRAY_STATIC' undeclared here (not in a function) // my_sd_id128_to_string(void *f, sd_id128_t boot_id, char s[_SD_ARRAY_STATIC SD_ID128_STRING_MAX]) ^ In file included from /usr/include/systemd/sd-journal.h:31:0, from pkg/mod/github.com/coreos/go-systemd/v22#v22.2.0/sdjournal/journal.go:27: pkg/mod/github.com/coreos/go-systemd/v22#v22.2.0/sdjournal/journal.go:313:77: error: expected ']' before numeric constant // my_sd_id128_to_string(void *f, sd_id128_t boot_id, char s[_SD_ARRAY_STATIC SD_ID128_STRING_MAX])
I want to know , how to compile for a specific version of any dependency module if this is not the way or I'm missing some option required to honor the version of dependency packages mentioned in go.mod
Thanks very much in advance and please pardon my golang knowledge.
Don't use -u. The purpose of -u is to allow Go to try to upgrade you to the latest minor or patch version:
The -u flag instructs get to update modules providing dependencies
of packages named on the command line to use newer minor or patch
releases when available.
If you're just trying to install dependencies, use go get.
I was trying to build podman for Centos7 and found this error, noticed that _SD_ARRAY_STATIC was not defined, so I just did a search in google and found this header file: https://code.woboq.org/qt5/include/systemd/_sd-common.h.html.
Also, by doing a search of this file in my docker I found this very old one: /usr/include/systemd/_sd-common.h, so I just decided to modify it and add that definition:
/*-*- Mode: C; c-basic-offset: 8; indent-tabs-mode: nil -*-*/
#ifndef foosdcommonhfoo
#define foosdcommonhfoo
/***
This file is part of systemd.
Copyright 2013 Lennart Poettering
...
...
#ifndef _SD_END_DECLARATIONS
# ifdef __cplusplus
# define _SD_END_DECLARATIONS \
} \
struct __useless_struct_to_allow_trailing_semicolon__
# else
# define _SD_END_DECLARATIONS \
struct __useless_struct_to_allow_trailing_semicolon__
# endif
#endif
#ifndef _SD_ARRAY_STATIC
# if __STDC_VERSION__ >= 199901L
# define _SD_ARRAY_STATIC static
# else
# define _SD_ARRAY_STATIC
# endif
#endif
#endif
And voila, got it working.
TL;DR, probably you have to update systemd package or at least the systemd C library.

How to correctly link the path of external library in CMake? [duplicate]

How to get CMake to link an executable to an external shared library that is not build within the same CMake project?
Just doing target_link_libraries(GLBall ${CMAKE_BINARY_DIR}/res/mylib.so) gives the error
make[2]: *** No rule to make target `res/mylib.so', needed by `GLBall'. Stop.
make[1]: *** [CMakeFiles/GLBall.dir/all] Error 2
make: *** [all] Error 2
(GLBall is the executable)
after I copied the library into the binary dir bin/res.
I tried using find_library(RESULT mylib.so PATHS ${CMAKE_BINARY_DIR}/res)
Which fails with RESULT-NOTFOUND.
arrowdodger's answer is correct and preferred on many occasions. I would simply like to add an alternative to his answer:
You could add an "imported" library target, instead of a link-directory. Something like:
# Your-external "mylib", add GLOBAL if the imported library is located in directories above the current.
add_library( mylib SHARED IMPORTED )
# You can define two import-locations: one for debug and one for release.
set_target_properties( mylib PROPERTIES IMPORTED_LOCATION ${CMAKE_BINARY_DIR}/res/mylib.so )
And then link as if this library was built by your project:
TARGET_LINK_LIBRARIES(GLBall mylib)
Such an approach would give you a little more flexibility: Take a look at the add_library( IMPORTED) command and the many target-properties related to imported libraries.
I do not know if this will solve your problem with "updated versions of libs".
Set libraries search path first:
link_directories(${CMAKE_BINARY_DIR}/res)
And then just do
target_link_libraries(GLBall mylib)
I assume you want to link to a library called foo, its filename is usually something link foo.dll or libfoo.so.
1. Find the library
You have to find the library. This is a good idea, even if you know the path to your library. CMake will error out if the library vanished or got a new name. This helps to spot error early and to make it clear to the user (may yourself) what causes a problem.
To find a library foo and store the path in FOO_LIB use
find_library(FOO_LIB foo)
CMake will figure out itself how the actual file name is. It checks the usual places like /usr/lib, /usr/lib64 and the paths in PATH.
You already know the location of your library. Add it to the CMAKE_PREFIX_PATH when you call CMake, then CMake will look for your library in the passed paths, too.
Sometimes you need to add hints or path suffixes, see the documentation for details:
https://cmake.org/cmake/help/latest/command/find_library.html
2. Link the library
From 1. you have the full library name in FOO_LIB. You use this to link the library to your target GLBall as in
target_link_libraries(GLBall PRIVATE "${FOO_LIB}")
You should add PRIVATE, PUBLIC, or INTERFACE after the target, cf. the documentation:
https://cmake.org/cmake/help/latest/command/target_link_libraries.html
If you don't add one of these visibility specifiers, it will either behave like PRIVATE or PUBLIC, depending on the CMake version and the policies set.
3. Add includes (This step might be not mandatory.)
If you also want to include header files, use find_path similar to find_library and search for a header file. Then add the include directory with target_include_directories similar to target_link_libraries.
Documentation:
https://cmake.org/cmake/help/latest/command/find_path.html
and
https://cmake.org/cmake/help/latest/command/target_include_directories.html
If available for the external software, you can replace find_library and find_path by find_package.
Let's say you have an executable like:
add_executable(GLBall GLBall.cpp)
If the external library has headers, give the path to its include folder:
target_include_directories(GLBall PUBLIC "/path/to/include")
Add the library directory path:
target_link_directories(GLBall PUBLIC "/path/to/lib/directory")
Finally, link the library name
target_link_libraries(GLBall mylib)
Note that the prefix and extension of the library file are removed:
libmylib.a ➜ mylib
mylib.so ➜ mylib
One more alternative, in the case you are working with the Appstore, need "Entitlements" and as such need to link with an Apple-Framework.
For Entitlements to work (e.g. GameCenter) you need to have a "Link Binary with Libraries"-buildstep and then link with "GameKit.framework". CMake "injects" the libraries on a "low level" into the commandline, hence Xcode doesn't really know about it, and as such you will not get GameKit enabled in the Capabilities screen.
One way to use CMake and have a "Link with Binaries"-buildstep is to generate the xcodeproj with CMake, and then use 'sed' to 'search & replace' and add the GameKit in the way XCode likes it...
The script looks like this (for Xcode 6.3.1).
s#\/\* Begin PBXBuildFile section \*\/#\/\* Begin PBXBuildFile section \*\/\
26B12AA11C10544700A9A2BA \/\* GameKit.framework in Frameworks \*\/ = {isa = PBXBuildFile; fileRef = 26B12AA01C10544700A9A2BA \/\* GameKit.framework xxx\*\/; };#g
s#\/\* Begin PBXFileReference section \*\/#\/\* Begin PBXFileReference section \*\/\
26B12AA01C10544700A9A2BA \/\* GameKit.framework xxx\*\/ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = GameKit.framework; path = System\/Library\/Frameworks\/GameKit.framework; sourceTree = SDKROOT; };#g
s#\/\* End PBXFileReference section \*\/#\/\* End PBXFileReference section \*\/\
\
\/\* Begin PBXFrameworksBuildPhase section \*\/\
26B12A9F1C10543B00A9A2BA \/\* Frameworks \*\/ = {\
isa = PBXFrameworksBuildPhase;\
buildActionMask = 2147483647;\
files = (\
26B12AA11C10544700A9A2BA \/\* GameKit.framework in Frameworks xxx\*\/,\
);\
runOnlyForDeploymentPostprocessing = 0;\
};\
\/\* End PBXFrameworksBuildPhase section \*\/\
#g
s#\/\* CMake PostBuild Rules \*\/,#\/\* CMake PostBuild Rules \*\/,\
26B12A9F1C10543B00A9A2BA \/\* Frameworks xxx\*\/,#g
s#\/\* Products \*\/,#\/\* Products \*\/,\
26B12AA01C10544700A9A2BA \/\* GameKit.framework xxx\*\/,#g
save this to "gamecenter.sed" and then "apply" it like this ( it changes your xcodeproj! )
sed -i.pbxprojbak -f gamecenter.sed myproject.xcodeproj/project.pbxproj
You might have to change the script-commands to fit your need.
Warning: it's likely to break with different Xcode-version as the project-format could change, the (hardcoded) unique number might not really by unique - and generally the solutions by other people are better - so unless you need to Support the Appstore + Entitlements (and automated builds), don't do this.
This is a CMake bug, see http://cmake.org/Bug/view.php?id=14185 and http://gitlab.kitware.com/cmake/cmake/issues/14185
It has been a long time since the question was posted but I am leaving this one just for reference.
I have a blog post describing step-by-step almost what you (or anyone else) were trying to do.
Please check here: https://michae9.wordpress.com/2022/09/01/shared-lib-to-be-used-by-client-programs-with-cmake/

Type mis-match for included library in go

I'm running into an issue with a library I made that I want to include in multiple projects
# github.com/pcs-services/message-queue-operator/pkg/controller/messagequeue
pkg/controller/messagequeue/messagequeue_controller.go:167:129: cannot use instance.ObjectMeta.GetUID() (type "github.com/pcs-services/message-queue-operator/vendor/k8s.io/apimachinery/pkg/types".UID) as type "github.com/pcs-services/grafana/vendor/k8s.io/apimachinery/pkg/types".UID in argument to grafana.DeployGrafana
The files are the same in the main project to the
$ md5 k8s.io/apimachinery/pkg/types/uid.go
MD5 (k8s.io/apimachinery/pkg/types/uid.go) = cc286eae550982db7f93c079e0df1f52
$ md5 k8s.io/apimachinery/pkg/types/uid.go
MD5 (k8s.io/apimachinery/pkg/types/uid.go) = cc286eae550982db7f93c079e0df1f52
k8s.io/apimachinery is vendored in the main app but not the library.
The error says:
pkg/controller/messagequeue/messagequeue_controller.go:167:129:
cannot use instance.ObjectMeta.GetUID() (type
"github.com/pcs-services/message-queue-operator/vendor/k8s.io/apimachinery/pkg/types".UID)
as type
"github.com/pcs-services/grafana/vendor/k8s.io/apimachinery/pkg/types".UID
in argument to grafana.DeployGrafana
So, you're using github.com/pcs-services/grafana/vendor/k8s.io/apimachinery/pkg/types.UID but imported github.com/pcs-services/message-queue-operator/vendor/k8s.io/apimachinery/pkg/types.UID. Check your imports and possibly run dep ensure (if you're using dep) to copy the correct dependencies to your vendor folder.

Protocol Buffers packages issue

I am new to Protocol Buffers, and I have just added the library / build requirements for my Maven project. I now have a .proto file in my source repository that has little to nothing in it:
package com.christopher.kade;
option java_package= "protocol";
message Protocol {
required int32 id = 1;
required string name = 2;
}
But I've found myself facing a problem when it comes to packages, the following file creates a protocol package in my com.christopher.kade one and I get an error message stating that:
Package name 'protocol' does not correspond to the file path 'com.christopher.kade.protocol'.
What is the good approach in order to generate my class in my current package? Therefore I would have:
com.christopher.kade
|- client.proto
|- MyGeneratedClass.java
|- MyClass.java
The mistake is in this line
option java_package= "protocol";
Change it to
option java_package= "com.christopher.kade";
and you are good!

OpenVRML in snow-leopard (from macports)

Hey, I just Downloaded openvrml from macports
(port install openvrml)
Now I have a Sample program (pretty_print.cpp from openvrml at sourceforge) that begins like this:
# ifdef HAVE_CONFIG_H
# include <config.h>
# endif
# include <openvrml/vrml97_grammar.h>
# include <openvrml/browser.h>
# include <fstream>
...
then in Xcode, I added the following path and check "recursive" for the Header search path and Lib Search Path:
/opt/local/var/macports/software
And all '***.h file not found' errors disappeared, but now I have the following two:
complex.h 943 '__pow_helper' is not a member of std
c++locale.h 71 'vsnprintf' is not a member of std
/Developer/SDKs/MacOSX10.6.sdk/usr/include/c++/4.2.1/complex: In function 'std::complex<_Tp> std::pow(const std::complex<_Tp>&, int)':
/Developer/SDKs/MacOSX10.6.sdk/usr/include/c++/4.2.1/complex:943: error: '__pow_helper' is not a member of 'std'
both errors come from system files.
I wonder what is causing this errors...
Can anyone advice me on how to use openvrml samples on Macs?
thanks in advance.
I've had a similar problem. I defined "recursive" flag for an '/opt/local/include' path. This pulled in some strange c++ headers from boost compatiblity includes.
In general, you do not want "recursive" flag on your include paths.
Try unchecking "recursive" from your paths.
if you put recursive on a path containing boost headers you'll use some random boost headers, which are likely designed to be used in different environment and/or different compiler, instead of standard C++ headers, meaning, for example, you'll include TR1 header instead of standard header. This is likely to be the cause of your problem (it happened to me too).
Just locate the directory which contains the headers you need and put only that in header search path instead of being lazy and using "recursive" flag, since there are a lot of header files which have same name but differ in location only.

Resources