I have a folder protobuf with a lot of .proto files which I can compile with
protoc -I=protobuf filename.proto --grpc-web_out=import_style=commonjs,mode=grpcwebtext:output
This generates the grpc_web_pb.js into the /output folder but I'm looking for a way to not have to call protoc for every single file, is there something like a wildcard?
I tried
protoc -I=protobuf *.proto --grpc-web_out=import_style=commonjs,mode=grpcwebtext:output
but that doesn't work, fails with no matches found: *.proto
This works for me, maybe just add path to your .proto file:
protoc -I . --grpc-web_out=import_style=commonjs,mode=grpcwebtext:. ./*.proto
Related
Unable to figure out why I'm getting a file not found error when I'm running a protoc command to generate the relevant go files.
Problem: Trying to figure out what the appropriate protoc command is while trying to import a proto file from a different directory.
I've set my proto-path in GoLand to be ~/go-workspace/github.com/xyz/project/internal
Structure
project
- internal
- song
- proto
- *.proto
- search
- proto
- *.proto
Song.proto
syntax = "proto3";
package song;
option go_package = "github.com/xyz/project/internal/song/proto";
......
Search.proto
syntax = "proto3";
package search;
option go_package = "github.com/xyz/project/internal/search/proto";
import "song/proto/song.proto";
Makefile:
generate:
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
proto/*.proto
Error:
song/proto/song.proto: File not found.
proto/search.proto:6:1: Import "song/proto/song.proto" was not found or had errors.
proto/search.proto:16:12: "song.Song" is not defined.
proto/search.proto:20:12: "song.Artist" is not defined.
As mentioned in a comment, you could use the --proto_path or -I option with the protoc compiler and you need to start your import with internal.
Another solution that I can see and that would not require multiple Makefiles, is putting the following Makefile at the project level:
generate:
protoc --go_out=. \
--go_opt=module=github.com/xyz/project \
internal/search/proto/*.proto \
internal/song/proto/*.proto
Note the the --go_opt=module that will trim your package name and help you generate the protobuf code inside the respective proto directories and note that I'm passing the entire paths to the proto directories.
and then search.proto like:
syntax = "proto3";
package search;
option go_package = "github.com/xyz/project/internal/search/proto";
import "internal/song/proto/song.proto";
I have the following structure:
rootfolder/
foldA
A1.proto
foldB
B1.proto
foldC
C1.proto
what I tried is:
for /r %g in (*.proto) do protoc -I=rootfolder --python_out=. %g
Ideally I would want to compile them all in a folder named protos at rootfolder level. Also I can't hardcode them because im supposed to do a command which would compile other proto files added in the future as well.
The error I get with my command is:
<full_path_to_proto_file> File does not reside within any path specified using --proto_path (or -I). You must specify a --proto_path which encompasses this file. Note that the proto_path must be an exact prefix of the .proto file name -- protoc is too dumb to figure out when two paths (eg absolute and relative) are equivalent.
Thanks to the kind user in the comment section of my post
for /D %J in ("rootfolder\*") do for %I in ("%~J\*.proto") do protoc --proto_path="%~dpI." --python_out="protos" "%~nxI"
I have 2 .proto files in the same directory such that second.proto is dependent on first.proto
second.proto
import "first.proto"
enum ThingINeed {
...something
}
I have no problem running these commands:
$ protoc --go_out=generatedsources/first -I. first.proto
$ protoc --go_out=generatedsources/second -I. second.proto
so the directory structure looks something like
src
|-first.proto
|-second.proto
|-generatedsources
|-first
|-first.pb.go
|-second
|-second.pb.go
My problem is that when I run
$ cd generatedsources/second
$ go build second.pb.go
I recieve a "./second.pb.go: Undefined: ThingINeed" since second.pb.go uses ThingINeed from first.pb.go (seen in first.proto as well)
I notice that second.pb.go doesn't have an import . "generatedsources/first" line in it. When I put it in manually, it works just fine. But I mean, I'm not supposed to edit these .pb.go files, so was wondering how to fix this. I also would very rather not edit these .proto files.
Would appreciate any help!
I'm wondering how to properly reference external proto files. Say I've got a .proto file which references standard protobuf types such as Timestamp:
syntax = "proto3";
package api;
import "google/protobuf/timestamp.proto";
message ServerTimeResponse {
google.protobuf.Timestamp ts = 1;
}
Easy. Timestamp is automatically available when compiling.
Now I add an external
type, say google.rpc.Status:
syntax = "proto3";
package api;
import "google/protobuf/timestamp.proto";
import "google/rpc/status.proto";
message ServerTimeResponse {
google.protobuf.Timestamp ts = 1;
google.rpc.Status status = 2;
}
Of course we have to tell protoc how to find this file where it is via -I/--proto_path.
My question is this: What is the best practice for actually referencing this file, in particular to make version control happy? There appears not to be a go mod equivalent for protobufs. I've seen it copied verbatim into projects (such as in grpc-gateway) or just referenced from the local filesystem.
I think you sort of answered your own question here. I've done both successfully: manually copied the necessary files in verbatim (from https://github.com/googleapis/googleapis/tree/master/google and https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf), and referenced local copies of the files.
If you want to do this and make version control happy, you could add these two repositories as git submodules inside your repository. Just make sure to pass the right locations to protoc using -I. E.g.:
cd $PROJECT_DIR
mkdir third_party && cd third_party
git submodule add https://github.com/googleapis/googleapis/tree/master/google
cd $PROJECT_DIR
<git commit the change>
protoc -I third_party/google <the rest of your protoc command>
As for referencing local copies of the files, and making sure they're present before attempting to build, you may find that adding something like the following to your Makefile will help (this is in a Go build environment):
go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway
go get -u github.com/golang/protobuf/protoc-gen-go
grpc_gateway_path=$(go list -m -f '{{.Dir}}' github.com/grpc-ecosystem/grpc-gateway)
googleapis_path="$grpc_gateway_path/third_party/googleapis"
protoc -I $googleapis_path --go_out=. <list of input files>
I am having a project where suppose:
Project structure is root/p1/mod1/include/test.hrl and root/p2/mod2/foo.erl
I have a erlang hrl file in root/p1/mod1/include and i have included it in erlang file (.erl) in root/p2/mod2
When I am compiling file foo.erl it gives following error:
foo.erl:16: can't find include file "test.hrl"
I tried including it in -I flag of erlc such as below:
user[root/p2/mod2]$ erlc foo.erl -I "/local/user/root/p1/mod1/include"
foo.erl:16: can't find include file "test.hrl"
But it does not seem to solve it.
Can anyone help on how to resolve it?
In Erlang, the command should be below:
erlc flags file1.ext file2.ext...
Compiles one or more files. The files must include the extension, for example, .erl for Erlang source code, or .yrl for Yecc source code. Erlc uses the extension to invoke the correct compiler
The following flags are supported:
-I 'Directory'
So just move your -I before erl file:
erlc -I "/local/user/root/p1/mod1/include" foo.erl