Starting with micro and everything works fine. I dont have VisualStudio, but instead I have make through chocolatey(dont know if could be the issue)
I bootstrap the service with micro new like this.
λ micro new my.test
Creating service my.test
.
├── micro.mu
├── main.go
├── generate.go
├── handler
│ └── my.test.go
├── proto
│ └── my.test.proto
├── Dockerfile
├── Makefile
├── README.md
├── .gitignore
└── go.mod
download protoc zip packages (protoc-$VERSION-$PLATFORM.zip) and install:
visit https://github.com/protocolbuffers/protobuf/releases
download protobuf for micro:
go get -u github.com/golang/protobuf/proto
go get -u github.com/golang/protobuf/protoc-gen-go
go get github.com/micro/micro/v3/cmd/protoc-gen-micro
compile the proto file my.test.proto:
cd my.test
make proto
Installed dependencies and everything was fine. Then I moved to my.test and after make proto this error shows up
protoc --proto_path=. --micro_out=. --go_out=:. proto/proto.test.proto
proto/proto.test.proto:7:14: Expected "{".
make: *** [Makefile:16: proto] Error 1
I have all dependencies and my PATH is fine, but I dnt know what the problem is.
Edit: This is my proto, conveniently generated by micro
syntax = "proto3";
package proto.test;
option go_package = "./proto;proto.test";
service Proto.Test {
rpc Call(Request) returns (Response) {}
rpc Stream(StreamingRequest) returns (stream StreamingResponse) {}
rpc PingPong(stream Ping) returns (stream Pong) {}
}
message Message {
string say = 1;
}
message Request {
string name = 1;
}
message Response {
string msg = 1;
}
message StreamingRequest {
int64 count = 1;
}
message StreamingResponse {
int64 count = 1;
}
message Ping {
int64 stroke = 1;
}
message Pong {
int64 stroke = 1;
}
Go package names cannot contain dots - it will just not compile:
$ cat pkg.go
package dot.test
$ go build
./pkg.go:1:12: syntax error: unexpected .
so one has to ensure generated code produces a valid Go package name.
From the Go spec, the package clause is defined as:
A package clause begins each source file and defines the package to
which the file belongs.
PackageClause = "package" PackageName .
PackageName = identifier .
and an identifier as:
... a sequence of one or more letters and digits. The first
character in an identifier must be a letter.
identifier = letter { letter | unicode_digit } .
After some reading of the basics I found the problem. Even if the proto file is generated by micro this code service Proto.Test at line seven is the problem, because of the dot. I mean, replacing that for service Test solved the issue. Now I dont know why. Any explanation would be preciated. I am in windows by the way
Related
syntax = "proto3";
package model;
import "google/protobuf/Empty.proto";
message User {
string id = 1;
string name = 2;
string email = 3;
string alamat = 4;
string password = 5;
}
message UserList {
repeated User list = 1;
}
message userId {
string id = 1;
}
message UserUpdate {
string id = 1;
User user = 2;
}
service Users {
rpc getUserList(google.protobuf.Empty) returns (UserList) {}
rpc getUserById(userId) returns (User) {}
rpc inserUser(User) returns (google.protobuf.Empty) {}
rpc updateUser(UserUpdate) returns (google.protobuf.Empty) {}
rpc deleteUser(userId) returns (google.protobuf.Empty) {}
}
above is my proto file. I get error google/protobuf/Empty.proto: File not found.
when trying to compile the proto file above. can someone help me ?
First of all, the correct import is import "google/protobuf/empty.proto";
second, for generating a proto file run this code:
protoc --proto_path={proto_directory_address} --proto_path={proto_directory_name} --go-grpc_out={generated_directory_path} --go_out={generated_directory_path} {proto_directory_address}/{proto_file_name}.proto
hi there / i had the same issue for a long time .. this process worked for me i hope it dose for you too :
navigate to this directory using your cmd(command line) :
cd .local/include
this directory normally should contain some folder named "google" copy this folder and paste it to this directory :
/usr/local/include
and now try the protoc engine again to generate your project and if the error still exists then try the rest of process :
navigate to that specific directory and check if its been copied or not . if it is then try to navigate to the folder from where you are (which it should be /usr/local/include) if the error says you have no permission to get in the folder
use this command to get the permission
$ sudo chmod o+r -R ./google
and then try to get permission to get in protobuf folder in the same directory using above command again
when its all done . check the protoc generator again /// hope works for you because it dose for me
I need to include some generated files (for example output from go-swagger) in a binary. Without bazel you can do that using go:generate in combination with go:embed:
package demo
//go:generate swagger generate spec -w . -o ./openapi.json
import "embed"
// Content contains openapi.json file generated via go-swagger from spec
//go:embed openapi.json
var Content embed.FS
I am trying to do the same thing with bazel. As a simple test I have have this BUILD.bazel file:
genrule(
name = "hellodata",
srcs = ["hello.go"],
outs = ["hello.txt"],
cmd = "cat $(SRCS) | tr A-Za-z N-ZA-Mn-za-m > $#",
)
go_library(
name = "hello",
srcs = ["hello.go"],
importpath = "wiggy.net/hello",
visibility = ["//visibility:public"],
embedsrcs = [":hellodata"],
)
with hello.go looking like this:
package hello
import (
_ "embed"
"io"
)
//go:embed hello.txt
var greeting []byte
func Hello(out io.Writer) error {
_, err := out.Write(greeting)
return err
}
The intention here is to have Hello output the rot13 of it's own source. When I try to compile this it successfully generates hello.txt (located in bazel-out/darwin_arm64-fastbuild/bin/hello.txt), but the compiler can not find it:
❯ bazel build //...
INFO: Analyzed 5 targets (0 packages loaded, 0 targets configured).
INFO: Found 5 targets...
ERROR: /Users/wichert/Hack/bzl/BUILD.bazel:14:11: GoCompilePkg hello.a failed: (Exit 1): builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_arm64 -src hello.go -embedsrc bazel-out/darwin_arm64-fastbuild/bin/hello.txt -importpath wiggy.net/hello ... (remaining 12 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
compilepkg: /private/var/tmp/_bazel_wichert/e7573342ee9452df4c3dfa671d399a16/sandbox/darwin-sandbox/76/execroot/__main__/hello.go:8:12: could not embed hello.txt: no matching files found
INFO: Elapsed time: 0,112s, Critical Path: 0,04s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully
I noticed -embedsrc bazel-out/darwin_arm64-fastbuild/bin/hello.txt in the command line for the failing action in your question, so as a hunch I tried the equivalent on my machine:
//go:embed bazel-out/k8-fastbuild/bin/hello.txt
var greeting []byte
And that seemed to work.
This is not so great because configuration information gets embedded in the source files (indeed, on your mac machine it's darwin_arm64-fastbuild and on my linux machine it's k8-fastbuild), so now your source code is artificially platform dependent.
It looks like this feature is relatively new (https://github.com/bazelbuild/rules_go/issues/2775, https://github.com/bazelbuild/rules_go/issues/2986). I would file an issue with rules_go about this.
There also appears to be go_embed_data which might behave differently:
https://github.com/bazelbuild/rules_go/blob/master/docs/go/extras/extras.md#go_embed_data
To answer my own question: the trick is to use genrule to generate the file(s) to be embedded, and then use go_embed_data to embed them. A work BUILD.bazel looks like this:
genrule(
name = "hellodata",
srcs = ["hello.go"],
outs = ["hello.txt"],
cmd = "cat $(SRCS) | tr A-Za-z N-ZA-Mn-za-m > $#",
)
go_embed_data(
name = "hello_embed",
src = ":hellodata",
package = "hello",
var = "greeting",
)
go_library(
name = "hello",
srcs = [
"hello.go",
":hello_embed",
],
importpath = "wiggy.net/hello",
visibility = ["//visibility:public"],
)
I have the following bazel BUILD configured to
gazelle(name = "gazelle")
go_embed_data(
name = "static_files",
srcs = glob(["static/**/*"]),
package = "main",
var = "staticFS",
)
go_library(
name = "kmdr_lib",
srcs = ["main.go"],
importpath = "github.com/myorg/myrepo",
visibility = ["//visibility:private"],
deps = [
"//api",
"//cmd",
],
)
With the following embed tag
package main
import (
"embed"
"github.com/myorg/myrepo/api"
"github.com/myorg/myrepo/cmd"
)
//go:embed static/*
var staticFS embed.FS
func main() {
api.StaticFS = staticFS
cmd.Execute()
}
However, when running...
bazel run //:gazelle
bazel build //...
I receive the following error that the static files tagged in the go main package could not be matched.
ERROR: GoCompilePkg kmdr_osx_amd64.a failed: (Exit 1): builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_amd64 -src main.go -arc ... (remaining 17 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
compilepkg: baf0dff8fcdeaf74ff5ba5ff8921e77f/sandbox/linux-sandbox/425/execroot/__main__/main.go:22:12: could not embed static/*: no matching files found
INFO: Elapsed time: 1.249s, Critical Path: 0.15s
INFO: 5 processes: 5 internal.
FAILED: Build did NOT complete successfully
The go_embed_data documentation doesn't give much details on how to use the library. I've also tried referencing the :static_files in the go_library srcs however, gazelle rewrites that.
bazel will rewrite the go_library if I reference go_emebed_data in the library srcs
go_embed_data generates a .go file that contains data from a file or a list of files. It should be consumed in the srcs list of one of the core go rules.
go_library(
name = "kmdr_lib",
srcs = ["main.go", ":static_files"],
importpath = "github.com/myorg/myrepo",
visibility = ["//visibility:private"],
deps = [
"//api",
"//cmd",
],
)
EDIT:
go build will parse the tag and embed the data as expected
It looks like go_embed_data is not the right method.
There was a PR made to address this issue https://github.com/bazelbuild/rules_go/pull/2806#issuecomment-784690934
Adding embedsrcs to your go_library will respect the go:embed directive
In Gradle, I'm trying to copy a list of absolute paths into a single directory, but I want to keep the last component of the absolute path as a subfolder of the destination directory.
Currently I have:
def copyTask = tasks.create(name: "copyFolders", type: Copy, overwrite: true) {
from ["/sources/a-1/first", "/sources/a-3/b-1/c-1/second"]
into "/destination"
}
but this results in the files contained in /sources/foo/bar and /sources/one/two/three/path being copied into /destination. Instead, I want /destination to contain both bar and path folders (with their contents inside them).
It should be able to handle this case correctly:
sources
├── a-1
│ └── fist
│ └── lorem.txt
| └── subfolder
│ └── lorem2.txt
├── a-2
│ └── fist-b
│ └── lorem.txt
| └── subfolder
│ └── lorem2.txt
└── a-3
└── b-1
└── c-1
└── second
└── ipsum.log
->
destination
├── first
│ └── lorem.txt
│ └── subfolder
│ └── lorem2.txt
└── path
└── ipsum.log
Based on #szymon answer I got:
def copyTask = tasks.create(name: "copyToDestDir", type: Copy, overwrite: true) {
includeEmptyDirs = false
def fromDir = "/sources"
def sourcePaths = ["/sources/a-1/first", "/sources/a-3/b-1/c-1/second"]
from fileTree(fromDir).dir.listFiles()
eachFile { copySpec ->
def shouldBeCopied = false
sourcePaths.find {
if (copySpec.getFile().getCanonicalPath().startsWith(it)) {
shouldBeCopied = true
return true // break iteration
}
}
if (shouldBeCopied) {
String[] parts = file.path.split('/')
copySpec.path = parts.size() > 1 ? "${parts[parts.size() - 2]}/${parts[parts.size() - 1]}" : parts[0]
} else {
copySpec.exclude()
}
}
into destDir
}
but this wrongly results in:
destination
├── first
│ └── lorem.txt
└── subfolder
│ └── lorem2.txt
└── path
└── ipsum.log
How to achieve this?
Try following Gradle task:
task copyFiles(type: Copy, overwrite: true) {
includeEmptyDirs = false
from fileTree("sources").dir.listFiles()
eachFile { file ->
String[] parts = file.path.split('/')
file.path = parts.size() > 1 ? "${parts[parts.size() - 2]}/${parts[parts.size() - 1]}" : parts[0]
}
into "destDir"
}
The whole part happens in eachFile {} closure where we specify a target file.path - I used this imperative approach, but it can be replaced with fancy regex that keeps last folder and file name and replaces all other parts with empty string (I tried to come up with such regex but I failed).
How it works
My input directory looks like this:
sources
├── foo
│ └── bar
│ └── lorem.txt
└── one
└── two
└── three
└── path
└── ipsum.log
I run a task:
#: ./gradlew copyFiles
:copyFiles UP-TO-DATE
BUILD SUCCESSFUL
And the destDir looks like this:
destDir
├── bar
│ └── lorem.txt
└── path
└── ipsum.log
Tested with Gradle 2.14.1, 3.5-rc-2 and 4.2.1
I've finally solved this doing this. Note that the top level into directive is needed or it won't work, and the nested into directives are relative to the top-level one.
def copyTask = tasks.create(name: "copyToDestDir", type: Copy, overwrite: true) {
includeEmptyDirs = false
def fromDir = "/sources"
def sourcePaths = ["/sources/a-1/first", "/sources/a-3/b-1/c-1/second"]
def destDir = "/destination"
sourcePaths.each { sourcePath ->
from(sourcePath) {
into sourcePath.substring(sourcePath.lastIndexOf('/') + 1)
}
}
into destDir
dependsOn cleanUpDestDir
}
Thanks to #SzymonStepniak for the inspiration.
Is there a good way to integrate protobuf compiles with the go build command?
goprotobuf "provides Go support, in the form of a library and protocol compiler plugin, for Google's protocol buffers".
The README at the goprotobuf library has some good info.
From https://code.google.com/p/goprotobuf/source/browse/README#106 :
Consider file test.proto, containing
package example;
enum FOO { X = 17; };
message Test {
required string label = 1;
optional int32 type = 2 [default=77];
repeated int64 reps = 3;
optional group OptionalGroup = 4 {
required string RequiredField = 5;
}
}
To build a package from test.proto and some other Go files, write a Makefile like this:
include $(GOROOT)/src/Make.$(GOARCH)
TARG=path/to/example
GOFILES=\
test.pb.go\
other.go
include $(GOROOT)/src/Make.pkg
include $(GOROOT)/src/pkg/code.google.com/p/goprotobuf/Make.protobuf