I am using wgo for dependency management in Golang (although I think wgo has little to do with this), wgo has a folder structure like this
project/
.gocfg/
gopaths
vendor.json
vendor/
src/
github.com_or_whatever/
I have a library I coded myself which uses an nsq-go type in one of the exported methods:
func AddNsqSubscription(
topic, channel string,
handler nsq.Handler,
config *nsq.Config) error { }
The library is called messi and I import the nsq-go like so "messi/vendor/src/github.com/bitly/go-nsq"
The problem comes when I try to use this library in another project. For instance, in a project called scribe I have the following code (notice the imports):
import (
"scribe/vendor/src/github.com/bitly/go-nsq"
"scribe/vendor/src/messi"
)
//...
nsqHandler := nsq.HandlerFunc(func(message *nsq.Message) error {
msgHandler(MessiMessage{message})
return nil
})
return messi.AddNsqSubscription(destination, subdestination, nsqHandler, nsq.NewConfig())
When I go build the following error is returned:
cannot use nsqHandler (type "scribe/vendor/src/github.com/bitly/go-nsq".HandlerFunc) as type "messi/vendor/src/github.com/bitly/go-nsq".Handler in argument to messi.AddNsqSubscription:
"scribe/vendor/src/github.com/bitly/go-nsq".HandlerFunc does not implement "messi/vendor/src/github.com/bitly/go-nsq".Handler (wrong type for HandleMessage method)
have HandleMessage("scribe/vendor/src/github.com/bitly/go-nsq".Message) error
want HandleMessage("messi/vendor/src/github.com/bitly/go-nsq".Message) error
Why? I do not really know what is going on. The code go-nsq imported is exactly the same, yet golang wants that this code comes from the same folder?
What am I doing wrong?
Packages in Go are identified by full import path, not by name.
For example in the standard library there are two different packages with the same name template but different import paths: text/template and html/template.
You should make sure that go-nsq package is imported using the same path.
Related
I am using the jackc/pgx driver alongside the GORM library to interface with a PostgreSQL database.
I have an instance where I have to check the PostgreSQL error code and handle a certain error type differently. When using the pgx driver, the GORM methods return a *pgconn.PgError type as the error, which contains a field with the specific error code.
In order to access that field, I must cast the error to a *pgconn.PgError, but for some reason this is failing:
res := tx.Take(&f, "id = ?", id)
if res.Error != nil {
if pqErr, ok := res.Error.(*pgconn.PgError); ok {
// does not reach here
} else {
fmt.Printf("Error type: %T\n", res.Error)
// Output: "Error type: *pgconn.PgError"
}
}
Notes:
The pgx and pgconn packages are inside the same project, so it's not the case that they are returning different versions of a type with the same name. In other words, I only have one import in my go.mod.
The returned value is not nil.
A debugger reveals that the type is a *pgconn.PgError.
You've solved your own issue, but here's some perhaps helpful background, and how I found the source.
Packages of the same name can exist in the same program, so long as they have different import paths. For example, the standard library has both math/rand and crypto/rand, each called rand. This is the first hint of how *pgconn.PgError and *pgconn.PgError are not the same: they come from different import paths.
When modules in Go make major revisions, they are supposed to change their import path. This is to preserve backwards compatibility with respect to import paths. Note that this is usually done by updating the module declaration in the go.mod file, rather than actually moving the code into a sub-directory. For example, see this commit where pgx was bumped from v4 to v5. This is the second hint: code from the pgx project is available under multiple import paths (due to the multiple major versions).
With this background in mind, I used the git tags to view the repository at the latest v4.x.x release. I noticed that oddly, the pgconn package did not exist in v4. This seemed to rule out the idea of a github.com/jackc/pgx/v4/pgconn vs github.com/jackc/pgx/v5/pgconn conflict. I then Google searched for "pgconn" and found the github.com/jackc/pgconn repository, where I saw in the README:
This version is used with pgx v4. In pgx v5 it is part of the https://github.com/jackc/pgx repository.
From the other information you've given, your mistake may have been using the import path "github.com/jackc/pgx/pgconn". As shown in the example code for pgx, the current import path you should be using for the base module is "github.com/jackc/pgx/v5", and packages within it would be specified similarly, e.g., "github.com/jackc/pgx/v5/pgconn".
As #HymnsForDisco correctly pointed out in the comments, both github.com/jackc/pgconn and github.com/jackc/pgx/pgconn exist. It turns out that the returned error was from the former, whereas I was importing the latter in my code.
To confirm, I added the following line:
fmt.Println("Error path: ", reflect.TypeOf(res.Error).Elem().PkgPath())
// Output: "Error path: github.com/jackc/pgconn"
Changing my import to "github.com/jackc/pgconn" resolved the issue.
My current Directory is setup as follows:
-- /webserver
|
/ftp
|-> ftp.go
|-> go.mod
|->server.go
|->go.mod
Server.go page heading
package main
import (
[..]
"../webserver/ftp" // issue package
)
func main() {
[..]
ftp.test()
}
ftp.go page
package ftp
import "fmt"
func test() {
fmt.Println("YES!")
}
Here's the part I cant seem to understand in the console "PROMBLEMS" tab. The ftp.go file gets saved. Before I save the server.go file with "../webserver/ftp/" the console displays this:
When I save server.go it removes "./webserver/ftp", I went as far as adding it into the go src to make it "ftp" instead of "../webserver/ftp". I get the same result.
Not sure what I am doing wrong.
Basically what i want to accomplish is:
have a functions folder and have the functions in that folder (functions.go) to keep the server.go page as clean as possible.
You have several problems here.
ftp/go.mod should not exist. While it is possible to have multiple Go modules in a single repository, this is very advanced usage, and is rarely useful. Unless you absoultely know that you need this capability, and are already exremely comfortable with creating and publishing Go modules, don't do this.
Go does not support relative imports. import "../webserver/ftp" is an invalid import path*. Use the fully qualified import path, i.e. import "github.com/username/projectname/webserver/ftp"
The path to use here depends on the module name you've declared on the module line of your your top-level go.mod file.
You cannot import unexported values.
You've defined func test() string in ftp.go. This is unexported, by virtue of beginning with a lowercase letter. Only functions, variables, or other symbols which begin with a capital letter are exported in Go. Try func Test() string instead.
*Some will say "But I used import ".../foo" and it worked!". Yes, it can work under very specific circumstances. But its behavior is not reliable, so you should never rely on this, except in very specific testing types of situations.
I'm new to Protocol Buffers and gRPC stuff. Now I'm trying to build a client/server architecture with grpc + grpc-gateway in Go.
I tried to follow some examples but I always end up with the same problem.
After generating the code with protoc i run go build and I get this error:
proto/helloworld/hello_world.pb.gw.go:64:2: cannot use msg (type *HelloReply) as type protoreflect.ProtoMessage in return argument:
*HelloReply does not implement protoreflect.ProtoMessage (missing ProtoReflect method)
proto/helloworld/hello_world.pb.gw.go:98:2: cannot use msg (type *HelloReply) as type protoreflect.ProtoMessage in return argument:
*HelloReply does not implement protoreflect.ProtoMessage (missing ProtoReflect method)
This is go.mod:
module github.com/riccardopedrielli/grpc-gateway-test
go 1.15
require (
github.com/golang/protobuf v1.4.3
github.com/grpc-ecosystem/grpc-gateway/v2 v2.2.0
google.golang.org/genproto v0.0.0-20210207032614-bba0dbe2a9ea
google.golang.org/grpc v1.35.0
google.golang.org/protobuf v1.25.0
)
This is hello_world.proto:
syntax = "proto3";
package helloworld;
import "google/api/annotations.proto";
option go_package = "github.com/riccardopedrielli/grpc-gateway-test/proto/helloworld";
// Here is the overall greeting service definition where we define all our endpoints
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {
option (google.api.http) = {
get: "/v1/example/echo/{name}"
};
}
}
// The request message containing the user's name
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
This is the link to the repository: https://github.com/riccardopedrielli/grpc-gateway-test
A difference I see between the generated go files is that they are importing different protobuf libraries.
The one generated by protoc-gen-go imports github.com/golang/protobuf/proto.
The one generated by protoc-gen-grpc-gateway imports google.golang.org/protobuf/proto.
Could this be the cause of the problem?
Still it's not clear to me which one should be used and how to force the same in both the generators.
I'm new to grpc and quite lost at this point, so I could have omitted some important informations. Any suggestion will be welcomed.
Thank you
Ok I solved the issue.
I had installed protoc via snap and the stable channel had version 3.11.4
Now I upgraded to 3.14.0 and everything is working well.
For generating the stubs, we can use either protoc or buf. protoc is the more classic generation experience used widely in the industry. Still, it has a pretty steep learning curve. buf is a newer tool built with user experience and speed in mind. It also offers linting and breaking change detection, and something protoc doesn’t provide.
You should check out the tutorial series on gRPC-Gateway, i.e., https://grpc-ecosystem.github.io/grpc-gateway/docs/tutorials/. Also, you can refer to my simple hello world program, which uses gRPC-Gateway, i.e., https://github.com/iamrajiv/helloworld-grpc-gateway.
Goal: I want to reuse many Go functions from two Go functions with HTTP triggers.
What I have tried and steps to reproduce the problem:
In GCP, create a new Go 1.11 Cloud Function, HTTP Trigger
Name it: MyReusableHelloWorld
In function.go, paste this:
package Potatoes
import (
"net/http"
)
// Potatoes return potatoes
func Potatoes(http.ResponseWriter, *http.Request) {
}
In go.mod, paste this: module example.com/foo
In function to execute, paste this: Potatoes
Click on deploy. It works.
Create another Go serverless function in GCP
In function. go, paste this:
// Package p contains an HTTP Cloud Function.
package p
import (
"encoding/json"
"fmt"
"html"
"net/http"
"example.com/foo/Potatoes"
)
// HelloWorld prints the JSON encoded "message" field in the body
// of the request or "Hello, World!" if there isn't one.
func HelloWorld(w http.ResponseWriter, r *http.Request) {
var d struct {
Message string `json:"message"`
}
if err := json.NewDecoder(r.Body).Decode(&d); err != nil {
fmt.Fprint(w, "error here!")
return
}
if d.Message == "" {
fmt.Fprint(w, "oh boy Hello World!")
return
}
fmt.Fprint(w, html.EscapeString(d.Message))
}
In go.mod, paste this: module example.com/foo
In function to execute, paste this: HelloWorld
Click on deploy. It doesn't work. You have the error: unknown import path "example.com/foo/Potatoes": cannot find module providing package example.com/foo/Potatoes
I have also tried all kinds of combinations for the module/packages to import.
I have tried without the example.com/ part.
Other smaller issue:
The functions I want to reuse could all be in the same file and don't really need any trigger, but it doesn't seem that having no trigger is possible.
Related questions and documentation with which I could not achieve my goal:
How can I use a sub-packages with Go on Google Cloud Functions?
https://github.com/golang/go/wiki/Modules , section go.mod
You can’t invoke a cloud function from another one, because each function is in its own container independently.
So If you want to deploy the function with a dependency that can't be downloaded from a package manager you need to put the code together like here and deploy using the CLI
It is likely each cloud function defined in the console is independent of the other. If you want code reuse, it's best to structure it as per the following document and deploy it using gcloud command.
https://cloud.google.com/functions/docs/writing/#structuring_source_code
You are mixing things: package management and function deployment.
When you deploy a Cloud Function, if you want to (re)use it, you have to call if with http package.
If you build a package that you want to include in your source code, you have to rely on package manager. With Go, Git repository, like Github, is the best way to achieve this (Don't forget to perform a release and to name it as expected by Go mod: vX.Y.Z)
Here your code can't work without more engineering and package publication/management.
I achieve the same things but with a Dockerfile and I deplored my code in Cloud Run (that I recommend you if you aren't event oriented and only HTTP oriented. I wrote a comparison on Medium)
Root
go.mod
pkg/foo.go
pkg/go.mod
service/Helloworld.go
service/go.mod
In my helloworld.go, I can reuse the package foo. For this I perform this in my service/go.mod file
module service/helloworld
go 1.12
require pkg/foo v0.0.0
replace pkg/foo v0.0.0 => ../pkg
Then when you build your container, you run your go build service/Helloworld.go from the root directory.
For a contrived example:
I have 2 packages, repo.com/alpha/A & repo.net/beta/B. package A uses package B, both structured as example.
A:
main.go
B:
b.go
templates \
1.tmpl
2.tmpl
In main.go of package A, I'd need to access the templates directory of package B.
b.go
var templates string
templates = templatepath
func init(){
templatepath, _ = filepath.Abs("./templates")
}
main.go
import(
repo.net/beta/B
)
func main(){
fmt.Printf("%s", B.templates)
}
So the problem being in my more complex use case & the contrived example here is that B.templates will be in the directory for package A, where I need to establish and reference the directory of the imported path. This is part of learning and navigating the Go way of doing things, and my understanding is probably basic, so I need to understand how to do this in a Go context.
My use case is a package that uses other packages that do things for the base package, and these external packages may use standard web resources files(css, html, js) the problem being I'm having immediate trouble packaging and referencing them abstractly enough for what I want to do.
You can't, you have to either use something like go-bindata or so, or simply embed the templates in your B package as consts.
tmpl1.go:
const tmpl1 = `........`