I have created a basic service with grpc-gateway. This is what my .proto file looks like
syntax = "proto3";
package health;
option go_package = "github.com/userl/brw/api/health";
import "google/api/annotations.proto";
service Health {
// Sends a greeting
rpc Ping (HealthRequest) returns (HealthReply) {
option (google.api.http) = {
get: "/ping"
};
}
}
// The request message containing the user's name
message HealthRequest {
}
// The response message containing the greetings
message HealthReply {
string message = 1;
}
I generate and start the service with the following code
mux := runtime.NewServeMux()
opts := []grpc.DialOption{grpc.WithInsecure()}
err := pHealth.RegisterHealthHandlerFromEndpoint(ctx, mux, ":3002"), opts)
if err != nil {
log.Fatal(err)
}
http.ListenAndServe(":3001"), mux)
Now when I try to curl command for example on
curl http://localhost:3001/ping I get the following error
curl: (7) Failed to connect to localhost port 3003: Connection refused
When I tried grpcurl
grpcurl --plaintext localhost:3002 list I get the following error
failed to dial target host "localhost:3002": dial tcp 127.0.0.1:3002: connect: connection refused
Not sure what is going wrong here or what I need to do.
Related
I'm using Fiber as an HTTP client to make some requests to an http server, however I'm being rate limited. On my vm I configured 5 different IP addresses (public/private) and have confirmed that they are indeed connected to the internet.
curl --interface 10.0.0.4 ipinfo.io/json
curl --interface 10.0.0.5 ipinfo.io/json
...
curl --interface 10.0.0.8 ipinfo.io/json
each one returns a different public facing ip address.
Now I'm interested in making round-robin requests using these local addresses but I'm not so sure how to go about it.
Is there some sort of property or function I can set/call to change where the outgoing request is coming from?
I've looked around at fasthttp.HostClient which fiber.Agent extends but I didn't see anything useful.
Thanks guys.
a := fiber.AcquireAgent()
req := a.Request()
req.Header.SetMethod(fiber.MethodGet)
req.SetRequestURI(fmt.Sprintf(formatUrl, args...))
if err := a.Parse(); err != nil {
h.Logger.Error("%v", err)
return fiber.StatusInternalServerError, nil, []error{err}
}
customDialer := fasthttp.TCPDialer{
Concurrency: 1000,
LocalAddr: &net.TCPAddr{
IP: h.IPPool[atomic.AddUint32(&h.IpIdx, 1)%uint32(len(h.IPPool))],
},
}
a.HostClient.Dial = func(addr string) (net.Conn, error) {
return customDialer.Dial(addr)
}
Creating a custom dialer and dial func allows you to change the local address associated with the http request.
UPDATE After much debugging, I uncovered Get "https://acme-v02.api.letsencrypt.org/directory": x509: certificate signed by unknown authority and suspect (!?) this results from the recent expiration of Let's Encrypt's root cert.
I accept that "This package is a work in progress and makes no API stability promises." but, if it no longer works (and it's much more likely that my code|deployment is at issue), then perhaps the repo can be marked e.g. Here be dragons.
The code results in an acme_account+key (EC PRIVATE KEY) but no certs I'm challenged to get autocert to disclose (log) its magic in order to understand where I'm going wrong.
The code is essentially the repo's Manager example with input from this answer. I assume that GetCertificate blocks on the completion of the ACME flow.
Code:
package main
import (
"crypto/tls"
"flag"
"fmt"
"log"
"net"
"net/http"
"golang.org/x/crypto/acme/autocert"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/health"
healthpb "google.golang.org/grpc/health/grpc_health_v1"
)
const (
email string = "my#email.com"
var (
host = flag.String("host", "foo.example.org", "Fully-qualified domain name")
port = flag.Uint("port", 443, "gRPC service port")
path = flag.String("path", "", "Folder location for certificate")
)
func main() {
flag.Parse()
if *host == "" {
log.Fatal("Flag --host is required")
}
log.Printf("Host: %s", *host)
log.Printf("Port: %d", *port)
if *path == "" {
log.Fatal("Flag --path is required")
}
log.Printf("Path: %s", *path)
addr := fmt.Sprintf(":%d", *port)
lis, err := net.Listen("tcp", addr)
if err != nil {
log.Fatalf("failed to listen: %s", err)
}
m := &autocert.Manager{
Cache: autocert.DirCache(*path),
Prompt: autocert.AcceptTOS,
HostPolicy: autocert.HostWhitelist(*host),
Email: email,
}
go func() {
log.Println("Starting HTTP server w/ autocert handler")
if err := http.ListenAndServe(":http", m.HTTPHandler(nil)); err != nil {
log.Fatalf("HTTP failure\n%s", err)
}
}()
tlsConfig := &tls.Config{
ClientAuth: tls.RequireAndVerifyClientCert,
GetCertificate: func(hello *tls.ClientHelloInfo) (*tls.Certificate, error) {
cert, err := m.GetCertificate(hello)
if err != nil {
log.Fatalf("GetCertificate\n%s", err)
}
return cert, err
},
}
opts := grpc.Creds(credentials.NewTLS(tlsConfig))
server := grpc.NewServer(opts)
healthcheck := health.NewServer()
healthpb.RegisterHealthServer(server, healthcheck)
log.Println("Starting gRPC server")
if err := server.Serve(lis); err != nil {
log.Fatalf("gRPC failure\n%s", err)
}
}
I'm deploying to a (Google Compute Engine) Container VM, the equivalent docker command is:
docker run \
--name=autocert \
--detach \
--net=host \
--volume=/tmp/certs:/certs \
${IMAGE} \
--host=${HOST} \
--port=${PORT} \
--path=/certs
And container logs:
2021/11/25 17:30:00 Host: [HOST]
2021/11/25 17:30:00 Port: 443
2021/11/25 17:30:00 Path: /certs
2021/11/25 17:30:00 Starting gRPC server
2021/11/25 17:30:00 Starting HTTP server
The host's /tmp/certs directory receives acme_account+key (which I've struggled to find explained by Google) but suspect (!?) is the initial phase of Domain Validation. It contains a private key (BEGIN EC PRIVATE KEY).
Even after some time with the server running, no further files are persisted.
I receive no emails from Let's Encrypt at the configured email address.
Unfortunately, while easy to use, autocert produces little logging and I've been unable to determine whether I can log the ACME flow that's (hopefully) taking place.
Since adding the anonymous function for GetCertificate, acme_account+key is no longer created (I removed the previous file to check whether it's recreated) and so I'm unable to gather any logging from it but the function is never invoked. Either this is because my anonymous function is incorrect or because I've exceeded requests against the ACME endpoint. Removing the function and reverting to m.GetCertificate does not result in recreation of acme_account+key so I'm at a loss.
The autocert Manager type documents an *acme.Client field which I'm not setting. The comment describes "if the Client.Key is nil, a new ECDSA P-256 key is generated" which is perhaps what I'm experiencing but it doesn't explain what I should do about it. Should I set this value to the content of acme_account+key?:
UPDATE I tried decoding the private key, creating a crypto.Signer and passing this in &acme.Client{Key: key} but it made no evident difference
// Client is used to perform low-level operations, such as account registration
// and requesting new certificates.
//
// If Client is nil, a zero-value acme.Client is used with DefaultACMEDirectory
// as the directory endpoint.
// If the Client.Key is nil, a new ECDSA P-256 key is generated and,
// if Cache is not nil, stored in cache.
//
// Mutating the field after the first call of GetCertificate method will have no effect.
Client *acme.Client
Evidently, I'm using this incorrectly. I'm not receiving a cert from Let's Encrypt and so I'm unable to get a cert from the endpoint and unable to invoke the gRPC endpoint:
openssl s_client -showcerts -connect ${HOST}:${PORT}
grpcurl \
-proto health.proto \
${HOST}:${PORT} \
grpc.health.v1.Health/Check
Failed to dial target host "${HOST}:${PORT}": remote error: tls: internal error
Guidance would be appreciated.
🤦♂️
Ugh :-(
I'd started using SCRATCH and hadn't copied the CA certificates
Once the container had CA certs, everything worked almost flawlessly.
I continue to have problems trying to use:
tlsConfig := &tls.Config{
ClientAuth: tls.RequireAndVerifyClientCert,
GetCertificate: m.GetCertificate
}
And am using m.TLSConfig()
So, autocert works like a treat (though it's difficult to debug self-inflicted errors 😊)
I'm using a Go API and I need to connect it to my Cassandra DB on a Google Cloud server.
This is my Go connection code:
func SetupDBConnection() {
cluster := gocql.NewCluster("XX.XXX.XXX.XXX")
cluster.Keyspace = "afterstr"
cluster.Consistency = gocql.Quorum
s, err := cluster.CreateSession()
if err != nil {
fmt.Println(err)
}
}
But, when I run this code, I have this error:
gocql: unable to create session: control: unable to connect to initial hosts: dial tcp xx.xxx.xxx.xxx:9042: i/o timeout
This is my Google Cloud configuration:
I also modified the cassandra.yaml file in my server to change the listen address and the RPC address to put my IP with the port I use to run my Go program that I opened.
(It is opened 3 times because I was just testing something)
Try:
cluster.Authenticator = gocql.PasswordAuthenticator{Username: username, Password: password}
I'm trying to send a custom Proto in Error Response of my grpc calls. As per this, it should be possible.
Here is the code.
st := status.Newf(codes.NotFound, "user %s doesn't exist", req.Name)
desc := "The username Doesn't exist, please give a valid username"
customProtoError := &data.Error{
Message: "Check username",
Code: 1404,
Type: "Service",
DetailedMessage: desc,
}
st, err := st.WithDetails(customProtoError)
if err != nil {
panic(fmt.Sprintf("Unexpected error attaching metadata: %v", err))
}
I've both grpc & http Handlers for these GRPC APIs.
The Marshaler used in HTTP server is github.com/gogo/gateway
mux := runtime.NewServeMux(
runtime.WithMarshalerOption(JSONContentType, &gateway.JSONPb{}))
The configuration for HTTP and GRPC server is available here.
When I try to access the API using HTTP calls, here is what I get in response.
HTTP Error:
{
"error": "user kishore1 doesn't exist",
"code": 5,
"message": "user kishore1 doesn't exist",
"details": [
{
"type_url": "type.googleapis.com/data.Error",
"value": "Cg5DaGVjayB1c2VybmFtZRD8ChoHU2t5ZmxvdyI4VGhlIHVzZXJuYW1lIERvZXNuJ3QgZXhpc3QsIHBsZWFzZSBnaXZlIGEgdmFsaWQgdXNlcm5hbWU="
}
]
}
The type_url is coming with google, as its hardcoded in the golang proto code. But this says Any in JSON will be deserialized to embedded messages.
Grpc Client Code:
resp, err := client.GetUser(ctx, req)
if err != nil {
st := status.Convert(err)
for _, detail := range st.Details() {
switch t := detail.(type) {
case *pb.Error:
fmt.Println("Oops! Your request was rejected by the server.")
...
default:
fmt.Println("Error Received - ", detail, reflect.TypeOf(detail))
}
}
Grpc Error:
Error Received - proto: not found *errors.prefixError
Any idea what could be causing this mess? HTTP doesn't deserialize as an embedded message and GRPC gives proto.NotFound
Could this be due to gogo proto being the gateway? I even tried the same with googleapis proto errdetails.BadRequest but still the same.
After spending about a day, I just figured out the issue was due to incompatible versions.
So here is what worked for me, when using gogo proto.
For the GRPC errors, I had to upgrade my grpc-gateway version and use gogo/status instead of grpc/status
github.com/gogo/status v1.1.0
github.com/grpc-ecosystem/grpc-gateway v1.14.5
For HTTP errors, I had to downgrade my golang/protobuf
github.com/golang/protobuf v1.3.2
I am trying to get DNS requests to tunnel through my company's proxy by using the HTTP CONNECT method.
The library I am using to make the DNS requests supports setting a custom net.Dialer on an instance of dns.Client. I found another library that claims to act as a drop-in replacement for net.Dialer that supports initiating TCP connections through a proxy using the HTTP CONNECT method.
However, I can't figure out how to get this to work with these two libraries.
I have tried setting the Dialer field of dns.Client, but it complains about incompatible types:
client := &dns.Client{
Net: "tcp",
Timeout: time.Duration(forwarder.Timeout) * time.Second,
}
if forwarder.Proxy != nil {
client.Dialer = http_dialer.New(forwarder.Proxy, http_dialer.WithDialer(client.Dialer))
}
Yields:
cannot use http_dialer.New(forwarder.Proxy, http_dialer.WithDialer(client.Dialer)) (type *http_dialer.HttpTunnel) as type *net.Dialer in assignment
So I tried casting it:
client := &dns.Client{
Net: "tcp",
Timeout: time.Duration(forwarder.Timeout) * time.Second,
}
if forwarder.Proxy != nil {
client.Dialer = net.Dialer(*http_dialer.New(forwarder.Proxy, http_dialer.WithDialer(client.Dialer)))
}
But that yields:
cannot convert *http_dialer.New(forwarder.Proxy, http_dialer.WithDialer(client.Dialer)) (type http_dialer.HttpTunnel) to type net.Dialer
Finally, I tried setting the Dial function of dns.Client.Dialer to the Dial function in the http_dialer.HttpTunnel returned by http_dialer#New:
client := &dns.Client{
Net: "tcp",
Timeout: time.Duration(forwarder.Timeout) * time.Second,
}
if forwarder.Proxy != nil {
client.Dialer.Dial = http_dialer.New(forwarder.Proxy, http_dialer.WithDialer(client.Dialer)).Dial
}
But that yielded:
cannot assign to client.Dialer.Dial
So I how do I set the Dialer of my DNS client?