i have a question. Is it possible to extract via metadata path from user request.
Here i have my proto file with defined method.
rpc AllPath(google.protobuf.Empty) returns (google.protobuf.Empty) {
option (google.api.http) = {
get: "/*",
};
}
rpc Auth(google.protobuf.Empty) returns (TokenRender) {
option (google.api.http) = {
get: "/auth"
};
}
}
In AllPath function in my server file im using something like this, found on grpc-gateway ecosystem website.
path := make(map[string]string)
if pattern, ok := runtime.HTTPPathPattern(ctx); ok {
path["pattern"] = pattern // /v1/example/login
}
fmt.Printf("Current path is: %v", path["pattern"])
but my current pattern/path is like i defined in proto file: Current path is: /*
If anyone have idea how to deal with this thing i would appreciate it :)
Best, Kacper
gRPC-Gateway passes various bits of information from the originating HTTP request via gRPC metadata. I don't believe the raw path is supplied, however. It is still possible to get the path passed through by registering a metadata annotator.
When calling github.com/grpc-ecosystem/grpc-gateway/v2/runtime.NewServeMux(), leverage the WithMetadata option func:
mux := runtime.NewServeMux(runtime.WithMetadata(func(_ context.Context, req *http.Request) metadata.MD {
return metadata.New(map[string]string{
"grpcgateway-http-path": req.URL.Path,
})
}))
Then in your gRPC service implementation, you can retrieve the value via the incoming context:
func (s *server) AllPath(ctx context.Context, _ *emptypb.Empty) (*emptypb.Empty, error) {
md, _ := metadata.FromIncomingContext(ctx)
log.Printf("path: %s", md["grpcgateway-http-path"][0])
return &emptypb.Empty{}, nil
}
When hitting, e.g. /foo, this should log:
2022/10/25 15:31:42 path: /foo
Related
I am using oapi-codegen to generate my server code and Echo Labstack as the server.
When I pass a Group instance to Openapi.RegisterHandlers instead of an Echo instance, I always get a 400 error with {"message":"no matching operation was found"} for any request in that group:
swagger, err := Openapi.GetSwagger()
if err != nil {
fmt.Fprintf(os.Stderr, "Error loading swagger spec\n: %s", err)
os.Exit(1)
}
// Use oapi validation middleware to check all requests against the
// OpenAPI schema.
g := e.Group("/api", middleware.OapiRequestValidator(swagger))
Openapi.RegisterHandlers(g, &MyApi{})
If send request /api/foo, where foo is an API endpoint defined in the generated server code, I get a 400 error. If I do /api/<some undefined api> I also get 400. If I do send a request for /baz, I get 404 as expected, since that isn't a defined route. If I don't pass a prefix to Group(), I get a 400 error for every request. I get the same behavior if I use RegisterHandlersWithBaseURL()
There seems to be a bug where if you specify the a base path, either to the Group() function or to RegisterHandlersWithBaseURL(), theOapiRequestValidator middle ignores the base path when checking the request path against the routes. It uses the routes defined in the OpenAPI spec without the base path. To work around this, I overwrote the inline.tmpl template and hacked the GetSwagger() function to include this at the bottom:
func GetSwagger(pathPrefix string) (swagger *openapi3.T, err error) {
...
var updatedPaths openapi3.Paths = make(openapi3.Paths)
for key, value := range(swagger.Paths) {
updatedPaths[pathPrefix + key] = value
}
swagger.Paths = updatedPaths
}
The key in the Path map is the route. I just append the base path to every key.
I have a GRPC server with APIs that are authorized like:
func (s *MyServer) MyAPI(ctx context.Context, req MyAPIRequest) (MyAPIResponse, error) {
isAuthorized, err = s.IsAuthorized(ctx, req.UserId, Role.User) // other APIs may use a different authorization function than IsAuthorized
if err != nil {
return nil, err
}
if !isAuthorized {
return nil, status.Error(codes.PermissionDenied, "not authorized")
}
// rest of API code
}
I'd like to know how to:
Simplify usage of the authorization logic, like an annotation. I'm more familiar with Java where it would be like #Authorize(ctx = ctx, req = req, role = Role.User) above the func.
Require authorization checks for APIs, so that builds fail if at least one API is missing authorization. I'm using bazel. Note that not all func (s *MyServer) will be APIs.
Here's my idea:
.
Create a YAML file with key value pairs of method name to authorization rule. Example is MyAPI: IsUserIdAuthorizedAsUser which would translate to s.IsAuthorized(ctx, req.UserId, Role.User).
Create an interceptor that looks up authz rule for the request's method name and calls the corresponding authz function.
Have a bazel build rule that parses proto files for rpc, which are all the API method names, and fails if not all of them are in the rule list. I don't know how to do this yet.
Would appreciate any suggestions on my idea or better ways.
I would use an interceptor (grpc.UnaryInterceptor) to handle the authentication / authorization process. It is similar to the classic spring filter (java world).
You can read about interceptors here: https://shijuvar.medium.com/writing-grpc-interceptors-in-go-bf3e7671fe48
You can easily chain multi interceptors or having interceptor per grpc method.
This below is an interceptor that I wrote some months ago (it uses JWT as auth mechanism). You can use it as example:
func (jwt JwtInterceptor) Interceptor(
ctx context.Context,
req interface{},
info *grpc.UnaryServerInfo,
handler grpc.UnaryHandler) (interface{}, error) {
md, _ := metadata.FromIncomingContext(ctx)
token := md["jwt"]
if token == nil {
return nil, errors.New("token not present")
}
apiClaims, err := jwt.decoder.Parse(token[0])
if err != nil {
return nil, errors.New("token signature not valid")
}
return handler(context.WithValue(ctx, "jwt", apiClaims), req)
}
I have been looking for an example GAE script in go to get my image that I got from the resulted screenshot of PageSpeed Insights and saved it as json_decode object using Kohana/Cache to Google Cloud Storage (GCS).
The reason of this method is simply because I found this Kohana model is the most convenient way writing files to GCS, although I am seeking also other way like this to write files to GCS using Blobstore to serve them while the Go API Files has been deprecate as documented here.
Here is the form of stored object containing the screenshot image data (base64) which is saved as public in default application bucket with object name images/thumb/mythumb.jpg:
stdClass Object
(
[screenshot] => stdClass Object
(
[data] => _9j_4AAQSkZJRgABAQAAAQABAAD_...= // base64 data
[height] => 240
[mime_type] => image/jpeg
[width] => 320
)
[otherdata] => Array
(
[..] => ..
[..] => ..
)
)
I want to get this image that set as public using my customized url as below that to be proceed through go module and also I need it to be expired in a certain time because I have managed to update the image content itself regularly:
http://myappId.appspot.com/image/thumb/mythumb.jpg
I have set in disptach.yaml to send all image request to my go module as below:
- url: "*/images/*"
module: go
and set the handler in go.yaml to proceed the image request as below:
handlers:
- url: /images/thumb/.*
script: _go_app
- url: /images
static_dir: images
Using this directive I have got that all /images/ request (other than /images/thumb/ request) serve images from the static directory and that /images/thumb/mythumb.jpg goes to the module application.
So left what code I have to use (see ????) in my application file named thumb.go as below:
package thumb
import(
//what to import
????
????
)
const (
googleAccessID = "<serviceAccountEmail>#developer.gserviceaccount.com"
serviceAccountPEMFilename = "YOUR_SERVICE_ACCOUNT_KEY.pem"
bucket = "myappId.appspot.com"
)
var (
expiration = time.Now().Add(time.Second * 60) //expire in 60 seconds
)
func init() {
http.HandleFunc("/images/thumb/", handleThumb)
}
func handleThumb(w http.ResponseWriter, r *http.Request) {
ctx := cloud.NewContext(appengine.AppID(c), hc)
???? //what code to get the string of 'mythumb.jpg' from url
???? //what code to get the image stored data from GCS
???? //what code to encoce base64 data
w.Header().Set("Content-Type", "image/jpeg;")
fmt.Fprintf(w, "%v", mythumb.jpg)
}
I have taken many codes from some examples like this, this or this but could not get one works so far. I have also tried a sample from this which is almost close to my case but also found no luck.
So in generally t was mainly due to lack on what are the correct code to be put on the line that I marked by ???? as well the relevant library or path to be imported. I have also checked the GCS permission if something have been missing as described here and here.
I shall thank you much for your help and advise.
From what I've read in your description, it seems that the only relevant parts are the ???? lines in the actual Go code. Let me know if that's not the case.
First ????: "what code to get the string of 'mythumb.jpg' from url"?
From reading the code, you're looking to extract mythumb.jpg from a url like http://localhost/images/thumb/mythumb.jpg. A working example is available at the Writing Web Applications tutorial:
package main
import (
"fmt"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
Such that
http://localhost:8080/monkeys
Prints
Hi there, I love monkeys!
Second ????: "what code to get the image stored data from GCS"?
The API method you're probably looking to use is storage.objects.get.
You did link to one of the JSON API Go Examples for Google Cloud Storage, which is a good general reference, but is not related to the problem you're trying to solve. That particular example is put together for Client-side applications (hence the redirectURL = "urn:ietf:wg:oauth:2.0:oob" line). Additionally, this sample uses deprecated/out-of-date oauth2 and storage packages.
One of the cleanest (and non-deprecated) ways to do this for an application which wants to access its own buckets on behalf of itself would be to use the golang/oauth2 and Google APIs Client Library for Go packages.
An example of how to authenticate with JSON Web Token auth with the golang/oauth2 package is available in the repo:
func ExampleJWTConfig() {
conf := &jwt.Config{
Email: "xxx#developer.com",
// The contents of your RSA private key or your PEM file
// that contains a private key.
// If you have a p12 file instead, you
// can use `openssl` to export the private key into a pem file.
//
// $ openssl pkcs12 -in key.p12 -out key.pem -nodes
//
// It only supports PEM containers with no passphrase.
PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
Subject: "user#example.com",
TokenURL: "https://provider.com/o/oauth2/token",
}
// Initiate an http.Client, the following GET request will be
// authorized and authenticated on the behalf of user#example.com.
client := conf.Client(oauth2.NoContext)
client.Get("...")
}
Next, instead of using the oauth2 client directly, use that client with the Google APIs Client Library for Go mentioned earlier:
service, err := storage.New(client)
if err != nil {
fatalf(service, "Failed to create service %v", err)
}
Notice the similarity to the out-of-date JSON API Go Examples?
In your handler, you'll want to go out and get the related object using func ObjectsService.Get. Assuming that you know the name of the object and bucket, that is.
Straight from the previous example, you can use code similar to what's below to retrieve the download link:
if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
} else {
fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}
Then, fetch the file, or do whatever you want with it. Full example:
import (
"golang.org/x/oauth2"
"golang.org/x/oauth2/jwt"
"google.golang.org/api/storage/v1"
"fmt"
)
...
const (
bucketName = "YOUR_BUCKET_NAME"
objectName = "mythumb.jpg"
)
func main() {
conf := &jwt.Config{
Email: "xxx#developer.com",
PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
Subject: "user#example.com",
TokenURL: "https://provider.com/o/oauth2/token",
}
client := conf.Client(oauth2.NoContext)
service, err := storage.New(client)
if err != nil {
fatalf(service, "Failed to create service %v", err)
}
if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
} else {
fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}
// Go fetch the file, etc.
}
Third ????: "what code to encoce base64 data"?
Pretty simple with the encoding/base64 package. SO simple, that they've included an example:
package main
import (
"encoding/base64"
"fmt"
)
func main() {
data := []byte("any + old & data")
str := base64.StdEncoding.EncodeToString(data)
fmt.Println(str)
}
Hope that helps.
I'm trying to write a twitter reader that resolves the final URLs of link shorteners etc, but gives me a URL along the way for a list of manually defined host patterns. The reason to do this is that i don't want to end up with the paywall URL but the one before.
As far as i can tell the way to do this is write my own client based on the default RoundTripper because returning an error from a custom CheckRedirect function aborts the client without yielding a response.
Is there a way to use the default client and record a list of URLs/specific URL from a custom checkRedirect function?
The client request will actually still return the last valid Response in cases where your custom CheckResponse yields an error (As mentioned in the comments).
http://golang.org/pkg/net/http/#Client
If CheckRedirect returns an error, the Client's Get method returns both the previous Response and CheckRedirect's error (wrapped in a url.Error) instead of issuing the Request req.
If you maintain a list of "known" paywall-urls, you can abort the paywall-redirect in your CheckResponse with a custom error type (Paywalled in the example below).
Your error handling code later has to consider that error type as a special (non-erroneous) case.
Example:
package main
import (
"errors"
"fmt"
"net/http"
"net/url"
)
var Paywalled = errors.New("next redirect would hit a paywall")
var badHosts = map[string]error{
"registration.ft.com": Paywalled,
}
var client = &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
// N.B.: when used in production, also check for redirect loops
return badHosts[req.URL.Host]
},
}
func main() {
resp, err := client.Get("http://on.ft.com/14pQBYE")
// ignore non-nil err if it's a `Paywalled` wrapped in url.Error
if e, ok := err.(*url.Error); (ok && e.Err != Paywalled) || (!ok && err != nil) {
fmt.Println("error: ", err)
return
}
resp.Body.Close()
fmt.Println(resp.Request.URL)
}
I'm investigating using the gorilla web toolkit to create a simple RPC API. I'm using the example from their documentation and I'm testing using Advanced Rest Client in Chrome and use
http://localhost:1111/api/
and POST the following RAW JSON payload:
{"method":"HelloService.Say","params":[{"Who":"Test"}]}
This reaches the server, I know this as I'm logging it (see code below) and I get a 200 OK response. However I'm getting "Response does not contain any data"
I'm expecting the JSON reply message that is defined in the Say method below. Does anyone have any suggestions as to what the problem is?
package main
import (
"gorilla/mux"
"gorilla/rpc"
"gorilla/rpc/json"
"log"
"net/http"
)
type HelloArgs struct {
Who string
}
type HelloReply struct {
Message string
}
type HelloService struct{}
func (h *HelloService) Say(r *http.Request, args *HelloArgs, reply *HelloReply) error {
log.Printf(args.Who)
reply.Message = "Hello, " + args.Who + "!"
log.Printf(reply.Message)
return nil
}
func main() {
r := mux.NewRouter()
jsonRPC := rpc.NewServer()
jsonCodec := json.NewCodec()
jsonRPC.RegisterCodec(jsonCodec, "application/json")
jsonRPC.RegisterCodec(jsonCodec, "application/json; charset=UTF-8") // For firefox 11 and other browsers which append the charset=UTF-8
jsonRPC.RegisterService(new(HelloService), "")
r.Handle("/api/", jsonRPC)
http.ListenAndServe(":1111", r)
}
It's because gorilla/rpc/json implements JSON-RPC, which requires three parameters in the request: method, params and id.
Requests without id in JSON-RPC are called notifications and do not have responses.
Check specification for more details.
So, in your case, you need to use following JSON:
{"method":"HelloService.Say","params":[{"Who":"Test"}], "id":"1"}