I am processing a GET request in Go using fasthttp.
The query parameter test in this request is .%2A%2Ftoday%2F.%2A.
I am using POSTMAN to create the request, and the URL generated is:
http://localhost:3000/apiname/?test=.%252A%252Ftoday%252F.%252A
ctx.QueryArgs().Peek("test") gives me .*/today/.* instead of the original .%2A%2Ftoday%2F.%2A
I know I cannot partially encode/decode the request URL. Is there any way to get the original param as is?
Are you sure? I've just tested it and I'm getting the result you want.
This is the minimal working example:
package main
import (
"fmt"
"log"
"github.com/fasthttp/router"
"github.com/valyala/fasthttp"
)
func Test(ctx *fasthttp.RequestCtx) {
ctx.Response.SetBodyString(string(ctx.QueryArgs().Peek("test")))
fmt.Println(string(ctx.QueryArgs().Peek("test")))
}
func main() {
r := router.New()
r.GET("/test", Test)
log.Fatal(fasthttp.ListenAndServe("127.0.0.1:8080", r.Handler))
}
This is the command line output after the GET request:
$ go run main.go
.%2A%2Ftoday%2F.%2A
and this is the response in Postman:
Related
I'm creating an endpoint using Go's Gin web framework. I need full server URL in my handler function. For example, if server is running on http://localhost:8080 and my endpoint is /foo then I need http://localhost:8080/foo when my handler is called.
If anyone is familiar with Python's fast API, the Request object has a method url_for(<endpoint_name>) which has the exact same functionality: https://stackoverflow.com/a/63682957/5353128
In Go, I've tried accessing context.FullPath() but that only returns my endpoint /foo and not the full URL. Other than this, I can't find appropriate method in docs: https://pkg.go.dev/github.com/gin-gonic/gin#Context
So is this possible via gin.Context object itself or are there other ways as well? I'm completely new to Go.
c.Request.Host+c.Request.URL.Path should work but the scheme has to be determined.
package main
import (
"fmt"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
r.GET("/foo", func(c *gin.Context) {
fmt.Println("The URL: ", c.Request.Host+c.Request.URL.Path)
})
r.Run(":8080")
}
You can determine scheme which also you may know already. But you can check as follows:
scheme := "http"
if c.Request.TLS != nil {
scheme = "https"
}
If your server is behind the proxy, you can get the scheme by c.Request.Header.Get("X-Forwarded-Proto")
You can get host part localhost:8080 from context.Request.Host and path part /foo from context.Request.URL.String().
package main
import (
"fmt"
"net/http"
"github.com/gin-gonic/gin"
)
func main() {
r := gin.Default()
r.GET("/foo", func(c *gin.Context) {
c.String(http.StatusOK, "bar")
fmt.Println(c.Request.Host+c.Request.URL.String())
})
// Listen and Server in 0.0.0.0:8080
r.Run(":8080")
}
And you can get http protocol version by context.Request.Proto, But it will not determine http or https. you need to get it from your service specifications.
I'm trying to build a Go package with the build flag -buildmode=c-shared. I'm expecting to get two files myfile.so and myfile.h. However, I'm only getting the .so file. Why is this and how can I fix it?
The full command I am running is:
go build -o myfile.so -buildmode=c-shared myfile.go
I found my "instructions" here as I am planning on calling myfile from Python.
This is my Go code:
package main
import (
"C"
"bytes"
"log"
"encoding/json"
"net/http"
)
func call_request(arg1, arg2, arg3 string) {
// simple golang code to submit a http post request
}
func main () {
}
This is a basic summary of my code, without posting my whole code. However, it may be useful to note that running the example in the link above created a .so and .h file.
As #JimB said, the issue was there was not a header file:
Updated code:
package main
import (
"C"
"bytes"
"log"
"encoding/json"
"net/http"
)
//export call_request
func call_request(arg1, arg2, arg3 string) {
// simple golang code to submit a http post request
}
func main () {
}
I have been looking for an example GAE script in go to get my image that I got from the resulted screenshot of PageSpeed Insights and saved it as json_decode object using Kohana/Cache to Google Cloud Storage (GCS).
The reason of this method is simply because I found this Kohana model is the most convenient way writing files to GCS, although I am seeking also other way like this to write files to GCS using Blobstore to serve them while the Go API Files has been deprecate as documented here.
Here is the form of stored object containing the screenshot image data (base64) which is saved as public in default application bucket with object name images/thumb/mythumb.jpg:
stdClass Object
(
[screenshot] => stdClass Object
(
[data] => _9j_4AAQSkZJRgABAQAAAQABAAD_...= // base64 data
[height] => 240
[mime_type] => image/jpeg
[width] => 320
)
[otherdata] => Array
(
[..] => ..
[..] => ..
)
)
I want to get this image that set as public using my customized url as below that to be proceed through go module and also I need it to be expired in a certain time because I have managed to update the image content itself regularly:
http://myappId.appspot.com/image/thumb/mythumb.jpg
I have set in disptach.yaml to send all image request to my go module as below:
- url: "*/images/*"
module: go
and set the handler in go.yaml to proceed the image request as below:
handlers:
- url: /images/thumb/.*
script: _go_app
- url: /images
static_dir: images
Using this directive I have got that all /images/ request (other than /images/thumb/ request) serve images from the static directory and that /images/thumb/mythumb.jpg goes to the module application.
So left what code I have to use (see ????) in my application file named thumb.go as below:
package thumb
import(
//what to import
????
????
)
const (
googleAccessID = "<serviceAccountEmail>#developer.gserviceaccount.com"
serviceAccountPEMFilename = "YOUR_SERVICE_ACCOUNT_KEY.pem"
bucket = "myappId.appspot.com"
)
var (
expiration = time.Now().Add(time.Second * 60) //expire in 60 seconds
)
func init() {
http.HandleFunc("/images/thumb/", handleThumb)
}
func handleThumb(w http.ResponseWriter, r *http.Request) {
ctx := cloud.NewContext(appengine.AppID(c), hc)
???? //what code to get the string of 'mythumb.jpg' from url
???? //what code to get the image stored data from GCS
???? //what code to encoce base64 data
w.Header().Set("Content-Type", "image/jpeg;")
fmt.Fprintf(w, "%v", mythumb.jpg)
}
I have taken many codes from some examples like this, this or this but could not get one works so far. I have also tried a sample from this which is almost close to my case but also found no luck.
So in generally t was mainly due to lack on what are the correct code to be put on the line that I marked by ???? as well the relevant library or path to be imported. I have also checked the GCS permission if something have been missing as described here and here.
I shall thank you much for your help and advise.
From what I've read in your description, it seems that the only relevant parts are the ???? lines in the actual Go code. Let me know if that's not the case.
First ????: "what code to get the string of 'mythumb.jpg' from url"?
From reading the code, you're looking to extract mythumb.jpg from a url like http://localhost/images/thumb/mythumb.jpg. A working example is available at the Writing Web Applications tutorial:
package main
import (
"fmt"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
Such that
http://localhost:8080/monkeys
Prints
Hi there, I love monkeys!
Second ????: "what code to get the image stored data from GCS"?
The API method you're probably looking to use is storage.objects.get.
You did link to one of the JSON API Go Examples for Google Cloud Storage, which is a good general reference, but is not related to the problem you're trying to solve. That particular example is put together for Client-side applications (hence the redirectURL = "urn:ietf:wg:oauth:2.0:oob" line). Additionally, this sample uses deprecated/out-of-date oauth2 and storage packages.
One of the cleanest (and non-deprecated) ways to do this for an application which wants to access its own buckets on behalf of itself would be to use the golang/oauth2 and Google APIs Client Library for Go packages.
An example of how to authenticate with JSON Web Token auth with the golang/oauth2 package is available in the repo:
func ExampleJWTConfig() {
conf := &jwt.Config{
Email: "xxx#developer.com",
// The contents of your RSA private key or your PEM file
// that contains a private key.
// If you have a p12 file instead, you
// can use `openssl` to export the private key into a pem file.
//
// $ openssl pkcs12 -in key.p12 -out key.pem -nodes
//
// It only supports PEM containers with no passphrase.
PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
Subject: "user#example.com",
TokenURL: "https://provider.com/o/oauth2/token",
}
// Initiate an http.Client, the following GET request will be
// authorized and authenticated on the behalf of user#example.com.
client := conf.Client(oauth2.NoContext)
client.Get("...")
}
Next, instead of using the oauth2 client directly, use that client with the Google APIs Client Library for Go mentioned earlier:
service, err := storage.New(client)
if err != nil {
fatalf(service, "Failed to create service %v", err)
}
Notice the similarity to the out-of-date JSON API Go Examples?
In your handler, you'll want to go out and get the related object using func ObjectsService.Get. Assuming that you know the name of the object and bucket, that is.
Straight from the previous example, you can use code similar to what's below to retrieve the download link:
if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
} else {
fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}
Then, fetch the file, or do whatever you want with it. Full example:
import (
"golang.org/x/oauth2"
"golang.org/x/oauth2/jwt"
"google.golang.org/api/storage/v1"
"fmt"
)
...
const (
bucketName = "YOUR_BUCKET_NAME"
objectName = "mythumb.jpg"
)
func main() {
conf := &jwt.Config{
Email: "xxx#developer.com",
PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
Subject: "user#example.com",
TokenURL: "https://provider.com/o/oauth2/token",
}
client := conf.Client(oauth2.NoContext)
service, err := storage.New(client)
if err != nil {
fatalf(service, "Failed to create service %v", err)
}
if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
} else {
fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}
// Go fetch the file, etc.
}
Third ????: "what code to encoce base64 data"?
Pretty simple with the encoding/base64 package. SO simple, that they've included an example:
package main
import (
"encoding/base64"
"fmt"
)
func main() {
data := []byte("any + old & data")
str := base64.StdEncoding.EncodeToString(data)
fmt.Println(str)
}
Hope that helps.
I'm trying to write a twitter reader that resolves the final URLs of link shorteners etc, but gives me a URL along the way for a list of manually defined host patterns. The reason to do this is that i don't want to end up with the paywall URL but the one before.
As far as i can tell the way to do this is write my own client based on the default RoundTripper because returning an error from a custom CheckRedirect function aborts the client without yielding a response.
Is there a way to use the default client and record a list of URLs/specific URL from a custom checkRedirect function?
The client request will actually still return the last valid Response in cases where your custom CheckResponse yields an error (As mentioned in the comments).
http://golang.org/pkg/net/http/#Client
If CheckRedirect returns an error, the Client's Get method returns both the previous Response and CheckRedirect's error (wrapped in a url.Error) instead of issuing the Request req.
If you maintain a list of "known" paywall-urls, you can abort the paywall-redirect in your CheckResponse with a custom error type (Paywalled in the example below).
Your error handling code later has to consider that error type as a special (non-erroneous) case.
Example:
package main
import (
"errors"
"fmt"
"net/http"
"net/url"
)
var Paywalled = errors.New("next redirect would hit a paywall")
var badHosts = map[string]error{
"registration.ft.com": Paywalled,
}
var client = &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
// N.B.: when used in production, also check for redirect loops
return badHosts[req.URL.Host]
},
}
func main() {
resp, err := client.Get("http://on.ft.com/14pQBYE")
// ignore non-nil err if it's a `Paywalled` wrapped in url.Error
if e, ok := err.(*url.Error); (ok && e.Err != Paywalled) || (!ok && err != nil) {
fmt.Println("error: ", err)
return
}
resp.Body.Close()
fmt.Println(resp.Request.URL)
}
I'm investigating using the gorilla web toolkit to create a simple RPC API. I'm using the example from their documentation and I'm testing using Advanced Rest Client in Chrome and use
http://localhost:1111/api/
and POST the following RAW JSON payload:
{"method":"HelloService.Say","params":[{"Who":"Test"}]}
This reaches the server, I know this as I'm logging it (see code below) and I get a 200 OK response. However I'm getting "Response does not contain any data"
I'm expecting the JSON reply message that is defined in the Say method below. Does anyone have any suggestions as to what the problem is?
package main
import (
"gorilla/mux"
"gorilla/rpc"
"gorilla/rpc/json"
"log"
"net/http"
)
type HelloArgs struct {
Who string
}
type HelloReply struct {
Message string
}
type HelloService struct{}
func (h *HelloService) Say(r *http.Request, args *HelloArgs, reply *HelloReply) error {
log.Printf(args.Who)
reply.Message = "Hello, " + args.Who + "!"
log.Printf(reply.Message)
return nil
}
func main() {
r := mux.NewRouter()
jsonRPC := rpc.NewServer()
jsonCodec := json.NewCodec()
jsonRPC.RegisterCodec(jsonCodec, "application/json")
jsonRPC.RegisterCodec(jsonCodec, "application/json; charset=UTF-8") // For firefox 11 and other browsers which append the charset=UTF-8
jsonRPC.RegisterService(new(HelloService), "")
r.Handle("/api/", jsonRPC)
http.ListenAndServe(":1111", r)
}
It's because gorilla/rpc/json implements JSON-RPC, which requires three parameters in the request: method, params and id.
Requests without id in JSON-RPC are called notifications and do not have responses.
Check specification for more details.
So, in your case, you need to use following JSON:
{"method":"HelloService.Say","params":[{"Who":"Test"}], "id":"1"}