I'm interfacing from my code certain 3rd party software (particularly chirpstack v4) which provides API via GPRC. When deployed locally, it answers via HTTP and I get the response all right.
However in cluster we have the same server deployed with HTTPS (with letsencrypt certificate, not something private) so I'm trying to add corresponding transport layer security settings, but to my surprise I got then
rpc error: code = Internal desc = unexpected HTTP status code received from server: 400 (Bad Request); malformed header: missing HTTP content-type
I tried adding SetHeader to context with content-type: application/grpc but this won't change anything so I'm not sure it is really about header (moreover that it works with plain HTTP). So I wonder, perhaps anyone can point me some mistake in initialization of transport layer security?
// this is used with plain HTTP
//opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
// this for case of working via HTTPS
opts := []grpc.DialOption{grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{}))}
// also tried NewClientTLSFromCert(x509.SystemCertPool()) - the same
serverAddr := "our-public-address:443"
ctx := context.Background()
conn, err := grpc.Dial(serverAddr, opts...)
if err != nil {
println("Dial error:", err.Error())
return
}
cli := api.NewInternalServiceClient(conn)
req := &api.LoginRequest{Email: "admin", Password: "admin"}
resp, err := cli.Login(ctx, req) // error arises here
if err != nil {
println("login failed", err.Error())
}
Thanks in advance for hints and suggestions!
Related
I connect a client (or a couple of clients) to the websockets endpoint in API Gateway.
Then, I try to post a message back to the client using these guidelines: https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-connections.html
// Send sends a message to a connection ID.
func Send(domain, stage, connectionID, message string) (events.APIGatewayProxyResponse, error) {
session := session.Must(session.NewSession())
endpoint := fmt.Sprintf("https://%s/%s/#connections/%s", domain, stage, connectionID)
apiClient := apigatewaymanagementapi.New(session, aws.NewConfig().WithEndpoint(endpoint))
connectionInput := apigatewaymanagementapi.PostToConnectionInput{
ConnectionId: aws.String(connectionID),
Data: []byte(message),
}
_, err := apiClient.PostToConnection(&connectionInput)
if err != nil {
fmt.Println(err.Error())
return events.APIGatewayProxyResponse{StatusCode: 500}, err
}
return events.APIGatewayProxyResponse{StatusCode: 200}, nil
}
It doesn't matter whether I invoke the Send function locally or a client sends a message and API Gateway invokes my publish Lambda where I loop through the connections and invoke Send for each of them.
The result is always the same.
NotFoundException:
status code: 404, request id: 7bb1546a-c2a7-4e98-92a0-fcc7ae175d7c
Things I've tried:
Escaped #connections and the actual connectionID
Made sure the client connection hasn't timed out
Made sure I have the correct AWS credentials in my environment variables
Made sure my Lambda has permissions to invoke API Gateway
Made sure the endpoint is in the correct format: https://{api-id}.execute-api.{region}.amazonaws.com/{stage}/#connections/{connection_id}
How can I successfully send messages to the clients?
Turns out this line
endpoint := fmt.Sprintf("https://%s/%s/#connections/%s", domain, stage, connectionID)
needs to turn into this
endpoint := fmt.Sprintf("https://%s/%s/", domain, stage)
I am trying to route my requests through a proxy and also sending cert.pem in TLS config. Below code is throwing this error - proxyconnect tcp: tls: first record does not look like a TLS handshake. When I change the proxy URL from https to HTTP, the same code works. However proxy URL with https works in python. Below is my code so far
certs := x509.NewCertPool()
pemFile, err := ioutil.ReadFile("cert.pem")
if err != nil {
return
}
certs.AppendCertsFromPEM(pemFile)
tlsConfig := &tls.Config{
RootCAs: certs,
}
proxyUrl, err := url.Parse("https://someproxyurlhere.com:8080")
if err != nil {
return
}
t := &http.Transport{
TLSClientConfig: tlsConfig,
Proxy: http.ProxyURL(proxyUrl),
}
client := http.Client{
Transport: t,
}
reqBody := "some JSON body here"
buff, err := json.Marshal(reqBody)
if err != nil {
return
}
req, err := http.NewRequest(http.MethodPost, "https://someurlhere.com", bytes.NewBuffer(buff))
if err != nil {
return
}
res, err := client.Do(req)
if err != nil {
// Error here - proxyconnect tcp: tls: first record does not look like a TLS handshake
return
}
defer res.Body.Close()
Python code
import requests
os.environ['HTTPS_PROXY'] = 'https://someproxyurlhere.com:8080'
response = requests.post("https://someurlhere.com",
json={'key': 'value'},
verify='cert.pem')
print(str(response.content))
When I change the proxy URL from https to HTTP, the same code works.
This is because you are using a HTTP proxy which need to be accessed by HTTP and not HTTPS even for https://.. URLs. This is how HTTPS proxying usually works. Note that while in theory a proxy could sniff if it gets a plain or a TLS connection in practice proxies (and servers) use different ports for HTTP and HTTPS - thus if it works on one port with HTTP it will very likely not work on the same port with HTTPS.
proxyconnect tcp: tls: first record does not look like a TLS handshake.
This is because the proxy answers with an plain HTTP error to the strange HTTP request (which is actually the start of the TLS handshake).
However proxy URL with https works in python.
While it looks that it works it actually doesn't. Python requests will still use plain HTTP to the proxy even if https:// was given as URL.
You can try to do a simple TLS connection to the proxy itself, for example with Python or with openssl s_client. It will very likely fail with some handshake error because the proxy isn't actually expecting TLS.
I am currently using fasthttp for sending my requests my question is, is there a way to have a persistent session? I need the cookies and data to stick.
c := fasthttp.Client{ Name: "Add To Cart",}
store, err := session.Start() // ?????
args := fasthttp.AcquireArgs()
defer fasthttp.ReleaseArgs(args)
args.Add("pid", sizepid)
args.Add("options", "[]")
args.Add("quantity", "1")
statusCode, body, err := c.Post(nil, "URL", args)
if err != nil {
panic(err)
}`
Based on your question I think this is already clear to you, but just in case:
Sessions aren't started on the client, they are started on the server. The server checks to see if a specific cookie exists; if it does it resumes the session that the cookie identifies; if it doesn't it creates a new session and sends the identifier back to the client as a cookie. All the client needs to do is send the correct cookie to the server.
So, you need to read and write cookies. The fasthttp.Client.Post() interface doesn't allow you to do that. So instead of that nice interface, things become rather ugly.
You need to ask fasthttp for both a Request and Response object before you do the request. Once you've done the initial request, you need to either look all cookies, or read out a specific cookie. You can now use those values for your next request.
I've written a short example of how you would do this.
func main() {
c := fasthttp.Client{}
// Create a request
req := fasthttp.AcquireRequest()
defer fasthttp.ReleaseRequest(req)
req.SetRequestURI(`https://www.google.com/`)
// Create a response
resp := fasthttp.AcquireResponse()
defer fasthttp.ReleaseResponse(resp)
// Execute the request, writing to the response object
err := c.Do(req, resp)
if err != nil {
panic(err)
}
// Loop over all cookies; usefull if you want to just send everything back on consecutive requests
resp.Header.VisitAllCookie(func(key, value []byte) {
log.Printf("Cookie %s: %s\n", key, value)
})
// Read a specific cookie
nid := fasthttp.AcquireCookie()
defer fasthttp.ReleaseCookie(nid)
nid.SetKey(`NID`)
if resp.Header.Cookie(nid) {
log.Println("Value for NID Cookie: " + string(nid.Value()))
// Create a second request and set the cookie from the first
req2 := fasthttp.AcquireRequest()
defer fasthttp.ReleaseRequest(req2)
req2.SetRequestURI(`https://www.google.com/`)
req2.Header.SetCookie(`NID`, string(nid.Value()))
// Now you can execute this request again using c.Do() - don't forget to acquire a new Response!
}
}
Note: you can chose to skip the fasthttp.AcquireXXX() and defer fasthttp.ReleaseXXX(yyy) steps - but that would negate much (maybe most) of the performance benefits over using standard net/http, so if you go that route maybe just ditch fasthttp all together.
I'm trying to retrieve an access token, in order to authenticate users using Oauth2. I'm using mostly code found on google's HOW-TO page for using the Calendar API with golang. The problem is that whenever I try to obtain a token, google sends back this:
Response: {
"error" : "invalid_grant"
}
With the error oauth2: cannot fetch token: 400 Bad Request
As I said, I'm using some code got from google's howto, just slightly modified to fit my needs.
//Somewhere...
authURL = config.AuthCodeURL("state-token", oauth2.AccessTypeOffline)
//Somewhere else...
func getClient(ctx context.Context, config *oauth2.Config, code string) *http.Client {
cacheFile := tokenCacheFile()
tok, err := tokenFromFile(cacheFile)
if err != nil {
log.Printf("Google auth code not cached. Obtaining from the web...")
tok, err = getTokenFromWeb(code) //This returns an error
if err == nil {
log.Printf("Got token!")
saveToken("calendar-go-quickstart.json", tok)
} else { //Prevent saving token when error
log.Printf("Couldn't get OAUTH2 token! %s", err)
}
}
return config.Client(ctx, tok)
}
The error occurs at "getTokenFromWeb(code)" (if I understood correctly, code must be some random string, no matter its value, it just needs to be the same during the whole process).
This is the problematic code:
func getTokenFromWeb(code string) (*oauth2.Token, error) {
tok, err := config.Exchange(context.Background(), code)
return tok, err
}
After executing, what I see is that error. I even get the exact same error when trying to copy-paste google's own example code!
Any idea? I really can't find a solution online.
Extra details: using IRIS web framework; using the latest version of google calendar api; using the latest version of Golang; I've created a client ID for OAuth2 on Google Cloud Console; The website has got a trusted SSL cert; it listens on port 80 (HTTP) and 4433 (HTTPS);
Here is Google's example:
// getTokenFromWeb uses Config to request a Token.
// It returns the retrieved Token.
func getTokenFromWeb(config *oauth2.Config) *oauth2.Token {
authURL := config.AuthCodeURL("state-token", oauth2.AccessTypeOffline)
fmt.Printf("Go to the following link in your browser then type the "+
"authorization code: \n%v\n", authURL)
var code string
if _, err := fmt.Scan(&code); err != nil {
log.Fatalf("Unable to read authorization code %v", err)
}
...
}
code is an authorization code given to the user after visiting the displayed link. fmt.Scan() is going to scan the input from the user.
If you're going to be acting on a different user's behalf, you will have to do something similar to this example.
If you're only acting as yourself, you should be able to authenticate as yourself without the code.
Either way, code cannot be a random string.
I'm building 2 apis. One make request to another.
To call the api that receives requests, we need to pass a X-Token Header. I'm doing this with Golang
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
},
}
req, err := http.NewRequest("GET", "https://localhost:8086/v2/example", nil)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"Error": err.Error()})
}
req.Header.Add("accept", "application/json")
req.Header.Add("content-type", "application/json")
req.Header.Add("x-token", "a2e63ee01401aaeca78be023dfbb8c59")
resp, err := client.Do(req)
In the other API, i get the http header with gin like this:
token := c.Request.Header.Get("x-token")
I dont know why my header arrives with another value and no X-Token. Thanks!
Result of fmt.Printf("%+v", c.Request.Header):
map[User-Agent:[Go-http-client/1.1] Referer:[https://localhost:8086/v2/example] Accept-Encoding:[gzip]]
I don't know where is my x-token, accept and content-type headers....
IMPORTANT
If i make a request with x-token header on Postman to the requested API i get the right header.
If i change the request address on the API that makes requests, e.g httpbin, i get the right header too....
Helo, guys! i found the solution....
I don't know why yet... but i think golang don't handle no trailing slash url's....
https://localhost:8086/v2/example
is different of
https://localhost:8086/v2/example/
That was my problem....
I just copy and past the golang generated code of postman... and that was the "biggest" difference....
Thanks mr. Postman...