I want to get the IP of the user(client) based on his interaction with the Pods, (I'm thinking about getting the user IP and locate him based on his IP)
I made the figure below for a better explanation of my question,
the only thing I was able to find to maybe improve the situation was to patch the service and set externalTrafficPolicy to "Local" so the service will preserve the IP of the user.
But still not sure how or even at which part should I check the IP of the user. is it possible to monitor the activity of the user from outside of pod? any idea?
(I'm using golang)
update: to make it more clear, i'm not creating the pods, in the scenario below clients are responsible and can create different pods and containers even the services they need, it's like a test-bed for them, so i cannot edit their containers file but i may be able to bring up another container beside their conatiner and then maybe use the answer at this post https://stackoverflow.com/a/27971761/9350055 to find the ip of the client. do you think this will work?
Sure, that will work but keep in mind that you will only receive traffic on the nodes where you have pods for the particular service (in your case Service NodePort).
If you are using Golang
Now, this should work with either L4 or L7 traffic. If you are using Golang an example of how to get it is looking at the X-Forwarded-For HTTP header:
package main
import (
"encoding/json"
"net/http"
)
func main() {
http.HandleFunc("/", ExampleHandler)
if err := http.ListenAndServe(":8080", nil); err != nil {
panic(err)
}
}
func ExampleHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Add("Content-Type", "application/json")
resp, _ := json.Marshal(map[string]string{
"ip": GetIP(r),
})
w.Write(resp)
}
// GetIP gets a requests IP address by reading off the forwarded-for
// header (for proxies) and falls back to use the remote address.
func GetIP(r *http.Request) string {
forwarded := r.Header.Get("X-FORWARDED-FOR")
if forwarded != "" {
return forwarded
}
return r.RemoteAddr
}
Also, here's an example of how to get for L4 services (TCP).
✌️
Related
I am trying to configure a RabbitMQ broker using the go-micro framework. I have noticed that the broker interface in go-micro has a broker.SubscriberOptions struct which allows configuring the parameters I am looking for (AutoAck, Queue name and so on) however I am unable to figure out how to pass this when starting a broker.
This is how a simple rabbit go-micro setup would look like
package main
import (
"log"
"github.com/micro/go-micro/server"
"github.com/micro/go-plugins/broker/rabbitmq"
micro "github.com/micro/go-micro"
)
func main() {
// Create a new service. Optionally include some options here.
service := micro.NewService(
micro.Name("go-micro-rabbit"),
micro.Broker(rabbitmq.NewBroker()),
)
// Init will parse the command line flags.
service.Init()
// Register handler
proto.RegisterGreeterHandler(service.Server(), new(Greeter))
micro.RegisterSubscriber("micro-exchange", service.Server(), myFunc, server.SubscriberQueue("my-queue"))
// Run the server
if err := service.Run(); err != nil {
log.Fatal(err)
}
}
The micro.RegisterSubscriber method takes in a list of server.SubscriberOption but does not allow me to set the broker.SubscriberOptions and the rabbitmq.NewBroker allows setting broker.Options but once again, not broker.SubscriberOptions
I have dug in the code of go-micro but have been unable to figure out how the broker.Subscribe method (Which exposes the correct struct) is called or by who.
Is this possible at all? Is it maybe something not yet fully fleshed out in the API?
There are a lot of mux routers for golang. All of the ones I've found assume that I'm building my own HTTP server in go. However, I would like to use aws apigateway as the external layer and have it forward the method, path, query parameters to a lambda function that I have deployed with apex (go shim for aws lambda functions). All the API gateway endpoints will forward to one lambda function so that there are fewer things to hook up, like permissions and so forth.
So I would like to use nice mux libraries for their ability to parse regex or path variables, but use them inside the Lambda and be able to invoke the correct handler based on the url path.
Most of the mux routers have a usage like this:
router := NewRouter()
router.Add("GET", "/my_path/:id", MyHandler)
Where MyHandler is a type of http.HandlerFunc
Then the server is started with something like http.ListenAndServe(port, router)
But in AWS Lambda there is no server to start, I would just like to use the mux to find the handler that I should be calling.
I have created a lib for this purpose.
The basic idea is to transform apigateway request context which is a json object into http.Request
The most common approach is to use apex/gateway which is a drop in replacement for net/http on AWS Lambda. This is started by calling with the same parameters.
gateway.ListenAndServe(port, router)
Here is a sample showing it's use:
func main() {
http.HandleFunc("/", hello)
log.Fatal(gateway.ListenAndServe(":3000", nil))
}
func hello(w http.ResponseWriter, r *http.Request) {
// example retrieving values from the api gateway proxy request context.
requestContext, ok := gateway.RequestContext(r.Context())
if !ok || requestContext.Authorizer["sub"] == nil {
fmt.Fprint(w, "Hello World from Go")
return
}
userID := requestContext.Authorizer["sub"].(string)
fmt.Fprintf(w, "Hello %s from Go", userID)
}
It's also common to use this with the chi router.
Some other options include:
davyzhang/agw (listed in Davy's response)
davidsbond/lux
EtienneBruines/fasthttplambda
So I made a few changes to this example to make it work with gin-gonic
https://github.com/utiq/go-in-5-minutes/tree/master/episode4
The websocket handshake between many clients is succesful. The problem is that when a client sends a message, the message is not propagated to the rest of the clients.
I had a look on your commit changes of episode4.
My observations as follows:
You're creating hub instance on every incoming request at stream handler. hub instance used to keeps track connections, etc. so you're losing it on every request.
You have removed index/home handler (may be you wanted to convert to gin handler or something, I don't know).
Now, let's bring episode4 into action. Please do following changes (as always improve it as you like). I have tested your episode4 with below changes, it's working fine.
Make /ws handler work on server.go:
h := newHub()
wsh := wsHandler{h: h}
r.GET("/ws", func(c *gin.Context) {
wsh.ServeHTTP(c.Writer, c.Request)
})
Remove the stream handler on connection.go:
func stream(c *gin.Context) {
h := newHub()
wsHandler{h: h}.ServeHTTP(c.Writer, c.Request)
}
Adding index HTML handler on server.go: (added it to test episode4 at my end)
r.SetHTMLTemplate(template.Must(template.ParseFiles("index.html")))
r.GET("/", func(c *gin.Context) {
c.HTML(200, "index.html", nil)
})
I'm trying to insert a simple record as in GoDoc. But this returns,
rpc error: code = 7 desc = "User can't access project: tidy-groove"
When I searched for grpc codes, it says..
PermissionDenied Code = 7
// Unauthenticated indicates the request does not have valid
// authentication credentials for the operation.
I've enabled Big table in my console and created a cluster and a service account and recieved the json. What I'm doing wrong here?
package main
import (
"fmt"
"golang.org/x/net/context"
"golang.org/x/oauth2/google"
"google.golang.org/cloud"
"google.golang.org/cloud/bigtable"
"io/ioutil"
)
func main() {
fmt.Println("Start!")
put()
}
func getClient() *bigtable.Client {
jsonKey, err := ioutil.ReadFile("TestProject-7854ea9op741.json")
if err != nil {
fmt.Println(err.Error())
}
config, err := google.JWTConfigFromJSON(
jsonKey,
bigtable.Scope,
) // or bigtable.AdminScope, etc.
if err != nil {
fmt.Println(err.Error())
}
ctx := context.Background()
client, err := bigtable.NewClient(ctx, "tidy-groove", "asia-east1-b", "test1-bigtable", cloud.WithTokenSource(config.TokenSource(ctx)))
if err != nil {
fmt.Println(err.Error())
}
return client
}
func put() {
ctx := context.Background()
client := getClient()
tbl := client.Open("table1")
mut := bigtable.NewMutation()
mut.Set("links", "maps.google.com", bigtable.Now(), []byte("1"))
mut.Set("links", "golang.org", bigtable.Now(), []byte("1"))
err := tbl.Apply(ctx, "com.google.cloud", mut)
if err != nil {
fmt.Println(err.Error())
}
}
I've solved the problem. It's nothing wrong with the code, but config json itself. So anyone who out there want to authenticate and came here by google search... This code is correct and working perfectly. What I've done wrong is follows.
First I made a service account and got the json. But google warned me that im not an owner of project hence it wont be added to accept list but anyway it let me download the json.
Then I deleted that key from console and requested project owner to create a key for me.
There he has created another key with the same name I given.. And since he's the owner no error/warning msgs displayed and successfully json file was downloaded.
When I tried with that... my question begun. That's when i posted this question.
After that with no solutions. I asked owner to delete that key and create another key but with a different name..
Then it worked! It seems if you try to create a key with non-owner account and then again create with same name ( after deleting original of course ) has no effect. Hope this helps everyone out there :)
Take a look at: helloworld.go or search.go which uses GOOGLE_APPLICATION_CREDENTIALS environment variable.
For most environments, you no longer even need to set GOOGLE_APPLICATION_CREDENTIALS. Google Cloud Platform, Managed VMs or Google App Engine all have the right thing set for you. Your desktop environment will also be correct if you've used gcloud init or it's predecessor gcloud auth login followed by gcloud config set project <projectID>.
I've written a REST web service that uses mongo as the backend data store. I was wondering at this stage (before deployment), what the best practices were, considering a service that essentially runs forever(ish).
Currently, I'm following this type of pattern:
// database.go
...
type DataStore struct {
mongoSession *mgo.Session
}
...
func (d *DataStore) OpenSession () {
... // read setup from environment
mongoSession, err = mgo.Dial(mongoURI)
if err != nil {}
...
}
func (d *DataStore) CloseSession() {...}
func (d *DataStore) Find (...) (results...) {
s := d.mongoSession.Copy()
defer s.Close()
// do stuff, return results
}
In main.go:
func main() {
ds := NewDataStore()
ds.OpenSession()
defer ds.CloseSession()
// Web Service Routes..
...
ws.Handle("/find/{abc}", doFindFunc)
...
}
My question is - what's the recommended practice for recovery from session that has timed out, lost connection (the mongo service provider I'm using is remote, so I assume that this will happen), so on any particular web service call, the database session may no longer work? How do people handle these cases to detect that the session is no longer valid and a "fresh" one should be established?
Thanks!
what you may want is to do the session .Copy() for each incoming HTTP request (with deffered .Close()), copy again from the new session in your handlers if ever needed..
connections and reconnections are managed by mgo, you can stop and restart MongoDB while making an HTTP request to your web service to see how its affected.
if there's a db connection problem while handling an HTTP request, a db operation will eventually timeout (timeout can be configured by using DialWithTimeout instead of the regular Dial, so you can respond with a 5xx HTTP error code in such case.