I've built an application in Go that connects to a Google PubSub to publish messages. It works perfectly in localhost (using pubsub emulator), but for some reason the connection hangs in GAE.
It hangs on this piece of code (removed unrelated code to improve readability)
import (
"cloud.google.com/go/pubsub"
)
client, err := pubsub.NewClient(ctx, config.Project_ID)
if err != nil {
return nil, errors.Wrap(err, "failed to create client")
}
topic, err := client.CreateTopic(ctx, id) // HANGS HERE
This is the app.yaml
runtime: go114
instance_class: B1
manual_scaling:
instances: 1
handlers:
- url: /.*
script: auto
Also, when the code is running (and hanged) it keeps outputting the following error message
internal.flushLog: Flush RPC: service bridge HTTP failed: Post "http://appengine.googleapis.internal:10001/rpc_http": dial tcp 169.254.169.253:10001: i/o timeout
It seems that something is dropping the packets sent to GCP internal API, but I don't know why or how to fix it. What I find even more weird is the fact that I have another app running almost the same code (and connecting successfully to pubsub). The only difference is that the app that is working is not the default service in GAE and the one that's failing is.
It seems like a network problem but unfortunately I haven't been able to find a solution online.
Below are the direct dependencies used by the app (extracted from go.mod)
cloud.google.com/go
cloud.google.com/go/storage
github.com/dgrijalva/jwt-go
github.com/getsentry/sentry-go
github.com/go-ozzo/ozzo-validation
github.com/golang/gddo
github.com/google/uuid
github.com/gorilla/handlers
github.com/gorilla/mux
github.com/jackc/pgx
github.com/jinzhu/copier
github.com/joho/godotenv
github.com/lib/pq
github.com/ory/dockertest
github.com/pkg/errors
github.com/stretchr/testify
google.golang.org/appengine
google.golang.org/genproto
cloud.google.com/go/pubsub
github.com/satori/go.uuid
github.com/sirupsen/logrus
github.com/spf13/pflag
google.golang.org/grpc
Okay guys, I finally found out what has happened. Dumb mistake.
We were using the pubsub emulator to test the application locally and unfortunately we left the PUBSUB_EMULATOR_HOST=localhost:8085 environment set in our GAE app. This means that the application was trying to connect with the emulator inside the GAE and not the real Pub/Sub. Since the emulator only exists locally, the application hangs on the TCP connection.
At least I can tell some thing to help other people that might have similar issues:
Pub/Sub works with the GAE standard environment
Pub/Sub works with Go 1.14 (you don't need to downgrade to 1.11)
Check your environment variables :)
Related
I'm running into an issue when using Go's http client to download a zip or tar.gz file from Github.
I get a 403 with the message "Your access to this site has been restricted".
Curl works fine though.
I am running this in an EC2 instance on AWS in the us-west-2 region. In particular,
Ubuntu Server 16.04 LTS (HVM), SSD Volume Type - ami-0807918df10edc141 (64-bit x86) / ami-0c75fb2e6a6be38f6 (64-bit Arm)
Info
Go: go1.15.linux-amd64 (tried it on 14 too)
AWS - Ubuntu Server 16.04 LTS (HVM), SSD Volume Type - ami-0807918df10edc141 (64-bit x86) / ami-0c75fb2e6a6be38f6 (64-bit Arm)
Endpoint: https://github.com/kubeflow/manifests/archive/v1.0.2.tar.gz (I've tried it with others too, none work)
Sample code to reproduce:
package main
import (
"fmt"
"io/ioutil"
"net/http"
)
func main() {
endpoint := "https://github.com/kubeflow/manifests/archive/v1.0.2.tar.gz"
// or https://api.github.com/repos/kubeflow/manifests/zipball/v0.12.0
// Get the data
resp, err := http.Get(endpoint)
if err != nil {
fmt.Printf("[error] %v", err)
return
}
defer resp.Body.Close()
respData, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Printf("[error] %v", err)
return
}
// Returns a 403 and html error page
fmt.Printf("Resp:\n%v\n", string(respData))
}
Note: the above works fine on my local machine, it just seems to stop in the aws instance.
Thanks!
That particular error message means that GitHub is restricting you because you're making requests that match a pattern of abuse that's ongoing. GitHub is blocking this pattern because it causes availability concerns for other users.
You should always make your program use a custom User-Agent header because that distinguishes your actions from other people's. (After all, lots of people use Go.) You should acquire the URLs you're using via the API, not via github.com directly. You should also authenticate when possible (e.g., with a token), because GitHub will give authenticated requests higher limits, and if you cause a problem, GitHub can reach out to you. Finally, you should implement appropriate rate-limiting and throttling so that you don't make too many requests and back off or stop completely if you get a 403, 429, or 5xx error.
If you need to download many archives for the same repository, clone it and use git archive, which is far more efficient. Caching data instead of requesting it multiple times is also recommended.
If you do all of these things, you'll probably find that your requests work.
I've installed IBM MQ (8.0.0.4) on a local Windows VM and followed the instructions in the docs to enable AMQP (https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_8.0.0/com.ibm.mq.con.doc/tamqp_creating.htm), and also disabled authentication since this is just a local development environment. With that done, I am able to connect from my host OS using the node.js sample in IBM's mqlight package:
>npm install mqlight
...
>node node_modules/mqlight/samples/recv.js -s amqp://windows-10:5672
Connected to amqp://windows-10:5672 using client-id recv_126117c
Subscribed to pattern: public
However, when attempting to connect from a golang app using vcabbage/amqp (version 0.12.5), it returns an EOF error on the attempt to dial the host. Minimal example:
package main
import (
"fmt"
"pack.ag/amqp"
)
func main() {
_, err := amqp.Dial("amqp://windows-10:5672")
fmt.Println(err) // EOF
}
Nothing is appearing in the IBM MQ error logs.
Unfortunately Google does not turn up any relevant results for connecting to IBM MQ via AMQP 1.0 in golang, so I'm stuck. Does anyone have any ideas?
So the solution is apparently to use SASL Anonymous mode; this allows the client to connect.
package main
import (
"fmt"
"pack.ag/amqp"
)
func main() {
_, err := amqp.Dial("amqp://windows-10:5672", amqp.ConnSASLAnonymous())
fmt.Println(err) // nil
}
If anybody wants to try to make it work in "normal" mode, it appears that IBM MQ was closing the channel as soon as the initial header packet was sent. The EOF was bubbling up from the receiving goroutine in any case.
I am running a HTTPS server in Linux (RHEL 7). I am getting the below error as soon as I start the server.
2019/09/04 15:46:16 http: TLS handshake error from xx.xx.xx.xx:60206: EOF
2019/09/04 15:46:21 http: TLS handshake error from xx.xx.xx.xx:31824: EOF
This error is coming automatically and continuously in the terminal.
Below is the go code for creating https server -
package main
import (
"fmt"
"net/http"
"github.com/gin-gonic/gin"
)
func main() {
fmt.Println("Starting webserver")
router := gin.Default()
router.GET("/", func(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{
"success": true,
})
})
router.RunTLS(":9001", "server.pem", "server.key")
}
We have purchased and combined the server certificate, intermidate certificate and root certificate into a single file to make the server.pem file.
As this error is coming continuously and in the terminal as soon as I start the server, I think there is some configuration problem in the VM?
Please suggest what are the things I can check here.
NOTE: This error is specific to the Go. I have tested on the same server on the same port with same certificates in Node JS. And it works fine.
Also the IP in the error message is of the reverse proxy server (WAF) which is continuosly doing health monitoring of the web application server.
I would attack the problem from two angles:
What is this xx.xx.xx.xx address? I'd expect that when I start some random piece of software, there is nothing to connect to it all by itself, right?
Is there anything special about that 9001 port? Try running nc -l -p 9001 and see whether those unidentified connections happen as well.
Run tcpdump and see whether there is any incoming traffic from the clients making those connections: those EOFs (that's "end of file") reported by the TLS mchinery most probably mean those clients—whatever they are—close their side of the connection somewhere amidst the TLS handshake—while the server is expecting to read some data from them.
This hints at that those clients do not actually expect to see TLS protocol in the connection they open; and they pretty much may send some plaintext in it, so you'll be able to peek at it.
Googling for "9001 port" hints at that it's used for some "ETL service manager" protocol—whatever it is. This hints at that traffic on 9001 might be related to VoIP.
I have no idea what to do with this, but it might give you some lead for further research.
I am trying to call http.Get over http://example.com/path/to/index, and I have resolved example.com to one of my inner IP address.
My Go code:
resp, err := http.Get("http://example.com/path/to/index")
On Windows, this blocks forever until connection timeouts.
However, on Linux, it works totally fine.
What is wielder is that I have tried both curl http://example.com/path/to/index and translated the Go code to Nodejs one, they all work fine on Windows, which make me sure that domain resolving is ok on my Windows.
So I wonder why the behaviors of http.Get differ on Windows and Linux.
I try to use the slack-sample bot from this blogpost https://www.opsdash.com/blog/slack-bot-in-golang.html . I've successfully created my api token, but i can not connect to the websocket server (the rtm.start request passs normally). I've get the error message
dial tcp 54.242.95.213:443: connectex: No connection could be made because the target machine actively refused it
I've also tried to connect via a chrome app called Simple Web Socket Client and via a website based one tester. Both work well, i can establish a connection and i can send data.
I'm behind a proxy, but i only have troubles with golang's websocket.Dial function.
Does anybody know why this can happen?
I use:
- Windows 7 SP1 x64
- Golang 1.7.1 windows/amd64
Greetings
Tonka
If you are using gorilla/websocket, it has the ability to use a Proxy. From issue 107:
import "net/http"
...
var dialer = websocket.Dialer{
Proxy: http.ProxyFromEnvironment,
}
If you are using golang.org/x/net you should switch to gorilla/websocket.