Client timeout exceeded while awaiting headers - performance

I got below error, I am using go v1.10.4 linux/amd64.
I am not behind any firewall or whatsoever. New Relic in java server (same network segment) that we have runs fine.
We have tried:
Increasing the timeout to 60 seconds
Use http2 in the server
Using Postman return 503 with response:
{"exception":{"message":"Server Error","error_type":"RuntimeError"}}
troubleshooting with ./nrdiag says “No Issues Found”
Below is our code:
config := newrelic.NewConfig(os.Getenv("NEW_RELIC_APP_NAME"), os.Getenv("NEW_RELIC_KEY"))
config.Logger = newrelic.NewDebugLogger(os.Stdout)
app, err := newrelic.NewApplication(config)
if err != nil {
fmt.Println("Failed to create newrelic application", err)
os.Exit(1)
}
.................
httpListener, err := net.Listen("tcp", *httpAddr)
if err != nil {
oldlog.Print("Error: ", err)
logger.Log("transport", "HTTP", "during", "Listen", "err", err)
os.Exit(1)
}
g.Add(func() error {
logger.Log("transport", "HTTP", "addr", *httpAddr)
return http.Serve(httpListener, nrgorilla.InstrumentRoutes(httpHandler, app))
}, func(error) {
httpListener.Close()
})
}
However this what we got,note some_key was removed:
(28422) 2019/07/29 18:08:50.058559 {"level":"warn","msg":"application connect failure","context":{"error":"Post https://collector-001.eu01.nr-data.net/agent_listener/invoke_raw_method?license_key=some_key\u0026marshal_format=json\u0026method=connect\u0026protocol_version=17: net/http: request canceled (Client.Timeout exceeded while awaiting headers)"}}

I think it is due to DNS network timeout.
You can easily test this out by using the following steps (in Ubuntu)
Select the IPv4 Settings tab.
Disable the “Automatic” toggle switch and enter the DNS resolvers' IP addresses, separated by a comma. We’ll use the Google DNS nameservers:
8.8.8.8,8.8.4.4
If it works, then you may be able to reset the DNS to "Automatic"

On Windows OS, Running Linux Containers with WSL2, i followed the following steps,
Ran the command docker logout
Ran the command, docker network prune, so as to remove all the preconfigured settings of the network.
From Docker Settings, Enabled the DNS server configuration with 8.8.8.8
Restarted the Docker
Now executed login command with registry to login, docker login {registry}

Related

Server Error 500 when deploying Go backend with Gorm, using CloudSQL and Google App Engine

I'm trying to deploy a Go backend on Google App Engine, connecting to CloudSQL. Deploy magic is working, but when I try to call the API, I get a 500 status:
Error: Server Error
The server encountered an error and could not complete your request.
Please try again in 30 seconds.
When running gcloud app logs tail -s default, I have:
2020/06/07 21:39:26 server.go:52: dial unix /cloudsql/: connect: is a directory
var dbURI string
dbURI = fmt.Sprintf("%s:%s#unix(/cloudsql/%s)/%s", dbUser, dbPwd, dbHost, dbName)
// Open database connection
db, err := gorm.Open(
"mysql",
dbURI,
)
if err != nil {
log.Fatal(err) <-- line 52
}
My app.yaml is setup correctly (I think..):
env_variables:
CLOUDSQL_CONNECTION_NAME: 'XXXXX:europe-west1:XXXXXX'
DB_USER: 'root'
DB_PASS: 'XXXXX'
DB_NAME: 'XXXXX'
Any help would be appreciated :)
I was indeed not using the right ENV variable.. Thank you for pointing that out #Muffin Top

How to fix: Chromedriver Page Immediately Closes

I run a program that tries to open a webpage using agouti/chromedriver in go, but as soon as it calls the new page, it closes the page. Subsequent attempts to access the page result in an "invalid session id" error. How can I get the page to stay open?
This is using Go (version 1.13.3 darwin/amd64) on Mac (version 10.14.5), with chromedriver (version 76.0.3809.68). I have the latest version of Google Chrome installed as well. I have tried updating chrome and agouti to no improved results. I have quit and restarted Chrome, uninstalled and reinstalled Chrome, and restarted my computer, none of which have worked. I have run with several chromedriver options (seen below), and run without them. I have tried with (seen below) and without time.Sleep() between functions. My regular chrome browser works fine.
package main
import (
"fmt"
"time"
"github.com/sclevine/agouti"
)
func main() {
driver := agouti.ChromeDriver(
agouti.ChromeOptions("args", []string{
"--headless",
"--no-sandbox",
"--disable-dev-shm-usage",
"--disable-gpu",
"--whitelisted-ips",
"--detach",
}),
agouti.Debug,
)
fmt.Println("start")
err := driver.Start()
if err != nil {
fmt.Println("Error starting driver: " + err.Error())
return
}
page, err := driver.NewPage(agouti.Browser("chrome"))
if err != nil {
fmt.Println("Error creating new page: " + err.Error())
return
}
time.Sleep(1 * time.Second)
err = page.Navigate("https://www.google.com")
if err != nil {
fmt.Println("Error navigating to job post link: " + err.Error())
return
}
time.Sleep(1 * time.Second)
fmt.Println("end")
}
When not using the --headless tag, I can see the browser window open and close in less than a second.
Expected output:
start
Starting ChromeDriver 76.0.3809.68 (420c9498db8ce8fcd190a954d51297672c1515d5-refs/branch-heads/3809#{#864}) on port 53489
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
end
Actual Output:
start
Starting ChromeDriver 76.0.3809.68 (420c9498db8ce8fcd190a954d51297672c1515d5-refs/branch-heads/3809#{#864}) on port 53489
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
Error navigating to job post link: failed to navigate: request unsuccessful: invalid session id
I found the solution; turns out I needed to reinstall chromedriver. Looking back through my logs, the issue occurred due to Chrome automatically updating overnight, and no longer working with my previous version of chromedriver.

Unable to create online web-page

I am trying to create Golang web-pages...
Progress:
Ubuntu 18.04 installed both locally and on a Linode VPS.
Created and compiled a local Golang "Hello World" script that renders OK both locally and online.
Created a net/http Golang script that works OK when called locally http://localhost:8080/testing to see if it works
Uploaded the script to the Linode server and initial status messages appear but when calling http:123.456.789.32:8080/testing to see if it works the browser freezes.
//
// Golang - main.go
//
package main
import (
"net/http"
)
func sayHello(w http.ResponseWriter, r *http.Request) {
message := r.URL.Path
message = "Hello " + message
w.Write([]byte(message))
}
func main() {
http.HandleFunc("/", sayHello)
if err := http.ListenAndServe(":8080", nil); err != nil {
panic(err)
}
}
There are no errors or warnings rendered and unable to find any log references.
Can error and warnings similar to PHP error_reporting(-1), declare(strict_types=1) etc be logged or rendered?
A quick check with Nmap showed this result:
nmap -sV -p 8080 <yourIP>
Starting Nmap 7.70 ( https://nmap.org ) at 2019-07-04 07:45 CEST
Nmap scan report for <your-domain>.com (<yourIP>)
Host is up (0.032s latency).
PORT STATE SERVICE VERSION
8080/tcp filtered http-proxy
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 0.90 seconds
The state of "filtered" actually means that there was no response on that port as opposed to an outright rejection of the request.
Check the output of iptables -L -n. Presumably, you have a firewall running and blocking port 8080. Do not simply deactivate the firewall, but read up on how to open port 8080 in the firewall product you are using. Linode has guides for the commonly used/preinstalled firewalls of various Linux distributions.
If you plan to go into production, please have someone help you to ensure security and availability of your deployment.

Golang (gin) server is going down without any trace

I tried making an API with gin.
router := gin.Default()
router.GET("/search", controllers.Search)
router.POST("/handshake/", controllers.Handshake)
router.GET("/ping", func(c *gin.Context) {
c.JSON(200, gin.H{
"message": "pong",
})
})
router.Use(gin.Recovery())
router.Run(fmt.Sprintf(":%d", 8080))
I am creating the binary on my local machine running ubuntu 18.04 and distributing the binary to ubuntu 16.04 on AWS EC2. the service runs fine and stops suddenly without any trace of error or warning or panic on the log. very difficult to find out what's wrong?

How to connect to a Gremlin Websocket in Go?

I'm trying to connect to Gremlin which should be running on port 8182 ([INFO] GremlinServer$1 - Channel started at port 8182)
When I'm connecting to: ws://127.0.0.1:8182 I'm getting the response:
2017/11/03 17:20:04 ERROR: database with the name 'gremlin' gave an error when connecting: websocket: bad handshake.
The Gremlin server responds with:
[WARN] HttpGremlinEndpointHandler - Invalid request - responding with 400 Bad Request and no gremlin script supplied
I'm using Gorilla Websocket in Go to connect:
gremlinGrpcAddress := fmt.Sprintf("ws://%s:%d/", f.config.Host, f.config.Port)
var dialer *websocket.Dialer
clientConn, _, err := dialer.Dial(gremlinGrpcAddress, nil)
if err != nil {
return err
}
f.client = clientConn
UPDATE:
Connecting to echo.websocket.org works, so somehow it is something in the Gremlin server.
The answer lies in the config YAML file.
What is needed is the correct channelizer.
GOOD:
channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer
Connect to ws://localhost:8182/gremlin instead of ws://localhost:8182. There are several references to it in the Apache TinkerPop documentation and the go-gremlin README.

Resources