I tried downloading an Excel file from a URL using http/net by calling the GET method. I don't know if this is releveant, but as for my case, I use dropbox to store the file on the cloud (it's open for public, not restricted, it can be accessed on incognito).
But when I open the file that's saved on the local, it has no content at all. It has just an empty sheet. The file is supposed to have filled with lots of data in cell.
What's happening here? Anyone knows how to solve it? There's no error at all when I print it.
func main() {
filePath := "./file/filename.xlsx"
url := "http://www.dropbox.com/somethingsomething.xlsx"
out, err := os.Create(filePath)
if err != nil {
fmt.Println(err)
}
defer out.Close()
resp, err := http.Get(url)
if err != nil {
fmt.Println(err)
}
defer resp.Body.Close()
_, err = io.Copy(out, resp.Body)
if err != nil {
fmt.Println(err)
}
return
}
Does the dropbox URL have dl=0 query param?
If so, try changing it to dl=1 to force download the file.
I tried the same with one of my files and it worked.
Thanks!
When posting an issue to GitHub API V3 I am getting an unexpected response. Namely 422 Unprocessable Entity. However the detail of the error is for the Search endpoint rather that the POST create endpoint.
{"message":"Validation Failed","errors":[{"resource":"Search","field":"q","code":"missing"}],"documentation_url":"https://developer.github.com/v3/search"}
My instinct is that I have messed up the json but it is pretty simple and I can't see the issue.
I have tried various solutions posted here and elsewhere but not found what I am doing wrong. This is a coding exercise rather than anything intended for production but driving me mildly insane.
Tested in Debug what the Request body is just before being posted.
{"title":"Hello World","body":"dfsdfsdf\n"}
Tried removing the body as it is optional, same issue.
Tested in Debug that request is of type POST
Tested in Debug that authorization header is correct.
Removed authorization key and received 401 as expected.
The posting function:
func CreateIssue (issue *NewIssue) (*IssueDetailsResult, error){
issueJson, err := json.Marshal(issue)
if err != nil {
log.Fatal(err)
os.Exit(1)
}
req, err := http.NewRequest("POST", github.IssuesURL, bytes.NewBuffer(issueJson))
req.Header.Set("Authorization", "token "+os.Getenv("UPGITUSER"))
req.Header.Set( "Content-Type", "application/json")
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
os.Exit(1)
}
if resp.StatusCode != http.StatusCreated {
bodyBytes, _ := ioutil.ReadAll(resp.Body)
body := string(bodyBytes)
resp.Body.Close()
return nil, fmt.Errorf("create issue failed:%s", resp.Status + "\ntext: " + body)
}
var result IssueDetailsResult
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
resp.Body.Close()
return nil, err
}
resp.Body.Close()
return &result, nil
}
Would expect 201 from GitHubAPI.
The response is a strong indicator that the request is being sent to the wrong endpoint.
You can use net/http/httputil's DumpRequestOut to inspect the requests you are about to send and to ensure that they are what you expect them to be.
I have this piece of code:
ctx:=context.Background()
cliente, err := storage.NewClient(ctx)
if err != nil {
log.Fatal(err)
}
clienteCS := cliente.Bucket("prueba123456789")
w:=clienteCS.Object("prueba").NewWriter(ctx)
w.ContentType = "text/plain"
if _, err := w.Write([]byte("abcde\n")); err != nil {
log.Fatal(err)
}
attrs, err := clienteCS.Attrs(ctx)
fmt.Printf("atrr: %+v err:%v\n",attrs,err)
When I run this program, in the console I can see the attributes printed from my bucket, that would mean for me that there is nothing wrong with the configuration
, but the problem is that when I see on console.cloud.google.com I see no file on my bucket even on api dashboard does not seem like any api has been called(that peak before the red line is other api I was using) , even though no error is showing on my console that could indicate something went wrong
I just forgot about closing the client, this page alert this problem, always must close the client or the file won't be send, so just need to add some lines:
if err := w.Close(); err != nil {
log.Fatal(err)
return
}
I'm building out a simple Slackbot as a learning experience with Go, and I've hit my first snag: I'm unable to write a message back to the connection!
Here's my main func:
func main() {
conn, botId, err := slackInit(os.Getenv("SLACKBOT_TOKEN"))
if err != nil { return }
defer conn.Close()
for {
_, event, err := conn.ReadMessage()
if err != nil {
fmt.Println("Error processing message:", err)
return
}
fmt.Println(string(event))
message, err := slackGetMessage(event)
if strings.Contains(message, botId) {
fmt.Println("Bot was mentioned!")
resp := []byte("You talkin' to me?")
err = conn.WriteMessage(1, resp)
if err != nil {
fmt.Println("Error writing message:", string(resp))
return
}
}
}
}
This listens and reports events with no issue, but fails as soon as the conn.WriteMessage method is called. I get a fatal error from websocket with code 1006 & the message "unexpected closure".
I'm using Gorilla for my websocket library, and I suspect maybe this is a concurrency issue related to their "one reader, one writer" limit. I've tried a few tweaks, but honestly just don't know enough about the languge/library yet to really dive deep on debugging this. :-/
I'm sure I'm missing something obvious here! Any tips for managing my Writer to get my bot talking back? Thanks!
The message sent to the server is not JSON as expected by the server. The server closed the connection without a handshake.
I am trying to debug a very unusual error I am receiving for a simple REST library I wrote.
I am using the standard net/http package to make Get, Post, Put, Delete requests but my tests occasionally fail when I make multiple requests successively. My test looks like this:
func TestGetObject(t *testing.T) {
firebaseRoot := New(firebase_url)
body, err := firebaseRoot.Get("1")
if err != nil {
t.Errorf("Error: %s", err)
}
t.Logf("%q", body)
}
func TestPushObject(t *testing.T) {
firebaseRoot := New(firebase_url)
msg := Message{"testing", "1..2..3"}
body, err := firebaseRoot.Push("/", msg)
if err != nil {
t.Errorf("Error: %s", err)
}
t.Logf("%q", body)
}
And I am making the request like this:
// Send HTTP Request, return data
func (f *firebaseRoot) SendRequest(method string, path string, body io.Reader) ([]byte, error) {
url := f.BuildURL(path)
// create a request
req, err := http.NewRequest(method, url, body)
if err != nil {
return nil, err
}
// send JSON to firebase
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("Bad HTTP Response: %v", resp.Status)
}
defer resp.Body.Close()
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return b, nil
}
Sometimes it works, but most of the time I get 1 or 2 failures:
--- FAIL: TestGetObject (0.00 seconds)
firebase_test.go:53: Error: Get https://go-firebase-test.firebaseio.com/1.json: EOF
firebase_test.go:55: ""
--- FAIL: TestPushObject (0.00 seconds)
firebase_test.go:63: Error: Post https://go-firebase-test.firebaseio.com/.json: EOF
firebase_test.go:65: ""
FAIL
exit status 1
FAIL github.com/chourobin/go.firebase 3.422s
The failures happen when I make more than 1 request. If I comment out everything except for the PUT request, the tests consistently pass. Once I include a second test, such as GET, one or the other fails (sometimes both pass).
I experienced this reliably. You need to set Req.Close to true (the defer on resp.Body.Close() syntax used in the examples is not enough). Like this:
client := &http.Client{}
req, err := http.NewRequest(method, url, httpBody)
// NOTE this !!
req.Close = true
req.Header.Set("Content-Type", "application/json")
req.SetBasicAuth("user", "pass")
resp, err := client.Do(req)
if err != nil {
// whatever
}
defer resp.Body.Close()
response, err = ioutil.ReadAll(resp.Body)
if err != nil {
// Whatever
}
I agree with the assertion that you shouldn't be hitting outside servers in your unit tests, why not just use the built-in http.Server and serve up the content that you want to test. (There is actually the httptest package to help with this)
I recently ran into this same problem while trying to crawl sitemaps, and this is what I have found so far:
Go by default will send requests with the header Connection: Keep-Alive and persist connections for re-use. The problem that I ran into is that the server is responding with Connection: Keep-Alive in the response header and then immediately closing the connection.
As a little background as to how go implements connections in this case (you can look at the full code in net/http/transport.go). There are two goroutines, one responsible for writing and one responsible for reading (readLoop and writeLoop) In most circumstances readLoop will detect a close on the socket, and close down the connection. The problem here occurs when you initiate another request before the readLoop actually detects the close, and the EOF that it reads get interpreted as an error for that new request rather than a close that occurred prior to the request.
Given that this is the case the reason why sleeping in between requests works is that it gives readLoop time to detect the close on the connection before your new request and shut it down, so that your new request will initiate a new connection. (And the reason why it would intermittently fail is because there is some amount code running between your requests and depending of scheduling of goroutines, sometimes the EOF will be properly handled before your next request, sometimes not). And the req.Close = true, solution works because it prevents the connection from being re-used.
There is a ticket related to this situation: https://code.google.com/p/go/issues/detail?id=4677 (and a dupe ticket that I created that allowed me to reliably reproduce this: https://code.google.com/p/go/issues/detail?id=8122)
I'm going to guess there is no problem with your code. The most likely cause of your problem is because the server is closing the connection. Rate limiting is one possible reason for this.
Your test shouldn't be relying on an external service that's very brittle and not hermetic. Instead you should think about spinning up a test server locally.
My experience with this error was when I entered absolutely empty input for my JSON API!
I should send {} as empty JSON, but I sent so this error happened
I encountered this issue while sending an invalid body to a GET request
I could reproduce by making a request similar to below:
var requestBody interface{}
requestData, _ := json.Marshal(requestBody)
payload = strings.NewReader(string(requestData))
req, err := http.NewRequest("GET", url, payload)
...
I was developing an image download app when this problem occurs.
Tried request.Close=true but not work.
60% requests resulted in a EOF error.
I thought maybe it is an image server problem, not my code.
But php code works fine.
Then I use
var client = &http.Client{
Transport: &http.Transport{},
}
client.Do(request)
to make request, instead of
http.DefaultClient.Do(request)
problem gone.
Not sure why,I guess something with RoundTripper