I've got a server-like program in Windows 10 where the main program waits for the requests with http.Handle(dst, handlerStruct), and for every query does the corresponding operations and returns a result or just prints something through the screen.
It usually works fine, but sometimes, and just sometimes, when I leave the program running "waiting" for requests to come in for a long time (3 minutes up) and then I send a request, it just gets "stucked" until I press CTRL+C, and then it returns the values inmediately. So it looks like the program runs well, but the returning thread is having a break or something when I leave it a long time without requests.
This happens ever since I started developing the program, and not only when using functions with goroutines.
For me it looks like it has more to do with Windows 10 than with Golang itself, but it's just very annoying, and of course I don't want this to happen with my server...
Hope you can see what's going wrong here...
Thanks.
EDIT
This is the main program waiting for the requests:
func main() {
runtime.GOMAXPROCS(10000000)
http.Handle("/sameDayCombinations", CBR.SameDayCombinationsHandler{})
http.ListenAndServe(":8080", nil)
}
Now this is the code of ServeHTTP for the handler:
func (h SameDayCombinationsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
t1 := time.Now()
q := common.DecodeRequest(r)
json := common.EncodeAnswer((&h).RetrieveSpecificSolutions(q.Year, q.Month, q.Day, q.DepID, q.ArrID, q.Adults, q.Children, q.Infants), true)
fmt.Fprintf(w, json)
t2 := time.Now()
fmt.Println(t2.Sub(t1))
}
EDIT2
Here I add the stack of a program execution after the appearance of this weird behavior. So, after doing CTRL+C I got this; looks pretty normal to me.
http://pastebin.com/DduvtNZr
Though, now, by empirical demonstration, I'm sure it has to do with the console Prints (probably large prints?). That's a good new, cause it'll only happen while debugging, but stills a weird behavior...
Related
My problem can be summarized as the following snippet:
package main
import (
"fmt"
"time"
)
func main() {
done := make(chan int)
done2 := make(chan int)
go func() {
for {
fmt.Println("1")
time.Sleep(time.Duration(1) * time.Second)
}
done <- 1
}()
go func() {
for {
fmt.Println("2")
}
done2 <- 1
}()
<- done
<- done2
}
Where the go routine "1" never gets the chance to run again. After doing some research, it looks like it's because go routine "2" takes up all the CPU.
I had done something similar in java before, and thread "1" can always wake up approximately 1 sec later.
My question is how can I achieve the same behavior in go?(I'm transferring a socket program originally written in Java to Go)
I have also tried runtime.GOMAXPROCS(2), but didn't work
Go routines to someone new to Go can have a number of surprising side effects. One of the more important of these is the problems of empty infinite loops.
The way that the Go scheduler works, every function call should get a preemption point placed in the beginning of the call. This should be happening with your fmt.Println("2") call, meaning that every time a print is called, the scheduler in the background has the ability to move around what routines are running at any given time, increase memory for a goroutine, etc.
Under normal circumstances, even if it is not ideally written code, this should be enough. Given that you have stated in the comments that this is not the actual code that you are working with, its not really possible to answer why your other code might not be working.
If you are interested in learning more about how the go scheduler works, I would recommend this article which goes over preemption, and these articles which covers the scheduler at a lower level.
I would also recommend taking a look through the Tour of Go site. While some of the concepts are rather basic, there are some very complex things that you can do if you are aware of all the tools in your toolbox. I would specifically recommend the concurrency section, and the select statement, which might be able to help you refactor your program easier, and in a manner more fitting to go.
Besides a few tutorials on Go I have no actual experience in it. I'm trying to take a project written in Go and converting it into a windows service.
I honestly haven't tried anything besides trying to find things to read over. I have found a few threads and choosen the best library I felt covered all of our needs
https://github.com/golang/sys
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build windows
package main
import (
"fmt"
"strings"
"time"
"golang.org/x/sys/windows/svc"
"golang.org/x/sys/windows/svc/debug"
"golang.org/x/sys/windows/svc/eventlog"
)
var elog debug.Log
type myservice struct{}
func (m *myservice) Execute(args []string, r <-chan svc.ChangeRequest, changes chan<- svc.Status) (ssec bool, errno uint32) {
const cmdsAccepted = svc.AcceptStop | svc.AcceptShutdown | svc.AcceptPauseAndContinue
changes <- svc.Status{State: svc.StartPending}
fasttick := time.Tick(500 * time.Millisecond)
slowtick := time.Tick(2 * time.Second)
tick := fasttick
changes <- svc.Status{State: svc.Running, Accepts: cmdsAccepted}
loop:
for {
select {
case <-tick:
beep()
elog.Info(1, "beep")
case c := <-r:
switch c.Cmd {
case svc.Interrogate:
changes <- c.CurrentStatus
// Testing deadlock from https://code.google.com/p/winsvc/issues/detail?id=4
time.Sleep(100 * time.Millisecond)
changes <- c.CurrentStatus
case svc.Stop, svc.Shutdown:
// golang.org/x/sys/windows/svc.TestExample is verifying this output.
testOutput := strings.Join(args, "-")
testOutput += fmt.Sprintf("-%d", c.Context)
elog.Info(1, testOutput)
break loop
case svc.Pause:
changes <- svc.Status{State: svc.Paused, Accepts: cmdsAccepted}
tick = slowtick
case svc.Continue:
changes <- svc.Status{State: svc.Running, Accepts: cmdsAccepted}
tick = fasttick
default:
elog.Error(1, fmt.Sprintf("unexpected control request #%d", c))
}
}
}
changes <- svc.Status{State: svc.StopPending}
return
}
func runService(name string, isDebug bool) {
var err error
if isDebug {
elog = debug.New(name)
} else {
elog, err = eventlog.Open(name)
if err != nil {
return
}
}
defer elog.Close()
elog.Info(1, fmt.Sprintf("starting %s service", name))
run := svc.Run
if isDebug {
run = debug.Run
}
err = run(name, &myservice{})
if err != nil {
elog.Error(1, fmt.Sprintf("%s service failed: %v", name, err))
return
}
elog.Info(1, fmt.Sprintf("%s service stopped", name))
}
So I spent some time going over this code. Tested it out to see what it does. It performs as it should.
The question I have is we currently have a Go program that takes in arguments and for our service we pass in server. Which spins up our stuff on a localhost webpage.
I believe the code above may have something to do with that but I'm lost at how I would actually get it spin off our exe with the correct arguements. Is this the right spot to call main?
Im sorry if this is vague. I dont know exactly how to make this interact with our already exisiting exe.
I can get that modified if I know what needs to be changed. I appreacite any help.
OK, that's much clearer now. Well, ideally you should start with some tutorial on what constitutes a Windows service—I bet tihis might have solved the problem for you. But let's try anyway.
Some theory
A Windows service sort of has two facets: it performs some useful task and it communicates with the SCM facility. When you manipulate a service using the sc command or through the Control Panel, you have that piece of software to talk with SCM on your behalf, and SCM talks with that service.
The exact protocol the SCM and a service use is low-level and complicated
and the point of the Go package you're using is to hide that complexity from you
and offer a reasonably Go-centric interface to that stuff.
As you might gather from your own example, the Execute method of the type you've created is—for the most part—concerned with communicating with SCM: it runs an endless for loop which on each iteration sleeps on reading from the r channel, and that channel delivers SCM commands to your service.
So you basically have what could be called "an SCM command processing loop".
Now recall those two facets above. You already have one of them: your service interacts with SCM, so you need another one—the code which actually performs useful tasks.
In fact, it's already partially there: the example code you've grabbed creates a time ticker which provides a channel on which it delivers a value when another tick passes. The for loop in the Execute method reads from that channel as well, "doing work" each time another tick is signalled.
OK, this is fine for a toy example but lame for a real work.
Approaching the solution
So let's pause for a moment and think about our requirements.
We need some code running and doing our actual task(s).
We need the existing command processing loop to continue working.
We need these two pieces of code to work concurrently.
In this toy example the 3rd point is there "for free" because a time ticker carries out the task of waiting for the next tick automatically and fully concurrently with the rest of the code.
Your real code most probably won't have that luxury, so what do you do?
In Go, when you need to do something concurrently with something else,
an obvious answer is "use a goroutine".
So the first step is to grab your existing code, turn it into a callable function
and then call it in a separate goroutine right before entering the for loop.
This way, you'll have both pieces run concurrently.
The hard parts
OK, that wasn't hard.
The hard parts are:
How to configure the code which performs the tasks.
How to make the SCM command processing loop and the code carrying out tasks communicate.
Configuration
This one really depends on the policies at your $dayjob or of your $current_project, but there are few hints:
A Windows service may receive command-line arguments—either for a single run or permanently (passed to the service on each of its runs).
The downside is that it's not convenient to work with them from the UI/UX standpoint.
Typically Windows services used to read the registry.
These days (after the advent of .NET and its pervasive xml-ity) the services tend to read configuration files.
The OS environment most of the time is a bad fit for the task.
You may combine several of these venues.
I think I'd start with a configuration file but then again, you should pick the path of the least resistance, I think.
One of the things to keep in mind is that the reading and processing of the configuration should better be done before the service signals the SCM it started OK: if the configuration is invalid or cannot be loaded, the service should extensively log that and signal it failed, and not run the actual task processing code.
Communication between the command processing loop and the tasks carrying code
This is IMO the hardest part.
It's possible to write a whole book here but let's keep it simple for now.
To make it as simple as possible I'd do the following:
Consider pausing, stopping and shutting down mostly the same: all these signals must tell your task processing code to quit, and then wait for it to actually do that.
Consider the "continue" signal the same as starting the task processing function: run it again—in a new goroutine.
Have a one-directional communication: from the control loop to the tasks processing code, but not the other way—this will greatly simplify service state management.
This way, you may create a single channel which the task processing code listens on—or checks periodically, and when a value comes from that channel, the code stops running, closes the channel and exits.
The control loop, when the SCM tells it to pause or stop or shut down, sends anything on that channel then waits for it to close. When that happens, it knows the tasks processing code is finished.
In Go, a paradigm for a channel which is only used for signaling, is to have a channel of type struct{} (an empty struct).
The question of how to monitor this control channel in the tasks running code is an open one and heavily depends on the nature of the tasks it performs.
Any further help here would be reciting what's written in the Go books on concurrency so you should have that covered first.
There's also an interesting question of how to have the communication between the control loop and the tasks processing loop resilient to the possible processing stalls in the latter, but then again, IMO it's too early to touch upon that.
func main() {
messages := make(chan string)
go func() { messages <- "hello" }()
go func() { messages <- "ping" }()
msg := <-messages
msg2 := <-messages
fmt.Println(msg)
fmt.Println(msg2)
The above code consistently prints "ping" and then "hello" on my terminal.
I am confused about the order in which this prints, so I was wondering if I could get some clarification on my thinking.
I understand that unbuffered channels are blocking while waiting for both a sender and a receiver. So in the above case, when these 2 go routines are executed, there isn't, in both cases,a receiver yet. So I am guessing that both routines block until a receiver is available on the channel.
Now... I would assume that first "hello" is tried into the channel, but has to wait... at the same time, "ping" tries, but again has to wait. Then
msg := <- messages
shows up, so I would assume that at that stage, the program will arbitrarily pick one of the waiting goroutines and allow it to send its message over into the channel, since msg is ready to receive.
However, it seems that no matter how many times I run the program, it always is msg that gets assigned "ping" and msg2 that gets assigned "hello", which gives the impression that "ping" always gets priority to send first (to msg). Why is that?
It’s not about order of reading a channel but about order of goroutines execution which is not guaranteed.
Try to ‘Println’ from the function where you are writing to the channel (before and after writing) and I think it should be in same order as reading from the channel.
In Golang Spec Channels order is described as:-
Channels act as first-in-first-out queues. For example, if one
goroutine sends values on a channel and a second goroutine receives
them, the values are received in the order sent.
It will prints which value is available first to be received on other end.
If you wants to synchronize them use different channels or add wait Groups.
package main
import (
"fmt"
)
func main() {
messages1 := make(chan string)
messages2 := make(chan string)
go func(<-chan string) {
messages2 <- "ping"
}(messages2)
go func(<-chan string) {
messages1 <- "hello"
}(messages1)
fmt.Println(<-messages1)
fmt.Println(<-messages2)
}
If you see you can easily receive any value you want according to your choice using different channels.
Go playground
I just went through this same thing. See my post here: Golang channels, order of execution
Like you, I saw a pattern that was counter-intuitive. In a place where there actually shouldn't be a pattern. Once you launch a go process, you have launched a thread of execution, and basically all bets are off at that point regarding the order that the threads will execute their steps. But if there was going to be an order, logic tells us the first one called would be executed first.
In actual fact, if you recompile that program each time, the results will vary. That's what I found when I started compiling/running it on my local computer. In order to make the results random, I had to "dirty" the file, by adding and removing a space for instance. Then the compiler would re-compile the program, and then I would get a random order of execution. But when compiled in the go sandbox, the result was always the same.
When you use the sandbox, the results are apparently cached. I couldn't get the order to change in the sandbox by using insignificant changes. The only way I got it to change was to issue a time.Sleep(1) command between the launching of the go statements. Then, the first one launched would be the first one executed every time. I still don't think I'd bet my life on that continuing to happen though because they are separate threads of execution and there are no guarantees.
The bottom line is that I was seeing a deterministic result where there should be no determinism. That's what stuck me. I was fully cleared up when I found that the results really are random in a normal environment. The sandbox is a great tool to have. But it's not a normal environment. Compile and run your code locally and you will see the varying results you would expect.
I was confused when I first met this. but now I am clear about this. the reason cause this is not about channel, but for goroutine.
As The Go Memory Model mention, there's no guaranteed to goroutine's running and exit, so when you create two goroutine, you cannot make sure that they are running in order.
So if you want printing follow FIFO rule, you can change your code like this:
func main() {
messages := make(chan string)
go func() {
messages <- "hello"
messages <- "ping"
}()
//go func() { messages <- "ping" }()
msg := <-messages
msg2 := <-messages
fmt.Println(msg)
fmt.Println(msg2)
}
I have been trying to get into Go from the more traditional languages such as Java and C and so far I've been enjoying the well-thought out design choices that Go offers. When I started my first "real" project though, I ran into a problem that almost nobody seems to have.
My project is a simple networking implementation that sends and receives packets. The general structure is something like this (of course simplified):
A client manages the net.Conn with the server. This Clientcreates a PacketReaderand a PacketWriter. These both run infinite loops in a different goroutine. The PacketReader takes an interface with a single OnPacketReceived function that is implemented by the client.
The PacketReader code looks something like this:
go func() {
for {
bytes, err := reader.ReadBytes(10) // Blocks the current routine until bytes are available.
if err != nil {
panic(err) // Handle error
}
reader.handler.OnPacketReceived(reader.parseBytes(bytes))
}
}()
The PacketWriter code looks something like this:
go func() {
for {
if len(reader.packetQueue) > 0 {
// Write packet
}
}
}()
In order to make Client blocking, the client makes a channel that gets filled by OnPacketReceived, something like this:
type Client struct {
callbacks map[int]chan interface{}
// More fields
}
func (c *Client) OnPacketReceived(packet *Packet) {
c.callbacks[packet.Id] <- packet.Data
}
func (c *Client) SendDataBlocking(id int, data interface{}) interface{} {
c.PacketWriter.QueuePacket(data)
return <-c.callbacks[id]
}
Now here is my problem: the reader.parseBytes function does some intensive decoding operation that creates quite a lot of objects (to the point that the GC decides to run). The GC however, pauses the reader goroutine that is currently decoding the bytes, and then hangs. A problem that seems similar to mine is described here. I have confirmed that it is actually the GC causing it, because running it with GOGC=off runs successfully.
At this point, my 3 routines look like this:
- Client (main routine): Waiting for channel
- Writer: Still running, waiting for new data in queue
- Reader: Set as runnable, but not actually running
Somehow, the GC is either not able to stop all routines in order to run, or it does not resume said goroutines after it stopped them.
So my question is this: Is there any way to fix this? I am new to Go so I don't really know if my design choices are even remotely conventional, and I'm all up with changing the structure of my program. Do I need to change the way I handle packet reading callbacks, do I need to try and make the packet decoder less intensive? Thanks!
Edit: I am running Go 1.5.1, I'll try to get a working example later today.
As per mrd0ll4rs comment, changed the writer to use channels instead of a slice (I don't even know why I did that in the first place). That seemed to give the GC enough "mobility" to allow the threads to stop. Adding in the runtime.Gosched() and still using slices also worked though, but the channels seemed more "go-esque".
I'm trying to write SDL app in go using https://github.com/klkblake/Go-SDL.
I created timer to call draw function on it:
render_timer := time.NewTicker(time.Second / 60)
Somewhere in event loop:
for running == true {
[...]
[process sdl events]
[...]
select {
case <-render_timer.C:
call_my_draw_function()
default:
some_default_actions()
}
[...]
}
If I run program after compiling this code nothing is drawn on screen. But if I place just:
fmt.Println("default")
in the default branch of select -- the code begin to work as I want it to(draws something in window); and if i remove println it again don't draw anything.
What am I doing wrong? Why is there such behaviour of select ?
Hm... Simplest testcase is:
package main
import (
"fmt"
"time"
)
func main() {
rt := time.NewTicker(time.Second / 60)
for {
select {
case <-rt.C:
fmt.Println("time")
default:
}
time.Sleep(1) // without this line 'case <-rt.C' is never executed
}
}
As for your example, your loop is a busy loop, constantly hitting the default: case. The scheduler in go is co-operative, and since you're in a busy loop, the go routine running the Ticker will never be scheduled to run, and thus never send anything on the channel. This will be the case even if your default: case is not empty, but do pure computations - that never make any calls that invoke the schudler.
However, when you do something else that in some form invokes the go scheduler, e.g. doing I/O , the scheduler will give the Ticker a chance to run.
You could import the runtime package and do
default:
runtime.Gosched()
to make the scheduler run, which will not starve the Ticker go routine.
I'm unsure how this leads to the problems you get when running SDL, as that would most likely involve I/O or something else that triggers the scheduler
See golang: goroute with select doesn't stop unless I added a fmt.Print()
Long story short, because of the default case, and because your GOMAXPROCS is probably set to 1 (the default value), it never schedules the goroutine to do its work. There are many options to fix this, depending on your needs (a sleep in default, a time.After() channel in the select instead of the default, a call to runtime.Gosched(), etc.).
Adding the fmt.Print makes it work because - my guess, not sure - it involves io and internally causes a Gosched() (that or something like Phlip's answer in the related question, I just read it).