I just started using go and wrote my first program but the output is not as expected. I have writtern a async routine addUrl which adds url to channel 5000 times and consumeUrl removes from the channel and prints it.
The routine runs only 9 time. Why is it? Below is the code and output
package main
import "fmt"
import "time"
var urlCount = 0
func main(){
urlHolder := make(chan string,5000)
fmt.Printf("Starting program")
go addUrls(urlHolder)
time.Sleep(time.Millisecond * 100)
go consumeUrls(urlHolder)
fmt.Printf("Done")
}
func addUrls(urlHolder chan string){
var myurl string = "https://example.com/"
for i:=0; i<5000 ; i++ {
urlHolder<-myurl
fmt.Printf(" %d url added \n",i)
time.Sleep(time.Millisecond * 10)
}
}
func consumeUrls(urlHolder chan string) {
urlCount++
urlsConsumed := <- urlHolder
fmt.Printf("Pulled url %d",urlCount," ",urlsConsumed,"\n")
time.Sleep(time.Millisecond * 20)
}
The output is
Starting program
0 url added
1 url added
2 url added
3 url added
4 url added
5 url added
6 url added
7 url added
8 url added
Done
Why is it terminating at 8 when loop is 5000?
You are using time.Sleep to wait for main to finish but you really should be using WaitGroups.
That way you don't have to try to pick some arbitrary time and hope it's enough for your program to finish, or worry about setting too much time and your program sits around doing nothing.
I've added the implementation of WaitGroups to your code here:
https://play.golang.org/p/1zn2JYefaA
Also, the way your consumeUrls function is written is not looping properly and you won't get everything in your channel printed. But since that wasn't your specific question I won't address it here.
Actually The problem was like Nipun said, the main got terminated early.
Related
I have a program that executes queries to an HTTP server, each request gets a goroutine.
I quickly found out it was too much for MacOS, as there is a file descriptor limit of 250.
I was wondering if I could limit the number of goroutines, or perhaps block until there are available goroutines, instead of failing.
Maybe a worker pool with 250 goroutines, and queue the rest of the requests?
What are your thoughts
package main
import "fmt"
const ROUTINE_LIMIT = 250
func main() {
channelCounter := make(chan bool, ROUTINE_LIMIT)
for {
select {
//will add till 250 after that it will block
case channelCounter <- true:
fmt.Println("added channel")
go startRoutine(channelCounter)
}
}
}
func startRoutine(channelCounter chan bool) {
/*
do your stuff
*/
//free the channel
<-channelCounter
}
You can limit your goruotine count using channels.
A channel to keep count of the number of go routines running ..
And once the job is done you can read the channel value to reduce the count.
The above program is like a rough sample code.. (i think it covers the general idea)
I'm trying to make a channel with never ending ticker, but it works only 2 times.
Could you help me to understand where is the problem?
Code:
package main
import (
"fmt"
"time"
)
var mark = [2]float64{8.9876, 762.098568}
func tick(out chan <- [2]float64){
c := time.NewTicker(time.Millisecond *500)
for range c.C{
out <- mark
}
}
func main() {
fmt.Println("Start")
md := make(chan [2]float64)
go tick(md)
for range <-md{
fmt.Println(<-md)
}
}
Output:
Start
[8.9876 762.098568]
[8.9876 762.098568]
Example: https://play.golang.org/p/P2FaUwbW-3
This:
for range <-md{
is not the same as:
for range md{
The latter ranges over the channel (what you want), while the former ranges over the value received from the channel when the loop starts, which happens to be a two-element array, hence the two executions. You're also ignoring the value received from the channel in the for statement, and reading from it again in the loop body, ignoring every other message on the channel (though this makes no difference in the example, since every value is identical, it would make a significant difference in practice). What you really want is:
for foo := range md{
fmt.Println(foo)
}
Here's a working version of your playground example, slightly modified to avoid "program took too long" errors because in its current form it never stops: https://play.golang.org/p/RSUJFvluU5
I'm playing around with the code presented on https://tour.golang.org/concurrency/5. My idea was that I could simplify the code by getting rid of the quit channel and still keep the correct program behavior - just for learning purposes.
Here is the code I've got (simplified it even more for better readability):
package main
import "fmt"
import "time"
func sendNumbers(c chan int) {
for i := 0; i < 2; i++ {
c <- i
}
}
func main() {
c := make(chan int)
go func() {
for i := 0; i < 2; i++ {
fmt.Println(<-c)
}
}()
time.Sleep(0 * time.Second)
sendNumbers(c)
}
In this code, the go routine I spawn should be able to receive 2 numbers before it returns. The sendNumbers() function I call next sends exactly 2 numbers to the channel c.
So, the expected output of the program is 2 lines: 0 and 1. However, what I'm getting when I run the code on the page, is just a single line containing 0.
Note the
time.Sleep(0 * time.Second)
though that I deliberately added before calling sendNumbers(c). If I change it to
time.Sleep(1 * time.Second)
The output becomes as expected:
0
1
So, I'm confused with what happens. Can someone explain what is going on? Shouldn't both sends and receives block regardless of how much time is passed before I call sendNumbers()?
In Go, the program exits as soon as the main function returns regardless of whether other goroutines are still running or not. If you want to make sure the main function does not exit prematurely, you need some synchronization mechanism. https://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-all-goroutines-to-finish-executing-before-continuing/
You don’t absolutely have to use synchronization primitives, you could also do it with channels only, arguably a more idiomatic way to do it.
I have a code like,
Routine 1 {
runtime.LockOSThread()
print something
send int to routine 2
runtime.UnlockOSThread
}
Routine 2 {
runtime.LockOSThread()
print something
send int to routine 1
runtime.UnlockOSThread
}
main {
go Routine1
go Routine2
}
I use run time lock-unlock because, I don't want that printing of
Routine 1 will mix with Routine 2. However, after execution of above
code, it outputs same as without lock-unlock (means printing outputs
mixed). Can anybody help me why this thing happening and how to force
this for happening.
NB: I give an example of print something, however there are lots of
printing and sending events.
If you want to serialize "print something", e.g. each "print something" should perform atomically, then just serialize it.
You can surround "print something" by a mutex. That'll work unless the code deadlock because of that - and surely it easily can in a non trivial program.
The easy way in Go to serialize something is to do it with a channel. Collect in a (go)routine everything which should be printed together. When collection of the print unit is done, send it through a channel to some printing "agent" as a "print job" unit. That agent will simply receive its "tasks" and atomically print each one. One gets that atomicity for free and as an important bonus the code can not deadlock easily no more in the simple case, where there are only non interdependent "print unit" generating goroutines.
I mean something like:
func printer(tasks chan string) {
for s := range tasks {
fmt.Printf(s)
}
}
func someAgentX(tasks chan string) {
var printUnit string
//...
tasks <- printUnit
//...
}
func main() {
//...
tasks := make(chan string, size)
go printer(tasks)
go someAgent1(tasks)
//...
go someAgentN(tasks)
//...
<- allDone
close(tasks)
}
What runtime.LockOSThread does is prevent any other goroutine from running on the same thread. It forces the runtime to create a new thread and run Routine2 there. They are still running concurrently but on different threads.
You need to use sync.Mutex or some channel magic instead.
You rarely need to use runtime.LockOSThread but it can be useful for forcing some higher priority goroutine to run on a thread of it's own.
package main
import (
"fmt"
"sync"
"time"
)
var m sync.Mutex
func printing(s string) {
m.Lock() // Other goroutines will stop here if until m is unlocked
fmt.Println(s)
m.Unlock() // Now another goroutine at "m.Lock()" can continue running
}
func main() {
for i := 0; i < 10; i++ {
go printing(fmt.Sprintf("Goroutine #%d", i))
}
<-time.After(3e9)
}
I think, this is because of runtime.LockOSThread(),runtime.UnlockOSThread does not work all time. It totaly depends on CPU, execution environment etc. It can't be forced by anyother way.
In my code there are three concurrent routines. I try to give a brief overview of my code,
Routine 1 {
do something
*Send int to Routine 2
Send int to Routine 3
Print Something
Print Something*
do something
}
Routine 2 {
do something
*Send int to Routine 1
Send int to Routine 3
Print Something
Print Something*
do something
}
Routine 3 {
do something
*Send int to Routine 1
Send int to Routine 2
Print Something
Print Something*
do something
}
main {
routine1
routine2
routine3
}
I want that, while codes between two do something (codes between two star marks) is executing, flow of control must not go to other goroutines. For example, when routine1 is executing the events between two stars (sending and printing events), routine 2 and 3 must be blocked (means flow of execution does not pass to routine 2 or 3 from routine 1). After completing last print event, flow of execution may pass to routine 2 or 3. Can anybody help me by specifying, how can I achieve this ? Is it possible to implement above specification by WaitGroup ? Can anybody show me by giving a simple example how to implement above specified example by using WaitGroup. Thanks.
NB: I give two sending and two printing options, in fact there is lots of sending and printing.
If I have gotten it correctly, what you want is to prevent simultaneous execution of some part of each function and other functions. The following code does this: fmt.Println lines won't happen as other routines are running. Here's what happens: when execution gets to the print section, it waits until other routines end, if they are running, and while this print line is executing other routines don't start and wait. I hope that is what you are looking for. Correct me if I'm wrong about this.
package main
import (
"fmt"
"rand"
"sync"
)
var (
mutex1, mutex2, mutex3 sync.Mutex
wg sync.WaitGroup
)
func Routine1() {
mutex1.Lock()
// do something
for i := 0; i < 200; i++ {
mutex2.Lock()
mutex3.Lock()
fmt.Println("value of z")
mutex2.Unlock()
mutex3.Unlock()
}
// do something
mutex1.Unlock()
wg.Done()
}
func Routine2() {
mutex2.Lock()
// do something
for i := 0; i < 200; i++ {
mutex1.Lock()
mutex3.Lock()
fmt.Println("value of z")
mutex1.Unlock()
mutex3.Unlock()
}
// do something
mutex2.Unlock()
wg.Done()
}
func Routine3() {
mutex3.Lock()
// do something
for i := 0; i < 200; i++ {
mutex1.Lock()
mutex2.Lock()
fmt.Println("value of z")
mutex1.Unlock()
mutex2.Unlock()
}
// do something
mutex3.Unlock()
wg.Done()
}
func main() {
wg.Add(3)
go Routine1()
go Routine2()
Routine3()
wg.Wait()
}
UPDATE: Let me explain these three mutex here: a mutex is, as documentation says: "a mutual exclusion lock." That means when you call Lock on a mutex your code just waits there if somebody else have locked the same mutex. Right after you call Unlock that blocked code will be resumed.
Here I put each function in its own mutex by locking a mutex at the beginning of the function and unlocking it the end. By this simple mechanism you can avoid running any part of code you want at the same time as those functions. For instance, everywhere you want to have a code that should not be running when Routine1 is running, simply lock mutex1 at the beginning of that code and unlock at at the end. That is what I did in appropriate lines in Routine2 and Routine3. Hope that clarifies things.
You could use sync.Mutex. For example everything between im.Lock() and im.Unlock() will exclude the other goroutine.
package main
import (
"fmt"
"sync"
)
func f1(wg *sync.WaitGroup, im *sync.Mutex, i *int) {
for {
im.Lock()
(*i)++
fmt.Printf("Go1: %d\n", *i)
done := *i >= 10
im.Unlock()
if done {
break
}
}
wg.Done()
}
func f2(wg *sync.WaitGroup, im *sync.Mutex, i *int) {
for {
im.Lock()
(*i)++
fmt.Printf("Go2: %d\n", *i)
done := *i >= 10
im.Unlock()
if done {
break
}
}
wg.Done()
}
func main() {
var wg sync.WaitGroup
var im sync.Mutex
var i int
wg.Add(2)
go f1(&wg, &im, &i)
go f2(&wg, &im, &i)
wg.Wait()
}
Another way would be to have a control channel, with only one goroutine allowed to execute at any one time, with each routine sending back to the 'control lock' whenever they are done doing their atomic operation:
package main
import "fmt"
import "time"
func routine(id int, control chan struct{}){
for {
// Get the control
<-control
fmt.Printf("routine %d got control\n", id)
fmt.Printf("A lot of things happen here...")
time.Sleep(1)
fmt.Printf("... but only in routine %d !\n", id)
fmt.Printf("routine %d gives back control\n", id)
// Sending back the control to whichever other routine catches it
control<-struct{}{}
}
}
func main() {
// Control channel is blocking
control := make(chan struct{})
// Start all routines
go routine(0, control)
go routine(1, control)
go routine(2, control)
// Sending control to whichever catches it first
control<-struct{}{}
// Let routines play for some time...
time.Sleep(10)
// Getting control back and terminating
<-control
close(control)
fmt.Println("Finished !")
}
This prints:
routine 0 got control
A lot of things happen here...... but only in routine 0 !
routine 0 gives back control
routine 1 got control
A lot of things happen here...... but only in routine 1 !
routine 1 gives back control
routine 2 got control
A lot of things happen here...... but only in routine 2 !
routine 2 gives back control
routine 0 got control
A lot of things happen here...... but only in routine 0 !
routine 0 gives back control
routine 1 got control
A lot of things happen here...... but only in routine 1 !
routine 1 gives back control
routine 2 got control
A lot of things happen here...... but only in routine 2 !
routine 2 gives back control
routine 0 got control
A lot of things happen here...... but only in routine 0 !
routine 0 gives back control
routine 1 got control
A lot of things happen here...... but only in routine 1 !
routine 1 gives back control
routine 2 got control
A lot of things happen here...... but only in routine 2 !
routine 2 gives back control
routine 0 got control
A lot of things happen here...... but only in routine 0 !
routine 0 gives back control
routine 1 got control
A lot of things happen here...... but only in routine 1 !
routine 1 gives back control
routine 2 got control
A lot of things happen here...... but only in routine 2 !
routine 2 gives back control
Finished !
You are asking for a library function that explicitly stop other routines ? Make it clear that, it is not possible in Go and also for most other languages. And sync, mutex case is not possible for your case.