What is the recommended way of using libssh2 to implement the “read data from whichever channel has it first” primitive?
E.g. I have a simple two-tab terminal program, where each tab corresponds to a LIBSSH2 channel. I want to wait until ANY of the 2 channels gets data and then print it.
The single-channel examples use libssh2_channel_read() in a non-blocking way like this:
while(not done) {
1. Try reading with libssh2_channel_read()
2. If returned LIBSSH2_ERROR_EAGAIN, wait with select()
}
The trivial way of extending this to two-channel case would be:
while(not done) {
1. Try reading channel 1
2. Try reading channel 2
3. If BOTH channels returned LIBSSH2_ERROR_EAGAIN, wait with select()
}
This leads to a rare bug when a packet with some data for channel 1 arrives just before reading channel 2. Then both calls return LIBSSH2_ERROR_EAGAIN, but as the attempt to read channel 2 actually recv()’d the data for channel 1, select() will now hang.
The workaround I am currently employing involves keeping raw data counters for the socket and using them to determine if any new data was consumed by libssh2, but I get the feeling of making a really complex workaround for a fairly simple problem. Am I missing something?
Is there some kind of libssh2_session_read_any_channel()?
Related
This page on a Go Tutorial about channels seems to be missing a word(s) or was just not edited. I can't tell what it is supposed to say about sending and receiving through channels.
By default, sends and receives block until the other side is ready.
Is a block something within Go? I haven't seen it before. Is block being used as a noun?
I tried searching for clarification. The only other page that has similar wording is educative.io
Moreover, by default, channels send and receive until the other side is ready
But it doesn't make sense. Do they mean:
Channels send and receive regardless of whether or not the other side is ready? Doesn't this seem wasteful?
Or is "don't" missing in the statement above?
"Block" means that the goroutine will wait. You could write it this way:
By default, sends and receives wait until the other side is ready.
"Block" is just the normal term for this. It is not specific to Go. It is possible to use a channel in Go in a non-blocking manner:
You can create a channel with a buffer. As long as there is space in the buffer, a write is non-blocking (but it will block if the buffer is full). As long as there is data in the buffer, a read is non-blocking (but it will block if the buffer is empty).
You can use a select statement with a default branch.
var readch chan int
var writech chan int
var value int
select {
case n := <- readch:
// Received data.
case writech <- value:
// Sent data.
default:
// Didn't send or receive data.
}
In this code, instead of blocking (waiting), the goroutine will go to the default branch.
I've wrapped a queue to implement the Writer and Reader interfaces (for pushing and popping, respectively).
I need to continuously listen to the queue, and handle every message that comes through. This is simple when the queue is represented as a channel, but more difficult otherwise:
loop:
for {
var data []byte
select {
case <-done:
break loop
case _, err := queue.Read(data):
fmt.Println(string(data))
}
}
What's the proper way to do this? Read here is blocking - it waits until the queue has a message.
Is there a better, more idiomatic way to achieve this?
It’s harder to take a synchronous API (like queue.Read as you described above) and make it asynchronous than it is to do the opposite.
The idea would be to create a new goroutine (using, for example go func() {...}) and have that goroutine execute the read and write the output to a channel.
Then the first goroutine would block on that channel and the one it’s already blocking on.
This has the potentially to leave orphaned resources for a little while if the read takes to long but if you have a synchronous API, it’s the best you can do.
Golang newbie here.
Is there a functional difference between
func randomNumberGenerator() <-chan int {
and
func randomNumberGenerator() chan int {
I've tried using both and they seem to work fine for me.
I've seen the former used by Rob Pike (one of Go creators) in his Go Concurrency Patterns talk at Google IO 2012. I've also seen it used in Go's official website. Why add 2 extra characters ("<-") when you can omit it? I've tried looking for the difference on the web, but couldn't find it.
Both will work indeed. But one will be more constraining. The form with the arrow pointing away from the chan keyword means that the returned channel will only be able to be pulled from by client code. No pushing allowed : the pushing will be done by the random number generator function. Conversely, there's a third form with the arrow pointing towards chan, that makes said channel write-only to clients.
chan // read-write
<-chan // read only
chan<- // write only
These added constraints can improve the expression of intent and tighten the type system : attempts to force stuff into a read-only channel will leave you with a compilation error, and so will attempts to read from a write-only channel. These constraints can be expressed in the return type, but they can also be part of the parameter signature. Like in :
func log(<-chan string) { ...
There you can know, just by the signature, that the log function will consume data from the channel, and not send any to it.
This is an example of a receive-only channel.
The optional <- operator specifies the channel direction, send or receive. If no direction is given, the channel is bidirectional. A channel may be constrained only to send or only to receive by conversion or assignment.
It's useful to tell the users of your API that they should only receive from that channel and never send, otherwise bad things happen. It is considered a good practice to specify the direction of your channels in public APIs. See also: Principles of designing Go APIs with channels.
I'm writing a program in the Go language, and I have a simple problem:
I have some goroutines in my program and channels with which goroutines use to communicate. From time to time I would like to check what is inside the channels. How could I achieve that without interrupting the goroutines' work? Do channels have any function to print their contents? Or should I somehow copy them?
var shelf chan int = make(chan int, 5)
go Depot(shelf)
go Shop(shelf)
var input string
fmt.Scanln(&input)
if (input == "print") {
//here print what on shelf
}
How could I achieve that without interrupting the goroutines' work?
The simple answer is that you can't, without interrupting. Channels are a synchronization primitive, meaning that they are what enables concurrent programs to communicate safely. If you take something out of a channel, that "taking out" happens atomically, nobody else can take the same item out of the same channel. And that's intended.
What you can do is take items out and put them back after printing them. The problem with this approach is that some elements might never be printed while others may be printed more than once as all goroutines involved race to grab items from the channel.
It sounds like you need something else than a channel.
This is more of a observation and also a suggestion for whats the best way to handle this scenario.
I have two threads one just pumps in data and another receives the data and does lot of work before sending it another socket. Both the threads are connected via a Domain socket. The protocol used here is UDP. I did not want to use TCP as it is stream based, which means if there is little space in the queue my data is split and sent. This is bad as Iam sending data that should not be split. Hence I used DGRAM. Interestingly when the send thread overwhelms the recv thread by pumping so much data, at some point the Domain socket buffer gets filled up and sendto() returns ENOBUFS. I was of the opinion that should this happen, sendto() would block until the buffer is available. This would be my desired behaviour. However this does not seem to be the case. I solve this problem in a rather weird way.
CPU Yield method
If I get ENOBUFS, I do a sched_yield(); as there is no pthread_yield() in OSX. After that I try to resend again. If that fails I keep doing the same until it is taken. This is bad as Iam wasting cpu cycles just doing something useless. I would love if sendto() blocked.
Sleep method
I tried to solve the same issue using sleep(1) instead of sched_yield() but this of no use as sleep() would put my process to sleep instead of just that send thread.
Both of them does not seem to work for me and Iam running out of options. Can someone suggest what is the best way to handle this issue? Is there some clever tricks Iam not aware of that can reduce unnecessary cpu cycles? btw, what the man page says about sentto() is wrong, based on this discussion http://lists.freebsd.org/pipermail/freebsd-hackers/2004-January/005385.html
The Upd code in kernel:
The udp_output function in /sys/netinet/udp_usrreq.c, seems clear:
/*
* Calculate data length and get a mbuf
* for UDP and IP headers.
*/
M_PREPEND(m, sizeof(struct udpiphdr), M_DONTWAIT);
if (m == 0) {
error = ENOBUFS;
if (addr)
splx(s);
goto release;
}
I'm not sure why sendto() isn't blocking for you... but you might try calling this function before you each call to sendto():
#include <stdio.h>
#include <sys/select.h>
// Won't return until there is space available on the socket for writing
void WaitUntilSocketIsReadyForWrite(int socketFD)
{
fd_set writeSet;
FD_ZERO(&writeSet);
FD_SET(socketFD, &writeSet);
if (select(socketFD+1, NULL, &writeSet, NULL, NULL) < 0) perror("select");
}
Btw how big are the packets that you are trying to send?
sendto() on OS X is really nonblocking (that is M_DONTWAIT flag for).
I suggest you to use stream based connection and just receive the whole data on the other side by using MSG_WAITALL flag of the recv function. If your data has strict structure than it would be simple, just pass the correct size to the recv. If not than just send some fixed-size control packet first with the size of the next chunk of data and then the data itself. On the receiver side you would be wait for control packet of fixed size and than the data of size from control packet.