I am developing a distributed system uisng ZeroMQ and Go. It's like a distributed ledger so you can get the contects and append. I have automated clients making GET and ADD requests. The program runs fine for a couple of seconds but then crashes with the error "panic: socket operation on non-socket".
I tried turning off the garbage collector using debug.SetGCPercent(-1) but im sure this solution is not entirely correct.
This is the server initialization code
package server
import (
"backend/config"
"backend/gset"
"backend/tools"
zmq "github.com/pebbe/zmq4"
)
type Server struct {
Zctx *zmq.Context
Peers map[string]*zmq.Socket
Receive_socket zmq.Socket
Id string
Gset map[string]string
Port string
My_init map[string]bool
My_echo map[string]bool
My_vote map[string]bool
Peers_echo map[string]bool
Peers_vote map[string]bool
}
func CreateServer(node config.Node, peers []config.Node, zctx *zmq.Context) *Server {
id := node.Host + node.Port
port := node.Port
server_sockets := make(map[string]*zmq.Socket)
my_gset := gset.Create()
my_init := make(map[string]bool)
my_echo := make(map[string]bool)
my_vote := make(map[string]bool)
peers_echo := make(map[string]bool)
peers_vote := make(map[string]bool)
receive_socket, _ := zctx.NewSocket(zmq.ROUTER)
receive_socket.Bind("tcp://*:" + node.Port)
tools.Log(id, "Bound tcp://*:"+node.Port)
// Connect my dealer sockets to all other servers' router
for i := 0; i < len(peers); i++ {
s, _ := zctx.NewSocket(zmq.DEALER)
s.SetIdentity(id)
s.Connect("tcp://localhost:" + peers[i].Port)
// append socket to socket list
server_sockets["tcp://localhost:"+peers[i].Port] = s
}
return &Server{
Peers: server_sockets,
Receive_socket: *receive_socket,
Id: id,
Port: port,
Gset: my_gset,
My_init: my_init,
My_echo: my_echo,
My_vote: my_vote,
Peers_echo: peers_echo,
Peers_vote: peers_vote,
}
}
And this is the function that contols the server
func Normal_listener_task(s *server.Server) {
for {
message, err := s.Receive_socket.RecvMessage(0)
if err != nil {
fmt.Println(zmq.AsErrno(err))
panic(err)
}
messaging.HandleMessage(s, message)
}
}
The entire code is in my github here
If anyone knows why this is happening you will save my thesis. Thank you
The problem was that I was declaring the receive_socket with Receive_socket zmq.Socket when it should have been *Receive_socket zmq.Socket. The pointer was just a copy, therefore being considered trash by the GC.
Related
I am trying to learn Golang and took on a simple project to call all the craigslist cities and query them for a specific search. In the code below I removed all the links in the listingmap but there are over 400 links there. So the loop is fairly large. I thought this would be a good test to put what I am learning into application but I am running into a strange issue.
Some of the times most of the Http.Get() get no response from the server while others it gets them all with no problem. So I started adding prints to show how many error out and we recovered and how many successfully made it through. Also while this is running it will randomly hang and never respond. The program doesn't freeze but the site just sits there trying to load and the terminal shows no activity.
I am making sure my Response body is closed by deferring the cleanup after the recover but it still seems to not work. Is there something that jumps out to anyone that maybe I am missing?
Thanks in advance guys!
package main
import (
"fmt"
"net/http"
"html/template"
"io/ioutil"
"encoding/xml"
"sync"
)
var wg sync.WaitGroup
var locationMap = map[string]string {"https://auburn.craigslist.org/": "auburn "...}
var totalRecovers int = 0
var successfulReads int = 0
type Listings struct {
Links []string `xml:"item>link"`
Titles []string `xml:"item>title"`
Descriptions []string `xml:"item>description"`
Dates []string `xml:"item>date"`
}
type Listing struct {
Title string
Description string
Date string
}
type ListAggPage struct {
Title string
Listings map[string]Listing
SearchRequest string
}
func cleanUp(link string) {
defer wg.Done()
if r:= recover(); r!= nil {
totalRecovers++
// recoverMap <- link
}
}
func cityRoutine(c chan Listings, link string) {
defer cleanUp(link)
var i Listings
address := link + "search/sss?format=rss&query=motorhome"
resp, rErr := http.Get(address)
if(rErr != nil) {
fmt.Println("Fatal error has occurs while getting response.")
fmt.Println(rErr);
}
bytes, bErr := ioutil.ReadAll(resp.Body)
if(bErr != nil) {
fmt.Println("Fatal error has occurs while getting bytes.")
fmt.Println(bErr);
}
xml.Unmarshal(bytes, &i)
resp.Body.Close()
c <- i
successfulReads++
}
func listingAggHandler(w http.ResponseWriter, r *http.Request) {
queue := make(chan Listings, 99999)
listing_map := make(map[string]Listing)
for key, _ := range locationMap {
wg.Add(1)
go cityRoutine(queue, key)
}
wg.Wait()
close(queue)
for elem := range queue {
for index, _ := range elem.Links {
listing_map[elem.Links[index]] = Listing{elem.Titles[index * 2], elem.Descriptions[index], elem.Dates[index]}
}
}
p := ListAggPage{Title: "Craigslist Aggregator", Listings: listing_map}
t, _ := template.ParseFiles("basictemplating.html")
fmt.Println(t.Execute(w, p))
fmt.Println("Successfully loaded: ", successfulReads)
fmt.Println("Recovered from: ", totalRecovers)
}
func indexHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "<h1>Whoa, Go is neat!</h1>")
}
func main() {
http.HandleFunc("/", indexHandler)
http.HandleFunc("/agg/", listingAggHandler)
http.ListenAndServe(":8000", nil)
}
I'm having trouble finding the golang mailing list discussion I was reading in reference to this, but you generally don't want to open up hundreds of requests. There's some information here: How Can I Effectively 'Max Out' Concurrent HTTP Requests?
Craigslist might also just be rate limiting you. Either way, I recommend limiting to around 20 simultaneous requests or so, here's a quick update to your listingAggHandler.
queue := make(chan Listings, 99999)
listing_map := make(map[string]Listing)
request_queue := make(chan string)
for i := 0; i < 20; i++ {
go func() {
for {
key := <- request_queue
cityRoutine(queue, key)
}
}()
}
for key, _ := range locationMap {
wg.Add(1)
request_queue <- key
}
wg.Wait()
close(request_queue)
close(queue)
The application should still be very fast. I agree with the other comments on your question as well. Would also try and avoid putting so much in the global scope.
You could also spruce my changes up a little by just using the wait group in the request pool and have each goroutine clean itself up and decrement the wait group. That would limit some of the global scope.
So I followed everyones suggestions and it seems to resolved my issue so I greatly appreciate it. I ended up removing the global WaitGroup like many suggested and had it passed in as a parameter(pointer) to clean up the code. As for the error issues before, it must have been maxing out the concurrent HTTP request like maxm had mentioned. Once I added a wait in between every 20 searches, I have not seen any errors. The program runs a little slower than I would like but for learning purposes this has been helpful.
Below is the major change the code needed.
counter := 0
for key, _ := range locationMap {
if(counter >= 20) {
wg.Wait()
counter = 0
}
wg.Add(1)
frmtSearch := key + "search/sss?format=rss&query=" + strings.Replace(p.SearchRequest, " ", "%20", -1)
go cityRoutine(queue, frmtSearch, &wg)
counter++
}
I have been learning Golang to move all my penetration testing tools to it. Since I like to write my own tools this is a perfect way to learn a new language. In this particular case I think something is wrong with the way I am using channels. I know for a fact that is not finishing the port mapping because the other tools I use that I wrote on ruby are finding all the open ports but my golang tool is not. Can someone please help me understand what I'm doing wrong? Are channels the right way to go about doing this?
package main
import (
"fmt"
"log"
"net"
"strconv"
"time"
)
func portScan(TargetToScan string, PortStart int, PortEnd int, openPorts []int) []int {
activeThreads := 0
doneChannel := make(chan bool)
for port := PortStart; port <= PortEnd; port++ {
go grabBanner(TargetToScan, port, doneChannel)
activeThreads++
}
// Wait for all threads to finish
for activeThreads > 0 {
<-doneChannel
activeThreads--
}
return openPorts
}
func grabBanner(ip string, port int, doneChannel chan bool) {
connection, err := net.DialTimeout(
"tcp",
ip+":"+strconv.Itoa(port),
time.Second*10)
if err != nil {
doneChannel <- true
return
}
// append open port to slice
openPorts = append(openPorts, port)
fmt.Printf("+ Port %d: Open\n", port)
// See if server offers anything to read
buffer := make([]byte, 4096)
connection.SetReadDeadline(time.Now().Add(time.Second * 5))
// Set timeout
numBytesRead, err := connection.Read(buffer)
if err != nil {
doneChannel <- true
return
}
log.Printf("+ Banner of port %d\n%s\n", port,
buffer[0:numBytesRead])
// here we add to map port and banner
targetPorts[port] = string(buffer[0:numBytesRead])
doneChannel <- true
return
}
Note: seems to find the first bunch ports but not the ones that are above a hight number example 8080 but it usually does get 80 and 443...
So I suspect something is timing out, or something odd is going on.
There are lots of bad hacks of code, mostly because I'm learning and searching a lot in how to do things, so feel free to give tips and even changes/pull requests. thanks
Your code has a few problems. In grabBanner you appear to be referencing openPorts but it is not defined anywhere. You're probably referencing a global variable and this append operation is not going to be thread safe. In addition to your thread safety issues you also are likely exhausting file descriptor limits. Perhaps you should limit the amount of concurrent work by doing something like this:
package main
import (
"fmt"
"net"
"strconv"
"sync"
"time"
)
func main() {
fmt.Println(portScan("127.0.0.1", 1, 65535))
}
// startBanner spins up a handful of async workers
func startBannerGrabbers(num int, target string, portsIn <-chan int) <-chan int {
portsOut := make(chan int)
var wg sync.WaitGroup
wg.Add(num)
for i := 0; i < num; i++ {
go func() {
for p := range portsIn {
if grabBanner(target, p) {
portsOut <- p
}
}
wg.Done()
}()
}
go func() {
wg.Wait()
close(portsOut)
}()
return portsOut
}
func portScan(targetToScan string, portStart int, portEnd int) []int {
ports := make(chan int)
go func() {
for port := portStart; port <= portEnd; port++ {
ports <- port
}
close(ports)
}()
resultChan := startBannerGrabbers(16, targetToScan, ports)
var openPorts []int
for port := range resultChan {
openPorts = append(openPorts, port)
}
return openPorts
}
var targetPorts = make(map[int]string)
func grabBanner(ip string, port int) bool {
connection, err := net.DialTimeout(
"tcp",
ip+":"+strconv.Itoa(port),
time.Second*20)
if err != nil {
return false
}
defer connection.Close() // you should close this!
buffer := make([]byte, 4096)
connection.SetReadDeadline(time.Now().Add(time.Second * 5))
numBytesRead, err := connection.Read(buffer)
if err != nil {
return true
}
// here we add to map port and banner
// ******* MAPS ARE NOT SAFE FOR CONCURRENT WRITERS ******
// ******************* DO NOT DO THIS *******************
targetPorts[port] = string(buffer[0:numBytesRead])
return true
}
Your usage of var open bool and constantly setting it, then returning it is both unnecessary and non-idiomatic. In addition, checking if someBoolVar != false is a non-idiomatic and verbose way of writing if someBoolVar.
Additionally maps are not safe for concurrent access but your grabBanner function is writing to map from many go routines in a concurrent fashion. Please stop mutating global state inside of functions. Return values instead.
Here's an updated explanation of what's going on. First we make a channel that we will push port numbers onto for our workers to process. Then we start a go-routine that will write ports in the range onto that channel as fast as it can. Once we've written every port available onto that channel we close the channel so that our readers will be able to exit.
Then we call a method that starts a configurable number of bannerGrabber workers. We pass the ip address and the channel to read candidate port numbers off of. This function spawns num goroutines, each ranging over the portsIn channel that was passed, calls the grab banner function and then pushes the port onto the outbound channel if it was successful. Finally, we start one more go routine that waits on the sync.WaitGroup to finish so we can close the outgoing (result) channel once all of the workers are done.
Back in the portScan function We receive the outbound channel as the return value from the startBannerGrabbers function. We then range over the result channel that was returned to us, append all the open ports to the list and then return the result.
I also changed some stylistic things, such as down-casing your function argument names.
At risk of sounding like a broken record I am going to emphasize the following again. Stop mutating global state. Instead of setting targetPorts you should accumulate these values in a concurrency-safe manner and return them to the caller for use. It appears your usage of globals in this case is ill-thought out a mixture of convenience and not having thought about how to solve the problem without globals.
I have a for-loop that calls a function runCommand() which runs a remote command on a switch and prints the output. The function is called in a goroutine on each iteration and I am using a sync.Waitgroup to synchronize the goroutines. Now, I need a way to capture the output and any errors of my runCommand() function into a channel. I have read many articles and watched a lot of videos on using channels with goroutines, but this is the first time I have ever written a concurrent application and I can't seem to wrap my head around the idea.
Basically, my program takes in a list of hostnames from the command line then asynchronously connects to each host, runs a configuration command on it, and prints the output. It is ok for my program to continue configuring the remaining hosts if one has an error.
How would I idiomatically send the output or error(s) of each call to runCommand() to a channel then receive the output or error(s) for printing?
Here is my code:
package main
import (
"fmt"
"golang.org/x/crypto/ssh"
"os"
"time"
"sync"
)
func main() {
hosts := os.Args[1:]
clientConf := configureClient("user", "password")
var wg sync.WaitGroup
for _, host := range hosts {
wg.Add(1)
go runCommand(host, &clientConf, &wg)
}
wg.Wait()
fmt.Println("Configuration complete!")
}
// Run a remote command
func runCommand(host string, config *ssh.ClientConfig, wg *sync.WaitGroup) {
defer wg.Done()
// Connect to the client
client, err := ssh.Dial("tcp", host+":22", config)
if err != nil {
fmt.Println(err)
return
}
defer client.Close()
// Create a session
session, err := client.NewSession()
if err != nil {
fmt.Println(err)
return
}
defer session.Close()
// Get the session output
output, err := session.Output("show lldp ne")
if err != nil {
fmt.Println(err)
return
}
fmt.Print(string(output))
fmt.Printf("Connection to %s closed.\n", host)
}
// Set up client configuration
func configureClient(user, password string) ssh.ClientConfig {
var sshConf ssh.Config
sshConf.SetDefaults()
// Append supported ciphers
sshConf.Ciphers = append(sshConf.Ciphers, "aes128-cbc", "aes256-cbc", "3des-cbc", "des-cbc", "aes192-cbc")
// Create client config
clientConf := &ssh.ClientConfig{
Config: sshConf,
User: user,
Auth: []ssh.AuthMethod{ssh.Password(password)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Timeout: time.Second * 5,
}
return *clientConf
}
EDIT: I got rid of the Waitgroup, as suggested, and now I need to keep track of which output belongs to which host by printing the hostname before printing its output and printing a Connection to <host> closed. message when the gorouttine completes. For example:
$ go run main.go host1[,host2[,...]]
Connecting to <host1>
[Output]
...
[Error]
Connection to <host1> closed.
Connecting to <host2>
...
Connection to <host2> closed.
Configuration complete!
I know the above won't necessarily process host1 and host2 in order, But I need to print the correct host value for the connecting and closing messages before and after the output/error(s), respectively. I tried defering printing the closing message in the runCommand() function, but the message is printed out before the output/error(s). And printing the closing message in the for-loop after each goroutine call doesn't work as expected either.
Updated code:
package main
import (
"fmt"
"golang.org/x/crypto/ssh"
"os"
"time"
)
type CmdResult struct {
Host string
Output string
Err error
}
func main() {
start := time.Now()
hosts := os.Args[1:]
clientConf := configureClient("user", "password")
results := make(chan CmdResult)
for _, host := range hosts {
go runCommand(host, &clientConf, results)
}
for i := 0; i < len(hosts); i++ {
output := <- results
fmt.Println(output.Host)
if output.Output != "" {
fmt.Printf("%s\n", output.Output)
}
if output.Err != nil {
fmt.Printf("Error: %v\n", output.Err)
}
}
fmt.Printf("Configuration complete! [%s]\n", time.Since(start).String())
}
// Run a remote command
func runCommand(host string, config *ssh.ClientConfig, ch chan CmdResult) {
// This is printing before the output/error(s).
// Does the same when moved to the bottom of this function.
defer fmt.Printf("Connection to %s closed.\n", host)
// Connect to the client
client, err := ssh.Dial("tcp", host+":22", config)
if err != nil {
ch <- CmdResult{host, "", err}
return
}
defer client.Close()
// Create a session
session, err := client.NewSession()
if err != nil {
ch <- CmdResult{host, "", err}
return
}
defer session.Close()
// Get the session output
output, err := session.Output("show lldp ne")
if err != nil {
ch <- CmdResult{host, "", err}
return
}
ch <- CmdResult{host, string(output), nil}
}
// Set up client configuration
func configureClient(user, password string) ssh.ClientConfig {
var sshConf ssh.Config
sshConf.SetDefaults()
// Append supported ciphers
sshConf.Ciphers = append(sshConf.Ciphers, "aes128-cbc", "aes256-cbc", "3des-cbc", "des-cbc", "aes192-cbc")
// Create client config
clientConf := &ssh.ClientConfig{
Config: sshConf,
User: user,
Auth: []ssh.AuthMethod{ssh.Password(password)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Timeout: time.Second * 5,
}
return *clientConf
}
If you use an unbuffered channel, you actually don't need the sync.WaitGroup, because you can call the receive operator on the channel once for every goroutine that will send on the channel. Each receive operation will block until a send statement is ready, resulting in the same behavior as a WaitGroup.
To make this happen, change runCommand to execute a send statement exactly once before the function exits, under all conditions.
First, create a type to send over the channel:
type CommandResult struct {
Output string
Err error
}
And edit your main() {...} to execute a receive operation on the channel the same number of times as the number of goroutines that will send to the channel:
func main() {
ch := make(chan CommandResult) // initialize an unbuffered channel
// rest of your setup
for _, host := range hosts {
go runCommand(host, &clientConf, ch) // pass in the channel
}
for x := 0; x < len(hosts); x++ {
fmt.Println(<-ch) // this will block until one is ready to send
}
And edit your runCommand function to accept the channel, remove references to WaitGroup, and execute the send exactly once under all conditions:
func runCommand(host string, config *ssh.ClientConfig, ch chan CommandResult) {
// do stuff that generates output, err; then when ready to exit function:
ch <- CommandResult{output, err}
}
EDIT: Question updated with stdout message order requirements
I'd like to get nicely formatted output that ignores the order of events
In this case, remove all print messages from runCommand, you're going to put all output into the element you're passing on the channel so it can be grouped together. Edit the CommandResult type to contain additional fields you want to organize, such as:
type CommandResult struct {
Host string
Output string
Err error
}
If you don't need to sort your results, you can just move on to printing the data received, e.g.
for x := 0; x < len(hosts); x++ {
r := <-ch
fmt.Printf("Host: %s----\nOutput: %s\n", r.Host, r.Output)
if r.Err != nil {
fmt.Printf("Error: %s\n", r.Err)
}
}
If you do need to sort your results, then in your main goroutine, add the elements received on the channel to a slice:
...
results := make([]CommandResult, 0, len(hosts))
for x := 0; x < len(hosts); x++ {
results = append(results, <-ch) // this will block until one is ready to send
}
Then you can use the sort package in the Go standard library to sort your results for printing. For example, you could sort them alphabetically by host. Or you could put the results into a map with host string as the key instead of a slice to allow you to print in the order of the original host list.
When I run this code an incoming UDP packet gets read in, however no packet gets sent back out. Why is this? (I verified this fact with wireshark). I want to be able to communicate two ways over a UDP connection, how do I achieve this with golang?
//Node 1
func main() {
addr := net.UDPAddr{
Port: 7000,
IP: net.ParseIP("127.0.0.1"),
}
conn, err := net.ListenUDP("udp", &addr)
defer conn.Close()
if err != nil {
panic(err)
}
for {
b := make([]byte, 10)
conn.Read(b)
fmt.Println(string(b[:]))
conn.Write([]byte("sending back"))
}
}
func main() {
sock, _ := net.Dial("udp", "127.0.0.1:7000")
buf := make([]byte, 10)
sock.Write([]byte("first send"))
sock.Read(buf)
fmt.Println(string(buf[:]))
}
Remember, UDP is connection-less. When you call conn.Write, your listener doesn't know where to send the packet. In your server code, you should be using UDPConn.ReadFromUDP and UDPConn.WriteToUDP to obtain and specify the client address, as mentioned in the documentation:
The returned connection's ReadFrom and WriteTo methods can be used to receive and send UDP packets with per-packet addressing.
Your modified Node 1 loop could then look something like the following:
for {
b := make([]byte, 10)
n, clientAddr, _ := conn.ReadFromUDP(b) // TODO: error check
fmt.Println(string(b[:n]))
conn.WriteToUDP([]byte("sending back"), clientAddr)
}
Note: I'm more interested in understanding general Go concepts/patterns, rather than solving this contrived example.
The Go (golang) WebSocket package provides a trivial echo server example, which condenses down to something like this:
func EchoServer(ws *websocket.Conn) { io.Copy(ws, ws); }
func main() {
http.Handle("/echo", websocket.Handler(EchoServer));
http.ListenAndServe(":12345", nil);
}
The server handles simultaneous connections, and I'm trying to upgrade it to a basic chat server by echoing the input to all connected clients.
How would I go about providing the EchoServer handler access to each of the open connections?
A quick little almost-functional example to give you an idea
var c = make(chan *websocket.Conn, 5) //5 is an arbitrary buffer size
var c2 = make(chan []byte, 5)
func EchoServer(ws *websocket.Conn) {
buff := make([]byte, 256)
c <- ws
for size, e := ws.Read(buff); e == nil; size, e = ws.Read(buff) {
c2 <- buff[0:size]
}
ws.Close()
}
func main() {
go func() {
var somekindofstorage
for {
select {
case newC := <-c:
somekindofstorage.Add(newC)
case msg := <-c2:
for _, v := range somekindofstorage {
if _, e := v.Write(msg); e != nil { //assuming the client disconnected on write errors
somekindofstorage.Remove(v)
}
}
}
}
}()
http.Handle("/echo", websocket.Handler(EchoServer));
http.ListenAndServe(":12345", nil);
}
This starts a goroutine that listens on two channels, one for new connections to add and one for messages to send to all active connection. somekindofstorage could be a map or a vector.
Edit:
Alternatively, you could just store all connections in a global map and write to each from EchoServer. But maps aren't designed to be accessed concurrently.