Go is fantastic for handling lightweight goroutines, the ability to spin up a thousand ‘parallel’ threads working simultaneously.[^1]

But goroutines seem to have one problem: how do you handle error return values from a goroutine? Or indeed, any return value at all?

const port = `:12345`
func mockServer() {
    http.Handle(`/`, http.FileServer(http.Dir(`.`)))
    http.ListenAndServe(port, nil)
}

func testUrl(u) {
    res, err := http.Get(u)
    if nil!=err {
    	// ?
    }
    res.Body.Close()
}

func main() {
    go mockServer()
    
    N := 10000    
    for i:=0; i<N; i++ {
        go testUrl(`http://localhost` + port)
    }
}

Firstly, this program won’t even work, because we exit before giving the goroutines a chance to finish their work. We can fix this with a sync.WaitGroup:

var wait sync.WaitGroup

func testUrl(u) {
    defer wait.Done()
    res, err := http.Get(u)
    if nil!=err {
        // ?
        return
    }
    res.Body.Close()
}

func main() {
    go mockServer()
    N := 10000
    wait.Add(N)
    for i:=0; i<N; i++ {
        go testUrl(&wait, `http://localhost`+port)
    }
    wait.Wait()
}

Now our program will make 10000 web requests, and then exit. But how will we know whether those requests succeeded?

Here’s a neat idea to use a channel to report the errors:

var wait sync.WaitGroup
var ERR chan error

func testUrl(u) {
    defer wait.Done()
    res, err := http.Get(u)
    if nil!=err {
        ERR <- err
        return
    }
	res.Body.Close()
}

func main() {
    go mockServer()
    N := 10000
    ERR = make(chan error)
    
    wait.Add(N)
    for i:=0; i<N; i++ {
        go testUrl(&wait, `http://localhost`+port)
    }
    wait.Wait()
    close(ERR)
    for err := range ERR {
        fmt.Fprintln(os.Stderr, err.Error())
    }
}

Sadly this won’t work. We don’t read from the ERR channel until after the wait.Wait, but a failed goroutine will be trying to write to the channel before it’s finished, and we’ll have a deadlock.

One way around this is give the ERR channel a buffer as large as the number of goroutines:

ERR = make(chan error, N)

That solves the problem, since this channel can buffer the maximum number of errors we might encounter, so that the goroutines can write to the channel without needing to have anything reading from the channel.

There’s a more elegant solution:

var wait sync.WaitGroup
var ERR chan error

func testUrl(u) {
    defer wait.Done()
    res, err := http.Get(u)
    if nil!=err {
        ERR <- err
        return
    }
    res.Body.Close()
}

func main() {
    go mockServer()
    
    N := 10000
    ERR = make(chan error)
    
    wait.Add(N)
    
    go func() {
        wait.Wait()
        close(ERR)
    }()

    for i:=0; i<N; i++ {
        go testUrl(&wait, `http://localhost`+port)
    }
    
    for err := range ERR {
        fmt.Fprintln(os.Stderr, err.Error())
    }
}

This solution has eliminated all our issues. We don’t close the ERR channel until every goroutine has finished. But we start reading from the channel before our goroutines are finished. We have a single goroutine dedicated to closing the channel when all the goroutines are completed.

We have goroutines that can report errors elegantly, and receiving those errors is built into our waiting for their completion.

[^1]: They don’t actually run simultaneously unless you’ve increased the number of CPU’s that your programme uses. See https://golang.org/pkg/runtime/#GOMAXPROCS