Go: Concurrency with Goroutines

Table of Contents

Following the Learn Go with Tests guide, exploring Go’s concurrency model with goroutines and channels.

Concurrency vs Parallelism
#

Concurrency: Dealing with multiple things at once

Parallelism: Doing multiple things at the same time (simultaneous execution)

How to Optimise
#

Let’s optimise a function that checks if websites are reachable:

type WebsiteChecker func(string) bool

func CheckWebsites(wc WebsiteChecker, urls []string) map[string]bool {
    results := make(map[string]bool)
    
    for _, url := range urls {
        results[url] = wc(url) 
        // Sequential - slow!
    }
    
    return results
}

Sequential execution is slow for many URLs. If each check takes 20ms, 100 URLs take 2+ seconds!

Benchmarking
#

func slowStubWebsiteChecker(_ string) bool {
    time.Sleep(20 * time.Millisecond)
    return true
}

func BenchmarkCheckWebsites(b *testing.B) {
    urls := make([]string, 100)
    for i := 0; i < len(urls); i++ {
        urls[i] = "a url"
    }
    
    for b.Loop() {
        CheckWebsites(slowStubWebsiteChecker, urls)
    }
}

Result: 2249228637 ns/op - about 2.25 seconds!

Goroutines
#

A goroutine is a lightweight thread

func CheckWebsites(wc WebsiteChecker, urls []string) map[string]bool {
    results := make(map[string]bool)
    
    for _, url := range urls {
        go func() {
            results[url] = wc(url) 
            // Race condition!
        }()
    }
    
    return results
}

go keyword starts a goroutine
Anonymous functions: func() { ... }()
() calls the anonymous function straight away

Race Condition
#

Running the goroutine version might result in:

--- FAIL: TestCheckWebsites (0.00s)
    CheckWebsites_test.go:31: Wanted map[...], got map[]

Or even worse:

fatal error: concurrent map writes

The function returns before goroutines finish, or multiple goroutines write to the map simultaneously.

Using the Race Detector
#

Go provides a built-in race detector:

go test -race

Channels
#

Channels provide safe communication between goroutines:

type result struct {
    string
    bool
}

func CheckWebsites(wc WebsiteChecker, urls []string) map[string]bool {
    results := make(map[string]bool)
    resultChannel := make(chan result)
    
    for _, url := range urls {
        go func() {
            resultChannel <- result{url, wc(url)} 
            // Send to channel
        }()
    }
    
    for i := 0; i < len(urls); i++ {
        r := <-resultChannel  
        // Receive from channel
        results[r.string] = r.bool
    }
    
    return results
}

make(chan result) creates a channel
channel <- value sends to channel
value := <-channel receives from channel
Channels are synchronous by default (blocking)

Performance Results
#

After fixing with channels:

BenchmarkCheckWebsites-8   100   23406615 ns/op

0.023 seconds - about 100x faster than the original!

Concurrency vs Parallelism #

How to Optimise #

Benchmarking #

Goroutines #

Race Condition #

Using the Race Detector #

Channels #

Performance Results #