This is the first step of my intention to write a series of articles where I would explain some common patterns used in Golang. Let’s start with generator pattern :
1type Generator func() <-chan int
Generator is a function or method which returns a sequence of values. In Golang, this usually means returning a channel of values of desired type. In our example, we will return a channel of integers. Let’s see an implementation of this:
1func randomGenerator() <-chan int {
2 count := 100
3 max := 1000
4 dc := make(chan int)
5 rand.Seed(time.Now().UnixNano())
6
7 go func() {
8 for i := 1; i <= count; i++ {
9 dc <- 1 + rand.Intn(max)
10 }
11 close(dc)
12 }()
13
14 return dc
15}
As you can see, we have a goroutine inside the function and the reason is that in Go, we usually want to do the things concurrently. This means the generator is going to generate some values and at the same time we are going to process the generated values in another goroutine. And this “another goroutine” may actually be a wait group which means a pool of workers where we wait for all workers to finish before we continue or end the execution. Now let’s see an example of Golang’s wait group:
1type WorkerPool struct {
2 Generator Generator
3 wg sync.WaitGroup
4 data <-chan int
5}
6
7func (p *WorkerPool) Run(concurrency int) {
8 p.data = p.Generator()
9
10 for i := 1; i <= concurrency; i++ {
11 p.wg.Add(1)
12 go p.worker(i)
13 }
14
15 p.wg.Wait()
16}
17
18func (p *WorkerPool) worker(i int) {
19 fmt.Printf("Worker %d started\n", i)
20 defer fmt.Printf("Worker %d exiting\n", i)
21 defer p.wg.Done()
22
23 for {
24 d, ok := <-p.data
25 if !ok {
26 // data channel closed
27 return
28 }
29
30 fmt.Printf("Worker %d processing data: %d\n", i, d)
31 // do some heavy lifting
32 time.Sleep(200 * time.Millisecond)
33 }
34}
Here we have a struct which contains one exported field, Generator and two non-exported fields, wg and data. Generator field is of Generator type described above in the article and is used to define a generator function to be used to generate a series of values. We run this method (p.Generator) early in the Run method in order to populate the data channel.
After that, in the Run method we start a group of goroutines controlled by sync.WaitGroup. Before we start each worker goroutine, we run p.wg.Add method adding 1 to it, meaning we tell the wait group that there is one more goroutine to wait for. Later, each worker calls p.wg.Done, which means we are telling the wait group that one worker finished or actually there is one worker less to wait for. Add and Done methods basically add and remove a worker (goroutine) to/from the wait group. Finally, p.wg.Wait will block the execution until all workers in the wait group finish the execution.
That would be all about the Run method and now let’s explain the things in non-exported method, worker. We have an infinite loop there, meaning we need some way to finish the goroutine and that is the data channel. This loop will end when p.data channel is closed and we check this condition using the ok variable when we receive a value from the channel. Closed channel continuosly emits zero values, but we may use second variable (ok in this case) to find out if we really received a value from the channel or is this just a dummy emition from the closed channel. In case this is just “dummy” value, we return but still deffered methods will be executed, meaning p.wg.Done will be called anaways. You may find more about the behaviour of the closed channels here .
Finally, let’s say something about function wrapping and closures in Golang. Generator type from above defines the signature fot this method, but in the real life we may need to pass some arguments to the function. How to do this when Generator type function has no arguments? The answer is function wrapping and we wrap a function which will call a closure in this case, because it uses variable from the wrapper function. This looks like the following:
1func random(count, max int) Generator {
2 return func() <-chan int {
3 dc := make(chan int)
4 rand.Seed(time.Now().UnixNano())
5
6 go func() {
7 for i := 1; i <= count; i++ {
8 dc <- 1 + rand.Intn(max)
9 }
10 close(dc)
11 }()
12
13 return dc
14 }
15}
So, function random returns a value of Generator type and this can be easily populate the exported, Generator field of WorkerPool struct. Inside the closure, we can used values passed to the random function and still return the right function of Generator type. This is very common in Golang and it is actually called functional programming.
Finally, we run all of this in the following way:
1func main() {
2 wp := WorkerPool{
3 Generator: random(100, 1000),
4 }
5 wp.Run(3)
6}
There is not much to explain here, but you may continue learning and testing this in Go Playground.