golang 40 lines of code implement a common pool
- 2020-06-15 09:17:38
- OfStack
Code warehouse
goroutine-pool
Covariance management for golang
The golang coroutine mechanism is very convenient to solve the problem of concurrent programming, but the coroutine is not without overhead, so it is necessary to appropriately limit the number under 1.
Code that does not use a pool (chan for the sample code)
func (p *converter) upload(bytes [][]byte) ([]string, error) {
ch := make(chan struct{}, 4)
wg := &sync.WaitGroup{}
wg.Add(len(bytes))
ret := make([]string, len(bytes))
// upload
for index, item := range bytes {
ch <- struct{}{}
go func(index int, imageData []byte) {
defer func() {
wg.Done()
<-ch
}()
link, err := qiniu.UploadBinary(imageData, fmt.Sprintf("%d.png", time.Now().UnixNano()))
if err != nil {
log.Println(" Failed to upload image ", err.Error())
return
}
ret[index] = link
}(index, item)
}
wg.Wait()
return ret, nil
}
There are two requirements that need to be implemented:
Limit the maximum covariance number, in this case 4
Wait for all coroutines to complete, in this case bytes slice length
Code that USES the pooling
func (p *converter) upload(bytes [][]byte) ([]string, error) {
ret := make([]string, len(bytes))
pool := goroutine_pool.New(4, len(bytes))
for index, item := range bytes {
index := index
item := item
pool.Submit(func() {
link, err := qiniu.UploadBinary(item, fmt.Sprintf("%d.png", time.Now().UnixNano()))
if err != nil {
log.Println(" Failed to upload image ", err.Error())
return
}
ret[index] = link
})
}
pool.Wait()
return ret, nil
}
The biggest difference you can see is that you only need to focus on the business logic, where concurrency control and waiting have been taken over by the pooling
conclusion