gopkg/pond
2
0
Fork 0
Code Pull Requests Activity

rename

Browse Source
This commit is contained in:
Сергей Солодягин
2024-04-09 00:01:28 +03:00
parent b9d24f45ff
commit 37ea64509a
19 changed files with 54 additions and 174 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/FUNDING.yml
-3
View File
@@ -1,3 +0,0 @@
# These are supported funding model platforms
github: alitto
.github/workflows/codeql-analysis.yml
-70
View File
@@ -1,70 +0,0 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"
on:
push:
branches: [ master ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]
schedule:
- cron: '40 15 * * 1'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://git.io/codeql-language-support
steps:
- name: Checkout repository
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language
#- run: |
# make bootstrap
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
.github/workflows/main.yml
-41
View File
@@ -1,41 +0,0 @@
on:
push:
branches:
- master
- main
pull_request:
name: Build
jobs:
test:
name: Test
strategy:
matrix:
go-version: [1.15.x, 1.16.x, 1.17.x, 1.18.x, 1.19.x]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Install Go
uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
uses: actions/checkout@v2
- name: Test
run: make test
codecov:
name: Upload coverage report to Codecov
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
uses: actions/checkout@v2
- name: Test
run: make coverage
- uses: codecov/codecov-action@v3
with:
files: coverage.out
fail_ci_if_error: true
verbose: true
README.md
+18 -16
View File
@@ -4,6 +4,7 @@
<a title="Go Report Card" target="_blank" href="https://goreportcard.com/report/github.com/alitto/pond"><img src="https://goreportcard.com/badge/github.com/alitto/pond"/></a>
# pond
Minimalistic and High-performance goroutine worker pool written in Go
## Motivation
@@ -12,8 +13,8 @@ This library is meant to provide a simple way to limit concurrency when executin
Some common scenarios include:
- Executing queries against a Database with a limited no. of connections
- Sending HTTP requests to a a rate/concurrency limited API
- Executing queries against a Database with a limited no. of connections
- Sending HTTP requests to a a rate/concurrency limited API
## Features:
@@ -40,20 +41,20 @@ Some common scenarios include:
## How to install
```bash
go get -u github.com/alitto/pond
go get -u git.company.lan/gopkg/pond
```
## How to use
### Worker pool with dynamic size
``` go
```go
package main
import (
"fmt"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func main() {
@@ -77,13 +78,13 @@ func main() {
### Worker pool with fixed size
``` go
```go
package main
import (
"fmt"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func main() {
@@ -107,13 +108,13 @@ func main() {
### Submitting a group of tasks
``` go
```go
package main
import (
"fmt"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func main() {
@@ -142,7 +143,7 @@ func main() {
This feature provides synchronization, error propagation, and Context cancelation for subtasks of a common task. Similar to `errgroup.Group` from [`golang.org/x/sync/errgroup`](https://pkg.go.dev/golang.org/x/sync/errgroup) package with concurrency bounded by the worker pool.
``` go
```go
package main
import (
@@ -150,7 +151,7 @@ import (
"fmt"
"net/http"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func main() {
@@ -197,7 +198,7 @@ func main() {
Specifies the minimum number of worker goroutines that must be running at any given time. These goroutines are started when the pool is created. The default value is 0. Example:
``` go
```go
// This will create a pool with 5 running worker goroutines
pool := pond.New(10, 1000, pond.MinWorkers(5))
```
@@ -206,14 +207,14 @@ pool := pond.New(10, 1000, pond.MinWorkers(5))
Defines how long to wait before removing idle worker goroutines from the pool. The default value is 5 seconds. Example:
``` go
```go
// This will create a pool that will remove workers 100ms after they become idle
pool := pond.New(10, 1000, pond.IdleTimeout(100 * time.Millisecond))
```
#### PanicHandler
Allows to configure a custom function to handle panics thrown by tasks submitted to the pool. The default handler just writes a message to standard output using `fmt.Printf` with the following contents: `Worker exits from a panic: [panic] \n Stack trace: [stack trace]`). Example:
Allows to configure a custom function to handle panics thrown by tasks submitted to the pool. The default handler just writes a message to standard output using `fmt.Printf` with the following contents: `Worker exits from a panic: [panic] \n Stack trace: [stack trace]`). Example:
```go
// Custom panic handler function
@@ -233,7 +234,7 @@ Configures the strategy used to resize the pool when backpressure is detected. Y
- **Balanced**: tries to find a balance between responsiveness and throughput. It's suitable for general purpose worker pools or those that will operate close to 50% of their capacity most of the time.
- **Lazy**: maximizes throughput at the expense of responsiveness. This strategy is meant for worker pools that will operate close to their max. capacity most of the time.
``` go
```go
// Example: create pools with different resizing strategies
eagerPool := pond.New(10, 1000, pond.Strategy(pond.Eager()))
balancedPool := pond.New(10, 1000, pond.Strategy(pond.Balanced()))
@@ -244,7 +245,7 @@ lazyPool := pond.New(10, 1000, pond.Strategy(pond.Lazy()))
Configures a parent context on this pool to stop all workers when it is cancelled. The default value `context.Background()`. Example:
``` go
```go
// This creates a pool that is stopped when myCtx is cancelled
pool := pond.New(10, 1000, pond.Context(myCtx))
```
@@ -260,6 +261,7 @@ As the name suggests, the "Eager" strategy always spawns an extra worker when th
### Stopping a pool
There are 3 methods available to stop a pool and release associated resources:
- `pool.Stop()`: stop accepting new tasks and signal all workers to stop processing new tasks. Tasks being processed by workers will continue until completion unless the process is terminated.
- `pool.StopAndWait()`: stop accepting new tasks and wait until all running and queued tasks have completed before returning.
- `pool.StopAndWaitFor(deadline time.Duration)`: similar to `StopAndWait` but with a deadline to prevent waiting indefinitely.
examples/dynamic_size/dynamic_size.go
+1 -1
View File
@@ -3,7 +3,7 @@ package main
import (
"fmt"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func main() {
examples/dynamic_size/go.mod
+3 -5
View File
@@ -1,9 +1,7 @@
module github.com/alitto/pond/examples/dynamic_size
module git.company.lan/gopkg/pond/examples/dynamic_size
go 1.19
require (
github.com/alitto/pond v1.7.1
)
require git.company.lan/gopkg/pond v0.0.0-00010101000000-000000000000
replace github.com/alitto/pond => ../../
replace git.company.lan/gopkg/pond => ../../
examples/fixed_size/fixed_size.go
+1 -1
View File
@@ -3,7 +3,7 @@ package main
import (
"fmt"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func main() {
examples/fixed_size/go.mod
+3 -5
View File
@@ -1,9 +1,7 @@
module github.com/alitto/pond/examples/fixed_size
module git.company.lan/gopkg/pond/examples/fixed_size
go 1.19
require (
github.com/alitto/pond v1.7.1
)
require git.company.lan/gopkg/pond v0.0.0-00010101000000-000000000000
replace github.com/alitto/pond => ../../
replace git.company.lan/gopkg/pond => ../../
examples/group_context/go.mod
+3 -5
View File
@@ -1,9 +1,7 @@
module github.com/alitto/pond/examples/group_context
module git.company.lan/gopkg/pond/examples/group_context
go 1.19
require (
github.com/alitto/pond v1.7.1
)
require git.company.lan/gopkg/pond v0.0.0-00010101000000-000000000000
replace github.com/alitto/pond => ../../
replace git.company.lan/gopkg/pond => ../../
examples/group_context/group_context.go
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"fmt"
"net/http"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func main() {
examples/pool_context/go.mod
+3 -3
View File
@@ -1,7 +1,7 @@
module github.com/alitto/pond/examples/pool_context
module git.company.lan/gopkg/pond/examples/pool_context
go 1.19
require github.com/alitto/pond v1.7.1
require git.company.lan/gopkg/pond v0.0.0-00010101000000-000000000000
replace github.com/alitto/pond => ../../
replace git.company.lan/gopkg/pond => ../../
examples/pool_context/pool_context.go
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"os/signal"
"time"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
// Pressing Ctrl+C while this program is running will cause the program to terminate gracefully.
examples/prometheus/go.mod
+3 -3
View File
@@ -1,9 +1,9 @@
module github.com/alitto/pond/examples/fixed_size
module git.company.lan/gopkg/pond/examples/fixed_size
go 1.19
require (
github.com/alitto/pond v1.7.1
git.company.lan/gopkg/pond v0.0.0-00010101000000-000000000000
github.com/prometheus/client_golang v1.11.1
)
@@ -19,4 +19,4 @@ require (
google.golang.org/protobuf v1.33.0 // indirect
)
replace github.com/alitto/pond => ../../
replace git.company.lan/gopkg/pond => ../../
examples/prometheus/prometheus.go
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"net/http"
"time"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
examples/task_group/go.mod
+3 -5
View File
@@ -1,9 +1,7 @@
module github.com/alitto/pond/examples/task_group
module git.company.lan/gopkg/pond/examples/task_group
go 1.19
require (
github.com/alitto/pond v1.7.1
)
require git.company.lan/gopkg/pond v0.0.0-00010101000000-000000000000
replace github.com/alitto/pond => ../../
replace git.company.lan/gopkg/pond => ../../
examples/task_group/task_group.go
+1 -1
View File
@@ -3,7 +3,7 @@ package main
import (
"fmt"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func main() {
go.mod
+1 -1
View File
@@ -1,3 +1,3 @@
module github.com/alitto/pond
module git.company.lan/gopkg/pond
go 1.19
group_blackbox_test.go
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"testing"
"time"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func TestGroupSubmit(t *testing.T) {
pond_blackbox_test.go
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"testing"
"time"
"github.com/alitto/pond"
"git.company.lan/gopkg/pond"
)
func assertEqual(t *testing.T, expected interface{}, actual interface{}) {