Eval: backon as standard retry/backoff utility (replaces ad-hoc retry across products) #65

New issue

Closed

opened 2026-07-04 23:23:31 -07:00 by pyr0ball · 0 comments

pyr0ball commented

2026-07-04 23:23:31 -07:00

Owner

Source

https://github.com/Llucs/backon — MIT license

What it is

Pure Python (zero stdlib dependencies), Python 3.10+. Modern evolution of the backoff
library with native async/await support and additional resilience primitives.

Wait strategies: exponential, constant, fibonacci, linear, decay, randomized variants,
chainable composition (wait_chain), full/random/no jitter.

API styles (all async-native):

Decorators: @on_exception, @on_predicate
Functional: retry() function
Context manager: Retrying class
Callable object: RetryingCaller

Beyond basic retry:

Circuit breaker — opens after N failures, blocks calls until cooldown
Hedging — fires N concurrent requests, returns first success (latency optimization)
Global retry toggle — disable all retries in test environments with one flag
Prometheus/OpenTelemetry metrics — retry counts, circuit state, latency

CF use cases

circuitforge-core (primary)

Add backon as the standard retry wrapper in cf-core modules that make external calls:

llm/ module — LLM endpoint calls (cloud tier: Anthropic, local tier: ollama/vllm)
affiliates/ module — affiliate link resolution/tracking
reranker/ module — reranker API calls
activitypub/ module — federated network calls

Expose a thin circuitforge_core.retry wrapper so products get consistent retry
behavior without importing backon directly.

cf-orch (secondary)

Hedging is particularly interesting for cf-orch worker dispatch:

Fire task to N available workers simultaneously, accept first completion
Reduces tail latency on slow workers without needing explicit load balancing
Circuit breaker useful for marking workers as degraded after repeated failures

Per-product use cases (consumed via cf-core)

Product	External calls that need retry
Peregrine	Job board scraping, LLM generation
Magpie	Reddit API (rate-limited), social posts
Snipe	eBay API, auction platform scraping
Kiwi	Barcode lookup APIs, receipt OCR
Osprey	Twilio, government IVR endpoints

Current state

Each product handles retry ad-hoc or not at all. Standardizing via cf-core eliminates
duplicated retry logic and ensures consistent backoff behavior across the menagerie.

License

MIT — clean for all CF tiers including commercial/cloud.

Eval steps

conda run -n cf pip install backon and test against cf-core's LLM module
Compare API ergonomics vs tenacity (current ad-hoc alternative in some products)
Prototype circuit breaker for cf-orch worker dispatch
If adopted: add backon to cf-core dependencies, add circuitforge_core/retry.py
wrapper, update affected modules

Note

Zero-dependency is a meaningful constraint — backon won't pull in transitive packages
that create conflicts across products' conda env.

## Source https://github.com/Llucs/backon — MIT license ## What it is Pure Python (zero stdlib dependencies), Python 3.10+. Modern evolution of the `backoff` library with native async/await support and additional resilience primitives. Wait strategies: exponential, constant, fibonacci, linear, decay, randomized variants, chainable composition (`wait_chain`), full/random/no jitter. API styles (all async-native): - Decorators: `@on_exception`, `@on_predicate` - Functional: `retry()` function - Context manager: `Retrying` class - Callable object: `RetryingCaller` Beyond basic retry: - **Circuit breaker** — opens after N failures, blocks calls until cooldown - **Hedging** — fires N concurrent requests, returns first success (latency optimization) - **Global retry toggle** — disable all retries in test environments with one flag - **Prometheus/OpenTelemetry metrics** — retry counts, circuit state, latency ## CF use cases ### circuitforge-core (primary) Add `backon` as the standard retry wrapper in cf-core modules that make external calls: - `llm/` module — LLM endpoint calls (cloud tier: Anthropic, local tier: ollama/vllm) - `affiliates/` module — affiliate link resolution/tracking - `reranker/` module — reranker API calls - `activitypub/` module — federated network calls Expose a thin `circuitforge_core.retry` wrapper so products get consistent retry behavior without importing backon directly. ### cf-orch (secondary) Hedging is particularly interesting for cf-orch worker dispatch: - Fire task to N available workers simultaneously, accept first completion - Reduces tail latency on slow workers without needing explicit load balancing - Circuit breaker useful for marking workers as degraded after repeated failures ### Per-product use cases (consumed via cf-core) | Product | External calls that need retry | |---------|-------------------------------| | Peregrine | Job board scraping, LLM generation | | Magpie | Reddit API (rate-limited), social posts | | Snipe | eBay API, auction platform scraping | | Kiwi | Barcode lookup APIs, receipt OCR | | Osprey | Twilio, government IVR endpoints | ## Current state Each product handles retry ad-hoc or not at all. Standardizing via cf-core eliminates duplicated retry logic and ensures consistent backoff behavior across the menagerie. ## License MIT — clean for all CF tiers including commercial/cloud. ## Eval steps 1. `conda run -n cf pip install backon` and test against cf-core's LLM module 2. Compare API ergonomics vs `tenacity` (current ad-hoc alternative in some products) 3. Prototype circuit breaker for cf-orch worker dispatch 4. If adopted: add `backon` to cf-core dependencies, add `circuitforge_core/retry.py` wrapper, update affected modules ## Note Zero-dependency is a meaningful constraint — backon won't pull in transitive packages that create conflicts across products' conda env.