perf: optimize index build — permanent caches + faster MB rate limit
Three changes to reduce subsequent index build times: 1. Positive image/rels cache TTL: 30 days → 365 days Artist url-rels and resolved image URLs rarely change. Already-indexed artists make zero API calls on rebuild. 2. Negative cache (misses) stays at 30 days so new images are discovered within a month of being added upstream. 3. MB rate limiter for background indexing: 1.0 → 1.5 req/s url-rels lookups are lightweight; 1.5/s is well within what MB handles (Picard and Kodi both use similar rates). Cuts the MB-bound portion of index build by ~33%. Also adds NewRateLimiterF for fractional rates and caches MB rels fetch failures (30-day miss TTL) to avoid retrying unreachable artists every build.
This commit is contained in:
@@ -38,6 +38,14 @@ func NewRateLimiterN(n int) *RateLimiter {
|
||||
}
|
||||
}
|
||||
|
||||
// NewRateLimiterF returns a rate limiter that allows f requests
|
||||
// per second with a burst of 1.
|
||||
func NewRateLimiterF(f float64) *RateLimiter {
|
||||
return &RateLimiter{
|
||||
limiter: rate.NewLimiter(rate.Limit(f), 1),
|
||||
}
|
||||
}
|
||||
|
||||
// Wait blocks until the rate limiter allows the caller to proceed
|
||||
// or the context is cancelled. Returns ctx.Err() if the context
|
||||
// expires before a token becomes available.
|
||||
|
||||
Reference in New Issue
Block a user