AI Gateway evicts cached request bridges after 15 minutes and currently gives in-flight requests only five seconds to finish, cutting off otherwise healthy model calls and streams. Give evicted bridges up to ten minutes to drain without changing the cache TTL. New acquisitions use a replacement bridge while the retired one drains. Pool shutdown retains a five-second cancellation grace period for cached and already-retired bridges, even when an acquisition is blocked initializing MCP connections. Requests still running at the drain deadline are cancelled. Refs [AIGOV-672](https://linear.app/codercom/issue/AIGOV-672/aibridged-15-minute-bridge-ttl-cancels-in-flight-ai-gateway-requests). The separate acquisition/admission race tracked in [AIGOV-609](https://linear.app/codercom/issue/AIGOV-609) is outside this change. --- Created by OpenCode on behalf of @dannykopping.
8 lines
186 B
Go
8 lines
186 B
Go
package aibridged
|
|
|
|
// Expose the production timeouts only to external tests.
|
|
const (
|
|
BridgeDrainTimeoutForTest = bridgeDrainTimeout
|
|
PoolShutdownTimeoutForTest = poolShutdownTimeout
|
|
)
|