AWS recommends [application inference
profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-application-inference-profiles.html)
for attributing Bedrock spend to a team or workload, which requires
invoking Bedrock with the profile ARN as the `modelId`. AI Gateway used
the configured Bedrock model for both the invocation target and model
capability detection, and capability detection is substring matching
against model IDs. Application inference profile ARNs are opaque, so
they matched nothing.
Reproduced against a profile wrapping `anthropic.claude-opus-4-8`: the
client sent `thinking.type: "adaptive"`, the gateway rewrote it to
`enabled` + `budget_tokens` because the ARN did not look like an
adaptive-capable model, and Bedrock returned
```text
400 "thinking.type.enabled" is not supported for this model. Use "thinking.type.adaptive" and "output_config.effort" to control thinking behavior.
```
Models that accept both shapes, such as Sonnet 4.6, did not fail but had
adaptive thinking silently downgraded. The ARN was also recorded as the
interception model, so those requests never matched a price and may
appear in the weekly Missing AI Model Prices report.
The gateway now resolves application inference profile ARNs through
`GetInferenceProfile` and uses the model behind them for capability
detection, usage records, pricing, and metrics, while still invoking the
profile so AWS records the cost attribution. Resolution runs only for
`application-inference-profile` ARNs, so plain model IDs and
system-defined inference profile ARNs, which already carry the model ID,
cost no AWS call and need no new permission. Deployments that do
configure a profile must grant `bedrock:GetInferenceProfile` to the
identity the gateway already uses, including any assumed role.
Resolution failure fails provider construction, matching how other
misconfigured providers behave: the failure is logged and the provider
is reported as errored rather than serving requests it would shape
incorrectly.
## Concerns
**Every provider reload re-resolves every application inference
profile.**
Any provider create/update/delete publishes on
`AIProvidersChangedChannel`, and each subscriber responds by rebuilding
its whole snapshot: `Reload` fetches all providers and
`BuildProvidersFromProto` constructs every one of them from scratch
before `ReplaceProviders` swaps the snapshot in.
That rebuild used to be purely local work. This PR makes it hit the
network: a Bedrock provider configured with an AIP ARN now performs a
synchronous `GetInferenceProfile` call during construction,
sequentially, before the snapshot is published. So editing an unrelated
OpenAI provider re-resolves every AIP in the deployment. Consequences:
- Reload latency grows with the number of AIP-configured providers,
bounded by the 30s resolution timeout per provider.
- A transient AWS failure during an unrelated provider edit drops the
Bedrock provider from the new snapshot until the next reload, surfacing
as a 404 for a provider whose configuration never changed.
An ARN-keyed resolution cache would remove this: the Bedrock API has no
`UpdateInferenceProfile`, so a profile's underlying model is fixed at
creation and repointing requires a new ARN, hence a config change and a
reload. A process-lifetime cache is therefore sound with no invalidation
logic. Left out of this PR to keep it scoped.
## Improvement Options
1. Keep it as is.
2. Add a per-process in-memory cache that maps AIP ARNs to models.
3. Add a database-level cache for AIP-to-model resolution.
4. Move AIP resolution to the per-LLM-request path and cache the result
there.
My choice would be either 3 or 4. See the relevant P1s:
- https://github.com/coder/coder/pull/28877#discussion_r3936255218
- https://github.com/coder/coder/pull/28877#discussion_r3936255211
Closes https://linear.app/codercom/issue/AIGOV-488
Created by Coder Agents on behalf of @evgeniy-scherbina.