Reliability

An LLM API should be treated as a variable external dependency. Responses can be slow, rate-limited, malformed or unavailable even when the rest of the application is healthy. Classify failures before retrying Retry transient network failures and explicit capacity responses with exponential...