Tutorial
Building a self-service LLM gateway with Amazon Bedrock and LiteLLM
AWS describes how to build a self-service LLM gateway using Amazon Bedrock Application Inference Profiles and LiteLLM running on AWS Fargate, exposing an OpenAI-compatible API for developer tools. The post covers per-user API keys, spend budgets, rate limits, inference-profile routing, and observability with CloudWatch and Langfuse. It also notes that self-service key issuance requires LiteLLM Enterprise and recommends adding responsible AI controls before production. The architecture is presented with illustrative configuration snippets and AWS CDK constructs.
Amazon BedrockLiteLLMLLM gateway