AI Agents on AWS Bedrock + NestJS: 2026 Production Architecture

Q: Why put a NestJS gateway in front of Bedrock instead of calling the SDK directly?

A direct SDK call works for prototypes and internal tools at low traffic. The moment you have multiple teams, multiple tenants, regulated data, cost SLAs, audit requirements, or production-grade observability, you need a wrapping layer that owns: authentication and per-tenant authorisation, request shaping and prompt-template versioning, data-classification routing, rate limiting per tenant and per agent, cost capping per tenant, retry and fallback to alternate models, structured response shaping, audit logging with PII redaction, observability with trace propagation, and graceful degradation when Bedrock is throttling. None of this belongs inside the agent itself; all of it belongs in a gateway that NestJS is well-suited for. Without the gateway, every team makes these decisions independently and the cost, audit, and operational stories all break.

Q: Should I use Bedrock Agents (AgentCore) or raw InvokeModel calls?

It depends on the workload shape. Raw InvokeModel or Converse calls are right when the application orchestrates the loop manually — single-step interactions, simple Q&A, classification, summarisation. Bedrock Agents (AgentCore) are right when you need multi-step reasoning with tool use and knowledge-base retrieval and you want Bedrock to manage the loop — most enterprise agent use cases. Bedrock Agents combined with Step Functions are right when the workflow is long-running (minutes to hours), needs durability across process restarts, or has complex orchestration that benefits from Step Functions native states. The gateway should expose a uniform interface across all three so the choice is made per-endpoint rather than baked into application code, and the workload can move from one to another as requirements change.

Q: How much does a typical Bedrock agent invocation cost?

A single agent invocation typically consumes 5,000-50,000 tokens; a complex multi-agent workflow can consume 100,000+ tokens. At Claude Sonnet pricing (~$3 per million input tokens, ~$15 per million output tokens) a 20K-token invocation is roughly $0.06-$0.30. At Haiku pricing (~$0.80 input, ~$4 output) the same invocation is $0.02-$0.08. At Nova Lite (~$0.06 input, ~$0.24 output) it is under $0.01. Knowledge base retrieval adds embedding cost (small, ~$0.10 per million tokens) plus vector storage (OpenSearch Serverless costs roughly $25-100/month for a small KB). Action group Lambda invocations add Lambda cost (negligible per call). The big variable is whether you cache prompts — Anthropic prompt caching reduces input cost by 90% on cache hits and is the single biggest cost lever after model selection. A typical mid-sized production agent costs $500-5000/month depending on traffic; without per-tenant caps it can spike unpredictably.

Q: What model should I use for what?

The rule that works in production: cheapest model that meets the quality bar for the specific endpoint. A typical layered selection: Claude Sonnet (or Llama 3 70B as fallback) for the main agent reasoning loop where quality matters most; Claude Haiku for high-volume routing decisions, classification, simple summarisation, and as the supervisor in multi-agent setups where it is just delegating; Amazon Nova Lite or Pro for AWS-native workloads where Bedrock-tight integration and competitive cost matter; Llama 3 70B as the cost-fallback when Sonnet is throttling or the tenant cost cap is approached; Titan or Cohere embeddings for knowledge-base ingestion (Titan is cheapest, Cohere has stronger multi-lingual). Implement model routing as per-tenant per-endpoint configuration so you can tune without code changes. Default-to-Sonnet across the board is the most expensive mistake teams make.

Q: How do action groups work and what are the security considerations?

Action groups are tools the agent can invoke during its reasoning loop. Each action group is defined by an OpenAPI schema (which the model reads to decide whether and how to invoke it) and implemented by a Lambda function (which executes the business logic). When the agent decides to use a tool, Bedrock dispatches to the Lambda with the parameters the model chose, the Lambda runs the logic, and returns a JSON observation that goes back into the agents context. Security considerations: every action group Lambda must authenticate the agents invocation context to the tenant whose data is being accessed (the gateway passes a tenant ID via session attributes, the Lambda validates it before any data access); the Lambdas IAM role grants only the minimum data access for that action; input must be validated against the OpenAPI schema because the agent can produce malformed requests especially when prompt-injected; every invocation must be audit-logged with inputs and outputs. The action group Lambda is the security boundary — the agent itself has no direct access to your data, so the Lambda is what gives it access and must be hardened accordingly.

Q: When should I use Bedrock Knowledge Bases vs a custom RAG pipeline?

Bedrock Knowledge Bases is the managed-RAG offering — documents in S3, automatic chunking, automatic embedding via Titan/Cohere, vectors in OpenSearch Serverless or Aurora pgvector or Pinecone, query API exposed to the agent. It is good enough for most workloads — broad corpus, standard chunking, standard retrieval — and saves weeks of work. Use it when you want to ship fast and the off-the-shelf retrieval quality meets your bar. Use a custom RAG pipeline when you need bespoke chunking (e.g., code-aware chunking for code search, structure-preserving chunking for tables), custom rerankers (Cohere Rerank or a custom cross-encoder), sophisticated context assembly (graph-aware retrieval, hybrid keyword+vector with custom scoring), or when your retrieval source is something other than documents (a graph DB, a search-engine-indexed data lake). The hybrid pattern is common: Bedrock KB for the bulk corpus, a custom retrieval action group for the precision cases.

Q: How do I prevent runaway costs on Bedrock agents?

Three layers of defence. (1) Per-tenant token budgets enforced at the gateway: every invocation reserves tokens against the tenants monthly cap stored in Redis; if the budget is exhausted, the request is rejected with 429 before any model invocation. (2) Per-request hard limits: max_tokens parameter set to a reasonable upper bound (typically 4000-8000 for chat, higher only when needed); max-iterations on the agent reasoning loop set to 10-15; timeouts at the gateway and Lambda level. (3) Prompt caching for the stable prefix (system prompt, tool descriptions, knowledge context summary) — Anthropic Claude prompt caching reduces input cost by 90% on cache hits, which is the single biggest lever after model selection. Plus: monitoring of "agents that hit iteration cap" as a quality signal (a single bad prompt that loops 10 times per request and is invoked 10K times/day is five-figure monthly), monthly cost-per-tenant review, and dashboards that surface top-spending agents and tenants for the on-call to act on.

Q: How do I observe a Bedrock agent end-to-end?

Three layers, one trace ID. (1) Gateway-level: trace ID assigned at ALB ingress, propagated through NestJS interceptors into the Bedrock SDK calls into the action group Lambda invocations and back. AWS X-Ray or OpenTelemetry to a collector handles propagation natively. (2) Bedrock-internal: enable the trace flag on InvokeAgent to retrieve the agents internal reasoning loop — the models thoughts, the tool selections, the knowledge base queries. This is essential for debugging "why did the agent do that". (3) Per-token / per-cost: every Bedrock response includes input and output token counts; the gateway converts to dollar cost using the model price table, emits as CloudWatch metric tagged with tenant ID and agent ID, writes into a cost-tracking time series. Plus application metrics: per-agent requests/sec, tokens-in/out, cost/hour, p50/p95/p99 latency, error rate by category; per-tenant cumulative usage vs cap; per-action-group invocation count, success rate, p99 latency. Without all three layers, debugging production agent issues is guesswork.

Q: What guardrails do I need beyond Bedrock Guardrails?

Bedrock Guardrails provides broad-strokes filtering — harmful content categories (hate, violence, sexual, misconduct), PII detection and redaction (with regional patterns including Indian PAN/Aadhaar when configured), denied topics, word filters, and contextual grounding checks for hallucination detection against retrieved context. The gateway adds the application-specific layer that Bedrock cannot know about: tenant-specific redaction rules (a healthcare tenant has stricter PII rules than a marketing tenant), audit-log obfuscation (don't log the users full input verbatim if it contains an account number — store a hash plus a redacted version), prompt-injection detection (basic patterns plus an LLM-based detector for sophisticated attempts), and output schema validation (if the response should be JSON conforming to a schema, validate before returning). The PII lifecycle pattern: input arrives, gateway runs PII detection, identified PII is hashed and replaced with a token before being sent to the model, model response references the token, gateway substitutes the original PII back in for the user response, both redacted prompt and redacted response are logged for audit, original values are not stored beyond request lifetime.

Q: What are the most common Bedrock agent failure modes in production?

Five recurring failure modes. (1) Bedrock throttling under burst load — per-region per-account quotas on requests-per-minute and tokens-per-minute hit during traffic spikes, surfacing as opaque agent failures. Mitigation: provisioned throughput for highest-traffic models, fallback model in different family, exponential backoff with jitter. (2) Long-tail latency from the reasoning loop — multi-step agents can take 20-60 seconds because each step is a serial round-trip. Mitigation: streaming response, async invocation pattern with workflow ID + polling/webhook, Step Functions for long-running workflows. (3) Action group Lambda cold starts — 200ms-3s per cold Lambda × 5-10 tools per workflow can dominate latency. Mitigation: small Lambda bundles, Node.js over Python/Java when latency matters, provisioned concurrency on hot action groups, batch multiple ops into one Lambda where possible. (4) Knowledge base staleness — Bedrock KB re-indexes on schedule, agent may reason over stale context. Mitigation: explicit refresh trigger when critical docs change, TTL on cached agent responses, observability on indexing lag. (5) Agent loops and runaway costs — poorly-prompted agent hits max-iterations while consuming 10K+ tokens per run. Mitigation: per-request token caps, conservative max-iterations, monitoring of cap-hits as quality signal, regular prompt review.

Satyam Kumar

返回博客

ai-architecture

AI Agents on AWS Bedrock + NestJS: Production Architecture (2026)

By Satyam KumarApril 28, 202620 min read

AWS Bedrock Bedrock Agents AgentCore NestJS AI agents action groups Bedrock Knowledge Bases Claude Sonnet Claude Haiku Amazon Nova Llama 3 Bedrock Guardrails multi-agent collaboration agent observability agent cost control prompt caching LLM gateway production AI architecture

AI Agents on AWS Bedrock + NestJS: Production Architecture (2026)

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

AI Agents on AWS Bedrock + NestJS: Production Architecture (2026)

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment