Async Parallel Enrichment Pattern: Fan-Out, Gather, Partial Results (2026)

Q: What is the async parallel enrichment pattern?

Async parallel enrichment is a pattern for assembling API responses that need data from multiple upstream services. Instead of calling each service sequentially (where total latency is the sum of all dependencies), the assembler dispatches every independent enrichment concurrently, gathers the responses with a wall-clock deadline, and assembles a partial-result-aware response. Wall-clock latency collapses from the sum of dependencies to the slowest single dependency (capped by the timeout). Failure of one optional enrichment becomes a missing widget rather than a failed request. The pattern is the workhorse of every assembled API endpoint in a modern microservices system — product detail pages, order summaries, dashboards, search results.

Q: Why use Promise.allSettled instead of Promise.all?

Promise.all rejects on the first failed promise; one failed optional enrichment causes the entire gather to reject and the endpoint returns 500 even though all the required data was present. Promise.allSettled waits for every promise to settle (resolve or reject) and returns an array of outcomes. The assembler walks the outcomes, treats fulfilled values as data, treats rejected values as missing, and decides what to do based on the required-vs-optional classification of each enrichment. The semantic difference is the difference between an endpoint that breaks on the first hiccup and one that ships a useful response under degraded conditions. Equivalents exist in every language — Go errgroup with collected errors, Java CompletableFuture with exceptionally, Python asyncio.gather with return_exceptions=True.

Q: How do I set the gather deadline and per-call timeouts?

Compose timeouts from the outside in. Start with the endpoint SLA (the latency you commit to clients — typically 1 second for a user-facing page); set the gather deadline shorter than the SLA (typically 600-900 ms for a page, 200-400 ms for an interactive widget); set per-call timeouts shorter than the gather deadline (0.5-0.7× the gather budget is a reasonable starting point); ensure downstream services' own per-call timeouts are shorter than yours. Inverted budgets — per-call timeouts longer than the gather budget, or downstream timeouts exceeding yours — are a common bug that produces "timed out before getting started" errors that look like cascading failures but are really configuration mistakes. Check the composition explicitly in code review.

Q: Why is cancellation important when the deadline expires?

When the gather deadline fires, the calls that have not returned must be cancelled — abort the HTTP request, release the connection, free the downstream worker. Without cancellation, the assembler returns the partial response to the user but the in-flight calls continue running in the background, holding connections and producing load on downstreams that nobody cares about anymore. Use AbortController in JavaScript, context.WithDeadline in Go, CancellationToken in .NET, deadline propagation in gRPC. Cancellation is what makes the pattern scale — without it the assembler accumulates orphaned in-flight work and degrades under load until the connection pool is exhausted. Cancellation should be mandatory, not optional, in the assembler's implementation.

Q: How do I classify enrichments as required versus optional?

Required enrichments must complete within budget or the response is 5xx — the data is essential to the page being usable (product details on a product page, order line items on an order summary, authentication context). Optional enrichments populate when they can; their absence degrades the experience without breaking it (reviews, recommendations, ads, related items, social proof). The classification is part of the API contract, written down explicitly, not buried in implementation. The required set caps the endpoint's availability — if you require five enrichments each at 99.9%, the endpoint cannot exceed 99.5% availability. Adding a new required enrichment increases the failure surface area; treat promotion from optional to required as a versioned contract change requiring (1) confirmation that the upstream SLA matches, (2) explicit alerting, and (3) phased rollout.

Q: What is request coalescing and why does it matter?

Multiple enrichments often need the same upstream data — three independent enrichments each fetching the same user-history payload triples load on the upstream for no benefit. Request coalescing keeps a map of in-flight calls keyed by URL and parameters; when a second caller asks for the same key while the first is still in flight, the second caller subscribes to the first call's promise rather than issuing a duplicate. The DataLoader pattern from GraphQL is the canonical implementation; equivalents exist in every language. Per-page load on the upstream goes from "number of widgets that need this data" to "one per page", typically a 3-5× reduction in upstream load with no architectural change. Coalescing should be scoped to the request, not global, to avoid cross-request data leakage.

Q: How do I handle caching with parallel enrichment?

Caching is the natural pair to parallel enrichment — a 60-second-cacheable enrichment removes itself from the gather budget on cache-hit requests. Decide caching per enrichment, not as a blanket policy: inventory cannot be cached (the buy button would lie about availability); personalisation tied to the user session cannot be cached across users; reviews and recommendations are typically safe to cache for 30-300 seconds. Use stale-while-revalidate semantics for soft enrichments — the assembler returns the cached value immediately and asynchronously refreshes the cache in the background, so the assembler never waits for a refresh. Document the per-enrichment caching decision in an enrichment registry; review changes to caching as you would review changes to data freshness contracts. Combined with request coalescing and per-request caching, this gives "fetch once per upstream key per incoming request" semantics for free.

Q: What observability do I need for parallel enrichment endpoints?

Endpoint-level latency hides the actual problem — the endpoint P99 is 1.2 seconds, the gather budget is 800 ms, and without per-dependency timing it is impossible to tell which enrichment is the cause. Emit per-enrichment timing, status, and partial-result outcome to the metrics pipeline. A dashboard that breaks endpoint latency down by dependency makes the cause visible: "the recommendation service P99 has doubled since Tuesday, dragging the product-page gather toward its budget." Partial-response rate (the percentage of responses missing one or more optional enrichments) is a primary SLI — at least as important as latency or availability. Trace propagation via OpenTelemetry lets the entire fan-out appear as one trace with each enrichment as a child span, so the shape of the trace immediately tells you whether parallelism is actually happening or whether the calls have accidentally chained.

Q: What is the partial-result contract and why is it explicit?

The response schema includes per-section status (present, missing, error) so the client can render appropriately. A widget that did not load gets a "couldn't load · retry" affordance; a section that returned partial data gets a clear visual indicator; a successful response gets the full render. Without an explicit partial-result contract, the endpoint ships responses with missing fields and no indication to the client; the client renders empty widgets without retry affordances; the user sees a half-broken page with no recovery path. The explicit contract is the single biggest difference between an enrichment endpoint that degrades gracefully under stress and one that flips between perfect and broken with no in-between. Define the per-section status enum in the API specification, not in the implementation; require clients to handle the missing-and-error states explicitly.

Q: When should I NOT use async parallel enrichment?

Three scenarios. (1) Sequenced enrichments with genuine data dependencies — if call B literally cannot start until A returns a token or ID it needs, the calls must chain; the answer is to look for opportunities to denormalise the upstream so the dependency goes away, not to pretend the chain can be parallelised. (2) Single-dependency endpoints — if the endpoint has only one upstream call, fan-out adds no value; the pattern is for assembled responses with multiple parallel sources. (3) Strong-consistency assembly — if the response requires all assembled data was read at exactly the same logical timestamp (rare, but real for some financial and audit use cases), parallel fan-out cannot satisfy the constraint without distributed-snapshot machinery; for these, route through a service that enforces snapshot semantics or denormalise into a read model that is consistent by construction.

Satyam Kumar

Back to Blog

ai-architecture

The Async Parallel Enrichment Pattern: Fan-Out, Gather, and Partial-Result Tolerance for Production APIs (2026)

By Satyam KumarApril 24, 202623 min read

async parallel enrichment microservices patterns fan-out gather promise allsettled request coalescing dataloader pattern per-call timeouts gather deadline partial result graceful degradation api assembly tail latency opentelemetry distributed systems system design

The Async Parallel Enrichment Pattern: Fan-Out, Gather, and Partial-Result Tolerance for Production APIs (2026)

Frequently Asked Questions

Share this article

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

AI & Cloud Architect. Helping teams build systems that scale to millions.

LinkedIn GitHub

The Async Parallel Enrichment Pattern: Fan-Out, Gather, and Partial-Result Tolerance for Production APIs (2026)

Frequently Asked Questions

Share this article

Comments

Leave a comment