Four AI System Anti-Patterns: Unclassified Query, Generic Prompt, Mono Safety, Confident Wrong (2026)

Q: What are the four most damaging AI system anti-patterns?

The unclassified query (treating every user input as a single category and routing through one one-size-fits-all flow), the generic single-prompt (one giant 4-8k token system prompt trying to cover every behaviour), monolithic safety (one safety configuration applied to every category and tenant regardless of differing requirements), and confident misclassification (the model is wrong but expresses high confidence so the system acts on the wrong answer with no calibration, novelty detection, or contradiction checking). The four share a structural theme — undifferentiated handling of inputs that have meaningfully different requirements — and they compound: each one makes fixing the others harder.

Q: Why does the unclassified query anti-pattern persist into production?

The flat flow is simpler to build, simpler to deploy, and simpler to reason about for the first few weeks. Adding a classification step feels like over-engineering until the team has lived through enough wrong-shaped responses to recognise the pattern — a how-to question gets a definition, a factual question gets an opinion, a transactional request gets a polite refusal because the system never gave it the tools it needed. By the time the recognition comes, the system has accumulated dependencies on the flat flow (prompt iterations, eval test sets, monitoring dashboards) that make migration expensive. The detection signature is wide variance in user satisfaction across query types — some categories at 80%+ thumbs-up, others at 30-40% — that the team cannot act on because the system has no concept of category.

Q: How do I know my system prompt has become a monolith?

Three signals. The prompt exceeds 4,000 tokens of layered instructions. Prompt edit regression rate exceeds 30% — a change to one section causes regressions in unrelated sections; clarifying tone causes a factuality regression; tightening safety causes a helpfulness regression. Engineering velocity on prompt iteration drops to a crawl because every change requires running the full eval suite to catch regressions. Token cost per call is high because the entire prompt is sent on every call regardless of which behaviour is needed. The fix is decomposition into small focused templates (one per category from the classification step), with shared sub-templates for cross-cutting concerns (tone, persona, refusal) included via composition; each template versioned independently with per-template eval suites.

Q: What is the detection signature for monolithic safety?

Refusal-rate uniformity across categories. A healthy production AI system has different refusal rates for different categories because the categories warrant different thresholds — a finance-vertical tenant should have higher refusal rates for certain query types than a consumer-vertical tenant; a children's-app category should have stricter content thresholds than an adult-fiction category. A flat refusal rate within 5 points across all categories is the smoking gun — the system is treating qualitatively different requirements as if they were the same. The over/under-strictness mismatch is invisible until it produces an incident or customer complaint, but the variance metric is detectable continuously as a leading indicator.

Q: Why is calibrated confidence different from raw model confidence?

Raw softmax probabilities from neural networks are not aligned with empirical accuracy — a model can be 90% confident on inputs where it is right 70% of the time. The reliability diagram (predicted confidence vs observed accuracy) shows the model is over-confident, often substantially. Calibrated confidence (after temperature scaling, isotonic regression, or per-class calibration curves on a held-out validation set) produces scores where "calibrated 80% confidence" actually means "right 80% of the time" — measurable, sweepable, comparable across model versions. Expected Calibration Error (ECE) above 5% indicates the model's confidence is meaningfully misaligned; below 3% after calibration is the production target. Without calibration, the confident-misclassification anti-pattern is invisible until the wrongness manifests externally.

Q: In what order should I unwind the four anti-patterns?

Start with classification (anti-pattern #1) because it unlocks every other fix — once inputs are categorised, the team can split prompts, apply per-category safety, and tune review thresholds per category. Decompose the prompt next (anti-pattern #2) because the team needs per-category prompts as the foundation for category-specific behaviour. Layer safety per category third (anti-pattern #3), with explicit per-profile audit trails so safety improvements are demonstrable to compliance reviewers. Calibrate confidence and add novelty/contradiction/policy triggers last (anti-pattern #4) because these depend on having stable per-category behaviour to calibrate against — calibrating against a flat flow with weekly regressions is meaningless because the underlying distribution keeps shifting.

Q: How do the four anti-patterns compound?

Without classification (anti-pattern #1), the team cannot have category-specific prompts (so anti-pattern #2 is forced) or category-specific safety (so anti-pattern #3 is forced) or category-specific review policies (so anti-pattern #4 is harder to address). Without per-category prompts, the single prompt has to encode all the safety logic, making anti-pattern #3 appear inside the prompt rather than in a separate safety layer. Without calibrated confidence, the team cannot reliably route uncertain cases to human review, so unclassified queries the model is unsure about are still answered automatically and incorrectly. The four are not independent — they form a tangle that gets harder to unwind the longer it persists; the dedicated positive patterns elsewhere in this series (Classification-First, Versioned Templates, Category-Aware Guardrails, HITL Escalation) are the constructions that prevent the tangle.

Q: When are these anti-patterns acceptable trade-offs?

Three scenarios. Genuinely single-category systems (a document summarisation tool that does only summarisation) does not need classification — anti-pattern #1 is not an anti-pattern when there is genuinely one category. Pre-product-market-fit prototypes optimising for iteration speed should not invest in classification, per-category prompts, per-category safety, and calibrated triggers before the team knows what the product is — premature optimisation; the graduation point is when traffic exceeds a few thousand calls per day, the team has more than 2-3 engineers, and the product has stabilised enough that categories are knowable. Internal-only or low-stakes deployments (engineer copilot at the company) can tolerate confident misclassification because the user is sophisticated enough to detect wrong answers; public-facing or high-stakes deployments cannot.

Q: How can I detect the anti-patterns before they cause incidents?

Four detection patterns. Anti-pattern #1: tag user feedback (thumbs up/down) with a coarse query-type label using a separate offline classifier; satisfaction variance > 20-30 points across categories indicates unclassified-query symptoms. Anti-pattern #2: track every prompt edit and the eval suite's before/after delta; regression rate > 30% means the prompt has become a monolith. Anti-pattern #3: aggregate refusal rates by query category over a rolling window; flat distribution within 5 points across all categories is suspicious. Anti-pattern #4: build a reliability diagram from a sample of predictions with ground-truth labels; Expected Calibration Error (ECE) > 5% indicates confidence is meaningfully misaligned with empirical accuracy. The variance metrics make the anti-patterns visible early enough to unwind without rebuilding the entire system.

Q: What real-world results come from unwinding these anti-patterns?

A customer-service AI platform serving 2M chats/day across 600 enterprise tenants unwound all four anti-patterns over six months. Outcomes: user satisfaction variance across query types narrowed from 35-82% to 71-86% (every category improved, worst categories improved most); prompt edit regression rate dropped from 41% to 9% (changes are now localised to specific templates); content incidents in regulated-vertical tenants went to zero in the following twelve months; high-confidence wrong-prediction rate on automated decisions dropped from 18% to 3%; percentage of decisions routed to human review increased from 4% to 11% but absolute number of incorrect automated decisions dropped by 70%; engineering velocity on AI features doubled because changes could be reasoned about locally rather than requiring full-system regression testing. Every team that has unwound the anti-patterns in production wishes they had started sooner; no team regrets doing the work.

Satyam Kumar

返回博客

ai-architecture

Four AI System Anti-Patterns: Unclassified Query, Generic Single-Prompt, Monolithic Safety, and Confident Misclassification (2026)

By Satyam KumarApril 24, 202619 min read

ai anti-patterns ai system design unclassified query generic single prompt monolithic safety confident misclassification classification first prompt decomposition category aware safety confidence calibration expected calibration error production llm failures microservices anti-patterns ai architecture ai governance

Four AI System Anti-Patterns: Unclassified Query, Generic Single-Prompt, Monolithic Safety, and Confident Misclassification (2026)

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

Four AI System Anti-Patterns: Unclassified Query, Generic Single-Prompt, Monolithic Safety, and Confident Misclassification (2026)

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment