8 AI Model Classes 2026: LLM LCM LAM MoE VLM SLM MLM SAM

Q: What are the eight specialised AI model classes that define the 2026 architecture stack?

The eight classes are LLM (Large Language Model, autoregressive token prediction — GPT-5, Claude 4 Sonnet, Llama 3.3 70B), LCM (Large Concept Model, sentence-level reasoning in SONAR embedding space — Meta’s 2024 research line), LAM (Large Action Model, post-trained on UI actions and tool calls — Anthropic Computer Use, OpenAI Operator, Rabbit r1), MoE (Mixture of Experts, sparse expert routing — GPT-5 family, DeepSeek V3/V4 671B with 37B active, Mixtral 8x22B), VLM (Vision-Language Model, joint image-and-text — GPT-4o/5 vision, Claude 4 vision, Gemini 2.5, FastVLM, MobileVLM), SLM (Small Language Model, 0.5B-4B dense — Phi-4 mini, Gemma 3n, Llama 3.2 1B/3B, Qwen 2.5 0.5B/1.5B), MLM (Masked Language Model, encoder-only — BGE-M3, E5-Large, Voyage-3, ModernBERT, DeBERTa-v3), and SAM (Segment Anything Model, pixel-mask segmentation — SAM 2, MedSAM, MobileSAM, EfficientSAM). Each class owns a distinct routing slot defined by what it models (tokens, sentences, actions, image patches, masks), how it is parameterised (dense vs sparse), and where it runs (cloud frontier, regional self-hosted, edge). Treating any one as the universal hammer is a 2024 mistake; 2026 production systems compose at least four of these classes per request, and the architecture decision is which class earns each slot. The eight-class taxonomy is the shortest list of legal answers, not a buyer’s guide — brands change quarterly while the classes are stable.

Q: Why is routing every retrieval through a generation LLM 50-200x more expensive than using an MLM encoder?

Because generation LLMs and MLM encoders solve different problems with different costs per unit. A generation LLM (autoregressive transformer) decodes one token at a time, paying the full parameter cost on each forward pass and producing a token stream you do not need for retrieval; embedding a query through a chat LLM is API-priced like generation, typically $1-15 per million tokens. An MLM encoder (BGE-M3, E5-Large, Voyage-3, Cohere Embed v3) is bidirectional, processes the entire sequence in a single forward pass, produces a fixed-dimensional vector, and is priced or self-hosted at $0.02-0.13 per million tokens encoded — typically 50-200x cheaper for the equivalent text. Worse, the generation LLM’s "embedding" is not a better representation for retrieval — it is often demonstrably worse, because it was trained to predict next tokens rather than to produce semantic similarity vectors, and the dimensionality, normalisation, and contrastive-training objectives that make embeddings work were not part of its loss. The single biggest cost lever in any 2024-2025 RAG stack that gets reviewed in 2026 is replacing the chat-LLM-as-embedding-model with a proper MLM encoder, which is a one-line code change and a 50-200x reduction in retrieval cost with equal or better recall — documented in our hybrid-search-and-reranking and retrieval-cache-hierarchy pieces. There is no architectural reason to embed with a generation model in 2026, and treating that as a meaningful tradeoff is the failure mode the matrix exists to prevent.

Q: What makes a Large Action Model (LAM) different from a chat LLM with function-calling enabled?

A LAM is post-trained — typically supervised plus reinforcement learning — on sequences of (screenshot, action, outcome) tuples or (tool-call, result, next-action) traces, so the model has learned the distribution of UI noise, partial-load states, ambiguous element positions, recovery from misclicks, and the proposal-confirmation patterns that real automation requires. A vanilla chat LLM with function-calling has none of that grounding; it has been trained to chat and post-trained to emit JSON tool calls when prompted, which is a strictly weaker capability. The observable failure modes when you swap a chat LLM into a LAM slot are: refusal to commit to the next action under uncertainty (the chat model will hedge instead of click), inability to recover from a UI state that does not match its expectation (the chat model will repeat the same failing action or hallucinate a new tool), poor screenshot understanding for non-text UI elements (icons, drag handles, zoomed-out layouts), and a much higher rate of dangerous-action commits because the chat model has not learned the irreversible-vs-reversible distinction the LAM post-training encodes. Anthropic’s Computer Use (Claude 4 family with computer-use post-training), OpenAI’s Operator, and Google’s Project Mariner are the public LAM-class examples in 2026; on benchmarks like OSWorld and WebArena, dedicated LAMs outperform vanilla chat LLMs by 5-10x in success rate at comparable cost, and that delta widens on real-world UI noise. The architectural rule is that a LAM slot needs a real LAM, paired with the sandbox + audit + irreversible-action-gate stack covered in our computer-use agent post.

Q: When should I pick a Mixture-of-Experts (MoE) model over a dense LLM, and when does dense win?

MoE wins on the cost-per-quality-unit ratio in hosted deployments, dense wins on operational predictability and self-hosted utilisation. MoE models (GPT-5 family, DeepSeek V3/V4 671B with 37B active, Mixtral 8x22B with ~40B active, Snowflake Arctic) decouple total parameter count from inference cost via a learned router that activates only a top-k subset of experts per token, so you pay 5-15 % of the parameters per forward pass while retaining the quality of a much larger model. For hosted-API consumption this is pure upside — frontier-grade quality at mid-tier per-output-token pricing — and MoE is the dominant frontier shape in 2026 for that reason. Dense models (Llama 3.3 70B, Qwen 2.5 72B, Mistral Large) win when you self-host because dense serving is operationally simpler: no expert parallelism, easier KV-cache management, predictable VRAM per request, cleaner cold-start, and the entire vLLM/SGLang ecosystem treats dense as the well-understood path. Self-hosting MoE in 2026 requires vLLM v0.7+ or SGLang with expert-parallel sharding, which is a non-trivial operational lift; if your fleet runs at >50% utilisation the dense self-host can be 30-60% cheaper than hosted MoE on $-per-quality, with the tradeoff that you ceiling at sub-frontier quality. The decision matrix is therefore: MoE via API for frontier quality at hosted economics, dense self-host for predictable workloads with steady utilisation, and never pin to a single SKU — frontier MoE upgrades happen every 6-10 weeks and your routing should follow.

Q: Why is a Vision-Language Model (VLM) better than OCR-then-text for documents with tables and charts?

Because the VLM reads the spatial layout, table structure, and chart geometry as part of its forward pass, while OCR-then-text discards every spatial relationship at the OCR boundary and forces the downstream LLM to reconstruct meaning from a flattened text stream. A VLM (GPT-4o/5 vision, Claude 4 vision, Gemini 2.5 multimodal) tokenises image patches alongside text and learns joint representations, so a question like "what was the Q3 revenue for the EMEA segment in this stacked bar chart?" gets answered against the image directly — the model sees the bar heights, the legend, the axis labels, the segment colour coding, and the numeric annotations as a unified scene. OCR-then-text on the same chart produces a list of disconnected numbers and labels with no preserved geometry; the LLM downstream has to guess which number belongs to which segment and frequently gets it wrong, especially on stacked bars, dual-axis charts, and non-standard layouts. The same applies to tables (cells, merged headers, multi-row labels), forms (field-and-value pairing), diagrams (boxes-and-arrows semantics), and scanned documents (text-with-position). The on-device VLM line (FastVLM, MobileVLM 1.4B/2.7B, Gemma 3n with MobileNet-V5 vision encoder) extends this advantage to edge deployments where OCR-then-text was the only option in 2024. The architectural rule is "VLM for layout-bearing inputs, OCR-only for clean machine-readable text" — our multimodal-RAG-with-ColPali piece shows the production pipeline that uses ColPali patch embeddings for retrieval plus a frontier VLM for grounded generation, with a citation-loop verifier closing the faithfulness gap on charts.

Q: When should I use a Small Language Model (SLM) instead of a frontier LLM in production?

Whenever the workload is narrow, latency-sensitive, edge-deployable, or the per-call cost of a frontier model dominates unit economics — which in 2026 is most of the routing fleet, not a niche. The 0.5B-4B SLM line (Phi-4 mini, Gemma 3n E2B effective, Llama 3.2 1B/3B, Qwen 2.5 0.5B/1.5B) earns its slot in three workload shapes. First, narrow tasks where a fine-tuned SLM beats frontier: intent classification, structured-output extraction, tool-argument synthesis, named-entity recognition, translation pairs, and any labelled task with thousands of examples — a fine-tuned 1B model trained on 5-50k task examples typically beats a 2023 70B base model on the task and runs at one-thirtieth the cost. Second, edge and on-device deployment: an SLM on a modern phone runs at 5-30ms per call with effectively zero marginal cost, no network round-trip, no cloud privacy boundary, and offline operation — the slot covered in our on-device-gallery-cleanup-agent and edge-AI-architecture pieces. Third, the front door of any classify-first architecture: a 1-3B SLM at the gateway sorts traffic into the cheapest qualifying class, escalating only on signal to MoE or frontier, which is the single largest cost lever in 2026 production routing. The 2026 rule is not "is this SLM good enough?" but "is an SLM already deployed for this task?" — the question has flipped, and frontier-LLM-by-default is the anti-pattern. The narrow caveat is open-ended generation requiring frontier reasoning (long-form analysis, multi-step planning, novel-domain synthesis) where SLMs still trail meaningfully — keep them out of those slots, but route everything else to them where the eval clears.

Q: What does a Large Concept Model (LCM) do differently from token-level autoregressive LLMs?

An LCM operates in concept space rather than token space — the unit of computation is a whole sentence embedded into Meta’s SONAR multilingual representation rather than a single token, the model predicts the next sentence concept (often via diffusion in concept space), and decoding back to text is a separate downstream step. The architectural significance is twofold. First, long-horizon coherence: token-level autoregression accumulates drift over thousands of tokens because each prediction conditions on the full token history, while sentence-level concept prediction operates at a coarser granularity that resists drift across long documents — useful for long-form drafting, multi-section reports, and structured planning where token-level models lose the thread. Second, language-agnostic reasoning: SONAR space carries 200+ languages in a unified representation, so an LCM plans the document in concept space and the same plan can be decoded into English, Polish, Japanese, or Arabic without re-planning — a genuine architectural advantage for multilingual document workflows that token-level models can only approximate via translation pipelines. Production deployment is still rare in 2026 — LCM is a research line introduced in Meta’s December 2024 paper, with limited public model releases and an ecosystem that is years behind LLM-class tooling — so most teams will not deploy an LCM in their first three architecture iterations. The decision rule is: if your planning horizon is short or single-language, stay with token-level MoE LLMs; if the workload is long-horizon multilingual document generation and you have the research-engineering budget to integrate a concept-space planner with a token-space generator, an LCM-plans + MoE-generates composition is the architectural answer that the eight-class matrix carves out.

Q: How does Segment Anything Model (SAM) fit alongside Vision-Language Models in a production vision stack?

SAM and VLM solve different vision problems and the production stack uses both. SAM (Meta’s Segment Anything Model 2 line, plus MedSAM, MobileSAM, EfficientSAM) is a promptable image segmentation model: given an image and a prompt (point, bounding box, or text), it produces pixel-accurate masks for the prompted object. SAM 2 extends to video tracking — mask one frame, track through the rest. Its slot is anywhere you need pixel-level masks: robotics perception (grasp planning, obstacle masks), manufacturing defect localisation (mask the flaw), medical imaging (organ and lesion masks for radiology workflow), video editing (per-frame object isolation), satellite and aerial imagery (per-pixel land-cover labels), and AR overlays (real-world object boundaries). A VLM can describe an image, answer questions about it, and reason over its content, but it cannot produce a pixel-accurate mask — the output shape is wrong. Conversely, SAM cannot tell you what an object is or why it matters — the input shape includes a prompt and the output is purely geometric. The composition pattern is "VLM for understanding, SAM for boundaries": a VLM identifies the object of interest in the scene and provides the prompt (text or bounding box), SAM produces the precise mask, and downstream logic acts on the mask. In edge deployments MobileSAM and EfficientSAM run locally on 2024+ SoCs at 50-200ms per image, making this composition viable on-device for AR, medical-imaging-at-the-bedside, and field-robotics workloads. Asking a VLM to "describe the segmentation" instead of running SAM for actual masks is one of the named anti-patterns — it is a wrong-tool failure that produces unusable outputs.

Q: What does a 2026 production RAG stack look like when it composes multiple model classes correctly?

A correctly composed 2026 RAG stack touches at least three of the eight model classes per query and routes the workload classify-first to keep cost per resolved query in the $0.002-0.01 band. The sequence: an inbound query hits a Small Language Model (1-3B fine-tuned on intent + language + allowed-tool labels) at the gateway, which classifies the query and decides the routing path in roughly 5-15ms. The classified query is then encoded by a Masked Language Model (BGE-M3 or Voyage-3 or Cohere Embed v3) into a fixed-dimensional vector for hybrid lexical-plus-dense retrieval against the corpus index; an MLM cross-encoder reranks the top 50 candidates to a top 10 in another 20-40ms. The retrieved chunks plus the question go to a Mixture-of-Experts LLM (GPT-5, Claude 4 Sonnet, DeepSeek V4) for generation with explicit citation requirements; a structured-output schema enforces that every claim cites a retrieved chunk. If the corpus contains charts, scanned forms, or layout-bearing documents, a Vision-Language Model runs over the relevant pages in parallel and a separate vision-aware retrieval path joins the text path under reciprocal rank fusion, with an on-device small VLM handling pre-classification of "does this page have a chart?" to avoid spending frontier-VLM tokens on pure text pages. A faithfulness verifier (a smaller MoE or dense LLM, never the same model as the generator to avoid self-preference bias) checks each cited claim against its retrieved chunk and rejects or repairs ungrounded claims. Total cost lands at $0.002-0.01 per resolved query, of which the MLM encoding is sub-$0.0001 and the MoE generation is the dominant component — a 50-200x cut versus routing every step through the frontier LLM. The same architecture extends to multimodal-RAG with ColPali patch embeddings for the vision retrieval path, covered in our multimodal-RAG piece.

Q: How should I plan model upgrades across all eight classes given that each class has a different drift profile?

Treat each class as a separate eval-driven upgrade lane with its own canary cadence, per-class golden eval sets, and per-class rollback policies — do not pin the whole stack to a single upgrade window. The drift profiles are genuinely different. Frontier MoE LLMs ship new snapshots every 6-10 weeks (Anthropic, OpenAI, Google) and can shift behaviour materially on tool-call schemas, refusal rates, and reasoning quality between point releases — your canary should be 24h shadow + 1%/24h + 5%/48h + 25%/48h + 100%, with eight drift signals (pairwise win rate, tool-call success, refusal rate, length, faithfulness, sentiment, CSAT, cost-per-task) sliced per intent class and tenant. MLM encoders move quarterly (BGE, E5, Voyage versions) and shifts are subtler but matter for retrieval recall; canary against a held-out retrieval eval set with NDCG@10 and recall@50 as the gates. SLMs upgrade on release cadence (Phi-4 → Phi-5, Gemma 3n → Gemma 4); fine-tuned SLMs require re-tuning per release and the drift tax is small if your training pipeline is automated. VLMs and LAMs upgrade on the underlying frontier model release plus separate post-training drops (computer-use post-training is independent of the base model); these need their own eval suites focused on screen-reading and action-success rates. SAM updates rarely (SAM, SAM 2, occasional MedSAM specialisations) and the drift is minimal across releases. LCM is research-only in 2026 — no production upgrade lane needed yet. The shared discipline is the eval-drift pattern documented in our eval-drift-silent-regression piece: per-class pin-then-track production posture, per-class shadow + canary, drift signals wired to auto-rollback, and the routing matrix itself versioned as a first-class artefact.

Satyam Kumar

返回博客

ai-architecture

Eight Specialised AI Model Architectures 2026 — LLM, LCM, LAM, MoE, VLM, SLM, MLM, SAM Decision Matrix

By Satyam KumarJune 10, 202622 min read

ai-model-taxonomy large-language-models large-concept-models large-action-models mixture-of-experts vision-language-models small-language-models masked-language-models segment-anything-model model-routing classify-first-architecture cost-engineering-llm on-device-ai computer-use-agent rag-architecture embedding-models eval-driven-development frontier-models

Eight Specialised AI Model Architectures 2026 — LLM, LCM, LAM, MoE, VLM, SLM, MLM, SAM Decision Matrix

Frequently Asked Questions

分享这篇文章

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

人工智能和云架构师。帮助团队构建可扩展到数百万的系统。

LinkedIn GitHub

Eight Specialised AI Model Architectures 2026 — LLM, LCM, LAM, MoE, VLM, SLM, MLM, SAM Decision Matrix

Frequently Asked Questions

分享这篇文章

Comments

Leave a comment