Japanese LLM Stack 2026: ELYZA, Stockmark, PLaMo vs Frontier

Q: When does a domestic Japanese LLM actually beat a frontier API in 2026?

Domestic Japanese LLMs beat frontier APIs on three structural axes that frontier price drops do not close. Tokenisation cost: Japanese-aware SentencePiece tokenisers (used by ELYZA, Stockmark, PLaMo) encode Japanese at roughly 1.0 to 1.2 tokens per character versus 1.5 to 2.5 tokens per character on cl100k_base, o200k_base, Claude BPE, or Gemini SentencePiece — a structural 70% to 100% per-request cost gap on Japanese workloads that holds across volume. Keigo reliability: domain-tuned domestic models score 85–95% on structured keigo test sets versus 70–85% for frontier models, with the gap concentrated in register consistency across multi-turn dialogues and in correct sonkeigo/kenjougo direction. Vertical vocabulary: jukugo and domain kanji compounds in finance, legal, and medical Japanese appear at higher frequency in domestic pre-training corpora, giving Stockmark and JMedLLM-family models a measurable accuracy edge over frontier models on vertical tasks. Above approximately 500K Japanese requests per month, the per-token economics make a self-hosted JP-aware model the default for general-purpose traffic and reserve frontier API spend for the quality-ceiling tail.

Q: When does the frontier API still win on Japanese?

Frontier APIs (GPT-4o, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 2.0) still win on quality ceiling for the most demanding 5–15% of Japanese requests where the response requires deep multi-step reasoning, broad world knowledge, or sophisticated cross-domain synthesis. They also win on multimodal Japanese (vision-plus-language tasks where Japanese-aware multimodal domestic options are sparse), on rapid prototyping where the operational overhead of self-hosting is not yet justified, on workloads below approximately 500K requests per month where the per-token gap does not amortise the operations cost of a self-hosted stack, and on use cases where the latest frontier capability (multi-step tool use, advanced reasoning modes) is the product differentiator. The right architectural posture is not to choose one or the other but to route per request envelope: send the high-volume general-purpose traffic to a self-hosted JP-aware model, send the quality-ceiling tail and multimodal cases to a frontier API, and measure the routing decisions against actual cost and quality telemetry.

Q: How big is the Japanese tokenisation cost penalty in real numbers?

For a representative Japanese business email of approximately 400 characters: cl100k_base (older OpenAI) encodes to roughly 700–900 tokens, o200k_base (newer OpenAI) to 600–750 tokens, Claude tokeniser to a similar range, Gemini SentencePiece to 500–650 tokens, and a Japanese-aware SentencePiece tokeniser (as used by ELYZA, Stockmark) to 400–500 tokens — close to the character-count floor. Scaled to one million Japanese requests per month with average input 800 characters and output 400 characters, the workload is approximately 2.4B tokens per month on o200k_base versus 1.4B tokens on a JP-aware tokeniser. At identical price-per-token the cost difference is 70% to 100%; the actual cost difference depends on the per-token pricing of the chosen frontier API versus the amortised inference cost of the self-hosted alternative, but the structural multiplier is real and survives any frontier price drop. The decision rule that follows: any Japanese workload above approximately 500K requests per month should route high-volume general-purpose traffic to a self-hosted JP-aware model and reserve frontier spend for the quality-ceiling tail.

Q: How do I evaluate keigo correctness in production?

Keigo correctness is evaluated by a structured test set with three sections: sonkeigo correctness (does the model use exalting forms when describing the listener's actions), kenjougo correctness (does the model use humbling forms when describing the speaker or speaker's organisation's actions), and register consistency (does the model maintain a single register through a multi-turn dialogue rather than slipping). Each item has a labelled correct register, a labelled incorrect-but-plausible register, and a labelled incorrect-and-obvious register, scored by a judge model with an explicit rubric and calibrated against human-labelled samples on a representative sub-set. The test set is owned by the AI evaluation team rather than by individual product teams, versioned alongside the application, and run as a gate in the AI-CICD pipeline on every prompt change and every model swap. The frontier models score 70–85% on keigo correctness depending on the test set construction; domain-tuned ELYZA and Stockmark score 85–95% on the same tests. The 10–15 point gap is what separates a B2B Japanese product that passes pilot with an enterprise customer from one that does not.

Q: What does the production routing layer actually look like?

The routing layer is a thin policy component in front of the inference call. The request envelope has three fields: domain (general/finance/medical/legal/multimodal), quality_floor (low/medium/high/ceiling), sovereignty_required (boolean) — populated by the upstream service or by a small classifier. A versioned routing table maps the envelope to a primary model and fallback chain. The router emits per-route AI events (which envelope, which model selected, which fallback if any, latency, token usage, cost) onto an OpenTelemetry-conformant stream. The cost-per-request is tracked per route so the table can be tuned against actual spend rather than against assumed spend. The most common implementation mistake is to build the router around "which model is best?" — which has no stable answer because the best model depends on the envelope. The right framing is "given this envelope, what is the cheapest model that meets the quality floor?" maintained as a per-(envelope, model) measurement so the answer can be computed rather than guessed. The companion AppScale Prompt Routing Pattern article covers the policy mechanics; the Multi-Provider Fallback Pattern covers resilience; the AI Observability with OpenTelemetry pattern covers the telemetry.

Q: What is the difference between ELYZA, Stockmark, PLaMo, and Sakana — when do I pick which?

ELYZA builds Japanese fine-tunes on top of open-weight foundations (Llama-3 lineage in 2026) with the ELYZA-japanese-Llama-3-8B and 70B as the production-relevant generation. Pick ELYZA for general-purpose Japanese text where you want self-hosted, open-weight, well-documented inference economics at sizes that fit on a single GPU or small cluster. Stockmark trains from scratch on a Japanese corpus heavy in business news and financial filings; the flagship Stockmark-13b and the Stockmark-LLM-100b extension are domain-biased toward business and finance vocabulary and writing register. Pick Stockmark when the use case is parsing securities filings, generating internal corporate documents in the appropriate register, or extracting from press releases and IR communications. PLaMo (Preferred Networks) is the closest to a sovereign Japanese frontier model — PLaMo-100B trained from scratch by a Japanese company on Japanese-purchased compute. Pick PLaMo when sovereignty is the operational requirement: government, defence, critical infrastructure, regulated industries with explicit cross-border restrictions. Sakana AI builds evolutionary model merges (Evo-Llava, EvoUkiyoe, etc.) that produce capable narrow artefacts without full retraining. Pick Sakana for narrow capability merges (Japanese-aware multimodal pipelines without fine-tuning from scratch); do not pick it as a production base model — the operational profile is research-grade.

Q: Should I fine-tune a domestic model on my own corpus, or use it as-is?

For vertical Japanese use cases the production-grade pattern in 2026 is rarely "use the domain-specialist model unmodified" and rarely "use the general model with prompt engineering." It is usually "fine-tune the domain-specialist base on the customer's own corpus." The recipe: take ELYZA-Llama-3-70B or Stockmark-13B as the base; apply LoRA or QLoRA fine-tuning on the customer's labelled domain corpus (typically 5K–50K labelled examples is sufficient for behavioural adaptation, more for capability extension); evaluate against the customer's held-out test set as the primary signal and the vertical benchmark as the secondary; deploy as a customer-specific artefact behind the routing layer. The companion AppScale article on LLM Fine-Tuning (LoRA, QLoRA, Full Fine-Tuning Compared) covers the technical depth of the fine-tuning choices. The architectural prerequisite is that the routing layer must accommodate per-customer model artefacts as first-class destinations rather than treating "the model" as a global singleton. Tenant-aware routing is the architectural primitive; LoRA-per-tenant is the deployment economy that makes it tractable.

Q: How do I think about sovereignty for a Japanese AI deployment?

Sovereignty in the Japanese AI context is the operational question of who controls the model weights, who controls the compute the model runs on, who controls the retraining pipeline, and what happens to the deployment if cross-border AI restrictions tighten. For most commercial use cases sovereignty is not the deciding factor and the choice can be made on quality and cost grounds. For government, defence, critical infrastructure (power, water, telecommunications), regulated financial services with explicit cross-border data restrictions, and any procurement where the contracting authority requires the AI supply chain to remain within Japan, sovereignty becomes the determining constraint. The architecturally clean answer is PLaMo (Preferred Networks) self-hosted on Japanese compute, with the routing layer separating the sovereign-required traffic from the general traffic so that only the regulated subset bears the operational profile of full sovereignty. The implementation pattern is sovereignty_required as a first-class field in the request envelope, a routing rule that hard-fails to PLaMo for true and routes by quality/cost for false, and audit logging that demonstrates the routing decision per request. Trying to make the entire stack sovereign on day one is rarely the right answer — the cost is high and the quality ceiling is below frontier on the requests that do not require sovereignty.

Q: How portable is this routing pattern outside Japan?

The routing pattern is highly portable; the Japanese case is the demanding instance of the general pattern. Korean, Mandarin, and Hindi share several of the structural characteristics that make Japanese hard: tokenisation cost penalty on Western-default tokenisers (Korean and Mandarin have similar 2x token-per-character ratios on cl100k_base; Hindi varies by script handling), domain-specific vocabulary that is sparser in web-scale English-dominant corpora, and register or honorific conventions that frontier models handle inconsistently (Korean has a similar honorific system to Japanese; Hindi has formal/informal pronoun distinctions). The architectural primitives transfer directly: request envelope with domain/quality/sovereignty fields, versioned routing table, per-route telemetry, tenant-aware fine-tunes as first-class destinations. The model choices change (Korean: HyperCLOVA X, EXAONE; Mandarin: Qwen, GLM, Doubao; Hindi: Krutrim, Sarvam, BharatGen) but the routing layer is the same. Building the router for the Japanese case is the highest-leverage way to get a routing architecture that survives expansion into other Asia markets, because Japanese has the tightest constraints on tokenisation cost and register handling.

Q: What is the maturity ladder for a Japanese LLM stack and where are most enterprises in 2026?

Stage 0 — Frontier-only on JP: the entire Japanese product runs on GPT-4o or Claude, quality acceptable, cost high, no measurement of where spend goes. Stage 1 — Frontier plus one domestic: a second model (typically ELYZA-Llama-3) added for high-volume general traffic, manual rule decides routing, cost improves materially. Stage 2 — Routing layer with envelope: request envelope formalised, routing table versioned, per-route cost telemetry, three to four models in active use, quality measured per (envelope, model) pair. Stage 3 — Tenant-aware routing plus LoRA-per-tenant: vertical and per-customer fine-tunes deployed as first-class routing destinations, customer-specific evaluation suites, cost-per-customer reportable. Stage 4 — Continuous routing optimisation: routing decisions updated automatically based on per-route cost and quality telemetry with safe-rollback on quality regression, per-tenant SLAs measured against actual spend, the router is a living system rather than a static config. Most Japanese enterprise LLM stacks in early 2026 are at Stage 0 or Stage 1. The infrastructure to move to Stage 2 is the highest-leverage investment in the JP product roadmap because it unlocks every subsequent optimisation and is the prerequisite for Stage 3 customer-fine-tunes to be operationally tractable.