When the Defender Is the Model: Adversarial Evasion of AI-Based Detection in Fraud, EDR and Anti-Abuse Systems (2026)


Rate limits and CAPTCHAs no longer stop bots riding residential proxies. The architecture that works scores intent from device, behaviour and network provenance.
Erasure law reaches PII memorised in model weights, but retraining per request is impossible. The architecture for lineage, scalable unlearning, and proof.
In February 2024 security researchers disclosed that a popular text-classification model on HuggingFace had been quietly modified to execute a reverse-shell payload at import time. Six months later a similar pattern hit a fork of a widely-used embedding model. Twelve months later JFrog disclosed more than a hundred malicious models sitting in plain sight on the public registry. The enterprise response in most cases was a shrug and a reminder to "be careful what you download" — which is the same response the software industry gave to npm and PyPI for a decade before SBOM-and-provenance discipline finally became table stakes. The AI supply chain is the next attack surface. This article is the deep-architecture guide: the four asset categories (weights, datasets, custom-code adapters, dep tree), the pickle problem precisely (why from_pretrained is arbitrary-code execution), the safetensors migration, CycloneDX ML-BOM, Sigstore/Cosign/in-toto for models, the Linux Foundation Model Signing project, the reference ingestion-gate architecture (picklescan + modelscan + CVE scan + provenance verify + private registry + fleet-side signature verification), dependency confusion in ML packages, embedded backdoors and stealth fine-tunes, dataset lineage and the EU AI Act, eight anti-patterns, five-stage maturity ladder, and the Monday-morning checklist.
A NIS2 compliance blueprint: map the Article 21 measures to cloud controls, engineer the 24h/72h reporting clock, and reuse one control set across GDPR, the EU AI Act, and DORA.
Advertise and partner
Reach engineering leaders and CTOs reading about AI, mobile and cloud architecture.
Sponsored mentions and co-authored articles or technical deep-dives. Relevant products only, always disclosed.