--- title: "Validation loop — 29 techniques" description: "Linters, type checks and audit tools the agent is told to run, turning “follow the system” into a feedback loop with failures it has to fix." url: "https://state-of-ai-in-design-systems.netlify.app/techniques/validation-loop.md" canonical: "https://state-of-ai-in-design-systems.netlify.app/techniques/validation-loop.md" type: "technique-category" id: "validation-loop" technique_count: 29 system_count: 19 data_collected: "2026-07-26/27" generated: "2026-07-28T06:01:02Z" report: "State of AI in Design Systems — July 2026" author: "Kaelig Deloumeau-Prigent" license: "CC-BY-4.0" citation: "Deloumeau-Prigent, K. (2026). State of AI in Design Systems. https://state-of-ai-in-design-systems.netlify.app/techniques/validation-loop.md" --- > Snapshot of 2026-07-27. Every claim below links to the source URL it was taken from. Check the source before citing. # Validation loop 29 of the 148 techniques in this study, across 19 of the 19 design systems. Read when asked how to make a design system's rules enforceable. Linters, type checks and audit tools the agent is told to run, turning “follow the system” into a feedback loop with failures it has to fix. Systems represented here: [Ant Design](https://state-of-ai-in-design-systems.netlify.app/systems/ant-design.md), [Atlassian Design System](https://state-of-ai-in-design-systems.netlify.app/systems/atlassian-design-system.md), [Carbon Design System](https://state-of-ai-in-design-systems.netlify.app/systems/carbon-design-system.md), [Chakra UI](https://state-of-ai-in-design-systems.netlify.app/systems/chakra-ui.md), [Cloudscape Design System](https://state-of-ai-in-design-systems.netlify.app/systems/cloudscape-design-system.md), [daisyUI](https://state-of-ai-in-design-systems.netlify.app/systems/daisyui.md), [Microsoft Fluent UI](https://state-of-ai-in-design-systems.netlify.app/systems/fluent-ui-microsoft.md), [HeroUI](https://state-of-ai-in-design-systems.netlify.app/systems/heroui.md), [Mantine](https://state-of-ai-in-design-systems.netlify.app/systems/mantine.md), [Material UI (MUI)](https://state-of-ai-in-design-systems.netlify.app/systems/material-ui.md), [Nord Design System](https://state-of-ai-in-design-systems.netlify.app/systems/nord-design-system.md), [Nuxt UI](https://state-of-ai-in-design-systems.netlify.app/systems/nuxt-ui.md), [PatternFly](https://state-of-ai-in-design-systems.netlify.app/systems/patternfly.md), [Primer](https://state-of-ai-in-design-systems.netlify.app/systems/primer-github.md), [React Spectrum / Spectrum 2 (S2)](https://state-of-ai-in-design-systems.netlify.app/systems/react-spectrum-s2.md), [Salesforce Lightning Design System](https://state-of-ai-in-design-systems.netlify.app/systems/salesforce-slds.md), [shadcn/ui](https://state-of-ai-in-design-systems.netlify.app/systems/shadcn-ui.md), [Shopify Polaris](https://state-of-ai-in-design-systems.netlify.app/systems/shopify-polaris.md), [U.S. Web Design System (USWDS)](https://state-of-ai-in-design-systems.netlify.app/systems/uswds.md) ## Mandatory post-edit lint loop (`antd lint`) Ant Design · full record: https://state-of-ai-in-design-systems.netlify.app/systems/ant-design.md Key Rule 5 requires the agent to run `antd lint` on changed files after every write or modification, specifically to catch deprecated API usage. Combined with `antd doctor` (project config diagnosis) and `antd env` (environment snapshot), this gives the agent a closed verification loop that does not depend on the host project having any antd-aware linting configured. ```markdown 5. **Lint after changes** — After writing or modifying antd code, run `antd lint` on the changed files to catch deprecated or problematic usage. **Workflow:** `antd env` → capture full environment → `antd doctor` → check configuration → `antd info --version X` → verify API against the user's exact version → `antd lint` → find deprecated or incorrect usage. ``` Source: https://raw.githubusercontent.com/ant-design/ant-design-cli/main/skills/antd/SKILL.md ## axe-core validation loop with capability-scoped tools Atlassian Design System · full record: https://state-of-ai-in-design-systems.netlify.app/systems/atlassian-design-system.md A three-step loop is baked into the tool graph: `ads_analyze_a11y` (JSX string → heuristic/axe findings) → `ads_suggest_a11y_fixes` (violation string → ADS-biased ‘recipe map’ fixes) → `ads_get_a11y_guidelines` (topic guidance). The descriptions are unusually honest about their own limits: ‘not every finding maps to a specific ADS component fix’, ‘Does not replace testing in a real browser with assistive technologies’, which reduces false confidence. The live-browser variant `ads_analyze_localhost_a11y` is gated by deployment: ‘it is **only exposed when this MCP runs locally**, not in the remote MCP deployment.’ The package depends on axe-core, @axe-core/playwright and @axe-core/puppeteer to back this. ```text ### ads_analyze_a11y Analyzes a **string of React/JSX** code for likely accessibility issues (heuristics and/or axe-related paths) and returns hints that often **point to** `ads_suggest_a11y_fixes` or generic axe/WCAG-style context—not every finding maps to a specific ADS component fix. LIMITATIONS: - Does not replace testing in a real browser with assistive technologies or full keyboard traversal. - For rendered UI, `ads_analyze_localhost_a11y` (live URL + axe) is preferable when available—it is **only exposed when this MCP runs locally**, not in the remote MCP deployment. ### ads_suggest_a11y_fixes WHAT YOU GET (varies by match): - **Curated hit:** ADS-biased examples and patterns from this server's recipe map (components, tokens, common fixes). - **No strong match:** Generic guidance (e.g. "use ADS components", labeling, testing)—still useful, but **not** guaranteed to be ADS-specific. ``` Source: https://mcp.atlassian.com/v1/ads/public/mcp ## Published head-to-head context-delivery evals Atlassian Design System · full record: https://state-of-ai-in-design-systems.netlify.app/systems/atlassian-design-system.md Atlassian ran an eval (agents generating a login screen) comparing four context strategies and published token usage, wall time, and turn counts: no context ~5% tokens / 4m19s / 43 turns; ADS MCP ~80% / 5m01s / 35.1 turns; ADS Skill ~80% / 5m23s / 36 turns; DESIGN.md ~30% / 6m46s / 45.3 turns. They report DESIGN.md consumed ‘92% more tokens’ than MCP with ‘2.7x the variance in token consumption between runs’, and (the most useful negative finding in the study) that DESIGN.md ‘frequently caused agents to re-create components rather than use the existing system’, because it is ‘a guide on how to re-implement’ rather than ‘an instruction manual to using the existing design system’. Separate blog figures for the structured-content/MCP work: ‘52% accuracy improvement in AI calls’, ‘34% faster on average across ADS specific tasks’. ```text However, DESIGN.md frequently caused agents to "re-create components rather than use the existing system," introducing technical debt by reimplementing instead of importing existing design system components. DESIGN.md consumed approximately "92% more tokens" than the MCP approach and showed "2.7x the variance in token consumption between runs." ``` Source: https://www.atlassian.com/blog/how-we-build/atlassians-design-md-is-here-what-we-learned-testing-portable-design-context-in-practice ## Self-check validation list of known silent failures Carbon Design System · full record: https://state-of-ai-in-design-systems.netlify.app/systems/carbon-design-system.md A ‘Result Validation — Critical Items’ checklist framed as ‘the non-obvious failures that slip through most often’. It is a post-generation lint pass the model runs against itself. It names exact response fields (use `example_clean`, not `example`), forbids a specific wrong remediation (stub variant → use `requery_hint`, never increase `size`), and encodes silent-failure knowledge like IBM Products’ `pkg.component.X = true` flags and DataTable being absent from the code index. ```markdown ## Result Validation — Critical Items The non-obvious failures that slip through most often: - [ ] Use `example_clean` for component JSX — **not** `example`, not `example_text`; for icons use `example` verbatim - [ ] Use `source.imports[]` verbatim — never construct import paths manually - [ ] Stub variant (`example_omitted: true`) → use `requery_hint`, **never increase `size`** - [ ] DataTable: not in code index — `docs_search` + generate from first principles - [ ] Charts: `get_charts` only — no `code_search`; all four assembly fields verbatim - [ ] Web Components tokens: never use `$spacing-*` / `$background` / `$layer-*` SCSS variables in component styles — they are compile-time only and produce no output at runtime; use `var(--cds-spacing-*)` / `var(--cds-background)` / `var(--cds-layer-*)` CSS custom properties instead - [ ] Web Components grid: default to CSS classes (`cds--grid` / `cds--row` / `cds--col-lg-*` on `
` elements) — never use `` (does not exist) - [ ] Accessibility: icon-only buttons have `iconDescription`; all inputs have `labelText`; no `tabIndex > 0`; no `div onClick` without `role` + keyboard handler ``` Source: https://carbondesignsystem.com/developing/carbon-mcp/files/carbon-builder.zip ## Migration validation loop: codemod dry-run → typecheck → lint → build → grep for leftovers Chakra UI · full record: https://state-of-ai-in-design-systems.netlify.app/systems/chakra-ui.md The strongest enforcement mechanism Chakra ships to consumers. The migrate skill ends with a checkbox list the agent must work through, closing with a literal grep it has to run to prove no v2 imports survive. That converts ‘did the migration work’ from a model judgement into a shell command with an observable exit condition. ````markdown ## Step 10 — Validation checklist Work through this after the migration is complete: - [ ] Reinstall dependencies: `npm install` / `pnpm install` - [ ] TypeScript: `npx tsc --noEmit` — resolve all type errors - [ ] Lint: `npm run lint` - [ ] Build: `npm run build` - [ ] Visually verify color mode toggle (light ↔ dark) - [ ] Test interactive components: Dialog, Drawer, Menu, Tabs, Accordion - [ ] Test form components: Checkbox, Select/NativeSelect, Input, Radio - [ ] Check for visual regressions across key pages - [ ] Search codebase for leftover v2 imports: ```bash grep -r "ColorModeScript\|useColorModeValue\|extendTheme\|styleConfig\|@chakra-ui/icons\|@chakra-ui/next-js" src/ ``` ```` Source: https://raw.githubusercontent.com/chakra-ui/chakra-ui/HEAD/skills/chakra-ui-migrate/SKILL.md ## Named the failure mode: “Your LLM ignores the instructions and skills” daisyUI · full record: https://state-of-ai-in-design-systems.netlify.app/systems/daisyui.md daisyUI’s marketing is an unusually explicit, receipts-first critique of the skill/rules paradigm every other system in this study relies on, complete with a mocked-up log of a model reading SKILL.md, truncating two reference files at ‘Lines 1 to 120’, partially ignoring four more and skipping quality-checks.md entirely. It is the clearest articulation in the dataset of *why* prohibition-style instruction files under-perform, and it is used to justify moving enforcement from prose into tool boundaries. ```text TRYING TO FIX THE SLOP WITH INSTRUCTIONS? Your LLM ignores the instructions and skills. Agent skills are collections of instructions and rules. Check the logs and the pattern is familiar: the model reads a few files, skips the rest, ignores inconvenient rules, and moves on. UI work needs more context than most tasks. Yet as the instruction set grows, the chance of selective reading grows with it. Skill files inspected Ignored 40% of the rules Skipped 23 files due to overconfidence Ignored the rules and treated them as suggestions only SKILL.md read references/components.md Lines 1 to 120 references/themes.md Lines 1 to 120 references/design.md partially ignored references/anti-patterns.md partially ignored references/accessibility.md partially ignored references/responsive.md partially ignored references/quality-checks.md skipped ! The model continued without following the rules it was expected to follow. Wasted time, wasted effort, wasted tokens ``` Source: https://daisyui.com/blueprint/ ## Builder-side: mandate an external MCP as the anti-hallucination source of truth daisyUI · full record: https://state-of-ai-in-design-systems.netlify.app/systems/daisyui.md On the maintainer side, daisyUI’s own Copilot instructions do not ask the model to be careful; they redirect it to a tool. Context7 MCP is named twice as the required lookup path for syntax and package information, paired with ‘do not make up answers’ and ‘Do not guess, do not hallucinate’. Workspace rules add hard directory prohibitions and a dependency-approval gate. ```markdown - `packages/bundle`: This directory contains generated bundle files. Do not read nor write code here. - `packages/logs`: This directory contains generated log files for performance analysis. Do not read nor write code here. # Our Stack - We use Bun.js as the runtime environment and package manager - We use latest stable versions of all packages, languages, and libraries - This project is a monorepo managed by Bun Workspaces - Do not add any new package or dependency without asking first ``` Source: https://raw.githubusercontent.com/saadeghi/daisyui/HEAD/.github/instructions/workspace.instructions.md ## Hook-enforced validation loop on every edit HeroUI · full record: https://state-of-ai-in-design-systems.netlify.app/systems/heroui.md Rather than asking the agent to remember to lint, .claude/hooks.mjs runs prettier check pre-edit and `pnpm lint --fix` + `tsc --noEmit --skipLibCheck` post-edit, feeding results back into the transcript; a denylist throws outright for protected files. Deterministic enforcement outside the model’s discretion. ```javascript // Run type checking on TypeScript files if (filePath.match(/\.(ts|tsx)$/)) { try { execSync(`npx tsc --noEmit --skipLibCheck "${filePath}"`, {stdio: "pipe"}); } catch (e) { console.log("⚠️ TypeScript errors detected - please review"); } } ``` Source: https://raw.githubusercontent.com/heroui-inc/heroui/HEAD/.claude/hooks.mjs ## Scaffolding-first + validation-loop checklist for contributor agents Nuxt UI · full record: https://state-of-ai-in-design-systems.netlify.app/systems/nuxt-ui.md AGENTS.md hands the agent a copy-able progress checklist whose first step is the CLI scaffolder (so structure is generated, not invented) and whose last three steps are the three commands that must pass. The same triad is repeated in ‘Before Submitting’. Contributor agents therefore cannot declare done without running lint, typecheck and tests. ````markdown ## Component Creation Workflow Copy this checklist and track progress when creating a new component: ``` Component: [name] Progress: - [ ] 1. Scaffold with CLI: nuxt-ui make component - [ ] 2. Implement component in src/runtime/components/ - [ ] 3. Create theme in src/theme/ - [ ] 4. Export types from src/runtime/types/index.ts - [ ] 5. Register in ThemeDefaults interface (src/runtime/composables/useComponentProps.ts) - [ ] 6. Write tests in test/components/ - [ ] 7. Create docs in docs/content/docs/2.components/ - [ ] 8. Add playground page - [ ] 9. Run pnpm run lint - [ ] 10. Run pnpm run typecheck - [ ] 11. Run pnpm run test ``` ```` Source: https://raw.githubusercontent.com/nuxt/ui/v4/AGENTS.md ## Eval-gated skills with a 1.0 no-false-positive gate PatternFly · full record: https://state-of-ai-in-design-systems.netlify.app/systems/patternfly.md The strongest technique found in this study so far: ai-helpers ships an eval harness (`eval//eval.yaml` + case workspaces with fixture .tsx/.scss files and `annotations.yaml`) run by the `Skill Evals` GitHub Action on any PR touching `plugins/*/skills/**`. Judges are Python `check` blocks; thresholds are explicit. Two judges demand a perfect 1.0 pass rate: `routes_to_subskills` and `gate_skip_non_pf`, the latter failing the build if the router mentions any `pf-*` skill inside a non-PatternFly project. MCP is explicitly denied during evals (`deny: mcp__*`) so the harness measures the prompt’s own influence, not retrieval. ```yaml permissions: allow: [] deny: - "mcp__*" ... - name: gate_skip_non_pf description: Non-PF project gets no PF-specific routing — generic advice only if: "annotations.get('expected_routing') == 'none'" check: | pf_subskills = [ "pf-figma-check", "pf-color-scan", "pf-import-check", "pf-component-check", "pf-test-gen", "pf-figma-token-check", "pf-css-migration-scan", "pf-project-gen", "pf-icon-finder", "pf-figma-design-mode", "pf-design-comments-setup", "pf-ai-guide" ] found_pf = [s for s in pf_subskills if s in text_lower] if found_pf: return False, f"Non-PF project got PF-specific routing: {found_pf}" return True, "No PF sub-skill routing for non-PF project — gate check working" thresholds: routes_to_subskills: min_pass_rate: 1.0 gate_skip_non_pf: min_pass_rate: 1.0 actionable_next_step: min_pass_rate: 0.67 ``` Source: https://raw.githubusercontent.com/patternfly/ai-helpers/main/eval/pf-assist/eval.yaml ## Deterministic grep commands instead of model judgment PatternFly · full record: https://state-of-ai-in-design-systems.netlify.app/systems/patternfly.md Audit skills hand the model exact regexes and ripgrep invocations rather than asking it to ‘look for problems’. pf-import-check ships three `rg` commands targeting the specific import mistakes PF’s packaging invites (charts must come from `/victory`, chatbot and component-groups from `dist/dynamic/*`), plus the corrected import lines verbatim. pf-color-scan specifies HEX/RGB/HSL regexes, the 148-name X11 list, a property filter, and an exception carve-out so token *definitions* are not flagged. ````markdown Before proposing import fixes, use the PatternFly MCP server to confirm current package paths and examples from the latest docs. ## What to check 1. Charts imported from `@patternfly/react-charts` root (invalid for Victory components). 2. Chatbot imports not using `@patternfly/chatbot/dist/dynamic/*`. 3. Component-group imports not using `@patternfly/react-component-groups/dist/dynamic/*`. 4. Missing package CSS imports for features in use. ## Validation commands ```bash rg "@patternfly/react-charts['\"]" src rg "@patternfly/chatbot['\"]" src rg "@patternfly/react-component-groups['\"]" src ``` ## Correct import examples ```tsx import { ChartDonut } from "@patternfly/react-charts/victory"; import { Chatbot } from "@patternfly/chatbot/dist/dynamic/Chatbot"; import { BulkSelect } from "@patternfly/react-component-groups/dist/dynamic/BulkSelect"; ``` ```` Source: https://raw.githubusercontent.com/patternfly/ai-helpers/main/plugins/react/skills/pf-import-check/SKILL.md ## ERROR/WARN severity report contract with fix strings PatternFly · full record: https://state-of-ai-in-design-systems.netlify.app/systems/patternfly.md pf-component-check enumerates ~13 families of structural violations as a lookup table (e.g. `` direct under ``, `` without ``, `` without dataLabel = WARN) and pins the output to a lint-like format with severity, path:line, found, and fix. It also constrains autonomy: only unambiguous structural fixes may be applied; anything design-affecting must be reported and asked about. ````markdown 5. If the user requests fixes, apply them. Only fix unambiguous structural issues — if a fix would change behavior or needs a design decision, report it and ask. ... ### Report format For each violation: ``` [ERROR|WARN] file/path.tsx:42 - has direct children that are not Found: