TMSH LLM Wiki Brain

Structured Knowledge Base  ·  Ground-Truth Reference  ·  Hallucination Guard

A locally-hosted, compounding knowledge base for F5 BIG-IP's Traffic Management Shell. Stored as an Obsidian vault of interlinked Markdown files, it forces LLMs to consult vetted source material instead of guessing from training data. Based on Andrej Karpathy's "LLM Wiki" pattern (See: References) — feed a curated knowledge base into a session so answers are grounded in reality, not probability.

The Problem It Solves

LLMs — including Claude — have unreliable and sometimes flat-out wrong knowledge of TMSH syntax. TMSH is purpose-built, highly structured, and full of nuances that general training data doesn't capture well:

  • Partition scoping changes behavior in ways that aren't obvious
  • list vs show have distinct semantic differences that matter
  • modify list replacement behavior can silently wipe configuration
  • Flag combinations, module hierarchies, and object paths have exact syntax that training data often garbles

Rather than accept hallucinated TMSH syntax, the wiki is the authoritative ground truth that agents are required to consult before answering any TMSH question.

Architecture

Vault Structure

  • raw/ — Immutable source documents, never modified by agents
  • wiki/ — All generated content derived from sources
  • commands/ — Pages for each TMSH verb (27 total)
  • modules/ — Pages for TMSH modules (ltm, net, sys, auth, cli)
  • objects/ — Pages for configuration object types
  • concepts/ — Architectural and conceptual topics
  • patterns/ — Recipes, workflows, and anti-patterns

Page Schema

  • YAML frontmatter: title, category, tags, sources, updated date
  • Obsidian wikilinks ([[Page Name]]) for cross-references
  • Obsidian callouts for warnings, notes, and tips
  • Code blocks tagged with tmsh language
  • Every page ends with a "See Also" section
  • Pages split when they exceed ~400 words

Operations

  • Ingest — Drop a source into raw/, agent reads it, creates source summary + relevant pages, updates index and log
  • Query — Agent reads index, reads up to 5 relevant pages, synthesizes answer with inline citations
  • Lint — Scans for contradictions, stale claims, orphan pages, missing cross-references, and out-of-date index entries

Ingest Sources & Coverage

📄

F5 KB K13225 Cheatsheet — Established the list vs show semantic distinction as a foundational concept. First ingest, creating the initial command pages.

📕

BIG-IP TMSH Reference Guide v17.0.0 — 1,843-page authoritative F5 document. Full ingest created 47 pages: 27 commands, 5 modules, 9 objects, 4 concepts, 1 pattern, 2 source summaries, 1 overview, and 1 index.

Current State

Total Pages 48 across commands, modules, objects, concepts, patterns, sources, and overview
Sources Ingested 2 — K13225 cheatsheet + full v17.0.0 TMSH reference PDF
TMSH Commands 27 verbs (cd, create, delete, list, modify, show, run, save, etc.)
Modules 5 (ltm, net, sys, auth, cli)
Objects 9 (virtual, pool, node, persistence, vlan, self, route, route-domain, trunk)
Concepts 4 (partitions, SCF, transactions, CLI scripting)
Outstanding Gaps cm, security, gtm/dns, apm, asm/waf, pem — awaiting additional source documents

Access Rules

The wiki enforces strict reading discipline:

  • Start at index.md — the navigation layer, always free (doesn't count toward page limits)
  • Read at most 5 content pages per question — if more are needed, the agent stops, reports which pages would help, and asks for permission
  • Never exceed 5 pages without explicit approval — even if the answer seems incomplete
  • Fallback is mandatory silence — if the wiki has no coverage, the agent states "The wiki does not currently cover [topic]" and suggests what source could fill the gap. It does not answer from training data as a substitute

The entire point is to prevent training-data guessing. Falling back silently would defeat the purpose.

Toolchain Context

TMSH Wiki

  • Purpose: TMSH command/syntax ground truth
  • Sterility: No constraint — no client data ever enters it
  • Access: Direct file reads, triggered by usage rules
  • No dedicated skill — the Local / Project CLAUDE.md is the enforcement boundary

TMSH Wiki Trigger

  • Purpose: Context Boundary
  • Location: Global CLAUDE.md
  • Access: Query must qualify with a specific call for a TMSH Command or a query that would result in a TMSH Command

Enhancement Opportunities

If this implementation seems limited, it is in its current state, but it does not have to stay that way. Imagine adding in or "teaching" it scripts that would provide solutions to requests like "I want to export every Virtual Server to tmsh create commands that are executable for say a platform migration?"

The possibilities of this type of resource for an F5 Administrator are endless and only limited by your imagination.

Details

Status Completed
Origin Date April 2026
Pattern Andrej Karpathy's "LLM Wiki" concept
Storage Obsidian vault (interlinked Markdown)
CLAUDE.md Gold-standard template for all other repo CLAUDE.md files

Reference

Andrej Karpathy's GitHub Article: LLM Wiki