10 KiB
English | 中文
llm-wiki
Based on Andrej Karpathy's llm-wiki methodology
The personal knowledge base that grows with you
Turn scattered information into a growing, interconnected knowledge base
Preview
Oriental editorial × digital landscape interactive knowledge graph — double-click the HTML file to explore in your browser. Search, community legend, focus filters, layered node grammar, hover previews, click-to-read, Shift multi-select, canvas zoom/pan, and minimap navigation all run offline.
30-Second Start
Give this repo link to your agent and let it install itself.
# Claude Code
bash install.sh --platform claude
# Codex
bash install.sh --platform codex
# OpenClaw
bash install.sh --platform openclaw
# Hermes
bash install.sh --platform hermes
Then just say:
"Help me initialize a knowledge base" "Digest this article: "
The key difference: knowledge is compiled once, maintained continuously — not re-derived from scratch every query.
Highlights
| Feature | Description | |
|---|---|---|
| 🗺️ | Digital Landscape Graph | Self-contained HTML with a three-column oriental editorial layout, draggable and zoomable canvas, minimap navigation, and readable side panels |
| ✨ | Graph Reading Polish | Nodes now separate map labels, index slips, and cinnabar annotations; the default view is lighter, hover previews stay available, and clicks open reading mode |
| 🎓 | Local Reading Flow | Community legend, focus filters, scoped search, the reader drawer, and the selection drawer stay connected to the visible graph; offline HTML keeps selected facts visible too |
| 📦 | Zero-config Init | One sentence to create a full knowledge base with directory structure and templates |
| 🔗 | Structured Wiki | Auto-generates entity pages, topic pages, source summaries with [[bidirectional links]] |
| 🏷️ | Confidence Annotation | EXTRACTED / INFERRED / AMBIGUOUS / UNVERIFIED — see at a glance what needs verification |
| 🔄 | Smart Caching | SHA256 deduplication + write-through cache + self-healing safety net |
| 🧠 | Conversation Crystallization | Turn valuable conversations into knowledge base pages directly |
| 📡 | Auto Context Injection | SessionStart hook makes the agent automatically sense the knowledge base every session |
| 📊 | Multi-format Analysis | Deep reports, comparison tables, and timeline views |
Source Support
| Category | Sources | How |
|---|---|---|
| Core | PDF, Markdown, Text, HTML, Plain text | Direct ingestion, no external dependencies |
| Optional | Web articles, X/Twitter, WeChat, YouTube, Zhihu | Auto-extract via adapters; manual fallback if extraction fails |
| Manual | Xiaohongshu | Paste content directly |
Optional adapters require one extra flag:
bash install.sh --platform claude --with-optional-adapters
Platform Entry Points
Each platform has its own setup guide:
Full Feature List
- Research Direction Guidance —
purpose.mdgives the agent a clear direction for organizing and querying - Two-step Ingestion — Analyze first, then generate; long content uses chained thinking
- Ingest Format Validation — Scripts auto-validate analysis results; even weak models won't produce broken data
- Smart Material Routing — Auto-selects the best extraction method based on URL domain
- Core-first Install — Default setup only includes the core pipeline; optional extractors enabled explicitly
- Claude Companion Upgrade Command —
/llm-wiki-upgradeincluded after installation - Material Deletion — Cascade-delete with automatic cleanup of associated pages, broken links, and cache
- Query Persistence — Save valuable comprehensive answers back to the knowledge base
- Batch Digestion — Give a folder path, process all files at once
- Knowledge Base Health Check — Scripts detect orphan pages, broken links, index consistency; plus AI-level contradiction and cross-reference checks
- Ingest Privacy Check — First-time ingestion reminds you to check for phone numbers, API keys, etc.
- Graph Relationship Vocabulary — Optional manual annotation vocabulary for more precise graph diagrams
- Obsidian Compatible — All content is local markdown, open directly in Obsidian
Installation Details
Default Install Locations
| Platform | Path |
|---|---|
| Claude Code | ~/.claude/skills/llm-wiki |
| Codex | ~/.codex/skills/llm-wiki |
| OpenClaw | ~/.openclaw/skills/llm-wiki |
| Hermes | ~/.hermes/skills/llm-wiki |
Updating
Already installed? Run from the repo directory:
bash install.sh --upgrade
Automatically: git pull → detect installed platforms → re-copy core files → existing hooks preserved.
For Claude Code with default install, you can also use /llm-wiki-upgrade directly.
Custom directories:
bash install.sh --upgrade --platform openclaw --target-dir <your-skill-dir>/llm-wiki
bash install.sh --upgrade --platform hermes --target-dir <your-skill-dir>/llm-wiki
Prerequisites
- Core: your agent can run shell commands and read/write local files
- Optional:
uvfor WeChat extraction;bunornpmfor web extraction; Chrome debug mode (port 9222) for authenticated sessions
Directory Structure
your-knowledge-base/
├── raw/ # Raw materials (immutable)
│ ├── articles/ # Web articles
│ ├── tweets/ # X/Twitter
│ ├── wechat/ # WeChat articles
│ ├── xiaohongshu/ # Xiaohongshu
│ ├── zhihu/ # Zhihu
│ ├── pdfs/ # PDF
│ ├── notes/ # Notes
│ └── assets/ # Attachments
├── wiki/ # AI-generated knowledge base
│ ├── entities/ # Entity pages
│ ├── topics/ # Topic pages
│ ├── sources/ # Source summaries
│ ├── comparisons/ # Comparisons
│ ├── synthesis/ # Synthesis
│ │ └── sessions/ # Conversation crystallization
│ └── queries/ # Saved queries
├── purpose.md # Research direction
├── index.md # Index
├── log.md # Operation log
├── .wiki-schema.md # Config
└── .wiki-cache.json # Dedup cache
FAQ
Is this Claude-only? No. Claude is one of multiple entry points. The same repo can be installed by Claude Code, Codex, OpenClaw, or Hermes.
Why does Hermes care about HERMES.md?
Hermes loads the repo root HERMES.md as its highest-priority project context. That file only defines the Hermes entry and install path; the shared workflow still lives in SKILL.md.
Can I update from within Claude Code?
Yes. /llm-wiki-upgrade updates the core. Add --with-optional-adapters to also refresh web/X/WeChat/YouTube/Zhihu extraction.
X/Twitter extraction failing?
Ensure optional adapters are installed (--with-optional-adapters). For authenticated content, start Chrome with debug port 9222. You can also paste content directly.
WeChat extraction failing?
Requires uv. Install it, then re-run with --with-optional-adapters.
Windows Users
Windows PowerShell 5.1 (bundled with Win10 / Win11) defaults to GB2312 for console encoding, ASCII (CP1252) for $OutputEncoding, and gbk for Python subprocess sys.stdout.encoding on Chinese locale. Running bash install.sh directly under PowerShell 5.1 garbles Chinese output and hook JSON (#16).
Option A — Use install.ps1 (recommended)
From the repo root:
powershell -ExecutionPolicy Bypass -File install.ps1 --platform claude
powershell -ExecutionPolicy Bypass -File install.ps1 --platform codex --dry-run
install.ps1 sets console / $OutputEncoding / PYTHONIOENCODING all to UTF-8, then delegates to bash install.sh.
Option B — Manually set PowerShell encoding
chcp 65001
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
$OutputEncoding = [System.Text.Encoding]::UTF8
$env:PYTHONIOENCODING = 'utf-8'
bash install.sh --platform claude
Option C — Upgrade to PowerShell 7+
PowerShell 7 defaults to UTF-8. Install: winget install Microsoft.PowerShell, then run bash install.sh --platform claude under pwsh directly.
Python command
Windows typically installs Python as python.exe, not python3.exe (the python3 on PATH via Microsoft Store is a stub that prompts installation instead of running). scripts/shared-config.sh auto-detects: tries python3 first, falls back to python. As long as Python 3.8+ is on PATH under either name, all scripts work.
Credits
This project builds on:
- Andrej Karpathy — llm-wiki gist, the core methodology
- baoyu-url-to-markdown by JimLiu — Web/X/Twitter content extraction
- youtube-transcript — YouTube subtitle extraction
- wechat-article-to-markdown — WeChat article extraction
License
MIT