Martin Vogel
efec2ceffe
- Update to current capabilities: 158 languages, 11 agents, 14 MCP tools, Hybrid LSP, infrastructure-as-code indexing, single static C binary - Add dedicated Hybrid LSP section and per-language resolution table - Remove the GitNexus comparison - Restructure with answer-first, question-based headings for AI answer engines - Add JSON-LD structured data (SoftwareApplication, Organization, WebSite, BreadcrumbList, FAQPage) with arXiv preprint citation - Add a live GitHub releases widget (client-side, graceful fallback) - Add prominent research-preprint callout under the hero - Centered nav, "by DeusData" subtitle, inline SVG favicon - Welcome AI crawlers in robots.txt, refresh sitemap, add llms.txt - Fix broken benchmark link
34 lines
539 B
Plaintext
34 lines
539 B
Plaintext
# codebase-memory-mcp — all crawlers welcome, including AI answer engines.
|
|
User-agent: *
|
|
Allow: /
|
|
|
|
# Explicitly allow AI / LLM crawlers used by answer engines.
|
|
User-agent: GPTBot
|
|
Allow: /
|
|
|
|
User-agent: OAI-SearchBot
|
|
Allow: /
|
|
|
|
User-agent: ChatGPT-User
|
|
Allow: /
|
|
|
|
User-agent: ClaudeBot
|
|
Allow: /
|
|
|
|
User-agent: Claude-Web
|
|
Allow: /
|
|
|
|
User-agent: PerplexityBot
|
|
Allow: /
|
|
|
|
User-agent: Google-Extended
|
|
Allow: /
|
|
|
|
User-agent: Applebot-Extended
|
|
Allow: /
|
|
|
|
User-agent: Bingbot
|
|
Allow: /
|
|
|
|
Sitemap: https://deusdata.github.io/codebase-memory-mcp/sitemap.xml
|