Building a Wiki LLM: A Knowledge Layer for AI Coding Agents

AI coding agents can explore repositories, understand code, generate implementations, and help developers move faster. But as projects grow, a recurring problem becomes difficult to ignore:

Every new task can require the agent to rediscover knowledge that the team already has.

Repository structure, architectural decisions, API contracts, database relationships, coding conventions, and implementation constraints are often scattered across source files, documentation, and previous conversations.

This creates an opportunity to improve how coding agents access engineering knowledge.

I have been experimenting with a Wiki LLM setup inspired by the idea of maintaining a structured, continuously evolving knowledge base for AI-assisted development.

The goal is simple: instead of making an agent reconstruct the entire system for every task, give it access to the relevant engineering context.

1. The context problem in AI-assisted development

Consider a task such as adding a new endpoint to an existing FastAPI service.

Before implementing it, an agent may need to understand:

The repository and module structure
Existing API patterns and authentication
Database models and relationships
Service dependencies
Architectural decisions
Error-handling conventions
Constraints that must not be violated

Some of this information is available directly in the code. Other information exists in documentation or is implicit in previous engineering decisions.

Without a reliable knowledge layer, the agent may repeatedly inspect files, infer relationships, and reconstruct context.

This can increase token consumption, latency, and the chance of inconsistent assumptions.

The issue is not simply that an agent has a limited context window. It is also that context needs to be selected, organized, and kept accurate.

2. What is a Wiki LLM?

A Wiki LLM is a structured knowledge base designed to work alongside an LLM or AI agent.

Rather than treating every interaction as an isolated task, the system maintains reusable knowledge about the project.

My setup organizes engineering knowledge into areas such as:

Architecture: System components and their responsibilities
Concepts: Core domain knowledge and technical explanations
ADRs: Architectural decisions and their rationale
Project knowledge: Project-specific implementation details
Relationships: Dependencies between services, modules, and components
Constraints: Rules and assumptions that implementations must respect
Diagrams: Visual representations of system architecture and flows
Audits: Broken links, inconsistencies, and documentation gaps

The wiki is not intended to replace the source code. It provides a navigable knowledge layer that helps an agent understand the codebase more efficiently.

3. Architecture: From repository exploration to knowledge retrieval

A conventional agent workflow often looks like this:

Developer Task
      |
      v
   AI Agent
      |
      v
Explore Repository
      |
      v
Reconstruct Context
      |
      v
Implement and Test

A knowledge-assisted workflow introduces a retrieval step:

Developer Task
      |
      v
   AI Agent
      |
      v
Retrieve Relevant Knowledge
      |
      +---- Architecture
      +---- ADRs and Constraints
      +---- Related Components
      |
      v
Inspect Relevant Source Code
      |
      v
Implement and Test
      |
      v
Update Knowledge When Needed

The distinction is important: the agent still inspects source code when necessary. The wiki reduces unnecessary rediscovery; it does not eliminate the need to verify implementation details.

The ideal workflow combines documented knowledge with the current state of the repository.

4. How can this reduce token usage?

Token savings come from avoiding repeated context loading and retrieving only the information relevant to a task.

Consider an illustrative example:

Context source	Without structured retrieval	With structured retrieval
Repository and documentation exploration	25,000 tokens	5,000 tokens
Relevant architecture and decisions	Included above	2,000 tokens
Total	25,000 tokens	7,000 tokens

In this example, the context reduction is:

(25,000 - 7,000) / 25,000 × 100 = 72%

This is an illustrative calculation, not a benchmark of my setup.

Actual savings depend on the repository, task complexity, retrieval quality, model, caching, and how much source code must still be inspected.

There is also an important distinction between reducing the tokens needed to understand a task and reducing total API usage. A workflow may save input tokens while still using substantial tokens for implementation, reasoning, and testing.

For a meaningful evaluation, I would measure:

Input tokens per task
Output tokens per task
Total API cost
Time to complete the task
Number of repository exploration steps
Implementation correctness
Rework required after review

The objective is not to minimize tokens at any cost. It is to provide the right context with less unnecessary overhead.

5. Why not just use RAG?

Retrieval-Augmented Generation (RAG) is a natural option for retrieving relevant information from a large collection of documents.

However, a structured engineering wiki and RAG solve slightly different problems.

RAG focuses on finding relevant information from a knowledge collection.

A structured wiki also focuses on how knowledge is organized, connected, maintained, and validated.

For example, an architecture decision may link to a service, a database model, and a specific implementation constraint. Those relationships can help an agent understand not only what a component does, but also why it was designed that way.

These approaches are complementary. A wiki can be used with keyword search, graph traversal, embeddings, or a RAG pipeline.

For a smaller project, Markdown files and targeted retrieval may be sufficient. A vector database is not automatically necessary.

6. Keeping the knowledge trustworthy

A knowledge base becomes less useful when it is outdated or contradicts the code.

This is one of the most important engineering challenges in the setup.

A useful knowledge workflow needs safeguards:

Source-of-truth boundaries

The repository remains authoritative for the current implementation. The wiki explains architecture, intent, and relationships, but code must be checked before implementation decisions are made.

Link integrity

References between wiki pages should be validated so that navigation does not silently break.

Consistency checks

Audits should identify contradictory statements, stale documentation, and missing coverage.

Controlled updates

Agents should not freely rewrite shared engineering knowledge without appropriate validation. Proposed changes can be reviewed before becoming part of the trusted wiki.

Knowledge freshness

Changes to APIs, database schemas, or architecture should trigger a review of the related knowledge.

These controls help prevent the wiki from becoming another source of outdated assumptions.

7. What I am exploring next

My experiments focus on making the wiki useful as part of an actual agentic development workflow:

Project-aware context retrieval
Better organization of architecture and domain knowledge
Automated broken-link and consistency audits
Architecture diagrams connected to written documentation
Context budgeting for individual tasks
Controlled knowledge updates
Evaluation of token usage and implementation quality

The longer-term idea is to create a feedback loop in which developers and agents use the same engineering knowledge, while changes to the system are reflected in the knowledge base through a validated process.

Conclusion

AI coding agents are becoming increasingly capable of generating and modifying code. But reliable engineering requires more than code generation.

Agents need access to the architecture, decisions, relationships, and constraints that shape a system.

A Wiki LLM provides one way to organize that knowledge and make it reusable across tasks.

The potential benefits go beyond token savings: better consistency, easier onboarding, less repeated exploration, and a clearer connection between implementation and architectural intent.

I am still evaluating the right retrieval strategy and the measurable impact on real development tasks.

Agents can write code. A well-maintained knowledge layer helps them understand why the code exists.

#AI #LLM #AIAgents #SoftwareEngineering #DeveloperTools

Command Palette