RAG has been the way to provide a knowledge base for AI agents. A lot of reports mentioned that RAG has a few weaknesses. The biggest weakness is that RAG requires the information to be converted to vectors and stored in a database that is like a black box to humen. All LLMs have been trained to use tools such as grep to search over texts. The LLMs don't know how to operate a RAG database.

I have been trying to build a data analytical agent for a while. In this project, I would need to build a knowledge base which stores two most important pieces of information: firstly instructions of how to use each datawarehouse table and secondly some business terminology and definitions. These are not heavily knowledge but essential to carry out analysis. When user asks for an analytical task, I would need the agent to search through the knowledge base, extract the relevant information and use the information to carry out the task.

In a realistic scenario, a business has large set of documents as knowledge base. These documents spread over different business units. One BU rarely needs to get all information to carry out its own works. For example, when Product team uses this analytical agent, it rarely needs knowledge from the Engineering department. Most likely, the product related metrics (such as monthly active users) and product features are most important. Engineering team needs different set of knowledge, such as the number of API calls or the memory consumption of a particular service. Dumping all knowledge to a single knowledge base is often unnecessary and may increase the chance for irrelvant information to be extracted for a particular task.

To my understanding, the best approach would be for each BU/team/personel to have the ability to define its own knowledge base. The knowledge base is more likely to be managed by non-technical individuals, such product managers and CRM managers. It is hard for non-technical people to operate a RAG database.

With this consideration, I used to try to use skills to hold this knowledge. However, I find out that it is infeasible to let skills take the role of being the knowledge base. The challengings are:

• Skills need to be drafted from scratch. In a real scenario, there are already existing documents/knowledges. You need a quick way to merge them into a knowledge base.
• Skills are better to describe 'way of doing things'. But not ideal to hold knowledges. Knowledges are better to be presented as entities and relations.
• Knowledge can be accumulated to considerable large. If knowledges are presented in skills, you still need to load the skill catalogs to context. When only one or two pieces of knowledges are required for a specific task, loading all knowledges' catalog is still a waste of context.
• If you mix real skills with knowledges, the agent may get confused and miss the scenarios where the real skill is needed.

When I am exploring and discovering a better approach to implement the knowledge base, Andrej Karpathy provides a new approach: use Obsidian to build a LLM driven personal wiki. I tried it out and think this may become a good way for my project. Wiki is naturally good to serve as a knowledge base. Andrej provides an approach to easily dump the raw resources to Obsidian and let LLM reform the raw information to wiki elements. LLM also manages conflict information, cross references and links. This approach fits my purpose in a few ways:

• The wiki form is good to hold knowledges.
• For existing knowledges, you can use Obsidian's web clip plugin to easily dump it to the knowledge and ask LLM to ingest it into the wiki. It is easy to utilize the existing knowledges.
• Everything is markdowns. People can see what's in the knowledge base. People can correct mistakes or inaccurate components. You can draft a new piece of document, dump to Obsidian and check the newly created wiki to make sure information is added with accuracy. Human can read through the entire knowledge base.
• There is no need to pre-load anything. Just provide a tool for wiki query and LLM can dnymaically fetch the information from the knowledge to carry out works.
• It is easy to combine semantic search and keyword search in the query tool. Keyword search just needs to match keywords in the index.md file. Semantic search can be implemented using qmd. You can combine the two query approaches easily by managing the query tool, which is fully self-implemented. The search behavior is easier to manage than the RAG approach.
• To leverage the knowledge base to any agent, you just need to write a single skill to tell the agent when and how to search the wiki. That's it. A single skill to open the door to your entire knowledge base to any agent.

I am not surprised that this is an innvoative and looking-elegant solution. It is from Andrej Karpathy anyway. But I am quite amazed to see how he delivers this idea to the public. He didn't provide any implementation. Instead, he just provided a single piece of markdown document (llm-wiki.md). He stated that this markdown is to be fed to your own vibe coding agent and the agent will decide the best approach to implement the idea. It is completely open for you to discuss and twist the ideas with your coding agent. There is no single line of code from him. Just this document that I can easily read through and understand what he is doing. I copy pasted it to Codex and it implemented the entire idea. After only a few twists, I found that it is working! I started dumping web clips to the knowledge base and Codex starts to build a wiki out of them. Then I try to draft a skill on how to use the wiki and provided the skill to Claude Code on another project. Then Claude Code started querying the wiki and got the instruction to carry out tasks. Everything just works.

I will continue to experiment working with this wiki approach for my analytical agent project and may share some experience in the near future. At the moment, I am thinking that Obsidian is not necessary here. After all, the wiki project is just a folder holding markdown files. Though Obsidian is naturally a good application to draft and manage markdowns, you can replace it with your own markdown management tool. The one advance of Obsidian is its web clip plug-in, which can convert html into markdown. I am sure big projects can re-invent their own convertors.

Andrej shows a brand new open source software engineering way: you no longer need to provide the implementation. Instead, you just need to provide a markdown document describing the idea. The document is not only to be read by human to understand what is your idea. It is also for the coding agents to implement the idea. You share the idea instead of your own implementation. Everyone can take your idea, twist it in their own way and implement it with their coding agents. Your software engineering skill is no longer the important skill. The way you describe your idea is the king. Andrej's document is not long, easy to read, and yet can be given to Codex for a complete workable implementation. This ability is invaluable. After I read his document, I asked myself this question: now I know what to build, could I draft a document as clear and succinct as his? Probably not. I suggest everyone read his draft a few times and try to learn why such a short document has such a power to allow a coding agent to come up with a completely working implementation. The ability to draft such blueprints is very important in the future software engineering.

By the way, if you want to get my implementation, you can find it here.