I have spent years building a second brain in Obsidian on my phone, and I trust it completely. Then I started failing, and it was entirely my fault.
I was writing a note about a setting for a book I have worked on for years. Halfway through, I noticed the idea was familiar, and that was because I had created the same place before.
I tried to find a related note by checking my links, but I found nothing. So I searched manually, and after an hour I found the note. I wrote it more than a year earlier and then forgot it.
The entire time, Obsidian stored my notes exactly how it was designed to. I needed something else to remove the headache of organizing them.
Why my Obsidian vault stopped working for me
A few hundred notes in, tagging became chaotic
I set the vault up carefully, and Obsidian ordered my chaotic thoughts for the first year or so. But somewhere past a few hundred notes, my system started drifting apart.
I found two tags covering the same topic and one label attached to three unrelated projects. A couple of minutes of checking turned up more broken things than I can write down.
Most of my daily logs don’t even have a category. They’re stored in the vault as loose text, connected to nothing I had organized on purpose.
Search only helps when you know what to search for
The notes I lost were the ones I forgot writing
Obsidian’s full-text search would have found that setting note in seconds if I had known to look for it, but I didn’t know it existed.
That is the gap I wanted to close. I needed something that could read the whole vault and tell me what connected without me naming the thing first.
Retrieval-augmented generation does that. A RAG app indexes your documents, then answers questions by pulling the relevant passages and showing you where each one came from.
This is where Obsidian continues to take the cake for me. Since it stores every note as a plain Markdown file on the device, any app with file access can read a vault.
Pointing a local model at my vault folder
The model doesn’t run on a separate server
When people say “local LLM,” they usually imagine a model running on a desktop. I had the same thought, and it decided where I looked first.
Obsidian has plugins for local LLMs. Smart Connections and RAG Chat both index a vault and answer questions against it without sending notes to a cloud provider.
However, neither one runs a model. They send requests to a server, usually Ollama, hosted at a fixed address on your desktop. On my phone, I would need a machine running at home and a route back to it every time I asked a question.
I had used PocketPal for offline LLM access, so I knew a phone could run a model on its own. I wanted that, plus document indexing, rather than a chat window.
LocalRAG! handles 25 file formats, including Markdown. It runs either through Anthropic’s Claude in the cloud or through a bundled on-device model.
The local model needs a download of about 3GB. After that, extraction, indexing, search, and answer generation all happen on the phone with no internet connection needed.
I added my vault through the app’s file picker and asked about the story setting first, since that’s what started all of this. The app returned three notes about the same place, written across a full year.
Every answer carried numbered citations I could tap. I opened each source note, read it, and added the links to Obsidian by hand. I worked through that folder in a couple of days, then started on the others.
What the setup costs
The pricing plans, and a free tool to try first
LocalRAG! runs a free plan at five questions per day, and that plan includes the on-device model. Standard costs $4.99 per month, while Pro costs $9.99, and both come with a one-week trial.
I pay for Standard. Five questions a day is not enough for a full vault audit, though it is enough to test whether the approach suits you.
You can also skip the subscription by bringing your own Anthropic API key, which unlocks every model and bills you through Anthropic.
If price is a blocker, AnythingLLM runs on-device RAG for free, and it is open source. I chose LocalRAG! because tapping a citation opens the exact passage behind it, and that’s what lets me repair links one note at a time.
The on-device model is small, and it’s noticeable. It answers questions about my notes quite well, but I still need a full assistant for more complex tasks.
The app finds the gaps, and I close them
Nothing here organizes my vault for me, and I would be careful of any tool that offers to do so. What changed is that I can ask my notes what I have written and get an answer with the source attached.
I still read the note, still decide whether the link belongs, and still type it in myself. Obsidian rewards deliberate work, and I have not stopped doing that work. I have just stopped losing years of it to my own memory.
