tobi/qmd

I use qmd for that on over 20 years of journaling and saved quotes, works great https://github.com/tobi/qmd

Exactly this, and this tool called qmd is what I use for the hybrid search portion. It also uses local LLMs to provide summaries on your own markdown data too. My agents use both depending on what type of search they are doing, and both provide good results. https://github.com/tobi/qmd

Definitely worth following for ideas. How did you solve memory in your system? I ended up using QMD + custom skills to keep it up to date as new data comes in. Essentially a cross-referenced set of markdown files forming a knowledge base: https://github.com/tobi/qmd

Most of my complex documents are, luckily, Markdown files. I can recommend https://github.com/tobi/qmd/ . It’s a simple CLI tool for searching in these kinds of files. My previous workflow was based on fzf, but this tool gives better results and enables even more fuzzy queries. I don’t use it for code, though.

I think that if I were to build this I might skip the LLM and just setup a nice interface to the source files using qmd style search. [0] That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly. I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers. [0] https://github.com/tobi/qmd

Have you looked at qmd by any chance? Thoughts on that for sqlite vs qmd in this case? https://github.com/tobi/qmd