Matthew MacDonald
1 min readMar 11, 2024

--

The problem is that you can't effectively train the LLM on new information (it tends to get drowned out by what the LLM already knows). Instead, you need to use a technique called RAG. First, you break your manual into chunks. Then you encode those chunks into a special numeric format, and store this information in a vector database. Then, when querying for information, you use a kind of semantic search to find the contextually related parts of your user manual in the database, and you feed those to the LLM in your prompt. I'm going to write an article about this soon.

Some newer, bigger LLMs have really big context windows, so you may be able to give them your whole user manual at once. See, for instance, Gemini. But we don't know whether that will be as practical and cost effective as RAG, and you can't run it locally on your computer.

--

--

Matthew MacDonald
Matthew MacDonald

Written by Matthew MacDonald

Teacher, coder, long-ago Microsoft MVP. Author of heavy books. Join Young Coder for a creative take on science and technology. Queries: matthew@prosetech.com

No responses yet