I am retiring soon after 30 years. I am making AI assissted Documentation. After 3 starts and many, many hours I am getting good output. But at this point it I realize it may all be a waste of time.
Part of the reason I get decent output is because I have alot of control over what gets sent to the LLM. Using the paid version of Gemini, I am using a postgres vector database and a local AI called SentenceTransformers to create the embeddings used for the first part of the AI process. The embeddings are mathematical arrays of numbers representing the documentation I have written. The pages of documentation is read as blocks, maybe half a page is a block. That block is stored as a vector.
Using a documentation program called Obsidian, my prompt is turned into a vector, then compared with the vector database. What is then passed to the LLM are the documents that relate to vectors found by an SQL query comparing the prompt vector with the stored documentation vectors. The query is augmented by a hints file that says things like, "if you see this word in the prompt, always send this file to the LLM." This is all part of a RAG (Retrieval-Augmented Generation) process.
Why is all this stuff important? Because the less you control the information sent to the LLM, the less satisfactory your answer will be. If you really want a good answer on your system components, attach text files containing the specs. (Don’t assume the AI can read other types of files) Attach images containing specs. Give it files of relevant reveiw, etc.
But be careful, you can dilute the prompt and information. Dilution occurs when you start sending too much. If you are wanting an honest review of amps in a certain price range, don’t include amps outside of that price range. That dilutes the data sent to the LLM by sending irrelevant information.
Also, be careful how you word things. If you consistently call an amplifier "amplifier" and suddenly throw in "receiver", the LLM may ’panic’ and hallucinate something trying to fit the ’receiver’ into the logic.
I have found, like many things, the more I learn about AI, the more I am just scratching the surface.