Initial considerations:
LLM tuning has advantages.
– It produces correct results on known question – answer pairs
– It has better performance, in some cases x 10 times
LLM tuning has disadvantages.
– Model is custom. There are investments in tuning to consider before changing models.
– There is a vendor lock.
– Need engineers who understand of LLM design
RAG has advantages:
– It can work with off-the-shelf model
– It is quicker to set up for existing data set
– Need to know little about LLM design to start
RAG has disadvantages:
– Results are worse quality than tuned models
– Performance is lower.
Another dimension to consider: data clustering
How well the data falls into clusters is an important consideration.
For dense clusters RAG is able to produce good enough results.
For sparse data tuning wins on quality with large gap.
Another dimension to consider: processing of input
Natural language input requires NLP parsing. Quality of parsed results depends on the format of original.
This propagates to output as per “garbage in, garbage out” rule.
Input of low quality needs to be mined. Dense data clusters help to have approximate hit. This favours RAG.
Conclusion:
RAG has a lower entry cost, it is a good starting point for LLM project.
When the project passes the prototype phase it prudent to consider tuning the model.
darlowie 