darlowie

Initial considerations:

LLM tuning has advantages.

 – It produces correct results on known question – answer pairs

 – It has better performance, in some cases x 10 times

LLM tuning has disadvantages.

 – Model is custom. There are investments in tuning to consider before changing models.

 – There is a vendor lock.

 – Need engineers who understand of LLM design

RAG has advantages:

 – It can work with off-the-shelf model

 – It is quicker to set up for existing data set

 – Need to know little about LLM design to start

 RAG has disadvantages:

  – Results are worse quality than tuned models

  – Performance is lower.

Another dimension to consider: data clustering

How well the data falls into clusters is an important consideration.

For dense clusters RAG is able to produce good enough results.

For sparse data tuning wins on quality with large gap.

Another dimension to consider: processing of input

Natural language input requires NLP parsing. Quality of parsed results depends on the format of original.

This propagates to output as per “garbage in, garbage out” rule.

Input of low quality needs to be mined. Dense data clusters help to have approximate hit. This favours RAG.

Conclusion:

RAG has a lower entry cost, it is a good starting point for LLM project.

When the project passes the prototype phase it prudent to consider tuning the model.

darlowie is attending BSides Canberra 2025 to hear the latest in SDL and cybersecurity best practice.

find us in the crowd, then read how darlowie is adding to the conversation by checking out some of our case studies, blog posts, or our methodology write-up

follow us on LinkedIN or subscribe to our RSS for more updates

see you in Canberra!

darlowie security consultancy turns recommendations into real action

we translate audit reports and security reviews into achievable steps for developers and software purchasers

security starts and ends with the code

read more about how, who we have helped, and how to start a conversation here