What is agentic RAG?

I would define agentic RAG as a retrieval-augmented generation system where the model controls part of the retrieval loop. In a basic RAG pipeline, I rewrite a query once, retrieve documents, and pass them to the model. In an agentic version, the model can decide whether it needs to search, choose what to look for, issue multiple queries, read the results, and decide whether to search again before it writes the answer.

That extra control is useful for questions that cannot be answered from one retrieval pass. For a multi-hop question, I might retrieve evidence about one entity, notice that another fact is missing, and issue a second query that connects the two. I can also ask for a narrower search if the first results look weak. The system therefore treats retrieval as an action the agent can iterate on, while the model uses the retrieved passages as evidence for the final response.

The tradeoff is that this can make the system more expensive, slower, and harder to evaluate than a fixed pipeline. Additional searches add retrieval overhead, and model calls to plan searches or process results can increase token cost. A model may also loop without converging. In production, I would cap the number of steps, log every query and retrieved document, and preserve a trace so I can diagnose both the answer and the path. I would test it on multi-hop tasks against a simpler baseline, measuring success as well as steps, cost, and latency rather than assuming the agentic design wins.