Rag Performance Prediction in Question Answering

Abstract: While retrieval-augmented generation has become a common approach for enhancing question answering systems, retrieval is not universally advantageous. We study the problem of predicting whether incorporating external retrieved information is likely to improve response quality for a given question. To this end, we evaluate a range of prediction methods that are based on retrieval signals, answer characteristics, and semantic consistency between generated responses and retrieved passages. In addition, we present a supervised predictor that jointly models interactions among the ques- tion, retrieved passages, and generated answer; the resultant prediction performance is state-of-the-art. We further devise a predictor that probes the LLM’s internal state. Its prediction performance significantly narrows the performance gap between post- generation methods which are computationally demanding and pre-generation (post- retrieval) methods. We use the prediction methods to devise a selective retrieval frame- work that dynamically chooses between retrieval and non-retrieval generation modes per question. Experimental results demonstrate that selectively applying retrieval aug- mentation yields answer quality that transcends that of using retrieval for all queries.