Hallucination Detection

[joelpaulkoch]

After generating the response, we can check if the response contains hallucinations.
In a RAG context, there are several points to consider:

• does the response answer the question?
• is it based on the provided context?
• are there obvious hallucinations?

There are different ways to implement hallucination detection:

1. Models specifically trained for this task, for instance from vectara
2. LLMs answering the points above
3. Other techniques, e.g. SelfCheckGPT

For now I've implemented a simple hallucination detection that makes use of an LLM in ef1763b.

There is the RAG Triad that is used to check for hallucinations.
This could be done by an LLM too, we can ask it to return a score for each of the criteria and classify them as hallucination based on thresholds.

[joelpaulkoch]

Hallucination detection is implemented for Rag.Evaluation.Http and Rag.Evaluation.Nx.
For both ways, we just ask an LLM to output YES if the response is correctly reflecting the output, and NO if it's not.
For any output that does not exactly equal YES, we say it's a hallucination.