EAGLE: Enhancing Uncertainty Estimation in LLMs with Expectation of Aggregated Internal Belief

🔮 Overview

This repository implements EAGLE , a novel self-evaluation-based calibration method for Large Language Models (LLMs) as presented in our paper. EAGLE improves uncertainty estimation by leveraging internal hidden states across multiple model layers to derive more accurate confidence scores.

Key features:

• Layer-wise hidden state aggregation for robust confidence estimation
• Expectation calculation over confidence score distributions
• Training-free approach compatible with various LLM architectures
• Superior calibration performance demonstrated across diverse tasks

🛠️ Installation

Installation Steps

pip install -r requirements.txt

🚀 Run Pipeline

./scripts/run.sh

📝 Citation

If you use this work, please cite our paper:

@misc{xiao2025enhancinguncertaintyestimationllms,
 title={Enhancing Uncertainty Estimation in LLMs with Expectation of Aggregated Internal Belief}, 
 author={Zeguan Xiao and Diyang Dou and Boya Xiong and Yun Chen and Guanhua Chen},
 year={2025},
 eprint={2509.01564},
 archivePrefix={arXiv},
 primaryClass={cs.CL},
 url={https://arxiv.org/abs/2509.01564}, 
}

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.