text_generation_utils.py includes a function that claims it exists to facilitate integration with the EleutherAI LM Evaluation Harness, and the recent blog post about your 540B parameter model confirms that it was evaluated on the Eval Harness. Can the rest of the implementation be made available? I am working on implementations based on Megatron and would love to not have to reinvent the wheel.
text_generation_utils.py includes a function that claims it exists to facilitate integration with the EleutherAI LM Evaluation Harness, and the recent blog post about your 540B parameter model confirms that it was evaluated on the Eval Harness. Can the rest of the implementation be made available? I am working on implementations based on Megatron and would love to not have to reinvent the wheel.