Skip to content

Restructure BBQ data#725

Merged
ArshaanNazir merged 3 commits intorelease/1.4.0from
improve-bbq-data
Aug 31, 2023
Merged

Restructure BBQ data#725
ArshaanNazir merged 3 commits intorelease/1.4.0from
improve-bbq-data

Conversation

@RakshitKhajuria
Copy link
Copy Markdown
Contributor

@RakshitKhajuria RakshitKhajuria commented Aug 25, 2023

Description

Previously when evaluating the BBQ data there were some false negative results.

This PR aims at restructuring the BBQ data and refining the prompt to give better and accurate evaluations.


Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • This change requires a documentation update

Usage

harness = Harness(task="question-answering", model={"model": "text-davinci-003","hub":"openai"}, data={"data_source" :"BBQ-test-tiny"})

Screenshots (if appropriate):

@RakshitKhajuria RakshitKhajuria added 💡Enhancements Something can be improved v2.1.0 Issue or request to be done in v2.1.0 release labels Aug 25, 2023
@RakshitKhajuria RakshitKhajuria linked an issue Aug 25, 2023 that may be closed by this pull request
@ArshaanNazir ArshaanNazir self-requested a review August 31, 2023 05:28
@ArshaanNazir ArshaanNazir merged commit e13717e into release/1.4.0 Aug 31, 2023
@ArshaanNazir ArshaanNazir deleted the improve-bbq-data branch August 31, 2023 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

💡Enhancements Something can be improved v2.1.0 Issue or request to be done in v2.1.0 release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve BBQ dataset evaluation

3 participants