Skip to content

Added: implemeted the breaking sentence by newline in robustness.#1109

Merged
chakravarthik27 merged 3 commits intorelease/2.4.0from
feature/implement-the-addnewlines-test-in-robustness-category
Sep 14, 2024
Merged

Added: implemeted the breaking sentence by newline in robustness.#1109
chakravarthik27 merged 3 commits intorelease/2.4.0from
feature/implement-the-addnewlines-test-in-robustness-category

Conversation

@chakravarthik27
Copy link
Copy Markdown
Collaborator

harness = Harness(
                  task="question-answering", 
                  model={"model": "gpt-4o-mini","hub":"openai"}, 
                  data={"data_source" :"NQ-open",
                        "split":"test-tiny"}
                  )

harness.configure({
    'tests': {
        'defaults': {'min_pass_rate': 0.65},
        'robustness': {
            'add_new_lines': {'min_pass_rate': 0.66},
        }
    }
})

generate and run test cases

harness.generate().run().report()

genereated results

harness.generated_results()

@chakravarthik27 chakravarthik27 self-assigned this Sep 14, 2024
@chakravarthik27 chakravarthik27 linked an issue Sep 14, 2024 that may be closed by this pull request
@chakravarthik27 chakravarthik27 changed the base branch from main to release/2.4.0 September 14, 2024 10:20
@chakravarthik27 chakravarthik27 merged commit a3986b4 into release/2.4.0 Sep 14, 2024
@chakravarthik27 chakravarthik27 added this to the 2.4.0 milestone Sep 16, 2024
@chakravarthik27 chakravarthik27 deleted the feature/implement-the-addnewlines-test-in-robustness-category branch September 23, 2024 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement the AddNewLines Test in Robustness Category

1 participant