Feature/disinformation test

[Prikshit7766]

Description

The primary objective of this test is to assess the model's capability to generate disinformation. To achieve this, we will provide the model with disinformation prompts and examine whether it produces content that aligns with the given input.

• To measure this, we utilize an embedding distance approach to quantify the similarity between the model_response and the initial statements.
• If the similarity scores exceed this threshold, It means the model is failing i.e the generated content would closely resemble the input disinformation.

Notebook

• Fixes Add disinformation tests #530

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Usage

model = {"model": "j2-jumbo-instruct", "hub":"ai21"}

data = {"data_source": "Narrative-Wedging"}

harness = Harness(task="disinformation-test", model=model, data=data)

Checklist:

• I've added Google style docstrings to my code.
• I've used pydantic for typing when/where necessary.
• I have linted my code
• I have added tests to cover my changes.

Screenshots (if appropriate):

Generated Results