Skip to content

Feature/explore prompt injection tests#708

Merged
ArshaanNazir merged 25 commits intorelease/1.3.0from
feature/explore-prompt-injection-tests
Aug 17, 2023
Merged

Feature/explore prompt injection tests#708
ArshaanNazir merged 25 commits intorelease/1.3.0from
feature/explore-prompt-injection-tests

Conversation

@chakravarthik27
Copy link
Copy Markdown
Collaborator

@chakravarthik27 chakravarthik27 commented Aug 15, 2023

Description

This particular PR pertains to the implementation of prompt injection attack tests, which fall under the security category. It's worth noting that these tests carry immense significance when it comes to testing the safety and precautionary measures of LLM or text generation models such as Text-DaVinci-003 and hugging face models, among others. Their usefulness cannot be overstated.


➤ Fixes # (issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Usage

Link to Notebook: Prompt_Injections_Tests.ipynb

from langtest import Harness
import os

# any llm API key 
os.environ["OPENAI_API_KEY"] = <YOUR_API_KEY>

# model security testing 
harness = Harness(
    task="security",
    model={
        'model': "text-davinci-003",
        "hub": "openai"
    },
    data={
        'data_source': 'Prompt-Injection-Attack'
    }
)


harness.generate().run().report()

Checklist:

  • I've added Google style docstrings to my code.
  • I've used pydantic for typing when/where necessary.
  • I have linted my code
  • I have added tests to cover my changes.

Screenshots (if appropriate):

image

@chakravarthik27 chakravarthik27 self-assigned this Aug 15, 2023
@ArshaanNazir ArshaanNazir self-assigned this Aug 15, 2023
@ArshaanNazir ArshaanNazir linked an issue Aug 15, 2023 that may be closed by this pull request
@ArshaanNazir ArshaanNazir added the v2.1.0 Issue or request to be done in v2.1.0 release label Aug 15, 2023
Copy link
Copy Markdown
Contributor

@ArshaanNazir ArshaanNazir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine. Kindly make sure that the prompts you have curated are working fine @chakravarthik27

@ArshaanNazir
Copy link
Copy Markdown
Contributor

@chakravarthik27 kindly add usage guideline snippet as well in PR description

@ArshaanNazir ArshaanNazir merged commit dfbfcf9 into release/1.3.0 Aug 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v2.1.0 Issue or request to be done in v2.1.0 release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explore prompt injection tests

2 participants