Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3,906 changes: 3,906 additions & 0 deletions demo/tutorials/llm_notebooks/Swapping_Drug_Names_Test.ipynb

Large diffs are not rendered by default.

6,484 changes: 6,484 additions & 0 deletions demo/tutorials/misc/Evaluation_with_Prometheus_Eval.ipynb

Large diffs are not rendered by default.

2,585 changes: 2,585 additions & 0 deletions demo/tutorials/misc/Misuse_Test_with_Prometheus_evaluation.ipynb

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions demo/tutorials/misc/Multi_Model_Multi_Dataset.ipynb

Large diffs are not rendered by default.

394 changes: 394 additions & 0 deletions docs/pages/docs/langtest_versions/release_notes_2_2_0.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,5 @@ The following table gives an overview of the different tutorial notebooks to tes
| [**LM Studio**](lm_studio): Running Hugging Face quantized models through LM-Studio and testing these models for a Question Answering task. | LM Studio | Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/LM-Studio-Demo.ipynb) |
| [**Question Answering Benchmarking**](question_answering_benchmarking): This notebook provides a demo on benchmarking Language Models (LLMs) for Question-Answering tasks. | Hugging Face Inference API | Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/benchmarks/Question-Answering.ipynb) |
| **Fewshot Model Evaluation**: This notebook provides a demo on Optimize and evaluate your models using few-shot prompt techniques | OpenAI | Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/Fewshot_QA_Notebook.ipynb) |
| **Evaluating NER in LLMs**:In this tutorial, we assess the support for Named Entity Recognition (NER) tasks specifically for Large Language Models (LLMs) | OpenAI | Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/NER%20Casual%20LLM.ipynb) |
| **Evaluating NER in LLMs**:In this tutorial, we assess the support for Named Entity Recognition (NER) tasks specifically for Large Language Models (LLMs) | OpenAI | Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/NER%20Casual%20LLM.ipynb) |
| **Swapping Drug Names Test**:In this notebook, we discussed implementing tests that facilitate the swapping of generic drug names with brand names and vice versa. This feature ensures accurate evaluations in medical and pharmaceutical contexts. | OpenAI | Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/Swapping_Drug_Names_Test.ipynb) |
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,6 @@ The following table gives an overview of the different tutorial notebooks. In th
| **Generic API-Based Model**: In this section, we discussed how to test API-based models hosted using Ollama, vLLM, and other tools. | Web |Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/llm_notebooks/Generic_API-Based_Model_Testing_Demo.ipynb) |
| **Data Augmenter**: In this Notebook, we can allows for streamlined and harness-free data augmentation, making it simpler to enhance your datasets and improve model robustness. | - |NER | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Data_Augmenter_Notebook.ipynb) |
| **Multi-Dataset Prompt Configs**: In this Notebook, we discussed about optimized prompt handling for multiple datasets, allowing users to add custom prompts for each dataset, enabling seamless integration and efficient testing. | OpenAI |Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/MultiPrompt_MultiDataset.ipynb) |
| **Multi-Model, Multi-Dataset**: In this Notebook, we discussed about testing on multiple models with multiple datasets, allowing users to allows for comprehensive comparisons and performance assessments in a streamlined manner. | OpenAI |Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Multi_Model_Multi_Dataset.ipynb.ipynb) |
| **Evaluation_with_Prometheus_Eval**: In this Notebook, we disscussed about integrating the Prometheus model to langtest brings enhanced evaluation capabilities, providing more detailed and insightful metrics for model performance assessment. | OpenAI |Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Evaluation_with_Prometheus_Eval.ipynb) |
| **Misuse_Test_with_Prometheus_evaluation**: In this Notebook, we discussed about new safety testing features to identify and mitigate potential misuse and safety issues in your models | OpenAI |Question-Answering | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/langtest/blob/main/demo/tutorials/misc/Misuse_Test_with_Prometheus_evaluation.ipynb) |
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "langtest"
version = "2.2.0"
version = "2.3.0"
description = "John Snow Labs provides a library for delivering safe & effective NLP models."
authors = ["John Snow Labs <support@johnsnowlabs.com>"]
readme = "README.md"
Expand Down