added LLM-based rubric auto-grade

source/instructors/authoring/assessments/add-assessment.rst

@@ -47,14 +47,14 @@ To add a new assessment for a course, follow these steps:
    .. image:: /img/guides/add_assessment.png
       :alt: Add Assessment
 
-3. Click **Create**.
+3. Fill the appropriate fields and click **Create**.
 
 While creating/editing the assessment, you can still see/edit the guide page.
 
 Assessment Auto-Generation
 ++++++++++++++++++++++++++
 
-Assessments can be auto-generated based on the text found on a guides page. Currently only :ref:`Multiple Choice <multiple-choice>`, :ref:`Fill in the Blanks <fill-in-blanks>`, :ref:`Free Text <free-text>` and :ref:`Parsons Puzzle <parsons-puzzle>` assessments can be auto-generated. For more information, please check out :ref:`Generating Assessments With AI <ai-assessment-generation>`
+Assessments can be auto-generated based on the text found on a guides page. Currently only :ref:`Multiple Choice <multiple-choice>`, :ref:`Fill in the Blanks <fill-in-blanks>`, :ref:`Free Text <free-text>`, :ref:`Standard Code Test <standard-code-test>`, and :ref:`Parsons Puzzle <parsons-puzzle>` assessments can be auto-generated. For more information, please check out :ref:`Generating Assessments With AI <ai-assessment-generation>`
 
 
 Refer to the specific topics for the type of assessment:
@@ -68,3 +68,4 @@ Refer to the specific topics for the type of assessment:
 - :ref:`Grade Book <grade-book>`
 - :ref:`Parsons Puzzle <parsons-puzzle>`
 - :ref:`Random Assessment <random>`
+- :ref:`LLM Rubric and LLM Rubric Autograde <llm-based-rubric>`

source/instructors/authoring/assessments/llm-based-rubric.rst

@@ -6,20 +6,27 @@
 
 LLM-Based Rubric Grader
 =======================
-You can find the **LLM Based Rubric** assessment in the **Manually Graded** section of the Assessments tab. More information about adding assessments can be found in our :ref:`assessment documentation <add-assessment>`.
 
-There are three steps to this process. 
+Codio has provided two versions of the **LLM-Based Rubric Grader**. One requires the instructor to approve the LLM-generated feedback, and the other automatically displays it to the student.
+
+The **LLM Rubric Autograde** assessment is in the **Auto-Graded** section of the assessments menu and the **LLM Rubric** assessment is in the **Manually Graded** section.
+More information about adding assessments can be found in our :ref:`assessment documentation <add-assessment>`.
+
+The first two steps are the same for both versions: 
 
 1. Rubric generation
 2. LLM-based grading using the generated rubric
+
+.. Note:: Step 3 is required only for the manually graded version - LLM feedback will not be released to the student until the instructor reviews and approves it.
+
 3. Manual approval of LLM-generated grades by the instructor
 
-.. Note:: Step 3 is required - LLM feedback will not be released to the student until the instructor reviews and approves it.
+
 
 LLM-Based Rubric Creation (Step 1)
 ----------------------------------
 
-Add an **LLM Based Rubric** assessment to your guide page and follow the steps below.
+Add an **LLM Rubric** or **LLM Rubric Autograde** assessment to your guide page and follow the steps below.
 
 
 1. On the **General** page, enter the following information:
@@ -34,62 +41,74 @@ Add an **LLM Based Rubric** assessment to your guide page and follow the steps b
 
    - Add a solution file (1) if you wish the rubric creation process to consider your solution.
 
-      .. image:: /img/guides/llmbasedrubric.png
-         :height: 600
-         :alt: Generate a rubric
+.. image:: /img/guides/llmbasedrubric.png
+   :height: 600
+   :alt: Generate a rubric
 
-   - Click the **Generate Rubrics** (2) button to initiate the process.
+- Click the **Generate Rubrics** (2) button to initiate the process.
 
-   The **Rubric Creation Agent** uses the following items to generate the rubric items:
+The **Rubric Creation Agent** uses the following items to generate the rubric items:
 
-   - The assessment name
-   - Instructions provided in the **General** tab of the assessment
-   - Content of the Guide Page where the assessment is being added
-   - Contents of the provided solution file
-   - The Course, Module, and Assignment name
-   - Requirements specified in the Rubric creation tab
-
-   .. Note:: If you do not add rubric requirements; the process will use general code grading norms to supply rubric items.
-
-   (optional) Add your requirements in the **Rubrics Requirements** dialog: 
+- The assessment name
+- Instructions provided in the **General** tab of the assessment
+- Content of the Guide Page where the assessment is being added
+- Contents of the provided solution file
+- The Course, Module, and Assignment name
+- Requirements specified in the Rubric creation tab
 
-      .. image:: /img/guides/llmrubricreqs.png
-         :height: 600
-         :alt: Area to add your rubric requirements
+.. Note:: If you do not add rubric requirements; the process will use general code grading norms to supply rubric items.
 
+Add your requirements in the **Rubric Requirements** dialog (optional): 
 
-   - Once you are done, click **Generate Using AI**. 
+.. image:: /img/guides/llmrubricreqs.png
+   :height: 600
+   :alt: Area to add your rubric requirements
 
 
-   - You can provide additional rubric items by clicking **Add Rubric** and entering information.
-   - Once you have reviewed the rubric items and other settings, click **Save** to save the assessment.
+- Once you are done, click **Generate Using AI**. 
+- You can provide additional rubric items by clicking **Add Rubric** and entering information.
+- Once you have reviewed the rubric items and other settings, click **Save** to save the assessment.
 
 
 LLM Grading Based on the Created Rubric (Step 2)
 ------------------------------------------------
 
-The LLM Grading agent uses the following to grade the student work:
+The grading occurs when the student clicks the **Check It** button. The LLM Grading agent uses the following to grade the student's work:
 
 - Instructions provided in the **General** tab of the assessment
 - Contents of the Guide page where the assessment is located
 - Contents of the specified solution file
 - The student file
 - The rubric generated in the previous step to identify the grading criteria
 
-.. Note:: The grading occurs when the student clicks the **Check It** button. The student receives information about whether they have passed or failed each rubric item, but does not see the rest of the LLM-generated feedback until after the instructor conducts their review.
+Sample feedback for the Auto-Graded Version:
+
+.. image:: /img/guides/rubricfinal.png
+    :height: 600
+    :alt: Final grading information displayed to the student
+
+
+Manual Approval by Instructor (Step 3 - only for the manually-graded version)
+-----------------------------------------------------------------------------
+.. Note::  The student receives information about whether they have passed or failed each rubric item, but does not see the rest of the LLM-generated feedback until after the instructor conducts their review.
+
+
+Sample student view of the feedback, before instructor approval:
 
 .. image:: /img/guides/rubricprelim.png
     :height: 600
     :alt: Preliminary grading information displayed to the student
 
+In the case of the manually graded version of this assessment, the instructor must open the student's assignment and select the passing test cases as part of the :ref:`grading process <grading>`. The instructor can also edit the comments generated by the LLM. Once this process is complete, click on **Apply Grade**. 
+
+.. Warning:: Once feedback is released to students, it cannot be modified.
 
-Manual Approval by Instructor (Step 3)
---------------------------------------
-The final step involves the instructor opening the student assignment and selecting the passing test cases as part of the :ref:`grading process <grading>`. The instructor can also edit the comments generated by the LLM. Once this process is complete, click on **Apply Grade**. Once the feedback is released to students, it cannot be modified.
+Sample Instructor view of LLM-generated feedback:
+
+.. image:: /img/guides/rubricapproval.png
+    :height: 600
+    :alt: Instructor view of LLM-generated feedback for approval
 
-   .. image:: /img/guides/rubricapproval.png
-      :height: 600
-      :alt: The teacher approves each comment generated by the LLM grader
 
 
 Rubric Requirements Example

source/instructors/getstarted/support/changelog.rst

@@ -18,6 +18,7 @@ We do not report small bug fixes or issues that affect individual users, who we
 - `[I] [24th]`: Pyret support has been deprecated.
 - `[I] [22nd]`: Pagination is now persistent across pages.
 - `[I] [20th]`: Added support for the Math markdown code block.
+- `[F] [17th]`: Add Ability to :ref:`Auto-Grade using an LLM-based rubric grader <llm-based-rubric>`
 - `[I] [14th]`: Behavior insight information is available in the assignment CSV file. :ref:`Assignment data <export-assignment>`
 - `[I] [9th]`: New Guides button for :ref:`uploading files<upload-file-button>`.