Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added source/img/guides/ACTGradingScreen.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified source/img/guides/assessment_act_exec_custom.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified source/img/guides/assessment_undelete.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified source/img/guides/duplicate-assessment.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified source/img/guides/editassessmentbutton.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 12 additions & 11 deletions source/instructors/authoring/assessments/add-assessment.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@

.. _add-assessment:

Auto-graded assessments
=======================
Auto-Graded Assessments
========================

.. toctree::
:caption: Auto-graded assessments
:caption: Auto-Graded Assessments
:hidden:

assessments
Expand All @@ -26,7 +26,6 @@ Auto-graded assessments
llm-based-rubric
math-assessments
multiple-choice
parameterized
parsons-puzzle
partial-points
random
Expand All @@ -36,10 +35,14 @@ Auto-graded assessments
ungraded-assessments


Codio offers a variety of auto-graded assessments to streamline your grading process. Below, you'll find instructions on how to add assessments to your assignment, along with links to pages detailing specific assessment types and their features. Some auto-graded assessments can also be generated automatically.

Add a New Assessment
--------------------
To add a new assessment for a course, follow these steps:
Assessments can be auto-generated based on the content found on a guides page. Currently, the following assessment types support auto-generation: :ref:`Multiple Choice <multiple-choice>`, :ref:`Fill in the Blanks <fill-in-blanks>`, :ref:`Free Text <free-text>`, :ref:`Standard Code Test <standard-code-test>`, and :ref:`Parsons Puzzle <parsons-puzzle>`. For more information, please check out :ref:`Generating Assessments With AI <ai-assessment-generation>`.


How to Add a New Assessment
----------------------------
To add a new assessment for an assignment, follow these steps:

1. Open the assignment in the course, and click the **Edit** button to open the Guide Editor. You can also click the **Tools** menu and choose **Guides > Edit**.
2. Click the **Assessments** button and choose the type of assessment you want to add to the assignment.
Expand All @@ -51,11 +54,9 @@ To add a new assessment for a course, follow these steps:

While creating/editing the assessment, you can still see/edit the guide page.

Assessment Auto-Generation
++++++++++++++++++++++++++

Assessments can be auto-generated based on the text found on a guides page. Currently only :ref:`Multiple Choice <multiple-choice>`, :ref:`Fill in the Blanks <fill-in-blanks>`, :ref:`Free Text <free-text>`, :ref:`Standard Code Test <standard-code-test>`, and :ref:`Parsons Puzzle <parsons-puzzle>` assessments can be auto-generated. For more information, please check out :ref:`Generating Assessments With AI <ai-assessment-generation>`

Auto-Graded Assessment Types
-----------------------------

Refer to the specific topics for the type of assessment:

Expand Down
174 changes: 86 additions & 88 deletions source/instructors/authoring/assessments/advanced-code-test.rst

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,37 @@ Important points to consider when auto-generating assessments:

This is a new feature, and if it does not work properly for you, please let us know via the support chat or by emailing help@codio.com. If you would like us to take a look, send us your course name, the module, the assignment, and the page.

.. note:: Not all assessments can be AI-generated. If you don't see a "Generate" button in the bottom right corner when you click on an assessment, that assessment type does not support AI generation.


How to Generate an Assessment with AI
-------------------------------------

Assessments can be auto-generated using the text on the current guides page as context. Follow the steps below to auto-generate an assessment:

1. Select an assessment from the Assessments list.

2. Press the **Generate** button at bottom right corner.

.. image:: /img/guides/generate-assessment-button.png
:alt: Generate assessment button

3. The Generation Prompt will open, press **Generate Using AI** to preview the generated assessment.

.. image:: /img/guides/assessment-generation-prompt.png
:alt: Assessment Generation Prompt


4. When you are satisfied with the result, press **Apply** and then **Create**.


.. important:: The generate assessment feature does not configure the page :ref:`layout; <layout>` you should specify the layout depending on how you want to present the information to the students.




Example Prompts
---------------
----------------
**Multiple choice**

- Create a question about how to refactor the code below with multiple correct answers: (include code)
Expand Down
118 changes: 100 additions & 18 deletions source/instructors/authoring/assessments/assessment-security.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,115 @@
.. _assessment-security:

Assessment Security
===================
====================

Data for assessments is stored in individual files in the ``.guides/assessments`` folder.
Other folders automatically created in the ``.guides`` directory are: ``.guides/img`` and ``.guides/content``.
Student assignments only receive the ``.guides`` folder and the ``.guides/img`` folder. If you have not provided student access to the file tree in the :ref:`page layout <page>`, the ``.guides`` folder will not be visible.
File Organization
-----------------

Assessment data is stored in individual files within the ``.guides/assessments`` folder. The ``.guides`` directory automatically contains the following subdirectories:

- ``.guides/assessments`` - Assessment configuration files
- ``.guides/img`` - Image assets
- ``.guides/content`` - Content files

Student Workspace Limitations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When students receive an assignment, they only get access to:

- The ``.guides`` folder
- The ``.guides/img`` folder

If you have not enabled student access to the file tree in the :ref:`page layout <page>`, the ``.guides`` folder will not be visible to them.



Using the .guides Folder
------------------------


The ``.guides`` folder is a good place to store bash files or data files because they can't be easily deleted by the student via the file tree. Keep in mind though, if students have access to the terminal they can use it to view any files in their Linux container, including the ``.guides`` folder.
Benefits and Considerations
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Files that are stored in the ``.guides`` folder can be updated even if students have already started an assignment.
When you publish an assignment that students have already started, pre-existing files in the student workspace ``\home\codio\workspace`` will not be updated in order to make sure student work is not overwritten.
More information on this topic may be found on the :ref:`modify assignments <modify-assignments>` page.
The ``.guides`` folder provides a secure location for storing bash scripts and data files, as students cannot easily delete them through the file tree interface. However, be aware that students with terminal access can view any files in their Linux container, including contents of the ``.guides`` folder.

Blocking student access to the terminal
Updating Published Assignments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Files stored in the ``.guides`` folder can be updated even after students have started an assignment. When you republish an assignment that students have already begun:

- Files in ``.guides`` **will be updated**.
- Pre-existing files in the student workspace (``/home/codio/workspace``) **will not be updated** to prevent overwriting student work.

For more information, see the :ref:`modify assignments <modify-assignments>` page.



Blocking Student Access to the Terminal
---------------------------------------

The :ref:`terminal <terminal>` can be accessed from the file tree, the menu or by using the key combination (Shift+Alt+T on a PC, Shift+Option+T on a Mac). If you want to block all student access to the terminal you need to hide the file tree in the page layout, :ref:`customize the IDE menu<custom-ide>` and block the keyboard short cut in :ref:`project preferences<project-prefs>`.

The Secure folder
The :ref:`terminal <terminal>` can be accessed through multiple methods:

- From the file tree
- From the menu
- Via keyboard shortcuts (Shift+Alt+T on PC, Shift+Option+T on Mac)

To completely block student access to the terminal, you must:

1. Hide the file tree in the page layout
2. :ref:`Customize the IDE menu <custom-ide>` to remove terminal access
3. Block the keyboard shortcut in :ref:`project preferences <project-prefs>`



The Secure Folder
-----------------

If you are creating your own testing scripts or unit tests you can create a folder named ``.guides\secure`` to safely store test files. The secure directory is not copied over to the student version of the assignment.
When a student runs a code test that references a file in the ``.guides\secure`` folder, an **ephemeral** container is created that is a combination of the student workspace and the ``.guides\secure`` folder the instructor created.
This container is referred to as **ephemeral** because it only exists during the execution of the assessment. If you open a student project to view their work, you will not have access to the secure folder because it does not exist in the student environment.
Overview
~~~~~~~~

The ``.guides/secure`` folder provides a protected location for storing testing scripts and unit tests. This folder offers enhanced security because it is **not copied** to the student version of the assignment.

How It Works
~~~~~~~~~~~~

When a student runs a code test that references files in ``.guides/secure``, an **ephemeral container** is created that combines:

- The student workspace
- The instructor's ``.guides/secure`` folder

This container is called "ephemeral" because it only exists during assessment execution. If you open a student project to review their work, you will not see the secure folder—it does not exist in the student environment.

Important Limitations
~~~~~~~~~~~~~~~~~~~~~


**Do not write to the secure folder** during test execution, as changes will not persist. All output must be conveyed through:

- Standard output (stdout)
- The feedback buffer (described in individual assessment documentation)

Assessment Types Supporting Secure Folder
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following assessment types can access the secure folder:

- :ref:`Advanced Code Test <advanced-code-test>`
- :ref:`Free Text Autograde <free-text-autograde>`
- :ref:`Assignment Level Scripts <auto-grade-scripts>`

Using Secure Folder with Standard Code Tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:ref:`Standard Code Tests <standard-code-test>` do not have access to the secure folder by default. However, you can force a Standard Code Test to run in an **ephemeral container** by including a file path with ``.guides/secure`` in the **Command** field on the **Execution** tab.

Implementation Methods
~~~~~~~~~~~~~~~~~~~~~~

You can trigger ephemeral container creation by:

Your test should not write any information to the ``.guides\secure`` folder because it does not persist.
All output should conveyed using either stdout or the feedback buffer that is described in the individual assessesment types that can use the secure folder, :ref:`Advanced Code Test <advanced-code-test>`, :ref:`Free Text Autograde <free-text-autograde>` and :ref:`Assignment Level Scripts <auto-grade-scripts>`.
- Passing a secure folder file name as a parameter
- Including the file name on the command line (it will be ignored by your program, but signals Codio to mount the secure folder)

Standard code tests do not have access to the secure folder by default but you *can* force a :ref:`Standard Code Test <standard-code-test>` to run in an **ephemeral** container by including a file name with the ``.guides\secure`` path in the **Command** portion on the **Execution** tab.
This could be done if you are passing a file name as a parameter or if you have no command line parameters, you can put the file name on the command line and it will be ignored by your program but Codio will run the test in a container with the Secure folder mounted. Then you can specify a file in ``.guides\secure`` as an input parameter.
Once the ephemeral container is created, you can specify files in ``.guides/secure`` as input parameters for your tests.
44 changes: 41 additions & 3 deletions source/instructors/authoring/assessments/assessments.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,55 @@

.. _assessments:


Assessments
===========
Assessments can be used to determine how well students understand course material, and can be automatically or manually graded. Codio offers a wide range of assessment types, including auto-graded code tests, multiple choice tests, fill in the blanks, drop-down selection, free text responses and assignment grading. Assessments can be interspersed throughout tutorial materials or stand alone using an :ref:`assignment level script <auto-grade-scripts>`.

Assessments can be used to determine how well students understand course material, and can be automatically or manually graded. Codio offers a wide range of assessment types, including auto-graded code tests, multiple choice tests, fill in the blanks, drop-down selection, free text responses, and assignment grading. Assessments can be interspersed throughout tutorial materials or stand alone using an :ref:`assignment level script <auto-grade-scripts>`.

You can view the results of assessments in a course from the teacher dashboard.

Review the following topics to gain an understanding of using assessments:
**Assessment Topics**

- :ref:`Ungraded Assessments <ungraded-assessments>`
- :ref:`Add a New Assessment <add-assessment>`
- :ref:`Edit an Assessment <edit-assessment>`
- :ref:`Edit Points for an Assessment <edit-assessment-points>`
- :ref:`Delete an Assessment <delete-assessment>`
- :ref:`Assessment Security <assessment-security>`
- :ref:`Assessment Security <assessment-security>`

Common Assessment Configuration
--------------------------------

The following configuration options are common to all assessment types:

General Settings
~~~~~~~~~~~~~~~~

.. image:: /img/guides/assessment_general.png
:alt: General

- **Name** - Enter a short name that describes the test. This name is displayed in the teacher dashboard, so it should clearly reflect the challenge. Toggle the **Show Name** setting to hide the name in the challenge text the student sees.

- **Instructions** - Enter text that is shown to the student using optional Markdown formatting.

Metadata (Optional)
~~~~~~~~~~~~~~~~~~~

.. image:: /img/guides/assessment_metadata.png
:alt: Metadata

- **Bloom's Level** - Select the appropriate level of `Bloom's Taxonomy <https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/>`_ for the current assessment.

- **Learning Objectives** - Specify the educational goals of the assessment. Objectives typically begin with "Students Will Be Able To (SWBAT)." For example: *SWBAT follow the flow of recursive execution*.

- **Tags** - The **Content** and **Programming Language** tags are provided and required. To add custom tags, click **Add Tag** and enter the name and values.

Files (Optional)
~~~~~~~~~~~~~~~~

.. image:: /img/guides/assessment_files.png
:alt: Files

Check the boxes for any additional external files to include with the assessment when adding it to an assessment library. Selected files appear in the **Additional content** list.

Loading