diff --git a/source/img/guides/ACTGradingScreen.png b/source/img/guides/ACTGradingScreen.png new file mode 100644 index 00000000..5c04fc82 Binary files /dev/null and b/source/img/guides/ACTGradingScreen.png differ diff --git a/source/img/guides/assessment_act_exec_custom.png b/source/img/guides/assessment_act_exec_custom.png index 9e6edf7b..9b05344d 100644 Binary files a/source/img/guides/assessment_act_exec_custom.png and b/source/img/guides/assessment_act_exec_custom.png differ diff --git a/source/img/guides/assessment_undelete.png b/source/img/guides/assessment_undelete.png index 7026f582..0d278b18 100644 Binary files a/source/img/guides/assessment_undelete.png and b/source/img/guides/assessment_undelete.png differ diff --git a/source/img/guides/duplicate-assessment.png b/source/img/guides/duplicate-assessment.png index 4e114982..6e474897 100644 Binary files a/source/img/guides/duplicate-assessment.png and b/source/img/guides/duplicate-assessment.png differ diff --git a/source/img/guides/editassessmentbutton.png b/source/img/guides/editassessmentbutton.png index 2da2d5bc..3d05dcc2 100644 Binary files a/source/img/guides/editassessmentbutton.png and b/source/img/guides/editassessmentbutton.png differ diff --git a/source/instructors/authoring/assessments/add-assessment.rst b/source/instructors/authoring/assessments/add-assessment.rst index d79702f1..d68505e0 100644 --- a/source/instructors/authoring/assessments/add-assessment.rst +++ b/source/instructors/authoring/assessments/add-assessment.rst @@ -3,11 +3,11 @@ .. _add-assessment: -Auto-graded assessments -======================= +Auto-Graded Assessments +======================== .. toctree:: - :caption: Auto-graded assessments + :caption: Auto-Graded Assessments :hidden: assessments @@ -26,7 +26,6 @@ Auto-graded assessments llm-based-rubric math-assessments multiple-choice - parameterized parsons-puzzle partial-points random @@ -36,10 +35,14 @@ Auto-graded assessments ungraded-assessments +Codio offers a variety of auto-graded assessments to streamline your grading process. Below, you'll find instructions on how to add assessments to your assignment, along with links to pages detailing specific assessment types and their features. Some auto-graded assessments can also be generated automatically. -Add a New Assessment --------------------- -To add a new assessment for a course, follow these steps: +Assessments can be auto-generated based on the content found on a guides page. Currently, the following assessment types support auto-generation: :ref:`Multiple Choice `, :ref:`Fill in the Blanks `, :ref:`Free Text `, :ref:`Standard Code Test `, and :ref:`Parsons Puzzle `. For more information, please check out :ref:`Generating Assessments With AI `. + + +How to Add a New Assessment +---------------------------- +To add a new assessment for an assignment, follow these steps: 1. Open the assignment in the course, and click the **Edit** button to open the Guide Editor. You can also click the **Tools** menu and choose **Guides > Edit**. 2. Click the **Assessments** button and choose the type of assessment you want to add to the assignment. @@ -51,11 +54,9 @@ To add a new assessment for a course, follow these steps: While creating/editing the assessment, you can still see/edit the guide page. -Assessment Auto-Generation -++++++++++++++++++++++++++ - -Assessments can be auto-generated based on the text found on a guides page. Currently only :ref:`Multiple Choice `, :ref:`Fill in the Blanks `, :ref:`Free Text `, :ref:`Standard Code Test `, and :ref:`Parsons Puzzle ` assessments can be auto-generated. For more information, please check out :ref:`Generating Assessments With AI ` +Auto-Graded Assessment Types +----------------------------- Refer to the specific topics for the type of assessment: diff --git a/source/instructors/authoring/assessments/advanced-code-test.rst b/source/instructors/authoring/assessments/advanced-code-test.rst index e7e309c4..d2b6ed36 100644 --- a/source/instructors/authoring/assessments/advanced-code-test.rst +++ b/source/instructors/authoring/assessments/advanced-code-test.rst @@ -7,118 +7,121 @@ Advanced Code Test ================== The advanced code test assessment type allows you to easily implement unit tests, style checkers, or write custom code tests in any language that grades student-written code. -To ensure that your test scripts run securely and to prevent student access to your testing files or executables, place them in the **.guides/secure** folder. This folder is not available to students in their assignments and they cannot access it from the command line. Only teachers with editing privileges have access to the **.guides/secure** folder. +To ensure that your test scripts run securely and to prevent student access to your testing files or executables, place them in the **.guides/secure** folder. Create a folder if one does not already exist. This folder is not available to students in their assignments and they cannot access it from the command line. Only teachers with editing privileges have access to the **.guides/secure** folder. - .. Note:: If your assignment will contain multiple assessments, Code files and Test Cases for individual assessments should be placed in separate folders to avoid compiling all files. +.. Note:: If your assignment will contain multiple assessments, Code files and Test Cases for individual assessments should be placed in separate folders to avoid compiling all files. -Complete each section to set up your advanced code test. -1. On the **General** page, enter the following information: +Complete each section to set up your advanced code test. For more information on **General**, **Metadata** and **Files** see :ref:`Assessments `. - .. image:: /img/guides/assessment_general.png - :alt: General - - - **Name** - Enter a short name that describes the test. This name is displayed in the teacher dashboard so the name should reflect the challenge and thereby be clear when reviewing. - - Toggle the **Show Name** setting to hide the name in the challenge text the student sees. - - - **Instructions** - Enter text that is shown to the student using optional Markdown formatting. +1. Complete **General**. 2. Click **Execution** in the navigation pane and complete the following information: - **Language Type** - Click the drop-down and choose the language. The following language-specific frameworks are explicitly supported: + - **Custom**: for a `custom`_ auto-grading script in any language - **Ruby**: `RuboCop`_ or `RSpec`_ - - **Java**: `JUnit`_ or `checkstyle`_ + - **Java**: `checkstyle`_ or `JUnit`_ - **Python**: `pycodestyle`_ or `UnitTest`_ - **JavaScript**: `JSHint and JSLint`_ - - **Custom**: for a `custom`_ auto-grading script in any language - - **Language Assessment Subtype** - Click the drop-down and choose a subtype for the selected language type, if applicable. - - - **Timeout** - Where you can amend the timeout setting for the code to execute. Arrows will allow you to set max 300 (sec), if you require longer, you can manually enter the timeout period. +.. Note:: For more information, see the :ref:`test-types` section or click any test name above to navigate directly to that section. -4. Click on the **Parameters** tab if you wish to edit/change **Parameterized Assessments** (deprecated) using a script. See :ref:`Parameterized Assessments ` for more information. New parameterized assessments can no longer be set up. +- **Language Assessment Subtype** - Click the drop-down and choose a subtype for the selected language type, if applicable. + +- **Timeout** - You can amend the timeout setting for the code to execute. Arrows will allow you to set max 300 (sec). If you require longer, you can manually enter the timeout period. -3. Click **Grading** in the left navigation pane and complete the following fields: +3. Click **Grading** in the top navigation pane and complete the following fields: - .. image:: /img/guides/Grading-new-feature1.png + .. image:: /img/guides/ACTGradingScreen.png :alt: Grading + :width: 500px - **Points** - The score given to the student if the code test passes. You can enter any positive numeric value. If this assessment should not be graded, enter 0 points. - - **Allow Partial Points** - Toggle to enable a percentage of total points to be given based on the percentage correctly answered. See :ref:`Allow Partial Points ` for more information. + - **Partial Points** - Toggle to enable a percentage of total points to be given based on the percentage correctly answered. Note that it's not enough to turn partial points on; the advanced code test has to be written to handle partial points. See :ref:`Partial Points ` for more information. - **Define Number of Attempts** - enable to allow and set the number of attempts students can make for this assessment. If disabled, the student can make unlimited attempts. - **Show Rationale to Students** - Toggle to display the rationale for the answer to the student. This guidance information will be shown to students after they have submitted their answer and any time they view the assignment after marking it as completed. You can set when to show this selecting from **Never**, **After x attempts**, **If score is greater than or equal to a % of the total** or **Always** - **Rationale** - Enter guidance for the assessment. This is always visible to the teacher when the project is opened in the course or when opening the student's project. - **Use maximum score** - Toggle to enable assessment final score to be the highest score attained of all runs. -5. Click **Metadata** in the left navigation pane and complete the following fields: +5. **(Optional)** Complete **Metadata**. - .. image:: /img/guides/assessment_metadata.png - :alt: Metadata +6. **(Optional)** Complete **Files**. - - **Bloom's Level** - Click the drop-down and choose the level of Bloom's Taxonomy: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ for the current assessment. - - **Learning Objectives** The objectives are the specific educational goal of the current assessment. Typically, objectives begin with Students Will Be Able To (SWBAT). For example, if an assessment asks the student to predict the output of a recursive code segment, then the Learning Objectives could be *SWBAT follow the flow of recursive execution*. - - **Tags** - The **Content** and **Programming Language** tags are provided and required. To add another tag, click **Add Tag** and enter the name and values. +7. Click **Create** to complete the process. -6. Click **Files** in the left navigation pane and check the check boxes for additional external files to be included with the assessment when adding it to an assessment library. The files are then included in the **Additional content** list. - .. image:: /img/guides/assessment_files.png - :alt: Files +See a Working Example +---------------------- +To see an example of a specific unit test or style checker, see the Starter Pack in the corresponding language: -7. Click **Create** to complete the process. +- Go to **Starter Packs** and search for **Advanced Features in Python** if not already loaded in your **My Projects** area. Click **Use Pack** and then **Create** to install it to your Codio account. + +- Information about C++ unit testing using GoogleTest is available in the **C++ Unit Testing Using GoogleTest** Starter Pack. + +- Additionally, Codio pre-populates a project in **My Projects** called **Demo Guides and Assessments** that contains examples for all assessment types and a guides authoring cheat sheet. If you do not see this project, go to **Starter Packs** and search for **Demo Guides and Assessments**. Click **Use Pack** and then **Create** to make a copy in your **My Projects** area. + + +.. _test-types: + +Test Types +---------- ---------------------- RuboCop ---------------------- - `RuboCop (website link)`_ is a Ruby Linter. +`RuboCop (website link)`_ is a Ruby Linter. - To check if RuboCop is already installed on your stack, simply run `rubocop` from the command line. If it is not installed, you can easily install it either via the command line (`gem install rubocop`) or using **bundler** (by adding `gem 'rubocop', require: false` to your Gemfile). +To check if RuboCop is already installed on your stack, simply run `rubocop` from the command line. If it is not installed, you can easily install it either via the command line (`gem install rubocop`) or using **bundler** (by adding `gem 'rubocop', require: false` to your Gemfile). - When using Rubocop in Codio, specify the Ruby files you'd like RuboCop to check under the **ADD CASE:** option. +When using Rubocop in Codio, specify the Ruby files you'd like RuboCop to check under the **ADD CASE:** option. - The student will need to follow all style conventions to earn full credit on the assessment. +The student will need to follow all style conventions to earn full credit on the assessment. .. _RuboCop (website link): https://rubocop.org/ + ---------------------- RSpec ---------------------- - `RSpec (website link)`_ is a Ruby testing suite. +`RSpec (website link)`_ is a Ruby testing suite. - To check if RSpec is already installed on your stack, simply run `rspec` from the command line. If it is not installed, you can easily install it either via the command line (`gem install rspec`) or using **bundler** by adding it to your Gemfile. +To check if RSpec is already installed on your stack, simply run `rspec` from the command line. If it is not installed, you can easily install it either via the command line (`gem install rspec`) or using **bundler** by adding it to your Gemfile. - When using RSpec in Codio, specify the ruby files containing RSpec tests you'd like to run under the **ADD CASE:** option. +When using RSpec in Codio, specify the ruby files containing RSpec tests you'd like to run under the **ADD CASE:** option. - If you have more then one test, by default, the student will need to pass all tests to earn the specified number of points. You can toggle on **ALLOW PARTIAL POINTS** to have Codio evenly weight each test. +If you have more than one test, by default, the student will need to pass all tests to earn the specified number of points. You can toggle on **ALLOW PARTIAL POINTS** to have Codio evenly weight each test. .. _RSpec (website link): https://rspec.info/ + ---------------------- JUnit ---------------------- - `JUnit (website link)`_ is a Java testing framework. Currently Codio supports JUnit 4 and JUnit 5 so you can choose any one of them as per your requirement. +`JUnit (website link)`_ is a Java testing framework. Currently Codio supports JUnit 4 and JUnit 5 so you can choose any one of them as per your requirement. - When using JUnit in Codio, specify the Java files containing JUnit tests you'd like to run under the **ADD CASE:** option. +When using JUnit in Codio, specify the Java files containing JUnit tests you'd like to run under the **ADD CASE:** option. - If you have more then one test, by default, the student will need to pass all tests to earn the specified number of points. You can toggle on **ALLOW PARTIAL POINTS** to have Codio evenly weight each test. +If you have more than one test, by default, the student will need to pass all tests to earn the specified number of points. You can toggle on **ALLOW PARTIAL POINTS** to have Codio evenly weight each test. - There are 4 *optional* configurations for more complex file structures: +There are 4 *optional* configurations for more complex file structures: - - **SOURCE PATH** - specifies where the student code being tested is - - **TESTS SOURCE PATH** - specifies where non-test-case test helper files are - - **LIBRARY PATH** - specifies where .jar files needed to run the student code or test code at - - **WORKING DIRECTORY** - specifies where in the file tree the actual test will run + - **SOURCE PATH** - Specifies the location of the student code being tested. + - **TESTS SOURCE PATH** - Specifies the location of non-test-case test helper files. + - **LIBRARY PATH** - Specifies the location of .jar files needed to run the student code or test code. + - **WORKING DIRECTORY** - Specifies where in the file tree the test will run. - All code files **Source path** will be compiled. Files that fail to compile successfully will cause the tests to fail, even if they are not used. - Codio has a :ref:`JUnit ` runner for building JUnit tests. +All code files **SOURCE PATH** will be compiled. Files that fail to compile successfully will cause the tests to fail, even if they are not used. +Codio has a :ref:`JUnit ` runner for building JUnit tests. +----------------------------------- Custom Feedback with JUnit in Codio ----------------------------------- - When using JUnit in Codio, you can add your own custom feedback to the standard feedback Junit returns to students. The feedback message is passed to the assert method as the first parameter. +When using JUnit in Codio, you can add your own custom feedback to the standard feedback Junit returns to students. The feedback message is passed to the assert method as the first parameter. `assertEquals(feedback, expected, actual)` @@ -128,11 +131,11 @@ Custom Feedback with JUnit in Codio checkstyle ---------------------- - `checkstyle (website link)`_ is a Java linter. +`checkstyle (website link)`_ is a Java linter. - When using checkstyle in Codio, specify the configuration file under **CONFIG PATH** -- you can use the `Google configuration`_, `Sun configuration`_, or `create your own configuration`_. +When using checkstyle in Codio, specify the configuration file under **CONFIG PATH** -- you can use the `Google configuration`_, `Sun configuration`_, or `create your own configuration`_. - Select the **CHECKSTYLE VERSION**, by default the appropriate version is selected according to your installed Java version but you can also select one of the available options: +Select the **CHECKSTYLE VERSION**, by default the appropriate version is selected according to your installed Java version but you can also select one of the available options: - Checkstyle v10.12(JRE 11 and above) @@ -143,9 +146,9 @@ checkstyle - Checkstyle v6.6(JRE 6 and 7) - Specify the Java files you'd like Checkstyle to check under the **ADD CASE:** option. +Specify the Java files you'd like Checkstyle to check under the **ADD CASE:** option. - The student will need to follow all style conventions to earn full credit on the assessment. +The student will need to follow all style conventions to earn full credit on the assessment. .. _checkstyle (website link): https://checkstyle.sourceforge.io/ .. _Google configuration: https://github.com/checkstyle/checkstyle/blob/2954d8723003ef229f5825510a433ab8c60f2774/src/main/resources/google_checks.xml @@ -156,7 +159,7 @@ checkstyle pycodestyle ---------------------- - If you want to use pycodestyle, you must first install it. Use the following commands to install pycodestyle: +If you want to use pycodestyle, you must first install it. Use the following commands to install pycodestyle: .. code:: ini @@ -173,21 +176,29 @@ To add individual Python source files whose style should be checked, either ente UnitTest ---------------------- - `UnitTest (website link)`_ is a python testing framework. - - When using python UnitTest in Codio, specify the python files containing UnitTest tests you'd like to run under the **ADD CASE:** option. - - Specify whether you are running python 2 (`python`) or python 3 (`python3`) under **PYTHON EXECUTABLE**. - - If you have more then one test, by default, the student will need to pass all tests to earn the specified number of points. You can toggle on **ALLOW PARTIAL POINTS** to have Codio evenly weight each test. - - There are 2 *optional* configurations for more complex file structures: - - - **PYTHON WORKING DIRECTORY** - specifies where in the file tree the actual test will run - - **STUDENT FOLDER** - specifies where the student code being tested is - +`UnitTest (website link)`_ is a python testing framework. + + +**Configuration** + +1. Under **ADD CASE**, specify the Python files containing your UnitTest tests. + +2. Under **PYTHON EXECUTABLE**, specify whether you are running Python 2 (``python``) or Python 3 (``python3``). + +3. **Scoring**: By default, students must pass all tests to earn the specified points. Toggle on **ALLOW PARTIAL POINTS** to evenly weight each test. + +**Optional Configurations** + +For more complex file structures, you can configure: + +- **PYTHON WORKING DIRECTORY** - Specifies where in the file tree the test will run +- **STUDENT FOLDER** - Specifies the location of the student code being tested + +**Working Example:** Advanced Features in Python `Starter Pack `_ + .. _UnitTest (website link): https://docs.python.org/3/library/unittest.html + ---------------------- JSHint and JSLint ---------------------- @@ -206,27 +217,14 @@ Custom If you choose **Custom**, enter the following information: - .. image:: /img/guides/assessment_act_exec_custom.png - :alt: Custom - - - **Command** - Enter the command that executes the student code. +.. image:: /img/guides/assessment_act_exec_custom.png + :alt: Custom + :width: 500px - .. Note:: If you store the assessment scripts in the **.guides/secure** folder, they run securely and students cannot see the script or the files in the folder. +- **Command** - Enter the command that executes the student code. - The files can be dragged and dropped from the File Tree into the field to automatically populate the necessary execution and run code. +The files can be dragged and dropped from the File Tree into the field to automatically populate the necessary execution and run code. - - **Timeout** - Enter the time period (in seconds) that the test runs before terminating. - - - **Allow Partial Points** - Toggle to enable partial points, the grade is then based on the percentage of test cases the code passes. See :ref:`Allow Partial Points ` for more information. - - -See a Working Example ----------------------- -To see an example of a specific unit test or style checker, see the Starter Pack in the corresponding language: - -Go to **Starter Packs** and search for **Advanced Features in Python** if not already loaded in your **My Projects** area. Click **Use Pack** and then **Create** to install it to your Codio account. - -Information about C++ unit testing using GoogleTest is available in the **C++ Unit Testing Using GoogleTest** Starter Pack. +- **Timeout** - Enter the time period (in seconds) that the test runs before terminating. -Additionally, Codio pre-populates a project in **My Projects** called **Demo Guides and Assessments** that contains examples for all assessment types and a guides authoring cheat sheet. If you do not see this project, go to **Starter Packs** and search for **Demo Guides and Assessments**. Click **Use Pack** and then **Create** to make a copy in your **My Projects** area. diff --git a/source/instructors/authoring/assessments/ai-assessment-generation.rst b/source/instructors/authoring/assessments/ai-assessment-generation.rst index 82d4f243..82046fdf 100644 --- a/source/instructors/authoring/assessments/ai-assessment-generation.rst +++ b/source/instructors/authoring/assessments/ai-assessment-generation.rst @@ -21,8 +21,37 @@ Important points to consider when auto-generating assessments: This is a new feature, and if it does not work properly for you, please let us know via the support chat or by emailing help@codio.com. If you would like us to take a look, send us your course name, the module, the assignment, and the page. +.. note:: Not all assessments can be AI-generated. If you don't see a "Generate" button in the bottom right corner when you click on an assessment, that assessment type does not support AI generation. + + +How to Generate an Assessment with AI +------------------------------------- + +Assessments can be auto-generated using the text on the current guides page as context. Follow the steps below to auto-generate an assessment: + +1. Select an assessment from the Assessments list. + +2. Press the **Generate** button at bottom right corner. + + .. image:: /img/guides/generate-assessment-button.png + :alt: Generate assessment button + +3. The Generation Prompt will open, press **Generate Using AI** to preview the generated assessment. + + .. image:: /img/guides/assessment-generation-prompt.png + :alt: Assessment Generation Prompt + + +4. When you are satisfied with the result, press **Apply** and then **Create**. + + +.. important:: The generate assessment feature does not configure the page :ref:`layout; ` you should specify the layout depending on how you want to present the information to the students. + + + + Example Prompts ---------------- +---------------- **Multiple choice** - Create a question about how to refactor the code below with multiple correct answers: (include code) diff --git a/source/instructors/authoring/assessments/assessment-security.rst b/source/instructors/authoring/assessments/assessment-security.rst index a56e0868..a3b29f83 100644 --- a/source/instructors/authoring/assessments/assessment-security.rst +++ b/source/instructors/authoring/assessments/assessment-security.rst @@ -4,33 +4,115 @@ .. _assessment-security: Assessment Security -=================== +==================== -Data for assessments is stored in individual files in the ``.guides/assessments`` folder. -Other folders automatically created in the ``.guides`` directory are: ``.guides/img`` and ``.guides/content``. -Student assignments only receive the ``.guides`` folder and the ``.guides/img`` folder. If you have not provided student access to the file tree in the :ref:`page layout `, the ``.guides`` folder will not be visible. +File Organization +----------------- + +Assessment data is stored in individual files within the ``.guides/assessments`` folder. The ``.guides`` directory automatically contains the following subdirectories: + +- ``.guides/assessments`` - Assessment configuration files +- ``.guides/img`` - Image assets +- ``.guides/content`` - Content files + +Student Workspace Limitations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When students receive an assignment, they only get access to: + +- The ``.guides`` folder +- The ``.guides/img`` folder + +If you have not enabled student access to the file tree in the :ref:`page layout `, the ``.guides`` folder will not be visible to them. + + + +Using the .guides Folder +------------------------ -The ``.guides`` folder is a good place to store bash files or data files because they can't be easily deleted by the student via the file tree. Keep in mind though, if students have access to the terminal they can use it to view any files in their Linux container, including the ``.guides`` folder. +Benefits and Considerations +~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Files that are stored in the ``.guides`` folder can be updated even if students have already started an assignment. -When you publish an assignment that students have already started, pre-existing files in the student workspace ``\home\codio\workspace`` will not be updated in order to make sure student work is not overwritten. -More information on this topic may be found on the :ref:`modify assignments ` page. +The ``.guides`` folder provides a secure location for storing bash scripts and data files, as students cannot easily delete them through the file tree interface. However, be aware that students with terminal access can view any files in their Linux container, including contents of the ``.guides`` folder. -Blocking student access to the terminal +Updating Published Assignments +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Files stored in the ``.guides`` folder can be updated even after students have started an assignment. When you republish an assignment that students have already begun: + +- Files in ``.guides`` **will be updated**. +- Pre-existing files in the student workspace (``/home/codio/workspace``) **will not be updated** to prevent overwriting student work. + +For more information, see the :ref:`modify assignments ` page. + + + +Blocking Student Access to the Terminal --------------------------------------- -The :ref:`terminal ` can be accessed from the file tree, the menu or by using the key combination (Shift+Alt+T on a PC, Shift+Option+T on a Mac). If you want to block all student access to the terminal you need to hide the file tree in the page layout, :ref:`customize the IDE menu` and block the keyboard short cut in :ref:`project preferences`. -The Secure folder +The :ref:`terminal ` can be accessed through multiple methods: + +- From the file tree +- From the menu +- Via keyboard shortcuts (Shift+Alt+T on PC, Shift+Option+T on Mac) + +To completely block student access to the terminal, you must: + +1. Hide the file tree in the page layout +2. :ref:`Customize the IDE menu ` to remove terminal access +3. Block the keyboard shortcut in :ref:`project preferences ` + + + +The Secure Folder ----------------- -If you are creating your own testing scripts or unit tests you can create a folder named ``.guides\secure`` to safely store test files. The secure directory is not copied over to the student version of the assignment. -When a student runs a code test that references a file in the ``.guides\secure`` folder, an **ephemeral** container is created that is a combination of the student workspace and the ``.guides\secure`` folder the instructor created. -This container is referred to as **ephemeral** because it only exists during the execution of the assessment. If you open a student project to view their work, you will not have access to the secure folder because it does not exist in the student environment. +Overview +~~~~~~~~ + +The ``.guides/secure`` folder provides a protected location for storing testing scripts and unit tests. This folder offers enhanced security because it is **not copied** to the student version of the assignment. + +How It Works +~~~~~~~~~~~~ + +When a student runs a code test that references files in ``.guides/secure``, an **ephemeral container** is created that combines: + +- The student workspace +- The instructor's ``.guides/secure`` folder + +This container is called "ephemeral" because it only exists during assessment execution. If you open a student project to review their work, you will not see the secure folder—it does not exist in the student environment. + +Important Limitations +~~~~~~~~~~~~~~~~~~~~~ + + +**Do not write to the secure folder** during test execution, as changes will not persist. All output must be conveyed through: + +- Standard output (stdout) +- The feedback buffer (described in individual assessment documentation) + +Assessment Types Supporting Secure Folder +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following assessment types can access the secure folder: + +- :ref:`Advanced Code Test ` +- :ref:`Free Text Autograde ` +- :ref:`Assignment Level Scripts ` + +Using Secure Folder with Standard Code Tests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:ref:`Standard Code Tests ` do not have access to the secure folder by default. However, you can force a Standard Code Test to run in an **ephemeral container** by including a file path with ``.guides/secure`` in the **Command** field on the **Execution** tab. + +Implementation Methods +~~~~~~~~~~~~~~~~~~~~~~ + +You can trigger ephemeral container creation by: -Your test should not write any information to the ``.guides\secure`` folder because it does not persist. -All output should conveyed using either stdout or the feedback buffer that is described in the individual assessesment types that can use the secure folder, :ref:`Advanced Code Test `, :ref:`Free Text Autograde ` and :ref:`Assignment Level Scripts `. +- Passing a secure folder file name as a parameter +- Including the file name on the command line (it will be ignored by your program, but signals Codio to mount the secure folder) -Standard code tests do not have access to the secure folder by default but you *can* force a :ref:`Standard Code Test ` to run in an **ephemeral** container by including a file name with the ``.guides\secure`` path in the **Command** portion on the **Execution** tab. -This could be done if you are passing a file name as a parameter or if you have no command line parameters, you can put the file name on the command line and it will be ignored by your program but Codio will run the test in a container with the Secure folder mounted. Then you can specify a file in ``.guides\secure`` as an input parameter. \ No newline at end of file +Once the ephemeral container is created, you can specify files in ``.guides/secure`` as input parameters for your tests. \ No newline at end of file diff --git a/source/instructors/authoring/assessments/assessments.rst b/source/instructors/authoring/assessments/assessments.rst index d59831e7..dff80c48 100644 --- a/source/instructors/authoring/assessments/assessments.rst +++ b/source/instructors/authoring/assessments/assessments.rst @@ -3,17 +3,55 @@ .. _assessments: + Assessments =========== -Assessments can be used to determine how well students understand course material, and can be automatically or manually graded. Codio offers a wide range of assessment types, including auto-graded code tests, multiple choice tests, fill in the blanks, drop-down selection, free text responses and assignment grading. Assessments can be interspersed throughout tutorial materials or stand alone using an :ref:`assignment level script `. + +Assessments can be used to determine how well students understand course material, and can be automatically or manually graded. Codio offers a wide range of assessment types, including auto-graded code tests, multiple choice tests, fill in the blanks, drop-down selection, free text responses, and assignment grading. Assessments can be interspersed throughout tutorial materials or stand alone using an :ref:`assignment level script `. You can view the results of assessments in a course from the teacher dashboard. -Review the following topics to gain an understanding of using assessments: +**Assessment Topics** - :ref:`Ungraded Assessments ` - :ref:`Add a New Assessment ` - :ref:`Edit an Assessment ` - :ref:`Edit Points for an Assessment ` - :ref:`Delete an Assessment ` -- :ref:`Assessment Security ` \ No newline at end of file +- :ref:`Assessment Security ` + +Common Assessment Configuration +-------------------------------- + +The following configuration options are common to all assessment types: + +General Settings +~~~~~~~~~~~~~~~~ + +.. image:: /img/guides/assessment_general.png + :alt: General + +- **Name** - Enter a short name that describes the test. This name is displayed in the teacher dashboard, so it should clearly reflect the challenge. Toggle the **Show Name** setting to hide the name in the challenge text the student sees. + +- **Instructions** - Enter text that is shown to the student using optional Markdown formatting. + +Metadata (Optional) +~~~~~~~~~~~~~~~~~~~ + +.. image:: /img/guides/assessment_metadata.png + :alt: Metadata + +- **Bloom's Level** - Select the appropriate level of `Bloom's Taxonomy `_ for the current assessment. + +- **Learning Objectives** - Specify the educational goals of the assessment. Objectives typically begin with "Students Will Be Able To (SWBAT)." For example: *SWBAT follow the flow of recursive execution*. + +- **Tags** - The **Content** and **Programming Language** tags are provided and required. To add custom tags, click **Add Tag** and enter the name and values. + +Files (Optional) +~~~~~~~~~~~~~~~~ + +.. image:: /img/guides/assessment_files.png + :alt: Files + +Check the boxes for any additional external files to include with the assessment when adding it to an assessment library. Selected files appear in the **Additional content** list. + diff --git a/source/instructors/authoring/assessments/autograde-free-text.rst b/source/instructors/authoring/assessments/autograde-free-text.rst index 9adf8de6..378c6abc 100644 --- a/source/instructors/authoring/assessments/autograde-free-text.rst +++ b/source/instructors/authoring/assessments/autograde-free-text.rst @@ -7,72 +7,63 @@ Free Text Autograde =================== The Free Text Autograde assessment is similar to the :ref:`Free Text ` assessment in that it allows students to answer questions in their own words, but includes a field for a command line to execute a script that allows autograding. The answer to the question is stored in the environment variable ``CODIO_FREE_TEXT_ANSWER``. -Follow these steps to set up an autograde free text assessment: +Follow these steps to set up an autograde free text assessment: -1. On the **General** page, enter the following information: +.. note:: For more information on **General**, **Metadata** and **Files** see :ref:`Assessments `. - .. image:: /img/guides/assessment_free_general.png - :alt: General - - - **Name** - Enter a short name that describes the test. This name is displayed in the teacher dashboard so the name should reflect the challenge and thereby be clear when reviewing. - - Toggle the **Show Name** setting to hide the name in the challenge text the student sees. - - - **Instruction** - Enter text that is shown to the student using optional Markdown formatting. +1. Complete **General**. 2. Click **Execution** in the left navigation pane and complete the following fields: .. image:: /img/guides/assessment_autofree_exec.png :alt: Execution - .. Note:: If you store the assessment scripts in the **.guides/secure** folder, they run securely and students cannot see the script or the files in the folder. - The files can be dragged and dropped from the File Tree into the field to automatically populate the necessary execution and run code. +.. Note:: If you store the assessment scripts in the **.guides/secure** folder, they run securely and students cannot see the script or the files in the folder. The files can be dragged and dropped from the File Tree into the field to automatically populate the necessary execution and run code. For more information see :ref:`Assessment Security `. - - **Command** - Enter the command that executes the student code. +- **Command** - Enter the command that executes the student code. - - **Timeout** - Enter the time period (in seconds) that the test runs before terminating. +- **Timeout** - Enter the time period (in seconds) that the test runs before terminating. 3. Click **Grading** in the navigation pane and complete the following fields: .. image:: /img/guides/assessment_free_grading.png :alt: Grading - - **Points** - The score given to the student if the code test passes. You can enter any positive numeric value. If this assessment should not be graded, enter 0 points. +- **Points** - The score given to the student if the code test passes. You can enter any positive numeric value. If this assessment should not be graded, enter 0 points. - - **Allow Partial Points** - Toggle to enable a percentage of total points to be given based on the percentage of answers they correctly answer. +- **Partial Points** - Toggle to enable a percentage of total points to be given based on the percentage of answers they correctly answer. - - **Preview Type** - Choose the input (plaintext or markdown) to be provided by the student. LaTex is also supported and is useful when students need to enter mathematical formulas in their answers. The following options are available: +- **Preview Type** - Choose the input (plaintext or markdown) to be provided by the student. LaTex is also supported and is useful when students need to enter mathematical formulas in their answers. The following options are available: - - **Plaintext** - Students enter ordinary text with no markdown formatting; there is no preview window. - - **Plaintext + LaTeX** - Students enter plaintext with no markdown formatting but offers support for LaTeX commands. A preview window is available where students can see the rendered LaTeX output. - - **Markdown + LaTeX** - Students enter markdown that also offers support for LaTex commands. A preview window is available where students can see the rendered markdown with LaTeX output. - - **Define Number of Attempts** - enable to allow and set the number of attempts students can make for this assessment. If disabled, the student can make unlimited attempts. - - **Show Rationale to Students** - Toggle to display the rationale for the answer, to the student. This guidance information will be shown to students after they have submitted their answer and any time they view the assignment after marking it as completed. You can set when to show this selecting from **Never**, **After x attempts**, **If score is greater than or equal to a % of the total** or **Always** - - **Rationale** - Enter guidance for the assessment. This is always visible to the teacher when the project is opened in the course or when opening the student's project. - - **Use maximum score** - Toggle to enable assessment final score to be the Highest score attained of all runs. ++---------------------------+-------------------------------------------------------------------------+ +| **Plaintext** | Students enter ordinary text with no markdown formatting; there is no | +| | preview window. | ++---------------------------+-------------------------------------------------------------------------+ +| **Plaintext + LaTeX** | Students enter plaintext with no markdown formatting but offers support | +| | for LaTeX commands. A preview window is available where students can | +| | see the rendered LaTeX output. | ++---------------------------+-------------------------------------------------------------------------+ +| **Markdown + LaTeX** | Students enter markdown that also offers support for LaTeX commands. A | +| | preview window is available where students can see the rendered | +| | markdown with LaTeX output. | ++---------------------------+-------------------------------------------------------------------------+ -4. Click on the **Parameters** tab if you wish to edit/change **Parameterized Assessments** (deprecated) using a script. See :ref:`Parameterized Assessments ` for more information. New parameterized assessments can no longer be set up. -5. Click **Metadata** in the left navigation pane and complete the following fields: +- **Define Number of Attempts** - enable to allow and set the number of attempts students can make for this assessment. If disabled, the student can make unlimited attempts. +- **Show Rationale to Students** - Toggle to display the rationale for the answer, to the student. This guidance information will be shown to students after they have submitted their answer and any time they view the assignment after marking it as completed. You can set when to show this selecting from **Never**, **After x attempts**, **If score is greater than or equal to a % of the total** or **Always** +- **Rationale** - Enter guidance for the assessment. This is always visible to the teacher when the project is opened in the course or when opening the student's project. +- **Use maximum score** - Toggle to enable assessment final score to be the Highest score attained of all runs. - .. image:: /img/guides/assessment_metadata.png - :alt: Metadata +4. **(Optional)** Complete **Metadata**. - - **Bloom's Level** - Click the drop-down and choose the level of Bloom's Taxonomy: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ for the current assessment. - - **Learning Objectives** The objectives are the specific educational goal of the current assessment. Typically, objectives begin with Students Will Be Able To (SWBAT). For example, if an assessment asks the student to predict the output of a recursive code segment, then the Learning Objectives could be *SWBAT follow the flow of recursive execution*. - - **Tags** - By default, **Content** and **Programming Language** tags are provided and required. To add another tag, click **Add Tag** and enter the name and values. - -5. Click **Files** in the left navigation pane and check the check boxes for additional external files to be included with the assessment when adding it to an assessment library. The files are then included in the **Additional content** list. - - .. image:: /img/guides/assessment_files.png - :alt: Files +5. **(Optional)** Complete **Files**. 6. Click **Create** to complete the process. -Example scripts for free-text auto-grade with all or nothing scoring -.................................................................... +Example Scripts for Free-Text Auto-Grade with All or Nothing Scoring +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. tabs:: @@ -137,8 +128,8 @@ Example scripts for free-text auto-grade with all or nothing scoring exit(1) -Example scripts for free-text auto-grade with partial points -............................................................ +Example Scripts for Free-Text Auto-Grade with Partial Points +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. tabs:: @@ -216,8 +207,9 @@ Example scripts for free-text auto-grade with partial points -Automatically grade a Free Text assessment correct -..................................................... +Automatically Grade a Free Text Assessment as Correct +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + This technique can be used to automatically mark the assessment correct for students who have submitted anything in the response. In the **Command** field on the **Execution** tab enter the command below: diff --git a/source/instructors/authoring/assessments/delete-assessment.rst b/source/instructors/authoring/assessments/delete-assessment.rst index 328201e5..a8b89199 100644 --- a/source/instructors/authoring/assessments/delete-assessment.rst +++ b/source/instructors/authoring/assessments/delete-assessment.rst @@ -24,7 +24,7 @@ To delete an assessment, follow these steps: 6. You can also use the **Filter By**, select **Not Used** and delete all unused assessments together. You can search for assessments by name,points or order in guides and order them up or down using the arrow buttons -.. Note:: You can restore a deleted assessment, if it is still on the Assessments tab, by clicking on the name and copying the unique identification (see image below). For this example, the line in your guide will be: ``{Check It!|assessment}(multiple-choice-1735957228)``. +.. Note:: You can restore a deleted assessment, if it is still on the Assessments tab, by clicking on the name and copying the unique identification (see image below). For this example, the line in your guide will be: ``{Check It!|assessment}(multiple-choice-3865028352)``. .. image:: /img/guides/assessment_undelete.png :alt: Assessment Undelete \ No newline at end of file