Skip to content
Snippets Groups Projects
Commit 96b2b41b authored by tuhe's avatar tuhe
Browse files

Moss integration

parent d99a8e59
No related branches found
No related tags found
No related merge requests found
Showing
with 28 additions and 14 deletions
# Unitgrade-private
# Unitgrade-devel
**Note: This is the development version of unitgrade. If you are a student, please see http://gitlab.compute.dtu.dk/tuhe/unitgrade.**
Unitgrade is an automatic report and exam evaluation framework that enables instructors to offer automatically evaluated programming assignments.
Unitgrade is build on pythons `unittest` framework so that the tests can be specified in a familiar syntax and will integrate with any modern IDE. What it offers beyond `unittest` is the ability to collect tests in reports (for automatic evaluation) and an easy and 100% safe mechanism for verifying the students results and creating additional, hidden tests. A powerful cache system allows instructors to automatically create test-answers based on a working solution.
- 100% Python `unittest` compatible
- No external configuration files: Just write a `unittest`
- No unnatural limitations: Use any package or framework. If you can `unittest` it, it works.
- Tests are quick to run and will integrate with your IDE
- Cache and hint-system makes tests easy to develop
- Granular security model:
- Students get public `unittests` for easy development of solutions
- Students get a tamper-resistant file to create submissions which are uploaded
- Students use a tamper-resistant file to create submissions which are uploaded
- Instructors can automatically verify the students solution using a Docker VM and run hidden tests
- Tests are quick to run and will integrate with your IDE
- Moss anti-plagiarism integration
**Note: This is the development version of unitgrade. If you are a student, please see http://gitlab.compute.dtu.dk/tuhe/unitgrade.**
- Automatic Moss anti-plagiarism detection
- CMU Autolab integration (Experimental)
# Using unitgrade
The examples can be found in the `/examples/` directory: https://gitlab.compute.dtu.dk/tuhe/unitgrade_private/examples
......@@ -26,9 +28,9 @@ Although not required, it is recommended you maintain two version of the code:
- A fully-working version (i.e. all tests pass)
- A public version distributed to students (some code removed))
In this example, I will use `snipper` (see http://gitlab.compute.dtu.dk/tuhe/snipper) to synchronize the two versions automatically.
Let's look at an example. You need three files
```
I use `codesnipper` (see http://gitlab.compute.dtu.dk/tuhe/snipper) to synchronize the two versions automatically.
Let's look at an example. Suppose our course is called `cs101`, in which case we make three files in our private folder `instructor`:
```terminal
instructor/cs101/homework.py # This contains the students homework
instructor/cs101/report1.py # This contains the tests
instructor/cs101/deploy.py # A private file to deploy the tests
......@@ -124,7 +126,7 @@ if __name__ == "__main__":
- The second line set up the students directory (remember, we have included the solutions!) and remove the students solutions. You can check the results in the students folder.
### Using the framework as a student
You can now upload the `student' directory to the students. The students can run their tests either by running `cs101.report1` in their IDE or by typing:
You can now upload the `student` directory to the students. The students can run their tests either by running `cs101.report1` in their IDE or by typing:
```
python -m cs101.report1
```
......@@ -133,13 +135,12 @@ in the command line. This produces a detailed output of the test and the program
python -m cs101.report1_grade
```
This runs an identical set of tests, but produces a `.token` file the students can upload to get credit.
- The reason to have a seperate `report1_grade.py` script is to avoid accidential removal of tests.
- The `report1_grade.py` includes all tests and the main parts of the framework and is obfuscated by default. You can apply a much strong level of protection by using e.g. `pyarmor`.
- The `report1_token.token` file includes the outcome of the tests, the time taken, and all python source code in the package. In other words, the file can be used for manual grading, for plagirism detection and for detecting tampering.
- You can easily use the framework to include output of functions.
- See below for how to validate the students results
### How safe is this?
### How safe is Unitgrade?
Cheating within the framework is probably best done by manually editing the `.token`-file or by creating a broken set of tests. This involves risk of being trivially detected, for instance because tests have the wrong runtime, but more importantly
the framework automatically pack all the used source code and so if a student is cheating, there is no way to hide it for an instructor who looks at the results. If the
program is used in conjunction with automatic plagiarism software, cheating therefore involves both breaking the framework, and creating 'false' solutions which statistically match other students solutions, and then hope nobody bothers to check the output.
......@@ -297,13 +298,26 @@ These steps compile a Docker image (you can easily add whatever packages you nee
The last lines load the result and compare the score -- in this case both will return 0 points, and any dissimilarity in the results should be immediate cause for concern.
- Docker prevents students from doing mailicious things to your computer and allows the results to be reproducible by TAs.
# Moss plagiarism detection
You can easily apply Moss to the students token files.
```python
from unitgrade_private2.plagiarism.mossit import moss_it, get_id
if __name__ == "__main__":
# Extract the moss id from the perl script:
id = get_id("../../../02465private/admin/moss.pl")
# moss_id should be a string containing an integer, i.e. "2434222134".
moss_it(whitelist_dir="whitelist", submissions_dir="student_submissions", moss_id=id)
```
# Citing
```bibtex
@online{unitgrade_devel,
title={Unitgrade-devel (0.0.1): \texttt{pip install unitgrade-devel}},
title={Unitgrade-devel (0.1.0): \texttt{pip install unitgrade-devel}},
url={https://lab.compute.dtu.dk/tuhe/unitgrade_private},
urldate = {2021-09-07},
urldate = {2021-09-08},
month={9},
publisher={Technical University of Denmark (DTU)},
author={Tue Herlau},
......
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
File deleted
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment