README.md 10.4 KB
Newer Older
Christian Rohlfing's avatar
Christian Rohlfing committed
1
2
# exam-scan

Deb's avatar
Deb committed
3
4
Exam-Scan is a command-line tool that helps to watermark and/or encrypt feedback files/scanned exams/additional exam materials and prepare them in Moodle uploadable format.
The tool is designed to handle zipped submissions downloadable from Moodle as well as scanned PDFs from your local scanner.
5

Amrita Deb's avatar
Amrita Deb committed
6
## Contents
7

Deb's avatar
Deb committed
8
9
10
* `handlemoodlesubmissions.py` (*for downloadable PDFs*) unzips files from submission zip file downloadable from Moodle(in case there are no scans) and renames it accordingly in the format *<Matriculation number>_<Lastname/first letter of Lastname>*
* `renamescans.py` (*for scanned PDFs*) Rename scanned PDFs, assuming scan order equal to alphabetical order of students in Moodle grading sheet, such that the file name is in the format in the format *<Matriculation number>_<first letter of Lastname>*. This only works if exams were scanned in alphabetical order. Optionally, each scanned PDF is searched for barcodes/QRs containing the matriculation number to double check.
* `supplements.py` renames and create copies of sample solutions(if any)/additional exam materials for every student.
11
* `watermark.py` watermarks each page of PDFs containing exam scans with matriculation number of the respective student
Deb's avatar
Deb committed
12
13
* `encrypt.py` encrypts PDFs either with a common password(passed as an argument) or a randomly generated password(when there is no argument)
* `preparemoodleupload.py` zips PDFs in the format acceptable for moodle upload via assign module as feedback file for each student
Amrita Deb's avatar
Amrita Deb committed
14
* `batch.py` executes all three programs as a singular batch job
15

Deb's avatar
Deb committed
16
17
Please note that `handlemoodlesubmissions.py`, `supplements.py`, `watermark.py`, `encrypt.py`, `preparemoodleupload.py` do not depend on each other.
If you want to use only a subset (or one) of the scripts, you can do so after installing the corresponding script's [dependancies](Dependancies.md).
Christian Rohlfing's avatar
Christian Rohlfing committed
18

19
Exemplary outputs can be downloaded:
20

Christian Rohlfing's avatar
Christian Rohlfing committed
21
22
* [moodle_feedbacks.zip](https://git.rwth-aachen.de/rwthmoodle/exam-scan/-/jobs/artifacts/master/raw/out/moodle_feedbacks.zip?job=test): The zip-Archive to be uploaded to Moodle containing the watermarked and encrypted PDFs for each student.
* [passwords.csv](https://git.rwth-aachen.de/rwthmoodle/exam-scan/-/jobs/artifacts/master/raw/out/passwords.csv?job=test): CSV file containing passwords for each PDF.
23

Deb's avatar
Deb committed
24
25
For more info please refer to the [Documentation](https://rwthmoodle.pages.rwth-aachen.de/exam-scan/)

Amrita Deb's avatar
Amrita Deb committed
26
## Instructions
27

28
29
### Prerequisites

Amrita Deb's avatar
Amrita Deb committed
30
31
32
33
* **Create and setup Moodle**
  * In your Moodle course room, create an `assign` module following this [guideline](https://help.itc.rwth-aachen.de/service/8d9eb2f36eea4fcaa9abd0e1ca008b22/article/0cfca4212fef4712ad2d432ac83eaf3e)
  * Download the grading table `Bewertungen.csv` from Moodle via: `Alle Angaben anzeigen` &#8594; `Bewertungsvorgang` &#8594; `Bewertungstabelle  herunterladen`

Amrita Deb's avatar
Amrita Deb committed
34
* **Create PDFs corresponding to each exam**
35
  * Scan the exams and save the scans as PDFs (each page should be A4). For most copy machines, you can save an A3 scan (double page of an exam) as two A4 pages.
Deb's avatar
Deb committed
36
37
38
  * You can either
      1. Order the scans following the same order as the grading worksheet
      1. Or rename the scans in the format `<Matriculation number>_<Lastname/first letter of Lastname>` (e.g. `123456_Nachname.pdf`/`123456_L.pdf`).
39

Amrita Deb's avatar
Amrita Deb committed
40
41
42
* **OR: Download submission zip file**
  * Download the submission zip file from (Assignment Main Page->View all submissions->Download all submissions)

Deb's avatar
Deb committed
43
44
* **Optional: Create ample solutions/additional exam materials (Refer [here](https://git.rwth-aachen.de/rwthmoodle/exam-scan/-/issues/3))**
  * Scan the sample solutions/additional exam materials and save the scans as PDFs (each page should be A4). For most copy machines, you can save an A3 scan (double page of an exam) as two A4 pages.
Amrita Deb's avatar
Amrita Deb committed
45
  * Place all PDFs in a folder, e.g. `supplements`.
46

Amrita Deb's avatar
Amrita Deb committed
47
* **Install the software dependancies**
48

Deb's avatar
Deb committed
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
  The current version of code was tested on Windows10, Ubuntu 20.04.1 LTS and macOS 10.14 Mojave to ensure platform independence.The code has the following software dependencies which needs to installed before the programs can be run successfully. 
  * Imagemagick (version: 7.0.10-33)
  * Ghostscript (version: 9.53.3)
  * Python (version: 3.8/3.9)
  * PIP (version: 21.0.1)
  * Additional Python modules:
    * wand (version 0.6.5)
    * pillow (version: 8.1.0)
    * PyPDF2 (version 1.26.0)
    * pwgen (version: 0.8.2.post0)
    * pikepdf (version 2.5.0)
    * zip (version 0.02)

  Instructions to install software dependencies based on your operating system:
  * Windows 10 : [Installation of Software Dependencies](swdependencies_win.md)
  * MacOS : [Installation of Software Dependencies](swdependencies_mac.md)
  * Linux : [Installation of Software Dependencies](swdependencies_linux.md)

 If you are an experienced user and familiar with the python `venv` (virtual environments) module and after having installed both ImageMagick (beware the `Policy Error` fix in [FAQs])and Ghostscript you can install the python dependencies in the virtual environment via pip with
68
69
70
71
72
73
74
75
76
77
78

  ```bash
  python -m venv venv
  source venv/bin/activate
  pip install -r requirements.txt 
  python ./watermark.sh --help
  ```

### Docker

If you are an experienced user familiar with Docker, you can use the provided `Dockerfile` to easily run the scripts.
Christian Rohlfing's avatar
Christian Rohlfing committed
79
80
81
82
83
84
85
Either use the already built image

```bash
docker run --name='examscan' --rm -v $(pwd):$(pwd) -w $(pwd) registry.git.rwth-aachen.de/rwthmoodle/exam-scan:master batch.py --help
```

or build the Dockerfile yourself locally
86
87
88

```bash
docker build -t examscan:latest .
Christian Rohlfing's avatar
Christian Rohlfing committed
89
docker run --name examscan --rm -v $(pwd):$(pwd) -w $(pwd) examscan:latest batch.py --help
90
```
Amrita Deb's avatar
Amrita Deb committed
91

Deb's avatar
Deb committed
92
## Commands
93
94

#### Unzip submission files from and rename them
Amrita Deb's avatar
Amrita Deb committed
95

Deb's avatar
Deb committed
96
Assuming that `./tests/assets/submissions.zip` is the zip file containing all submissions and `./tests/assets/Grades.csv` is the grading worksheet
Amrita Deb's avatar
Amrita Deb committed
97

98
```bash
Deb's avatar
Deb committed
99
python handlemoodlesubmissions.py ./tests/assets/submissions.zip  ./tests/assets/Grades.csv ./tests/assets/pdfs 
Amrita Deb's avatar
Amrita Deb committed
100
101
```

Deb's avatar
Deb committed
102
For more info on the scripts and additional arguments refer to the [Documentation](https://https://rwthmoodle.pages.rwth-aachen.de/exam-scan/handlemoodlesubmissions.html)
Deb's avatar
Deb committed
103

Deb's avatar
Deb committed
104
#### Rename scanned PDFs
Deb's avatar
Deb committed
105

Deb's avatar
Deb committed
106
Assuming that `./tests/assets/pdfs_scan` is the folder containing all scans, `./tests/assets/Grades.csv` is the grading worksheet and `./tests/assets/pdfs` is the output folder where all the renamed scans will be placed
Deb's avatar
Deb committed
107
108
109
110

```bash
python renamescans.py  ./tests/assets/pdfs_scan ./tests/assets/Grades.csv ./tests/assets/pdfs
```
Deb's avatar
Deb committed
111
For more info on the scripts and additional arguments refer to the [Documentation](https://https://rwthmoodle.pages.rwth-aachen.de/exam-scan/renamescans.html)
Deb's avatar
Deb committed
112

113
#### Prepare copies of Sample Solutions for each student (Optional)
Amrita Deb's avatar
Amrita Deb committed
114

Deb's avatar
Deb committed
115
Assuming that the folder `./tests/assets/supplements` holds the scans of the sample solution/additional materials, `./tests/assets/Grades.csv` is the grading worksheet and `./tests/assets/pdfs` is the output folder where all the renamed materials will be placed
Amrita Deb's avatar
Amrita Deb committed
116

117
```bash
Deb's avatar
Deb committed
118
python supplements.py ./tests/assets/supplements ./tests/assets/Grades.csv ./tests/assets/pdfs
Amrita Deb's avatar
Amrita Deb committed
119
120
```

Deb's avatar
Deb committed
121
For more info on the scripts and additional arguments refer to the [Documentation](https://https://rwthmoodle.pages.rwth-aachen.de/exam-scan/supplements.html)
Amrita Deb's avatar
Amrita Deb committed
122

Amrita Deb's avatar
Amrita Deb committed
123
#### Watermark the submissions
Amrita Deb's avatar
Amrita Deb committed
124

Deb's avatar
Deb committed
125
Assuming that the folder `./tests/assets/pdfs` holds the scans of the exams and filenames follow the format *<Matriculation number>_<Lastname/first letter of Lastname>*, `./tests/assets/pdfs_watermarked` is the output folder where all the watermarked PDFs will be placed and cores=2 indicate the number of cores for parallel processing
126

127
```bash
Deb's avatar
Deb committed
128
python watermark.py ./tests/assets/pdfs ./tests/assets/pdfs_watermarked --cores 2 --dpi 150 --quality 75
129
130
```

Christian Rohlfing's avatar
Christian Rohlfing committed
131
**TIP:** Play around with `dpi` and `quality` parameters according to your requirements. Higher values for these two will result in high resolution PDFs of bigger size (ideal for when the number of files is low). Lower values will result in PDFs having lower file size and low resolution (ideal when the number of files is high)
Deb's avatar
Deb committed
132

Deb's avatar
Deb committed
133
For more info on the scripts and additional arguments refer to the [Documentation](https://https://rwthmoodle.pages.rwth-aachen.de/exam-scan/watermark.html)
Amrita Deb's avatar
Amrita Deb committed
134

Amrita Deb's avatar
Amrita Deb committed
135
#### Encrypt the files
Christian Rohlfing's avatar
Christian Rohlfing committed
136

Deb's avatar
Deb committed
137
Assuming that the folder `./tests/assets/pdfs_watermarked` holds the files to encrypt, `./tests/assets/pdfs_encrypted` is the output folder where all the encrypted PDFs will be placed and providing a common password for all encrypted PDFs with the `--password` option
138

139
```bash
Deb's avatar
Deb committed
140
python  encrypt.py ./tests/assets/pdfs_watermarked ./tests/assets/pdfs_encrypted --password ganzgeheim
141
```
Deb's avatar
Deb committed
142
**TIP:** Omitting the `password` option, you can set randomly generated passwords for each PDF
143

Deb's avatar
Deb committed
144
For more info on the scripts and additional arguments refer to the [Documentation](https://https://rwthmoodle.pages.rwth-aachen.de/exam-scan/encrypt.html)
Christian Rohlfing's avatar
Christian Rohlfing committed
145

Amrita Deb's avatar
Amrita Deb committed
146
#### Prepare for Moodle batch upload
Christian Rohlfing's avatar
Christian Rohlfing committed
147

Deb's avatar
Deb committed
148
Assuming that the folder `./tests/assets/pdfs_encrypted` holds the scans of be uploaded to Moodle, `./tests/assets/Grades.csv` is the grading worksheet and `./tests/assets/out` is the output folder where zip archive will be palced
Christian Rohlfing's avatar
Christian Rohlfing committed
149

150
```bash
Deb's avatar
Deb committed
151
python preparemoodleupload.py ./tests/assets/pdfs_encrypted  ./tests/assets/Grades.csv  ./tests/assets/out
152
153
```

154
Then, you can upload `moodle_feedbacks.zip` in Moodle:
155
`Alle Angaben anzeigen` &#8594; `Bewertungsvorgang` &#8594; `Mehrere Feedbackdateien in einer Zip-Datei hochladen`
Christian Rohlfing's avatar
Christian Rohlfing committed
156

Deb's avatar
Deb committed
157
For more info on the scripts and additional arguments refer to the [Documentation](https://https://rwthmoodle.pages.rwth-aachen.de/exam-scan/preparemoodleupload.html)
158

Amrita Deb's avatar
Amrita Deb committed
159

Amrita Deb's avatar
Amrita Deb committed
160
#### Batch job
Christian Rohlfing's avatar
Christian Rohlfing committed
161

Deb's avatar
Deb committed
162
You can do all the above processes with one single scripts as follows:
163

164
```bash
Deb's avatar
Deb committed
165
python batch.py ./tests/assets/pdfs ./tests/assets/Grades.csv ./tests/assets/out --cores 2 --password ganzgeheim --suppinfolder ./tests/assets/supplements --supp
166
```
Christian Rohlfing's avatar
Christian Rohlfing committed
167

Deb's avatar
Deb committed
168
In this case the `./tests/assets/out` contains both passwords.csv and zip archive
Deb's avatar
Deb committed
169
For more info on the scripts and additional arguments refer to the [Documentation](https://https://rwthmoodle.pages.rwth-aachen.de/exam-scan/batch.html)
Christian Rohlfing's avatar
Christian Rohlfing committed
170

171
## Original Authors
172

173
174
Helmut Flasche, Jens Schneider, Christian Rohlfing, IENT RWTH Aachen\
Dietmar Kahlen, ITHE RWTH Aachen\
Amrita Deb's avatar
Amrita Deb committed
175
Amrita Deb, IT Center, RWTH Aachen University
176

Christian Rohlfing's avatar
Christian Rohlfing committed
177
178
## Who do I talk to?

Amrita Deb's avatar
Amrita Deb committed
179
180
Servicedesk IT-Center RWTH Aachen <[servicedesk@itc.rwth-aachen.de](mailto:servicedesk@itc.rwth-aachen.de)>

Amrita Deb's avatar
Amrita Deb committed
181
182
If you have any errors or problems while running the programs, make sure to first check if your error is listed in [FAQs]

Amrita Deb's avatar
Amrita Deb committed
183
[FAQs]: https://git.rwth-aachen.de/rwthmoodle/exam-scan/-/blob/master/FAQs.md