README.md 9.31 KB
Newer Older
Christian Rohlfing's avatar
Christian Rohlfing committed
1
2
# exam-scan

3
Preparing exam scans for ship out: Adding watermarks, encryption and preparing upload to Moodle.
4

Amrita Deb's avatar
Amrita Deb committed
5
## Contents
6

7
* `preparepdf.py` unzips files from submission zip file downloadable from Moodle(in case there are no scans) and renames it accordingly in the format (`<Matriculation number>_<Lastname>`)
Amrita Deb's avatar
Amrita Deb committed
8
* `supplements.py` renames and create copies of sample solutions(if any) for every student
9
10
11
* `watermark.py` watermarks each page of PDFs containing exam scans with matriculation number of the respective student
* `encrypt.py` encrypts PDF with password either with a common password(passed as an argument) or a randomly generated password(when there is no argument)
* `preparemoodle.py` prepares for uploading PDFs to moodle via assign module as feedback file for each student
Amrita Deb's avatar
Amrita Deb committed
12
* `batch.py` executes all three programs as a singular batch job
13

14
Please note that the three scripts `watermark.py`, `encrypt.py`, and `preparemoodle.py`do not depend on each other.
Amrita Deb's avatar
Amrita Deb committed
15
If you want to use only a subset (or one) of the scripts, you can find it [here](Dependancies.md).
Christian Rohlfing's avatar
Christian Rohlfing committed
16

17
Exemplary outputs can be downloaded:
18

Christian Rohlfing's avatar
Christian Rohlfing committed
19
20
* [moodle_feedbacks.zip](https://git.rwth-aachen.de/rwthmoodle/exam-scan/-/jobs/artifacts/master/raw/out/moodle_feedbacks.zip?job=test): The zip-Archive to be uploaded to Moodle containing the watermarked and encrypted PDFs for each student.
* [passwords.csv](https://git.rwth-aachen.de/rwthmoodle/exam-scan/-/jobs/artifacts/master/raw/out/passwords.csv?job=test): CSV file containing passwords for each PDF.
21

Amrita Deb's avatar
Amrita Deb committed
22
## Instructions
23

24
25
### Prerequisites

Amrita Deb's avatar
Amrita Deb committed
26
27
28
29
* **Create and setup Moodle**
  * In your Moodle course room, create an `assign` module following this [guideline](https://help.itc.rwth-aachen.de/service/8d9eb2f36eea4fcaa9abd0e1ca008b22/article/0cfca4212fef4712ad2d432ac83eaf3e)
  * Download the grading table `Bewertungen.csv` from Moodle via: `Alle Angaben anzeigen` &#8594; `Bewertungsvorgang` &#8594; `Bewertungstabelle  herunterladen`

Amrita Deb's avatar
Amrita Deb committed
30
* **Create PDFs corresponding to each exam**
31
32
33
  * Scan the exams and save the scans as PDFs (each page should be A4). For most copy machines, you can save an A3 scan (double page of an exam) as two A4 pages.
  * The filename of each PDF should start with the student's matriculation number (e.g. `123456_Nachname.pdf`).
  * Place all PDFs in a folder, e.g. `pdfs`.
34
  **OR: Download submission zip file**
Amrita Deb's avatar
Amrita Deb committed
35
  * Download the submission zip file from (Assignment Main Page->View all submissions->Download all submissions)
36

Amrita Deb's avatar
Amrita Deb committed
37
38
39
40
* **OR: Download submission zip file**
  * Download the submission zip file from (Assignment Main Page->View all submissions->Download all submissions)

* **Optional: Create Sample Solutions (Refer [here](https://git.rwth-aachen.de/rwthmoodle/exam-scan/-/issues/3))**
Amrita Deb's avatar
Amrita Deb committed
41
42
  * Scan the sample solutions and save the scans as PDFs (each page should be A4). For most copy machines, you can save an A3 scan (double page of an exam) as two A4 pages.
  * Place all PDFs in a folder, e.g. `supplements`.
43

Amrita Deb's avatar
Amrita Deb committed
44
* **Install the software dependancies**
45

46
47
48
49
50
51
52
53
54
  If you are an experienced user and familiar with the python `venv` (virtual environments) module and after having installed both ImageMagick (beware the `Policy Error` fix in [FAQs])and Ghostscript you can install the python dependencies in the virtual environment via pip with

  ```bash
  python -m venv venv
  source venv/bin/activate
  pip install -r requirements.txt 
  python ./watermark.sh --help
  ```

Christian Rohlfing's avatar
Christian Rohlfing committed
55
  The current version of code was tested on Windows10, Ubuntu 20.04.1 LTS and macOS 10.14 Mojave to ensure platform independence.The code has the following software dependencies which needs to installed before the programs can be run successfully. Specific version numbers along with categorisation based on script requirement is available [here](Dependancies.md):
56
57
58
59
60
61
62
63
64
65
  * Imagemagick
  * Ghostscript
  * Python 3.8/3.9
  * PIP
  * Additional Python modules:
    * wand
    * pillow
    * PyPDF2
    * pwgen
    * pikepdf
Amrita Deb's avatar
Amrita Deb committed
66

Amrita Deb's avatar
Amrita Deb committed
67
  Instructions to install software dependencies based on your operating system:
68
69
70
71
72
73
74
  * Windows 10 : [Installation of Software Dependencies](swdependencies_win.md)
  * MacOS : [Installation of Software Dependencies](swdependencies_mac.md)
  * Linux : [Installation of Software Dependencies](swdependencies_linux.md)

### Docker

If you are an experienced user familiar with Docker, you can use the provided `Dockerfile` to easily run the scripts.
Christian Rohlfing's avatar
Christian Rohlfing committed
75
76
77
78
79
80
81
Either use the already built image

```bash
docker run --name='examscan' --rm -v $(pwd):$(pwd) -w $(pwd) registry.git.rwth-aachen.de/rwthmoodle/exam-scan:master batch.py --help
```

or build the Dockerfile yourself locally
82
83
84

```bash
docker build -t examscan:latest .
Christian Rohlfing's avatar
Christian Rohlfing committed
85
docker run --name examscan --rm -v $(pwd):$(pwd) -w $(pwd) examscan:latest batch.py --help
86
```
Amrita Deb's avatar
Amrita Deb committed
87

Amrita Deb's avatar
Amrita Deb committed
88
89
90
91
## Scripts and how to run them

### Process

92
93
Run `preparepdf.py` (if you have submissions in a zip and not as scans), `supplements.py` (if you want to add watermarked sample solutions as well), `watermark.py`, `encrypt.py`, and `preparemoodle.py` (or run `batch.py` which runs all) as described in the sections below. In summary, these steps will

Christian Rohlfing's avatar
Christian Rohlfing committed
94
1. unzip all PDF files from the zip and rename them according to the schema `<Matriculation number>_<Lastname>`
95
96
97
98
1. prepare sample solution for each students
1. watermark each page of each PDF with the corresponding matriculation number,
1. encrypt each PDF with a password (global or per-student) and
1. construct a zip-archive enabling batch upload and assignment of each PDF to each student in Moodle.
Amrita Deb's avatar
Amrita Deb committed
99
100

Upload `moodle_feedbacks.zip` to Moodle
101
102
103

* `Alle Angaben anzeigen` &#8594; `Bewertungsvorgang` &#8594; `Mehrere Feedbackdateien in einer Zip-Datei hochladen`
* Moodle will check for consistency and prompt errors.
Amrita Deb's avatar
Amrita Deb committed
104
105

### Commands
Amrita Deb's avatar
Amrita Deb committed
106

107
108
109
Please note that the following commands may only work for you with either `python` or `python3`.

#### Unzip submission files from and rename them
Amrita Deb's avatar
Amrita Deb committed
110
111
112

Assuming that ./submissions.zip is the zip file containing all submissions and ./Bewertungen.csv is the grading worksheet

113
```bash
Amrita Deb's avatar
Amrita Deb committed
114
115
116
python preparepdf.py --inzip ./submissions.zip --outfolder ./pdfs --csv ./Bewertungen.csv
```

117
#### Prepare copies of Sample Solutions for each student (Optional)
Amrita Deb's avatar
Amrita Deb committed
118

119
We assume that the folder `./supplements` holds the scans of the sample solution.
Amrita Deb's avatar
Amrita Deb committed
120

121
```bash
Amrita Deb's avatar
Amrita Deb committed
122
123
124
125
126
python supplements.py
```

Folder `supplements_out` contains copies of the sample solutions for each student.

Amrita Deb's avatar
Amrita Deb committed
127
#### Watermark the submissions
Amrita Deb's avatar
Amrita Deb committed
128

129
We assume that the folder `./pdfs` holds the scans of the exams and .
Christian Rohlfing's avatar
Christian Rohlfing committed
130
The filename of each PDF should start with the matriculation number of the student, e.g. `./pdfs/123456_Lastname.pdf`.
131

132
```bash
133
python watermark.py --in ./pdfs --out ./pdfs_watermarked --cores 2
134
135
```

136
137
Folder `pdfs_watermarked` contains watermarked PDFs, with each page watermarked with the matriculation number of the student.

Christian Rohlfing's avatar
Christian Rohlfing committed
138
**TIP:** Play around with `dpi` and `quality` parameters according to your requirements. Higher values for these two will result in high resolution PDFs of bigger size (ideal for when the number of files is low). Lower values will result in PDFs having lower file size and low resolution (ideal when the number of files is high)
Deb's avatar
Deb committed
139

Amrita Deb's avatar
Amrita Deb committed
140
#### Watermark Sample solution copies
Amrita Deb's avatar
Amrita Deb committed
141
142
143

We assume that the folder `./supplements_out` holds the copies for every students

144
```bash
Amrita Deb's avatar
Amrita Deb committed
145
146
147
python watermark.py --in ./supplements_out --out ./pdfs_watermarked --cores 2
```

Amrita Deb's avatar
Amrita Deb committed
148
#### Encrypt the files
Christian Rohlfing's avatar
Christian Rohlfing committed
149

Christian Rohlfing's avatar
Christian Rohlfing committed
150
Use either a global password by specifying it with the `--password` option or per-student passwords by omitting `--password`.
151

152
```bash
153
python  encrypt.py --in ./pdfs_watermarked --out ./pdfs_encrypted --password ganzgeheim
154
155
```

Christian Rohlfing's avatar
Christian Rohlfing committed
156
157
Folder `./pdfs_encrypted` contains all encrypted PDFs as well as `passwords.csv`, mapping the password of each PDF to the matriculation number.

Amrita Deb's avatar
Amrita Deb committed
158
#### Prepare for Moodle batch upload
Christian Rohlfing's avatar
Christian Rohlfing committed
159

Christian Rohlfing's avatar
Christian Rohlfing committed
160
161
This step prepares the PDFs for upload to Moodle. First, the grading table `Bewertungen.csv` has to be downloaded from Moodle via:

162
`Alle Angaben anzeigen` &#8594; `Bewertungsvorgang` &#8594; `Bewertungstabelle herunterladen`.
Christian Rohlfing's avatar
Christian Rohlfing committed
163

164
This step is needed since Moodle does not only need matriculation number, but also last and first name as well as an internal user id, which is stored in `Bewertungen.csv`.
Christian Rohlfing's avatar
Christian Rohlfing committed
165

166
```bash
167
168
169
python preparemoodle.py --in ./pdfs_encrypted --csv ./Bewertungen.csv --out ./moodle_feedbacks.zip
```

170
Then, you can upload `moodle_feedbacks.zip` in Moodle:
171
`Alle Angaben anzeigen` &#8594; `Bewertungsvorgang` &#8594; `Mehrere Feedbackdateien in einer Zip-Datei hochladen`
Christian Rohlfing's avatar
Christian Rohlfing committed
172

Christian Rohlfing's avatar
Christian Rohlfing committed
173
Further remarks:
174

Christian Rohlfing's avatar
Christian Rohlfing committed
175
176
* Exemplary zip archive `moodle_feedbacks.zip` can be downloaded [here](https://git.rwth-aachen.de/IENT/exam-scan/-/jobs/artifacts/master/download?job=test).
* You can also conduct a dry run (neither folders nor zip file are created) via `./preparemoodle.sh --dry [...]`
Amrita Deb's avatar
Amrita Deb committed
177

Amrita Deb's avatar
Amrita Deb committed
178
#### Batch job
Christian Rohlfing's avatar
Christian Rohlfing committed
179

180
Or do everything in one step
181

182
```bash
Amrita Deb's avatar
Amrita Deb committed
183
python batch.py --in ./pdfs --out ./out --cores 2 --password ganzgeheim --csv ./Bewertungen.csv --supinfolder ./supplements --sup 1 --zip ./submissions.zip
184
```
Christian Rohlfing's avatar
Christian Rohlfing committed
185

186
with folder `out` containing `passwords.csv` and `moodle_feedbacks.zip`.
Christian Rohlfing's avatar
Christian Rohlfing committed
187

188
## Original Authors
189

190
191
Helmut Flasche, Jens Schneider, Christian Rohlfing, IENT RWTH Aachen\
Dietmar Kahlen, ITHE RWTH Aachen\
Amrita Deb's avatar
Amrita Deb committed
192
Amrita Deb, IT Center, RWTH Aachen University
193

Christian Rohlfing's avatar
Christian Rohlfing committed
194
195
## Who do I talk to?

Amrita Deb's avatar
Amrita Deb committed
196
197
Servicedesk IT-Center RWTH Aachen <[servicedesk@itc.rwth-aachen.de](mailto:servicedesk@itc.rwth-aachen.de)>

Amrita Deb's avatar
Amrita Deb committed
198
199
If you have any errors or problems while running the programs, make sure to first check if your error is listed in [FAQs]

Amrita Deb's avatar
Amrita Deb committed
200
[FAQs]: https://git.rwth-aachen.de/rwthmoodle/exam-scan/-/blob/master/FAQs.md