diff --git a/docs/docs/deployment.md b/docs/docs/deployment.md index b4daf7de1b3a2ffb1e1a521f47b8a3c8fc421fbf..5f9f265b548cdd6dab5f139363086ddbd4bae960 100644 --- a/docs/docs/deployment.md +++ b/docs/docs/deployment.md @@ -138,23 +138,21 @@ First, for each VM, the [deployment repository](https://git.rwth-aachen.de/polar Requirements 1. Machines must be able to access the Gitlab from Aachen, full internet access would be better 2. docker and docker compose must be installed 3. SSL certificates and domains must be available 4. Machines must be able to communicate with each other via HTTP and HTTPS -> Passwords should not contain special characters. +> Passwords should not contain special characters, since this often lead to problems. + +> Docker creates folder, if the corresponding volume not exists. ### Mongo (LRS) VM setup #### 1. Clone deployment repository +Clone the deployment repository for a quick start. ```console $ git clone https://git.rwth-aachen.de/polaris/entwicklung/deployment.git ``` -#### 2. Checkout `digitallearning` branch - -```console -$ git checkout digitallearning -``` - -#### 3. Configuration +#### 2. Configuration +Next, we update the .env file of the Mongo database. Settings for `MONGO_USER` and `MONGO_PASSWORT` need to match settings in `.env` for Rights Engine. @@ -165,32 +163,41 @@ $ vi .env ``` #### 3. Create Docker network +Next, we need to create a Docker network that enables a secure network connection between the mongo database and the rights engine of Polaris. ```console $ docker network create web ``` +If you like to check if the network is created, use the following command + +```console +$ docker network ls +``` + #### 4. Starting Docker services +Finally, start the Mongo database with the following command ```console $ docker compose up -d ``` -### Rights Engine VM setup - -#### 1. Clone deployment repository +Check, if the Mongo database is running: ```console -$ git clone https://git.rwth-aachen.de/polaris/entwicklung/deployment.git +$ docker compose ps ``` -#### 2. Checkout `digitallearning` branch +### Rights Engine VM setup + +#### 1. Clone deployment repository +Clone the deployment repository for a quick start. ```console -$ git checkout digitallearning +$ git clone https://git.rwth-aachen.de/polaris/entwicklung/deployment.git ``` -#### 3. Configuration +#### 2. Configuration ```console $ cd rights-engine @@ -198,29 +205,29 @@ $ cp .env.sample .env $ vi .env ``` -#### 4. Generate key pair +#### 3. Generate key pair ```console $ ssh-keygen -b 4096 -f id_rsa ``` -#### 5. Create Docker network +#### 4. Create Docker network ```console $ docker network create web ``` -#### 6. Install Traefik +#### 5. Install Traefik See section `Installing Traefik` -#### 7. Starting Docker services +#### 6. Starting Docker services ```console $ docker compose up -d ``` -#### 8. Migrate DB +#### 7. Migrate DB ```console $ docker compose exec -it rights-engine sh -c 'python3 manage.py sqlflush | sed s/TRUNCATE/DROP\ TABLE\ IF\ EXISTS/g | python3 manage.py dbshell && echo DROP\ TABLE\ IF\ EXISTS\ django_migrations\; | python3 manage.py dbshell && python3 manage.py migrate && python3 manage.py loaddata fixtures/initial_db.json' diff --git a/docs/docs/provider_schema.md b/docs/docs/provider_schema.md new file mode 100644 index 0000000000000000000000000000000000000000..6c06b6eefc98af4e10f782544693e7b7f4c699de --- /dev/null +++ b/docs/docs/provider_schema.md @@ -0,0 +1,311 @@ +# Provider Schema +A provider schema is a JSON file that contains information regarding the xAPI Verbs and Objects transmitted by the provider system. For instance, a Moodle system can transmit various verbs such as "accessed" or "viewed." Polaris requires knowledge of the structure and a list of these verbs in order to request users' consent. + +Furthermore, the schema includes descriptions and textual information explaining the purpose of the data collection. This allows the administrator to make updates to the schema within the administration interface of Polaris. + +## Structure +The provider schema is documented within a JSON file and can be uploaded via the administrative interface of Polaris. This schema adheres to a JSON schema, which is accessible in the rights engine repository at `src/static/provider_schema.schema.json`. In the subsequent section, we will guide you through a detailed, step-by-step explanation of the JSON file's structure. + +### Minimal example +The following example contains a schema that only contains the relevant information devoid of any xAPI verbs or objects. + +```json +{ + "id": "moodle-0", + "name": "Moodle", + "description": "Open-source learning management system", + "groups": [ + ... + ] +} +``` +The "id" should be the unique id of the provider system. The "id" should be the same if you like to update the schema. The "Name" and "Description" are presented to users as an explanation of the data collection carried out by this service. + +### Minimal example with verbs + +The following example contains a schema with a single group of xAPI Verbs. Each schema should contain at least one group of xAPI verbs + +```json +{ + "id":"moodle-0", + "name":"Moodle", + "description":"Open-source learning management system", + "groups":[ + { + "id":"default_group", + "label":"Default group", + "description":"default", + "showVerbDetails":true, + "purposeOfCollection":"Lorem Ipsum", + "verbs":[ + { + "id":"http://moodle.example.com/expapi/verbs/completed", + "label":"Completed", + "description":"Completed", + "defaultConsent":true + } + ] + } + ] +} +``` +The schema provided above represents the minimal practical example. It includes a single group labeled "Default group" and a single verb identified by the ID `http://moodle.example.com/expapi/verbs/completed`. It's essential for the "id" field to correspond to the ID used in the xAPI statement. When a user consents to data collection, Polaris specifically collects verbs with the ID `http://moodle.example.com/expapi/verbs/completed`. Any xAPI verbs not included in the schema will not be collected under any circumstances. + +You have the flexibility to update the schema whenever it becomes necessary to add new verbs. However, it's important to note that when new verbs and objects are introduced, users will need to provide their consent once again. + +### Groups + +Now, let's delve deeper into the group configuration options. A group serves as a straightforward means of organizing xAPI settings. The advantage of this grouping is that users can provide consent for all associated verbs with a single toggle within the Polaris user interface. + +```json +{ + "id":"default_group", + "label":"Default group", + "description":"default", + "showVerbDetails":true, + "purposeOfCollection":"Lorem Ipsum", + "verbs":[ + ... + ] +} +``` + +The "id" property serves as a simple and unique identifier for the group within the provider. The "label" functions as the headline of the group in the user consent view within the user interface. The "description" property, positioned beneath the label, provides an informative description of the group's purpose. Ideally, both the group label and description should offer a clear and enlightening explanation of the data collection's intent. + +The "purposeOfCollection" property serves a similar function but allows for additional information to be included for the data disclosure report. This text is automatically incorporated into a PDF report that users can generate in Polaris. + +The "showVerbDetails" property enables the option to toggle between displaying verbs in detail for the user or allowing the user to consent to data collection for the entire group. This feature provides flexibility in how users can interact with and provide consent for data collection. + +### Verbs + +Going one level deeper, we have the ability to configure individual xAPI verbs. In the subsequent sections, we will provide an explanation of the available options for each xAPI verb. + +```json +{ + "id":"http://moodle.example.com/expapi/verbs/completed", + "label":"Completed", + "description":"Completed", + "defaultConsent":true, + "allowAnonymizedCollection": false, + "allowAnonymizedCollectionMinCount": 4, + "objects": [] +} + +``` + +The "id" property corresponds to the ID of the verb as it appears in the xAPI Statement. Polaris determines user consent by cross-referencing the ID of the verb in the xAPI statement with the toggled options specified in the provider schema. This matching process is essential for recognizing and managing user consent effectively. + +The "label" and "description" fields contain user-friendly text that provides information about the xAPI verb. Here, you can describe when the xAPI verbs are triggered and what information is included in the corresponding statement. +For instance, let's consider the xAPI verb "graded." A suitable label could be "Graded," and the description might read as follows: "We collect information when you grade an activity in Moodle, such as a quiz or test. This includes data on the achieved score, whether the test was passed or failed, and detailed information about the individual questions." +In this way, the "label" and "description" fields serve to provide clear and understandable explanations of when and how the xAPI verbs are utilized, enhancing user comprehension and transparency regarding data collection practices. + +The "defaultConsent" is a toggle that allows you to specify whether the option is initially set to "on" or "off." This setting determines the default state of the option when users first encounter it, and they can subsequently adjust it as per their preferences or consent. + +The "allowAnonymizedCollection" is a toggle that controls whether Polaris can collect anonymized data related to the xAPI verb, even if the user has not explicitly granted consent. When this option is activated, user names are irreversibly anonymized within the xAPI statement before being stored in the Learning Record Store (LRS). +By default, the value is set to "false" if the field is left empty. In other words, unless specifically configured to allow anonymized data collection, the default behavior is to refrain from collecting such data without the user's explicit consent. This feature ensures compliance with privacy regulations and provides an extra layer of data protection for users. + +The "allowAnonymizedCollectionMinCount" setting can override the default threshold established by the rights engine. This threshold is designed to control the minimum number of distinct users who must have generated the xAPI verb within statements. When this minimum count is met, the verb is considered valid or allowed for use by analytic engines. +In other words, "allowAnonymizedCollectionMinCount" allows you to customize the specific threshold for a particular xAPI verb, potentially making it more or less restrictive than the default threshold set by the rights engine. This flexibility is useful when fine-tuning data collection and analysis requirements for specific verbs or use cases. + +The "objects" array comprises a list of objects associated with a particular xAPI verb. If the object list is empty, it signifies that the xAPI verb, in this context, accepts all objects. + +### Object Filtering +If you wish to expand the choices available to users within the consent view, you have the option to add objects to each verb. While not mandatory, this practice greatly enhances the transparency and comprehensibility of the consent process, making it more user-friendly and informative. This additional context can help users make more informed decisions regarding their data consent. + +There are two types of options available for matching the object against the xAPI statement: "id-based" and "definition-based" filtering. + +- Id-Based Filtering: With id-based filtering, Polaris verifies if the object's ID matches the one specified in the user consent. This method is particularly useful when you want to narrow down your analytics to focus on specific content, such as a particular video or a specific task. + +- Definition-Based Filtering: Definition-based filtering, on the other hand, involves Polaris checking if the ID within the object's definition corresponds to the one stipulated in the user consent. This approach is akin to matching xAPI verbs and is suitable for scenarios where you want to filter data based on the attributes or characteristics outlined in the object's definition, aligning it with the user's consent preferences. + + +#### Id-based filtering + +The provided JSON sample file contains a configuration designed for utilizing the "id-based" type filtering method. This configuration is tailored to facilitate the matching of objects based on their unique identifiers, aligning with specific data collection and analysis requirements. + +```json +{ + "id":"http://h5p.example.com/expapi/activity/QKGPPiIhI4zx9YAZZksLKigqyf7yW4WF", + "label":"1.1.1 Funktionen", + "defaultConsent":true, + "matching":"id", +} + +``` + +The "label" and "defaultConsent" properties within the object configuration function similarly to those found in the xAPI verb configuration. They serve to provide a descriptive label for the object and specify whether the default consent setting is enabled or disabled, just as they do in the verb configuration. This consistency in property usage ensures a uniform approach to user consent and settings throughout the configuration, making it more intuitive for users and administrators. + +The matching field controls the switch between the id-based filtering and defintion type-based filtering. +In id-based filtering, the "id" field must match against the object's ID in the statements to establish a match. This means the ID specified in the configuration must correspond to the ID found in the xAPI statements for the filtering to be effective. + +#### Definition type-based filtering + +The provided JSON sample file contains a configuration intended for the utilization of the "definition-based" type filtering. This configuration is designed to support the matching of objects based on the attributes and properties outlined in their definitions, aligning with the user consent preferences and specific data filtering needs. + +```json +{ + "id":"https://xapi.elearn.rwth-aachen.de/definitions/lms/activities/assign", + "label":"Assignment", + "defaultConsent":false, + "matching":"definitionType", + "definition":{ + "type":"https://xapi.elearn.rwth-aachen.de/definitions/lms/activities/assign", + "name":{ + "enUS":"An assignment object. Provides the students with a task description and further needed materials and allows to upload a submission." + } + } +} +``` + +The "label" and "defaultConsent" properties within the object configuration function similarly to those found in the xAPI verb configuration. They serve to provide a descriptive label for the object and specify whether the default consent setting is enabled or disabled, just as they do in the verb configuration. This consistency in property usage ensures a uniform approach to user consent and settings throughout the configuration, making it more intuitive for users and administrators. + +The "definition" section within the configuration should be copied from a sample statement and must match the definition part of the object in the xAPI statement generated by the user. If the definition in the configuration does not correspond to the definition in the user's xAPI statement, Polaris will reject the statement. + +### Essential verbs +Essential verbs are verbs that are always collected. The user can not disagree to this data collection, however Polaris will inform the user about the collection. The essential verbs are definited in the root node of the provider schema configuration. + +```json +{ + "id": "h5p-0", + "name": "H5P", + "description": "Open-source content collaboration framework", + "groups": [ + ... + ], + "essentialVerbs": [ + { + "id": "http://h5p.example.com/expapi/verbs/liked", + "label": "Liked", + "description": "Like interaction", + "defaultConsent": true, + "objects": [] + } + ] +} + +``` +The configuration of the verbs is identical to that of other verbs within the schema. You can also restrict the verbs to specific verbs. That allows the essential data collection for example for specific objects using the id-based filtering approach. + +## Validate Schema +It is possible to validate the schema againts the json schema from the repository of the rights engine (`src/static/provider_schema.schema.json`). You can use multiple online tools like https://www.jsonschemavalidator.net/ to validated the schema before uploading it to Polaris. + +## Update Schema + +The provider schema can be updated at any time, offering the flexibility to make changes such as adding or removing groups, verbs, and objects. You can also modify the status of verbs, shifting them between "essential" and "optional." However, it is advisable to avoid including the same verb in both the "essential" and "optional" sections within the same schema version to prevent confusion. + +Polaris has the capability to generate a comparison between different versions of the schema, making it easier for users to review the changes made. This streamlined process ensures that users only need to focus on the parts of the schema that have been altered, enhancing the efficiency of schema updates and user consent management. + +To update the schema, you can upload a new JSON file with the same identifier as an existing provider. This identifier serves as a reference to the specific provider, allowing you to replace the previous schema with the updated one, ensuring continuity in your data collection and consent management process. + +## i18n Support +In a later stage of development, we introduced comprehensive internationalization (i18n) support for the description fields within the provider, verbs, and objects. This enhancement enables Polaris to provide multi-language support, allowing users to interact with the platform in their preferred language, ensuring a more inclusive and user-friendly experience. + +You have the option to retain the existing schema, translate specific properties, or fully translate the entire schema. All of these variants are feasible and supported within the system's configuration. This flexibility ensures that you can adapt the schema to your language and localization requirements, providing a tailored experience for your users. + +## Useful online tools + +- JSON Formatter: https://jsonformatter.curiousconcept.com/ +- JSON Schema Validator: https://www.jsonschemavalidator.net/ + +## Extended example + + +```json +{ + "id": "moodle-0", + "name": "Moodle", + "description": "Open-source learning management system", + "groups": [ + { + "id": "default_group", + "label": "Default group", + "description": "default", + "showVerbDetails": true, + "purposeOfCollection": "Lorem Ipsum", + "verbs": [ + { + "id": "http://moodle.example.com/expapi/verbs/completed", + "label": "Completed", + "description": "Completed", + "defaultConsent": true, + "objects": [ + { + "id": "http://moodle.example.com/expapi/activity/programming-course-python", + "label": "Python Programming Course", + "defaultConsent": true, + "matching": "definitionType", + "definition": { + "type": "http://moodle.example.com/expapi/activity/programming-course-python", + "name": { + "enUS": "Python Programming Course" + } + } + }, + { + "id": "http://moodle.example.com/expapi/activity/foreign-language-course", + "label": "Foreign language course", + "defaultConsent": true, + "matching": "definitionType", + "definition": { + "type": "http://moodle.example.com/expapi/activity/foreign-language-course", + "name": { + "enUS": "Foreign language course" + } + } + } + ] + }, + { + "id": "http://moodle.example.com/expapi/verbs/started", + "label": "Started", + "description": "Started", + "defaultConsent": false, + "matching": "definitionType", + "objects": [] + }, + { + "id": "http://moodle.example.com/expapi/verbs/paused", + "label": "Paused", + "description": "paused", + "defaultConsent": false, + "matching": "definitionType", + "objects": [] + }, + { + "id": "http://moodle.example.com/expapi/verbs/created", + "label": "Created", + "description": "Created", + "defaultConsent": false, + "matching": "definitionType", + "objects": [] + }, + { + "id": "http://moodle.example.com/expapi/verbs/answered", + "label": "Answered", + "description": "Answered", + "defaultConsent": true, + "matching": "definitionType", + "objects": [ + { + "id": "http://moodle.example.com/expapi/activity/poll-preferred-course-level", + "label": "Poll: Preferred course level", + "defaultConsent": true, + "matching": "definitionType", + "definition": { + "type": "http://moodle.example.com/expapi/activity/poll-preferred-course-level", + "name": { + "enUS": "Poll: Preferred course level" + } + } + } + ] + } + ], + "isDefault": true + } + ], + "essentialVerbs": [] +} + + +``` \ No newline at end of file diff --git a/docs/docs/rights_engine.md b/docs/docs/rights_engine.md index 74142e09080771a9d4a3208f23f686581280402d..23b1807005f529205a8671147f17ad0ecc32a19a 100644 --- a/docs/docs/rights_engine.md +++ b/docs/docs/rights_engine.md @@ -4,6 +4,10 @@  +## Configuration + +Tips and tricks about the configuration can be found in the deployment chapter. If you need information how to create a schema for adding new plattform, look at the provider schema chapter. + ## Visualization Token Dashboards obtain data for different charts using a JWT visualization token. The token should be created in backend of the dashboard environment e.g. moodle backend. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 219e7060cf6b33c7c979153ecb914163c68543b4..8aa351113886147fa0bb5838ed607b9779e9596b 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -9,6 +9,7 @@ theme: nav: - Rights Engine: rights_engine.md + - provider Schema: provider_schema.md - Analytics Engine: analytics_engine.md - Dashboard: dashboard.md - Deployment: deployment.md diff --git a/src/users/models.py b/src/users/models.py index 8fa35a20f6a0ef7403629566c04e388ee3cdd507..681df55fbded27b14c555e456ad20aabba979072 100644 --- a/src/users/models.py +++ b/src/users/models.py @@ -15,6 +15,7 @@ class CustomUser(AbstractUser): accepted_provider_schemas = models.ManyToManyField(ProviderSchema) # Main field for authentication paused_data_recording = models.BooleanField(default=False) + general_privacy_policy = models.BooleanField(default=False) USERNAME_FIELD = 'email' REQUIRED_FIELDS = ['first_name', 'last_name']