Changes

Page history
started with updates from my master's thesis authored by Simon Frank's avatar Simon Frank
Show whitespace changes
Inline Side-by-side
home.md
View page @ 310efdc9
[<img src="https://git.rwth-aachen.de/medialab/19squared/-/raw/2.0.9/src/app/static/img/brand/banner3.png" alt="19squared" width="1000"/>](https://19squared.de)
# `19squared` - Just more than 360
**This wiki serves as a central repository for all information about deployment, configuration, architecture, and operational procedures of 19squared.**
......@@ -9,8 +7,6 @@
## Demo Video I
[![19squared - Demo Stand September 2024](https://img.youtube.com/vi/uk9fABJgWFI/0.jpg)](https://www.youtube.com/watch?v=uk9fABJgWFI)
## Project Handouts
- [`poster1.pdf`](https://git.rwth-aachen.de/medialab/19squared/-/raw/lars-dev/src/app/static/img/brand/poster1.pdf)
......@@ -72,16 +68,23 @@ This document outlines the technologies used to build this project.
* **Node.js/npm:** Used to install all JavaScript dependencies (https://nodejs.org/en).
* **Python/pip:** Used to install all Python dependencies and as the programming language for the backend (https://www.python.org/).
* **Flask:** Used as the backend framework (https://palletsprojects.com/p/flask/).
* FastAPI: An async backend Python framework (https://fastapi.tiangolo.com/).
### AI-Servers (optional)
* SearXNG: A meta search engine to fetch current data from the internet after model knowledge cut-off (https://docs.searxng.org/).
### Database
* **MariaDB:** Used as the database management system (https://mariadb.org/).
* **PostgreSQL:** Used as the database management system (https://www.postgresql.org/).
* MinIO: Object Storage to store assets (https://minio-docs.tf.fo/).
### Infrastructure
* **Docker:** Used to containerize all services and make the application deployable in a stable environment (https://www.docker.com/).
* **Nginx:** Used as the web server that delivers the required files to the user's web browser (https://www.f5.com/go/product/welcome-to-nginx).
![19squared_architecture.png](uploads/51b01ae62d2b92f65fe6bbcfb9f51e80/19squared_architecture.png){width=1417 height=805}
## Code Structure
......@@ -129,16 +132,15 @@ For details about the development server, please refer to the project's [README]
Before running the project, create a `.env` file from the `.env.template` to set database credentials, API endpoints, and more. Then, after running `docker compose up -d --build`, Docker is running the following containers by default:
| Container Name | Description | Port |
|:------------------|---------------------------------------------|-------:|
|----------------|-------------|-----:|
| `19sq-nginx` | Nginx Web Server | `5001` |
| `19sq-flask` | Flask Application | - |
| `19sq-mariadb` | MariaDB Database | - |
| `19sq-flask` | Flask Application | \- |
| `19sq-mariadb` | MariaDB Database | \- |
| `19sq-ladsh` | Learning Analytics Dashboard | `2000` |
## Routing Traffic
🔄 Reverse Proxy `80/443` >> 🖥️ 19sq-nginx `5001` >>_(via `frontend` network)_ 🐍 19sq-flask >>_(via `backend` network)_ 💾 19sq-mariadb
:arrows_counterclockwise: Reverse Proxy `80/443` \>\> :desktop: 19sq-nginx `5001` \>\>_(via `frontend` network)_ :snake: 19sq-flask \>\>_(via `backend` network)_ :floppy_disk: 19sq-mariadb
To make your application accessible, you'll want to route external traffic to the `19sq-nginx` container on port `5001`. [Nginx Proxy Manager](https://nginxproxymanager.com/) might simplify the setup of a reverse proxy. It provides an interface for:
......@@ -150,7 +152,7 @@ To make your application accessible, you'll want to route external traffic to th
A GitLab pipeline was developed inside the `.gitlab-ci.yml` for automatic testing and deploying on a stage and production server. After installing a GitLab runner with the `stage` and `main` tags on the according server and starting it with `cd / && gitlab-runner run &`, code will be tested and deployed fully automated. The config.toml could look like the following.
````
```
concurrent = 1
check_interval = 20
connection_max_age = "60m0s"
......@@ -169,18 +171,18 @@ shutdown_timeout = 0
token_expires_at = 0001-01-01T00:00:00Z
executor = "shell"
user = "gitlab-runner"
````
```
**Important:** When using a similar approach, make sure that the `gitlab-runner` user is allowed to execute Docker commands but no security relevant commands. The following commands create the user and grant the required permissions.
````
```
sudo useradd -m -s /bin/bash gitlab-runner
getent group docker | cut -d: -f3
sudo usermod -aG <group_id> gitlab-runner
sudo systemctl stop gitlab-runner
sudo systemctl daemon-reload
sudo systemctl start gitlab-runner
````
```
## Proposed (V-)Server Setup Guide
......@@ -230,7 +232,8 @@ When deploying your instance, consider employing Cloudflare to enhance performan
Cloudflare Tunnels offer a cost-effective and secure way to deploy a 19squared instance. With this method, you simply run a server on your local machine (any laptop or PC). Then, a tunnel is established, allowing any internet-connected user to access the server running on your device. In this secure setup, your service communicates with a Cloudflare server, so users do not access the local machine directly.
**Prerequisites:**
1. Top-Level Domain: The domain's registration location is irrelevant. However, it will require configuration changes so that Cloudflare manages its DNS requests. This domain can also support other services after that. Domains are affordable, even as low as ~$10 per year on [Cloudflare Registrar](https://www.cloudflare.com/de-de/products/registrar/).
1. Top-Level Domain: The domain's registration location is irrelevant. However, it will require configuration changes so that Cloudflare manages its DNS requests. This domain can also support other services after that. Domains are affordable, even as low as \~$10 per year on [Cloudflare Registrar](https://www.cloudflare.com/de-de/products/registrar/).
2. Reasonably Modern Laptop or PC: No need for top-of-the-line hardware, just acceptable performance. If your device isn't ancient, it should suffice for small user groups.
**Learn More:** [Cloudflare Docs](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/)
......@@ -239,8 +242,6 @@ Cloudflare Tunnels offer a cost-effective and secure way to deploy a 19squared i
The following video shows all steps needed to deploy the application using a Cloudflare tunnel.
[![Deploying 19squared on a Windows PC/Laptop with Cloudflare Tunnels](https://img.youtube.com/vi/t-RPpB83htg/0.jpg)](https://www.youtube.com/watch?v=t-RPpB83htg)
### Step-by-Step Guide
Perform the following steps on the machine you want to use as your server.
......@@ -273,23 +274,21 @@ This section explains how to generate client IDs and client secrets for Google,
1. **Create a Project:**
* Go to the [Google Cloud Console](https://console.cloud.google.com/).
* Create a new project or select an existing one.
2. **Enable the OAuth 2.0 API:**
* Navigate to "APIs & Services" -> "Library".
* Navigate to "APIs & Services" -\> "Library".
* Search for "OAuth 2.0 API" and enable it.
3. **Create OAuth 2.0 Credentials:**
* Go to "APIs & Services" -> "Credentials".
* Click "Create Credentials" -> "OAuth client ID".
* Go to "APIs & Services" -\> "Credentials".
* Click "Create Credentials" -\> "OAuth client ID".
* Select "Web application" as the application type.
* Provide a name for your OAuth client.
* Set the "Authorized JavaScript origins" to your application's domain (e.g., `http://localhost:5000`).
* Set the "Authorized redirect URIs" to your application's OAuth callback URL (e.g., `http://localhost:5000/callback`).
* Click "Create".
4. **Obtain Client ID and Client Secret:**
* The newly created OAuth client ID and client secret will be displayed.
* Copy these values and add them to your `.env` file:
```
GOOGLE_CLIENT_ID=your_client_id
GOOGLE_CLIENT_SECRET=your_client_secret
......@@ -301,16 +300,15 @@ This section explains how to generate client IDs and client secrets for Google,
* Go to your GitLab project or group settings.
* Navigate to "Applications" under "Settings".
* Click "New application".
2. **Configure Application:**
* Provide a name for your application.
* Set the "Redirect URI" to your application's OAuth callback URL (e.g., `http://localhost:5000/callback`).
* Select the necessary scopes (e.g., `read_user`).
* Click "Save application".
3. **Obtain Application ID and Secret:**
* The newly created application ID and secret will be displayed.
* Copy these values and add them to your `.env` file:
```
GITLAB_CLIENT_ID=your_application_id
GITLAB_CLIENT_SECRET=your_secret
......@@ -320,18 +318,17 @@ This section explains how to generate client IDs and client secrets for Google,
1. **Create an OAuth App:**
* Go to your GitHub account settings.
* Navigate to "Developer settings" -> "OAuth Apps".
* Navigate to "Developer settings" -\> "OAuth Apps".
* Click "New OAuth App".
2. **Configure OAuth App:**
* Provide a name for your application.
* Set the "Homepage URL" to your application's domain (e.g., `http://localhost:5000`).
* Set the "Authorization callback URL" to your application's OAuth callback URL (e.g., `http://localhost:5000/callback`).
* Click "Register application".
3. **Obtain Client ID and Client Secret:**
* The newly created client ID and client secret will be displayed.
* Copy these values and add them to your `.env` file:
```
GITHUB_CLIENT_ID=your_client_id
GITHUB_CLIENT_SECRET=your_client_secret
......@@ -355,23 +352,23 @@ We employ Alembic to streamline our MariaDB database migrations in the developme
To generate a new migration script for version `0.0.1`, execute:
````
```
cd src
py -m alembic revision --autogenerate -m v0.0.1
````
```
To apply all pending migrations and update your database to the latest schema, execute the following steps:
````
```
docker exec -it 19sq-flask bash
alembic upgrade head
````
```
Instead of `head`, a specific migration ID can also be passed to the upgrade command.
# MariaDB Manual Operations
````
```
docker exec -it 19sq-mariadb mariadb -u root -p
USE db;
````
```
\ No newline at end of file