Skip to content
Snippets Groups Projects
Commit c8872972 authored by Petar Hristov's avatar Petar Hristov :speech_balloon:
Browse files

Fix: Readme

parent 89b65c96
No related branches found
No related tags found
1 merge request!17BREAKING: Migrated the GraphDeployer to use the new API
...@@ -4,7 +4,13 @@ ...@@ -4,7 +4,13 @@
## 📝 Overview ## 📝 Overview
This repository handles the deployment of every static graph which gets included into the Coscine environment. It is a C# program that performs tasks related to deploying RDF graphs to a Virtuoso triple store. The program connects to a Virtuoso server and obtains configuration settings from a Consul configuration store. It then clones one or more Git repositories containing RDF graphs, merges the graphs into a single graph, and inserts the resulting graph into the Virtuoso triple store. The program logs its progress and any errors it encounters using the NLog logging framework. The Graph Deployer, intended to be used as a CRON job, is a .NET application designed to manage and deploy RDF graphs. It interacts with configured GitLab repositories to pull RDF graph definitions and updates a centralized repository if changes are detected or redeployment is triggered. The job uses a series of configurations to determine the operational parameters including the execution environment, repository access, and deployment specifications.
### Features
- **Configuration-driven**: Behavior driven by settings defined in configuration files and environment variables (see `appsettings.json`).
- **Supports Dummy Mode**: Can run in a non-operative mode for testing configurations and process flow without affecting live data (CLI arg `--dummy`).
- **Selective Redeployment**: Allows for forced redeployment of graphs regardless of changes (CLI arg `--redeploy`).
- **Logging**: Extensive logging capabilities to track the process and troubleshoot issues.
## ⚙️ Configuration ## ⚙️ Configuration
...@@ -15,6 +21,38 @@ Before you can run and use the script, you need to ensure that the following dep ...@@ -15,6 +21,38 @@ Before you can run and use the script, you need to ensure that the following dep
3. Every repository must include the key `coscine/local/graphs/{graphname}/repositoryurl` for specifying the URL (e.g. `git.rwth-aachen.de/coscine/applicationprofiles.git`) and can include the key `coscine/local/graphs/{graphname}/branch` which specifies the state of the repository being imported (e.g. `master`). 3. Every repository must include the key `coscine/local/graphs/{graphname}/repositoryurl` for specifying the URL (e.g. `git.rwth-aachen.de/coscine/applicationprofiles.git`) and can include the key `coscine/local/graphs/{graphname}/branch` which specifies the state of the repository being imported (e.g. `master`).
Once you have all the necessary dependencies and prerequisites in place, you should be able to run and use this script. Once you have all the necessary dependencies and prerequisites in place, you should be able to run and use this script.
### Configuration file
The deployment script uses a configuration class `GraphDeployerConfiguration` to manage settings such as:
- `IsEnabled`: Toggles the deployer on or off.
- `WorkingFolder`: Specifies the directory where repositories will be cloned.
- `Logger` Configuration: Specifies the logging level and output directory.
- `GitLab` Configuration: Contains information required to access GitLab, including host URL, token, and repository details.
### Example `appsettings.json`
```json
{
"GraphDeployerConfiguration": {
"IsEnabled": true,
"WorkingFolder": "./output/",
"Logger": {
"LogLevel": "Information",
"LogHome": "C:/Logs"
},
"GitLab": {
"HostUrl": "https://gitlab.example.com",
"Token": "YourTokenHere",
"Repositories": [
{
"Name": "RepositoryName",
"Url": "https://gitlab.example.com/group/project.git",
"Branch": "main"
}
]
}
}
}
```
## 📖 Usage ## 📖 Usage
...@@ -22,6 +60,31 @@ To get started with this project, you will need to ensure you have configured an ...@@ -22,6 +60,31 @@ To get started with this project, you will need to ensure you have configured an
1. Execute the built executable (`.exe`) 1. Execute the built executable (`.exe`)
To use the **GraphDeployer**, execute the main program with appropriate command-line arguments to control its operation.
### Command-Line Arguments
- `--redeploy`: Forces redeployment of all graphs.
- `--dummy`: Runs the deployer in dummy mode, making no actual changes.
### Running the Deployer
```sh
dotnet GraphDeployer.dll --dummy true --redeploy false
```
### Deployment Process
1. **Initialization**: Configurations are loaded and services are initialized.
2. **Repository Management**: For each configured repository, the latest content is cloned (`git clone`) into the specified working folder.
3. **Processing**: Turtle files (`*.ttl`) are processed to accumulate RDF graph changes. Accumulation is nedessary, as they may be split across multiple files.
4. **Deployment Decision**: Compares the current graph file hashes against stored versions to decide if deployment is necessary. Redeployment is enforced on graphs that have no changes, whenever the `--redeploy` CLI argument is set (value: `true`).
5. **Graph Deployment**: Updates are pushed to the central graph repository if changes are detected or redeployment is forced.
6. **Clean-up**: Cleans the working directory after processing each repository. The working directory will not be deleted, only its contents.
### Considerations
- **Security**: Ensure the security of the GitLab token and other sensitive data in configuration files. Furthermore, ensure the validity of the used GitLab token, as it may expire.
- **Performance**: Large graph datasets or numerous repositories may impact performance; consider scaling solutions or optimizing configurations.
- **Error Handling**: Robust error handling and logging are crucial for diagnosing issues during deployments.
## 👥 Contributing ## 👥 Contributing
As an open source plattform and project, we welcome contributions from our community in any form. You can do so by submitting bug reports or feature requests, or by directly contributing to Coscine's source code. To submit your contribution please follow our [Contributing Guideline](https://git.rwth-aachen.de/coscine/docs/public/wiki/-/blob/master/Contributing%20To%20Coscine.md). As an open source plattform and project, we welcome contributions from our community in any form. You can do so by submitting bug reports or feature requests, or by directly contributing to Coscine's source code. To submit your contribution please follow our [Contributing Guideline](https://git.rwth-aachen.de/coscine/docs/public/wiki/-/blob/master/Contributing%20To%20Coscine.md).
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment