Merge branch 'documentation_adapt_testing' into 'main'

Added an initial description of the CI-Pipeline

See merge request !2

docs/developer/testing.md

@@ -122,3 +122,138 @@ To reduce the workload of the developer, tests can be automated. For that, the a
 !!! attention
     The `testFramework` is currently under construction :construction: and still needs to be linked to the CI/CD pipeline
 
+The CI/CD pipeline is currently implemented for the aircraftDesign repository and consists of 3 stages:
+- setup
+- build
+- run
+
+Each of these stages have jobs for the implemented 3 implemented test levels:
+- black-box-test (Module)
+- integration-test
+- performance-test (System)
+
+For each test level we have an own gitlab-runner resource, which executes the respective test in a docker container (Currently only tested on ubuntu 24.04; image: ubuntu:latest)
+
+Unit tests and an own pipeline are planned to be implemented for the libraries repository.
+Testing on windows is planned as well.
+
+The CI / CD pipeline is implemented in gitlab via a YAML script: .gitlab-ci.yml
+
+For the aircraft design repository, the pipeline has a modular structure, i.e. for each stage of each test level there exist an own YAML script with instructions.
+Furthermore, there is one variables.yml script controlling constant variable namings and paths, which are used in the other YAML scripts.
+
+The YAML scripts are located in the folder .ci-scripts, currently within the aircraft design repository. The main script - .gitlab-ci.yml - however is always at the root of a repository, also with this fixed naming.
+
+#### Setup
+
+Within each setup step a clean unicado repository is cloned into the aircraft design repository.
+For the clone process at first necessary git and ssh packages are downloaded and installed in a fresh docker container based on the alpine:latest image.
+Why explicitly mention alpine:latest? It is a small unix based image, which is pretty fast and lightweight in installing packages (at least faster, than ubuntu images). But we will need the ubuntu image for the later stages - we will come to that later.
+
+Currently for all sub-repositories within the unicado repository the develop branch is checked out. The branch for each respective repository, however, can be set in the variables.yml script, i.e.
+- BRANCH_UNICADO: "main"
+- BRANCH_AIRCRAFT_DESIGN: "develop"
+- BRANCH_ADDITIONAL_SOFTWARE: "develop"
+- BRANCH_LIBRARIES: "develop"
+- BRANCH_AIRCRAFT_REFERENCES: "develop"
+
+But why should there be a setup script, for each test level, if all are doing the same?
+Simply, because they are not the same:
+E.g. for each test level different folders and tools are necessary from the unicado repository. For efficient cache management (We'll come to that later), it is a good idea, to keep the content of the cache as small as possible, i.e. it should contain only the necessary files for the respective test.
+
+##### The Blackbox-test needs
+- the modified tool
+- gnuplot and inkscape for plotting
+- an aircraft reference
+
+##### The integration-test needs
+- the modified tool
+- the un-modified tool
+- libraries for the un-modified tool
+- cmake build receipts for the un-modified tool
+- gnuplot and inkscape for plotting
+- designEvaluator to compare results
+- libraries for the design Evaluator
+- cmake build receipts for the the designEvaluator
+
+##### The performance-test needs
+- The modified tool
+- All un-modified aircraft design tools
+- cmake build receipts for all un-modified aircraft design tools
+- Libraries for the un-modified tools
+- all additional software tools
+- libraries for additional software
+- cmake build receipts for additional software
+- rce environment which has to be downloaded
+
+#### Build
+
+Within the build process we use docker container based on the ubuntu:latest (Currently 24.04) image.
+Why? Well .. Several reasons:
+- Not alpine, because the gcc install differs from the gcc install on ubuntu, which causes build failure. With ubuntu latest and gcc 14 it works. The rest are deeper technical details
+- Not ubuntu 24.10 (Most current ubuntu version at the moment), because python 3.11 can't be installed easily, because the deadsnakes ppa repository is currently not available for ubuntu 24.10. A manual make install of python 3.11 would take unnecessary much computational time, which is a good idea to be reduced as much as possible
+- Not ubuntu 24.10 because RCE needs python 3.11 - and the challenges of the previous bullet point
+
+##### Black-box-test build instructions:
+- Install necessary packages like cmake, gcc, python, git, eigen3, cgal, etc.
+- Create a project directory and copy the chosen aircraft reference to work on (Currently CSR-02 hard coded. A script probably needs to be added later, to modify the config. There is already a script, which modifies the designEvaluator config, which can be generalized)
+-  Copy gnuplot and inkscape into the working directory
+- Build the modified tool in black-box test mode as stated above.
+- Move black-box-test executable to modified tool folder
+
+##### Integration-test build instructions:
+- Install necessary packages like cmake, gcc, python, git, eigen3, cgal, etc.
+- Make gnuplot and inscape executable
+- Modify cmake build receipts of current aircraft design repo to build only modified tool
+- Build modified tool
+- Modify cmake build receipts of clean aircraft design repo within UNICADO repo to build only un-modified tool for comparison
+- Build clean tool
+- Build designEvaluator
+- Create a project directory within the aircraftDesign repo of the UNICADO repo (clean) and copy the chosen aircraft reference to work on (For limitations see instructions of Black-box-test above)
+-  Create a project directory within the repo (local) and copy the chosen aircraft reference to work on
+- Copy gnuplot an inkscape to the aircraft design repo of UNICADO
+- Copy gnuplot an inkscape to the current repo
+
+##### Performance-test build instructions:
+- Install necessary packages like cmake, gcc, python, git, eigen3, cgal, etc.
+- Install pipenv for necessary virtual environments
+- Download and install the most latest version of cmake directly from source
+- Install gcc-14 and g++-14 from ubuntu-toolchain-r/test repository, because build-essential package of ubuntu 24.04 comes with gcc version <14  but 14 is required
+- Make gnuplot and inscape executable
+- Create a release package by building the unicado installer (Here also a pipeline is planned, which solely tests the release of the installer)
+- Modify cmake build receipts of current aircraft design repo to build only modified tool
+- Build modified tool only
+- Build designEvaluator
+- Copy designEvaluator to current repo (local)
+
+#### Test
+
+After successful setup and build, the tests are ready to be executed, with the following steps
+
+##### Black-box-test test execution steps:
+- Enter modified tool folder and execute black-box-test
+
+##### Integration-test test execution steps:
+- Install python and some libraries via pip
+- Execute modified tool
+- Store aircraft reference result to be compared into a common project folder with postfix _post
+- Execute un-modified tool
+- Store aircraft reference result to be compared into a common project folder with postfix _pre
+- Adapt designEvaluator config to compare both aircraft reference designs
+- Execute designEvaluator
+- Evaluate report of designEvaluator
+
+##### Performance-test test execution steps:
+- Install python 3.11 from deadsnakes repository and some libraries via pip
+- Remove rce user folder
+- Make UNICADOinstaller executable
+- Execute UNICADOinstaller
+- Create workflow configuration file
+- Execute UNICADOworkflow with RCE in headless mode with un-modified tool
+- Store aircraft reference result to be compared into a common project folder with postfix _pre
+- Copy modified tool executable into UNICADOworklow
+- Execute UNICADOworkflow with RCE in headless mode with modified tool
+- Store aircraft reference result to be compared into a common project folder with postfix _post
+- Adapt designEvaluator config to compare both aircraft reference designs
+- Execute designEvaluator
+- Evaluate report of designEvaluator
\ No newline at end of file