@@ -122,12 +122,14 @@ To reduce the workload of the developer, tests can be automated. For that, the a
!!! attention
The `testFramework` is currently under construction :construction: and still needs to be linked to the CI/CD pipeline
The CI/CD pipeline is currently implemented for the aircraftDesign repository and consists of 3 stages:
The CI/CD pipeline is **currently implemented for the aircraftDesign repository** and consists of 3 stages:
- setup
- build
- run
Each of these stages have jobs for the implemented 3 implemented test levels:
- black-box-test (Module)
- integration-test
- performance-test (System)
...
...
@@ -137,33 +139,34 @@ For each test level we have an own gitlab-runner resource, which executes the re
Unit tests and an own pipeline are planned to be implemented for the libraries repository.
Testing on windows is planned as well.
The CI / CD pipeline is implemented in gitlab via a YAML script: .gitlab-ci.yml
The CI / CD pipeline is implemented in gitlab via a YAML script: `.gitlab-ci.yml`
For the aircraft design repository, the pipeline has a modular structure, i.e. for each stage of each test level there exist an own YAML script with instructions.
Furthermore, there is one variables.yml script controlling constant variable namings and paths, which are used in the other YAML scripts.
The YAML scripts are located in the folder .ci-scripts, currently within the aircraft design repository. The main script - .gitlab-ci.yml - however is always at the root of a repository, also with this fixed naming.
The YAML scripts are located in the folder .ci-scripts, currently within the aircraft design repository. The main script - `.gitlab-ci.yml` - however is always at the root of a repository, also with this fixed naming.
#### Setup
Within each setup step a clean unicado repository is cloned into the aircraft design repository.
Within each setup step a clean UNICADO repository is cloned into the aircraft design repository.
For the clone process at first necessary git and ssh packages are downloaded and installed in a fresh docker container based on the alpine:latest image.
Why explicitly mention alpine:latest? It is a small unix based image, which is pretty fast and lightweight in installing packages (at least faster, than ubuntu images). But we will need the ubuntu image for the later stages - we will come to that later.
Currently for all sub-repositories within the unicado repository the develop branch is checked out. The branch for each respective repository, however, can be set in the variables.yml script, i.e.
- BRANCH_UNICADO: "main"
- BRANCH_AIRCRAFT_DESIGN: "develop"
- BRANCH_ADDITIONAL_SOFTWARE: "develop"
- BRANCH_LIBRARIES: "develop"
- BRANCH_AIRCRAFT_REFERENCES: "develop"
Currently for all sub-repositories within the UNICADO repository the develop branch is checked out. The branch for each respective repository, however, can be set in the variables.yml script, i.e.
-`BRANCH_UNICADO`: "main"
-`BRANCH_AIRCRAFT_DESIGN`: "develop"
-`BRANCH_ADDITIONAL_SOFTWARE`: "develop"
-`BRANCH_LIBRARIES`: "develop"
-`BRANCH_AIRCRAFT_REFERENCES`: "develop"
But why should there be a setup script, for each test level, if all are doing the same?
Simply, because they are not the same:
E.g. for each test level different folders and tools are necessary from the unicado repository. For efficient cache management (We'll come to that later), it is a good idea, to keep the content of the cache as small as possible, i.e. it should contain only the necessary files for the respective test.
E.g. for each test level different folders and tools are necessary from the UNICADO repository. For efficient cache management (We'll come to that later), it is a good idea, to keep the content of the cache as small as possible, i.e. it should contain only the necessary files for the respective test.
##### The Blackbox-test needs
- the modified tool
- gnuplot and inkscape for plotting
-**gnuplot** and **inkscape** for plotting
- an aircraft reference
##### The integration-test needs
...
...
@@ -171,10 +174,10 @@ E.g. for each test level different folders and tools are necessary from the unic
- the un-modified tool
- libraries for the un-modified tool
- cmake build receipts for the un-modified tool
- gnuplot and inkscape for plotting
- designEvaluator to compare results
-**gnuplot** and **inkscape** for plotting
-`designEvaluator` to compare results
- libraries for the design Evaluator
- cmake build receipts for the the designEvaluator
- cmake build receipts for the the `designEvaluator`
##### The performance-test needs
- The modified tool
...
...
@@ -188,43 +191,44 @@ E.g. for each test level different folders and tools are necessary from the unic
#### Build
Within the build process we use docker container based on the ubuntu:latest (Currently 24.04) image.
Within the build process we use docker container based on the **ubuntu:latest (Currently 24.04)** image.
Why? Well .. Several reasons:
- Not alpine, because the gcc install differs from the gcc install on ubuntu, which causes build failure. With ubuntu latest and gcc 14 it works. The rest are deeper technical details
- Not ubuntu 24.10 (Most current ubuntu version at the moment), because python 3.11 can't be installed easily, because the deadsnakes ppa repository is currently not available for ubuntu 24.10. A manual make install of python 3.11 would take unnecessary much computational time, which is a good idea to be reduced as much as possible
- Not ubuntu 24.10 because RCE needs python 3.11 - and the challenges of the previous bullet point
- Not alpine, because the gcc install differs from the gcc install on **ubuntu**, which causes build failure. With **ubuntu latest** and gcc 14 it works. The rest are deeper technical details
- Not **ubuntu 24.10** (Most current ubuntu version at the moment), because **Python 3.11** can't be installed easily, because the `deadsnakes` ppa repository is currently not available for **ubuntu 24.10**. A manual make install of **Python 3.11** would take unnecessary much computational time, which is a good idea to be reduced as much as possible
- Not **ubuntu 24.10** because **RCE** needs **Python 3.11** - and the challenges of the previous bullet point
##### Black-box-test build instructions:
- Install necessary packages like cmake, gcc, python, git, eigen3, cgal, etc.
- Create a project directory and copy the chosen aircraft reference to work on (Currently CSR-02 hard coded. A script probably needs to be added later, to modify the config. There is already a script, which modifies the designEvaluator config, which can be generalized)
- Copy gnuplot and inkscape into the working directory
- Install necessary packages like `cmake`, `gcc`, **Python**, **git**, `eigen3`, `cgal`, etc.
- Create a project directory and copy the chosen aircraft reference to work on (Currently CSR-02 hard coded. A script probably needs to be added later, to modify the config. There is already a script, which modifies the `designEvaluator` config, which can be generalized)
- Copy **gnuplot** and **inkscape** into the working directory
- Build the modified tool in black-box test mode as stated above.
- Move black-box-test executable to modified tool folder
##### Integration-test build instructions:
- Install necessary packages like cmake, gcc, python, git, eigen3, cgal, etc.
- Make gnuplot and inscape executable
- Install necessary packages like `cmake`, `gcc`, **Python**, **git**, `eigen3`, `cgal`, etc.
- Make **gnuplot** and **inkscape** executable
- Modify cmake build receipts of current aircraft design repo to build only modified tool
- Build modified tool
- Modify cmake build receipts of clean aircraft design repo within UNICADO repo to build only un-modified tool for comparison
- Build clean tool
- Build designEvaluator
- Build `designEvaluator`
- Create a project directory within the aircraftDesign repo of the UNICADO repo (clean) and copy the chosen aircraft reference to work on (For limitations see instructions of Black-box-test above)
- Create a project directory within the repo (local) and copy the chosen aircraft reference to work on
- Copy gnuplot an inkscape to the aircraft design repo of UNICADO
- Copy gnuplot an inkscape to the current repo
- Copy **gnuplot** an **inkscape** to the aircraft design repo of UNICADO
- Copy **gnuplot** an **inkscape** to the current repo
##### Performance-test build instructions:
- Install necessary packages like cmake, gcc, python, git, eigen3, cgal, etc.
- Install pipenv for necessary virtual environments
- Install necessary packages like `cmake`, `gcc`, **Python**, **git**, `eigen3`, `cgal`, etc.
- Install `pipenv` for necessary virtual environments
- Download and install the most latest version of cmake directly from source
- Install gcc-14 and g++-14 from ubuntu-toolchain-r/test repository, because build-essential package of ubuntu 24.04 comes with gcc version <14 but 14 is required
- Make gnuplot and inscape executable
- Create a release package by building the unicado installer (Here also a pipeline is planned, which solely tests the release of the installer)
- Install gcc-14 and g++-14 from ubuntu-toolchain-r/test repository, because build-essential package of **ubuntu 24.04** comes with gcc version <14 but 14 is required
- Make **gnuplot** and **inkscape** executable
- Create a release package by building the UNICADO installer (Here also a pipeline is planned, which solely tests the release of the installer)
- Modify cmake build receipts of current aircraft design repo to build only modified tool
- Build modified tool only
- Build designEvaluator
- Copy designEvaluator to current repo (local)
- Build `designEvaluator`
- Copy `designEvaluator` to current repo (local)
#### Test
...
...
@@ -239,21 +243,21 @@ After successful setup and build, the tests are ready to be executed, with the f
- Store aircraft reference result to be compared into a common project folder with postfix _post
- Execute un-modified tool
- Store aircraft reference result to be compared into a common project folder with postfix _pre
- Adapt designEvaluator config to compare both aircraft reference designs
- Execute designEvaluator
- Evaluate report of designEvaluator
- Adapt `designEvaluator` config to compare both aircraft reference designs
- Execute `designEvaluator`
- Evaluate report of `designEvaluator`
##### Performance-test test execution steps:
- Install python 3.11 from deadsnakes repository and some libraries via pip
- Install **Python 3.11** from `deadsnakes` repository and some libraries via pip
- Remove rce user folder
- Make UNICADOinstaller executable
- Execute UNICADOinstaller
- Make `UNICADOinstaller` executable
- Execute `UNICADOinstaller`
- Create workflow configuration file
- Execute UNICADOworkflow with RCE in headless mode with un-modified tool
- Store aircraft reference result to be compared into a common project folder with postfix _pre
- Copy modified tool executable into UNICADOworklow
- Copy modified tool executable into UNICADOworkflow
- Execute UNICADOworkflow with RCE in headless mode with modified tool
- Store aircraft reference result to be compared into a common project folder with postfix _post
- Adapt designEvaluator config to compare both aircraft reference designs
- Execute designEvaluator
- Evaluate report of designEvaluator
\ No newline at end of file
- Adapt `designEvaluator` config to compare both aircraft reference designs
