Added small documentation on scenarios for at article.

Journal: at - Automatisierungstechnik
Title: Assessment of Reinforcement Learning Applications for Industrial Control Based on Complexity Measures
DOI: 10.1515/auto-2021-0118

README.md

 # Palette Transport System Control
+
+[[_TOC_]]
+
 This is an implementation of the [control component](https://wiki.eclipse.org/BaSyx_/_Documentation_/_API_/_ControlComponent) concept utilizing the [Unity ML-Agents Toolkit](https://github.com/Unity-Technologies/ml-agents) for reinforcement learning in a Unity simulation of a very simplified palette transport system (PTS):
 
 ![Screenshot PTS](img/screenshot-overview-pts.png)
@@ -18,7 +21,7 @@ This is an implementation of the [control component](https://wiki.eclipse.org/Ba
 
 ## Heuristic: How to test the PTS environment
 - Open PTSSim Project in Unity
-  - Open the Test scene in Unity: `Assets/Scenes/Test.unity`
+  - Open the Test scene in Unity: **Assets/Scenes/Test.unity**
   - Click *Play*
   - Use *arrow keys* to trigger the palettes actions
     - left/right
@@ -28,7 +31,7 @@ This is an implementation of the [control component](https://wiki.eclipse.org/Ba
 
 ## Training: How to train a model
 - Open PTSSim Project in Unity
-  - Open the Training scene in Unity: `Assets/Scenes/Training.unity`
+  - Open the Training scene in Unity: **Assets/Scenes/Training.unity**
 - Open terminal in this folder:
   - Activate ml-agents environment according to your python installation
     - For example: `conda activate ml-agents` or `source ~/python-envs/mlagents/bin/activate`
@@ -38,13 +41,54 @@ This is an implementation of the [control component](https://wiki.eclipse.org/Ba
   - Click *Play*
 - Watch the training process in Unity or [tensorboard](localhost:6006)
   - *Display 1* to *Display 3* allow for switching between single PTS, arena or agent view
+
 ![Screenshot Arena View](img/screenshot-overview-arena.png)
 
 ## Inference: How to test a trained model
 - Open PTSSim Project in Unity
-  - Open the Run scene in Unity: `Assets/Scenes/Run.unity`
-  - Navigate to the Palette game object in the hierarchy view: `PTS/PA01`
-  - Place your trainend model (from `results/<<uniqeNameForTraining>>/PalettePathFinder.onnx`) in the *Model* property of the *Behavior Parameters* component (in the inspector view).
-    - An example for a trained model can be found in `Assets/Brains/Example.onnx`
+  - Open the Run scene in Unity: **Assets/Scenes/Run.unity**
+  - Navigate to the Palette game object in the hierarchy view: **PTS/PA01**
+  - Place your trainend model (from *results/<<uniqeNameForTraining>>/PalettePathFinder.onnx*) in the *Model* property of the *Behavior Parameters* component (in the inspector view).
+    - An example for a trained model can be found in **Assets/Brains/Example.onnx**
   - Click *Play*
     - *Display 2* shows a view from the palette perspective
+
+___
+# AT-Paper on Complexity Considerations
+
+This repository was used to show the influence on the choice of different control task considering the operational equipment measure model in the following journal contribution:
+
+* Title: **Assessment of Reinforcement Learning Applications for Industrial Control Based on Complexity Measures**
+* Authors: Julian Grothoff, Nicolas Camargo Torres and Tobias Kleinert},
+* Journal: at - Automatisierungstechnik (ISSN: 2196-677X)
+* Date: Jan. 2022
+* DOI: [10.1515/auto-2021-0118](https://doi.org/10.1515/auto-2021-118)
+
+## Scenarios
+
+As shown below four different scenarios for the control task were defined in the paper:
+
+![Scenarios in the equipment measure model](img/at-paper-scenarios.png)
+
+Scenario 2 and 3 were realised.
+Therefore, [scenes](PTSSim/Assets/Scenes/AT-Paper), PTS environment [prefabs](PTSSim/Assets/Prefabs), operation mode [scripts](PTSSim/Assets/Scripts/Procedures) and [trainer](trainer) configurations were added besides some code changes: 6e9433b2...e6e13abb.
+The setup of the training environment [PTS](PTSSim/Assets/Scripts/PTS.cs) was changed to support the indipendent handling of RL agents and palettes with control components. 
+Moreover, a script to represent the physical [palette](PTSSim/Assets/Scriptes/Palette.cs) is introduced that handles the physical positioning of the palettes.
+
+### Scenario 2: Procedure for each Coil
+
+In scenario 2 the plant oriented ML palette control operation mode was moved from the palette control component, e.g. GCU 1 in figure above, to an own control component representing the product related measure (procedure) to position a coil.
+Hence, some default operation modes for the palette (LEFT, RIGHT, UP, DOWN) were implemented in the control components for palettes, as depicted in the following figure.
+The regular operation modes LEFT and RIGHT will also check, if the next conveyor is a shifttable and move it to the right position, as in the "real" process control application, realized with [ACPLT/RTE](https://github.com/acplt/rte) in [acplt](acplt).
+
+![Topology with schematic palette operation modes.](img/palette-operation-modes.png)
+
+In consequence a control component for each coil with one [operation mode](PTSSim/Assets/Scripts/Procedures/SingleCoilProcedure.cs) to position the palette with regular operation modes is dynamically created based on the parameterization of the PTS environment.
+To run or train scneario 2 use the corresponding [scenes](PTSSim/Assets/Scenes/AT-Paper).
+
+### Scenario 3: Procedure for all Coils
+
+For scenario 3 a single RL-agent controls all palettes in one PTS environment.
+It is realized as measure (procedure) control component with one [operation mode](PTSSim/Assets/Scripts/Procedures/MultiCoilProcedure.cs).
+
+To run or train scneario 3 use the corresponding [scenes](PTSSim/Assets/Scenes/AT-Paper).
\ No newline at end of file