Commit 784865b3 authored by Bernhard Rumpe's avatar Bernhard Rumpe
Browse files

BR

parent 468c5a73
<h2>Model-Driven Engineering of Intelligent Robot Architectures</h2>
<h3>Paper accompanying website</h3>
<a href="http://www.se-rwth.de/~kusmenko/">Evgeny Kusmenko</a>, Svetlana Pavlitskaya
<a href="http://www.se-rwth.de/~rumpe/">Bernhard Rumpe</a>,
and Thomas Timmermanns
<br><br>
<b>Contents for Supplementary Material according to Paper Outline:</b><br>
<a href="#IN">1 Intro</a><br>
<a href="#RW">3 Related Work</a><br>
<a href="#MA">4 MontiAnna</a><br>
<a href="#CNC">5 AI Driven Robot Architectures</a><br><br>
Video demonstration:
<iframe width="640" height="360" src="https://www.youtube.com/embed/XDeITQlUnA0" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
<br><b>
Detailed instructions on how to use EmbeddedMontiArc and MontiAnna to create a deep learning based robot software are given in
<a href="https://git.rwth-aachen.de/autonomousdriving/torcs_dl/tree/develop">here</a>.</b>
<br><br>
<a name="IN"></a>
<h4>1 Intro</h4>
<a href="https://rwth-aachen.sciebo.de/s/5LJgqqhyTgEiepU">EmbeddedMontiArc Studio - preconfigured Ubuntu VM</a>.
The EmbeddedMontiArc Studio is a web based IDE for EmbeddedMontiArc. It feature a sample project defining a neural network in
MontiAnna. To get seeing the first results quickly, the network is trained on the cifar-10 data set which takes several minutes to train
on a CPU or less than a minute on a GPU.<br>
Please follow the instruction on the desktop to run the project or watch <a href="https://www.youtube.com/watch?v=KK85jvMp55s">this video</a>. You are invited to customize the network or retrain it using your own labeled images.
<a name="RW"></a>
<h4>3 Related Work</h4>
The running example used in the paper to compare the deep learning frameworks is the so called
ResNet152 deep network. It is a convolutional neural network able to learn to extract
information from images, for instance to make a robot understand its environment.
Residual networks such as the ResNet152 try to tackle the vanishing gradient problem in deep architectures
by introducing so called residual blocks.
<br><b>
<a href="resnet152.rar" target="_blank">
The implementations of the ResNet152 we used for our framework comparison are available
here
</a>.</b><br>
In addition to the actual code, each implementation contains its origin as well as some remarks.
Note that there are always many ways to define a complex architecture such as the ResNet152.
Hence, shorter or more elegant examples might exist. However, we may conclude that using most frameworks the
ResNet152 needs several hundresds of lines of code. Caffe even needs over 6000 lines of codes. <br><br><b>The MontiAnna
description of ResNet152 contains only 33 lines of code!</b> This is one order of magnitude less than the other languages need.
<br><br>
Further network examples, e.g. the AlexNet can be found in our <a href="https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/languages/CNNArchLang/tree/master/src/test/resources/architectures">test model repository</a>.
<br>
<a name="MA"></a>
<h4>4 MontiAnna</h4>
MontiAnna is the umbrella term for a set of frameworks containing two languages and several code generators.
The modules are:
<ul>
<li><a href="https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/languages/CNNArchLang" target="_blank">CNNArc</a>: the language to define
the architecture of a deep artificial neural network. As the framework was originally intended for CNNs, the name still contains the term.
However, MontiAnna can handle any kind of deep layered networks.</li>
<li><a href="https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/languages/CNNTrainLang" target="_blank">CNNTrain</a>: the training language to define
the training hyperparameters, loss function, etc.
</li>
<li><a href="https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/generators/CNNArch2MXNet" target="_blank">CNNArch2MXNet</a>: The MontiAnna-to-MxNet
compiler. <a href="https://mxnet.apache.org/">MxNet</a> is a widely used deep learning framework used in industry applications, e.g. by Amazon.
Our compiler produces C++-code for deployment and Python code for training. Furthermore, we provide CMAKE files to facilitate the final
compilation process of the generated MxNet code.
</li>
</ul>
<a name="CNC"></a>
<h4>5 AI Driven Architectures</h4>
We embed MontiAnna into a component and connector (C&C) modeling language called EmbeddedMontiArc. EmbeddedMontiArc
allows us to decompose software architectures hierarchically and to implement the components using MontiMath, a Matlab-inspired
matrix-based language for math-heavy algorithms such as controllers (PID, MPC,...).
Having composed EmbeddedMontiArc with MontiAnna we make it possible to use neural networks as an alternative to MontiMath based
component implementaions.<br>
The composed language governing the sub-languages is
<a href="https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/languages/EmbeddedMontiArcDL">EmbeddedMontiArcDL</a>.<br>
The composed generator compiling the architecture model, the MontiMath behavior, as well as MontiAnna deep neural networls to C++ code as well as corresponding CMAKE files
for building the executable is <a href="https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/generators/EMADL2CPP">EMADL2CPP</a>.
<br><br>
To demonstrate our methodology in action, we developed a self-driving vehicle software based on the direct perception principle.
<br><b>
Detailed instructions on how to use EmbeddedMontiArc and MontiAnna to create a deep learning based robot software are given in
<a href="https://git.rwth-aachen.de/autonomousdriving/torcs_dl/tree/develop">here</a>.</b>
<br><br>
<b><a href="https://git.rwth-aachen.de/autonomousdriving/torcs_dl/blob/develop/doc/deep_driving_project.zip">The complete project is available here.</a>
The model sources can be found under src/main/dp. The main component is stored as plain text in the Mastercomponent.emadl file. Subcomponents can be found
in the subdirectory subcomponents. In particular the deep learning component is stored in <b>Dpnet.emadl</b> and its training is specified
in <b>Dpnet.cnnt</b>.
</b>
<br>
The system uses a deep neural network to extract a dozen of scalar affordance indicators from a front camera image. The affordance indicators
are scalar quantities, e.g. the distance to the front car, the distance to the lane marking, etc.
<br>
The affordance indicators extracted by the neural network are then fed into Kalman filters and a controller written in MontiMath which in turn computes
the actuator commands, i.e. steering, acceleration, and braking.
<br>
The graphical component and connector architecture of the autonomous vehicle is depicted below. The Deep Leaning component is highlighted in violet.
Other components are either implemented in MontiMath or are a composition of subcomponents. Of
course it is possible to have multiple deep learning components in an architecture:<br>
<img src="cnc.png"></img>
The workflow of the system design with EmbeddedMontiArc + MontiAnna is depicted in the following diagram.<br>
<img src="workflow.png"></img>
<br><br>
The generation workflow is depicted next:
<img src="generators.png"></img>
<br><br>
Of course, EmbeddedMontiArc+MontiAnna can be used to design a variety of other robotics applications beside the autonomous driving domain.
<?php
$ROOT_PATH = "../../";
$TITLE = "SE@RWTH : Model-Driven Engineering of Intelligent System Architecturess";
include ($ROOT_PATH."layout/lib.php");
include ($ROOT_PATH."layout/header.php");
include ("content.php");
include ($ROOT_PATH."layout/footer.php");
?>
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment