Commit 442f7ccf authored by Jean Meurice's avatar Jean Meurice Committed by Evgeny Kusmenko

Renaming, Direct/Emu simulator Types, Java Config objects for...

Renaming, Direct/Emu simulator Types, Java Config objects for Simulator/HardwareEmulator, JNIInterface, new Port structure
parent 30a5fdb9
......@@ -16,4 +16,6 @@ zydis/dependencies/zycore/Release/
target/
unicorn/
zydis/
pe-parse/
\ No newline at end of file
pe-parse/
src/main/resources/linux/
src/main/resources/windows/
......@@ -23,6 +23,8 @@ windows_build:
artifacts:
paths:
- hardware_emulator/build/Release/hardware-emulator-test.exe
- src/main/resources/windows/HardwareEmulator.dll
expire_in: 1 hour
windows_test:
stage: test
......@@ -53,6 +55,7 @@ linux_build:
paths:
- hardware_emulator/build/hardware-emulator-test
- src/main/resources/linux/HardwareEmulator.so
expire_in: 1 hour
linux_test:
stage: test
......@@ -76,6 +79,7 @@ linux_maven_master:
- master
dependencies:
- linux_build
- windows_build
linux_maven_branch:
stage: maven
......
......@@ -73,4 +73,56 @@ The components used to discover the ports of a *DynamicInterface* software and r
The main idea is to discover the name and types of the ports (performed by the `DynamicInterfaceResolver`), to store these informations in `PortInformation` structures, then depending on the actual simulator implementation used, allocate actual `Port` instances that can store and transfer the specific data type (`PortSimpleIntDirect`, `PortArrayIntEmu`, ...).
The templated `PortSimple` and `PortArray` implementations already define how the port communicates with the Java simulator (through JNI), but how the port data is given to the software is depending on how the software is loaded (as native library or in the Computer emulation), which is specified in the variants `Port...Emu` and `Port...Direct`.
\ No newline at end of file
The templated `PortSimple` and `PortArray` implementations already define how the port communicates with the Java simulator (through JNI), but how the port data is given to the software is depending on how the software is loaded (as native library or in the Computer emulation), which is specified in the variants `Port...Emu` and `Port...Direct`.
# TODO Adapt
# Dev Documentation
## Emulator configuration
The [emulator_configuration](docs/emulator_configuration.pdf) document shows the configuration possibilities for the autopilot emulators.
## HardwareEmulator JNI interface
The interface of between the RMIModelServer and the HardwareEmulator works with _Java Native Interface (JNI)_. In case of an update
of the interface (file: [src/main/java/simulator/integration/HardwareEmulatorInterface.java](src/main/java/simulator/integration/HardwareEmulatorInterface.java)),
the corresponding C header has to be generated.
Currently, this happens in the [hardware_emulator](https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/simulators/hardware_emulator) project under the `jni` folder,
where the HardwareEmulatorInterface.java has to be copied. `generate.bat` shows how to update the interface on Windows.
The java file has to be adapted to compile without its dependencies, since the JNI header will only
be generated from compilable java files. Since the actual header file only depends on the `native` functions from the java file, the other dependencies can
be safely deleted. The resulting header file then has to be renamed `emulator_server.h` and placed in the `hardware_emulator/src/emulator` source folder.
The corresponding implementations of the C interface have to be updated in `hardware_emulator/src/emulator/emulator_server.cpp`.
A better approach would be a script in the RMIModelServer project generating the header and copying it to the *hardware_emulator* project.
## New autopilot EMA port types
To communicate with the ports from the autopilot, an _AutopilotAdapter_ encapsulates the EMA model. For new port data types,
the *EMAM2CPP* generator and the *hardware_emulator* have to be updated to handle the new data type.
In the EMAM2CPP project, the `de.monticore.lang.monticar.generator.cpp.viewmodel.AutopilotAdapterDataModel` class is responsible to create the _AutopilotAdapter_.
In its `addInput` and `addOutput` functions, the new function definition and declarations for the data type have to be defined.
In the *hardware_emulator* projects, three places require updates for new data types.
1. The `hardware_emulator/src/emulator/function_values.h / .cpp` files need to updated with the new data type (`enum class VALUE_TYPE`), buffer entry (`struct FunctionValue`) and type name (`FunctionValue::get_type()`).
2. The `call_input()` and `call_output()` functions from the `HardwareEmulator` structure (`hardware_emulator/src/emulator/hardware_emulator.h`) must be updated. They need to know how to pass the arguments to the emulated autopilots.
3. The `Java_simulator_integration_HardwareEmulatorInterface_add_1one_1input()` and `Java_simulator_integration_HardwareEmulatorInterface_query_1outputs()` functions need to know how to pass and read port data to and from Java. (`hardware_emulator/src/emulator/emulator_server.h`)
## Unsupported autopilot functionalities.
The emulator used in this RMIModelServer only implements features encountered in previous autopilots.
New autopilots might trigger new unsupported features:
* Unsupported system calls (enable `debug=unsupported_syscalls` flag for the autopilot emulator). These are operating system functions for which no current emulation is implemented. These have to be added and registered in the `hardware_emulator/src/os_linux/linux_calls.h / .cpp` files for Linux and in the `hardware_emulator/src/os_windows/windows_calls.h / .cpp` files for Windows.
* Unsupported instruction time value. The time table (currently `hardware_emulator/src/timetable/skylake.txt`) describing the number of CPU cycles for every instruction is manually filled. It currently only contains entries (!=0) for encountered instructions. It can be filled with elements from the `hardware_emulator/docs/instruction_time.txt` file (which contains entries from the instruction_tables.pdf file).
To make sure the autopilot emulation is correct, allocate an autopilot emulator with the `test_real` flag. When loading an autopilot with the same OS as the RMIModelServer, the emulator will compare the outputs of the emulated autopilot with the outputs of the actual autopilot program.
% (c) https://github.com/MontiCore/monticore
%
% The license generally applicable for this project
% can be found under https://github.com/MontiCore/monticore.
\usepackage{graphicx}
%\usepackage{ngerman}
\usepackage[utf8x]{inputenc}
\usepackage{fancyvrb}
\usepackage{courier}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{helvet}
\usepackage{pbox}
\usepackage{xcolor}
\usepackage{pdfpages}
\usepackage[strict]{changepage}
\usepackage{calc}
\usepackage{enumitem}
\usepackage{array}
\usepackage{tabularx}
\usepackage{tabu}
\usepackage{multirow}
\usepackage{tikz}
\usetikzlibrary{tikzmark}
\usetikzlibrary{calc}
\usetikzlibrary{decorations.pathreplacing}
\usetikzlibrary{datavisualization}
\usepackage[htt]{hyphenat}
\usepackage{float}
\newfloat{example}{tbh}{exa}[chapter]
\floatname{example}{Example}
\usepackage{siunitx}
\usepackage{pgfplots}
%\usepackage{hyperref}
%\usepackage{url}
\pgfplotsset{compat=newest} % Allows to place the legend below plot
\usepgfplotslibrary{units} % Allows to enter the units nicely
%\the\textwidth = 426.79pt
%\the\arrayrulewidth = 0.4pt
\pdfoptionpdfminorversion=6
% \definecolor{se_dark_blue}{RGB}{0,103,166} % powerpoint
\definecolor{se_dark_blue}{RGB}{0,96,178} % website
% \definecolor{se_light_blue}{RGB}{119,158,201} % powerpoint
\definecolor{se_light_blue}{RGB}{129,160,225} % website
\graphicspath{{src/pic/}}
%%\newcommand{\tc}[1]{\Large{\fontfamily{ph v}\selectfont{#1}}}
\newcommand{\tc}[1]{\normalsize{#1}}
\newcommand{\tm}[1]{\scriptsize{#1}}
\newcommand{\ttt}[1]{\scriptsize{\texttt{#1}}}
\newcommand{\tsf}[1]{\scriptsize{\textsf{#1}}}
\newcommand{\tit}[1]{\scriptsize{\textit{#1}}}
%% setup listings
\usepackage{listings}
\lstset{
language=C++,
frame=none,
basicstyle=\small\ttfamily,
}
\lstset{
language=Java,
frame=none,
basicstyle=\small\ttfamily,
}
\lstdefinestyle{MontiArcStyle}{
numbers=left,
numberstyle=\tiny,
numbersep=5pt,
xleftmargin=11pt,
xrightmargin=4pt,
frame=single,
aboveskip=0pt,
belowskip=-6pt,
sensitive=true,
float=!t,
breaklines=false,
captionpos=b,
tabsize=2,
showstringspaces=false,
basicstyle=\small\ttfamily,
morecomment=[l]{//},
morecomment=[s][\itshape]{/**}{*/}
}
%% defines the listings laguage named 'MontiArc' derived from the language 'Java'
%% adding the below listed keywords. See
%% ftp://ftp.tex.ac.uk/tex-archive/macros/latex/contrib/listings/listings.pdf
%% for listings documentation
\lstdefinelanguage{MontiArc}[]{Java}{
morekeywords={component, port, in, out, inv, package, import, connect, autoconnect}
}
% Seite einrichten
\setlength{\voffset}{-1in}
\setlength{\hoffset}{-1in}
\setlength{\topmargin}{2.5cm}
\setlength{\headheight}{0cm}
\setlength{\headsep}{0cm}
\setlength{\oddsidemargin}{3,3cm} % innen ein wenig mehr Rand für die Klebebindung
\setlength{\evensidemargin}{2,7cm} % dafür außen ein wenig weniger
\setlength{\textwidth}{15cm}
\setlength{\textheight}{23,5cm}
\setlength{\parindent}{0cm}
\newcommand{\emptyLine}{{\LARGE ~\\}}
\begin{document}
\setcounter{page}{1}
\pagenumbering{arabic}
\section{Configuration}
\label{emuconfig}
The HardwareEmulator and the EmulatorManager support a simple fail-safe way of configuring them and making queries to them. A message is made of a String where each line (separated by a newline character) is a name followed by optional parameters.
The syntax is whitespace insensitive and as follows:
\begin{verbatim}
name = param1, param2, param...
or
name
\end{verbatim}
Whitespace is optional, and the only separating characters are the equal sign ($=$), the comma (,) and the newline character (\texttt{\textbackslash n}).
\subsection{Configuration}
The current settings supported when allocating an emulator are shown in Table~\ref{tab:emuconfig}.
\begin{table}[!htb]
\begin{tabularx}{\textwidth}{ | X |}
\hline
\texttt{autopilot=name} \\
Name of the autopilot program file, without extension.\\ \hline
\texttt{os=name} \\
Name of the operating system used. Currently \texttt{linux} or \texttt{windows}. This implies the extension of the autopilot file searched. If the os command is not given, the Emulator will search for any autopilot with the given name in the search folder. \\ \hline
\texttt{debug=flag1,flag2,\dots} \\
Enable the debugging outputs for the specified flags, where the flags are any of \texttt{mem}, \texttt{regs}, \texttt{reg\_update}, \texttt{syscalls}, \texttt{unsupported\_syscalls}, \texttt{code} or \texttt{call}. \\\hline
\texttt{cpu\_frequency=value} \\
The CPU frequency in Hertz used to evaluate the computation time. \\\hline
\texttt{memory\_frequency=value} \\
The RAM frequency in Hertz used to evaluate the computation time. \\\hline
\texttt{cache\_name=size,read\_ticks,write\_ticks} \\
Where \texttt{name} is \texttt{IL1}, \texttt{DL1}, \texttt{L2} or \texttt{L3}.
This setting enables and sets the parameters of the named cache layer. Set \texttt{size} to 0 to disable this cache level. The ticks refer to the CPU ticks.\\\hline
\texttt{no\_time} \\
Disables the delaying of the autopilot in the simulation. \\\hline
\texttt{test\_real} \\
If the RMIModelServer and the autopilot are of the same operating system type, this flag makes the emulator load the autopilot directly as a library (not emulated) and uses its outputs to validate those of the emulated instance. \\
\texttt{export} \\
Enables the \texttt{HardwareEmulator} to export the deviation from the planned trajectory and the evaluated execution time of the autopilot. Data is currently exported for the first 60 seconds of the simulation. The output files are \texttt{autopilot\_dist.txt} and \texttt{autopilot\_time.txt}.
\\\hline
\end{tabularx}
\caption{HardwareEmulator Configuration}
\label{tab:emuconfig}
\end{table}
The \texttt{EmulatorManager} itself also supports a config string on initialization. Table~\ref{tab:managerconfig} shows the supported config.
\begin{table}[!htb]
\begin{tabularx}{\textwidth}{ | X |}
\hline
\texttt{autopilots\_folder=folder} \\
Sets the folder in which to look for autopilot programs.\\ \hline
\end{tabularx}
\caption{EmulatorManager Configuration}
\label{tab:managerconfig}
\end{table}
\subsection{Queries}
The query system uses the same presented syntax, but also sends a response for every query, using the same syntax.
Queries supported by the \texttt{EmulatorManager} are shown in Table~\ref{tab:managerqueries}.
\begin{table}[!htb]
\begin{tabularx}{\textwidth}{ >{\em}l | X |}
\cline{2-2}
Query & \texttt{get\_error\_msg} \\
Response & \texttt{error\_msg=msg} \\
& Returns the last error message that occurred while allocating an autopilot emulator. \\ \cline{2-2}
Query & \texttt{get\_available\_autopilots} \\
Response & \texttt{available\_autopilots=autopilot1, autopilot2, \dots} \\
& Returns the list of autopilot files available in the autopilot folder. \\ \cline{2-2}
Query & \texttt{get\_available\_threads} \\
Response & \texttt{available\_threads=thread\_count} \\
& Returns the number of concurrent threads the machine running the RMIModelServer supports.\\ \cline{2-2}
Query & \texttt{get\_autopilots\_folder} \\
Response & \texttt{autopilots\_folder=folder} \\
& Returns the current autopilots folder.\\ \cline{2-2}
\end{tabularx}
\caption{EmulatorManager queries.}
\label{tab:managerqueries}
\end{table}
Queries supported by querying a specific emulator are shown in Table~\ref{tab:emuqueries}.
\begin{table}[!htb]
\begin{tabularx}{\textwidth}{ >{\em}l | X |}
\cline{2-2}
Query & \texttt{get\_avg\_runtime} \\
Response & \texttt{avg\_runtime=time} \\
& Returns the mean average time of autopilot execution.\\ \cline{2-2}
Query & \texttt{is\_computing} \\
Response & \texttt{computing=1 or 0} \\
& Returns 1 if the computer is considered to be computing in simulation time.\\ \cline{2-2}
Query & \texttt{get\_computer\_time} \\
Response & \texttt{computer\_time=time} \\
& Returns the time counter of the computer, in microseconds.\\ \cline{2-2}
\end{tabularx}
\caption{HardwareEmulator queries.}
\label{tab:emuqueries}
\end{table}
\end{document}
@REM
@REM (c) https://github.com/MontiCore/monticore
@REM
@REM The license generally applicable for this project
@REM can be found under https://github.com/MontiCore/monticore.
@REM
@echo off
set GENDIR=target\
set MAINNAME=emulator_configuration
set MAINTEX=%MAINNAME%.tex
set TARGET=%GENDIR%%MAINNAME%.pdf
mkdir %GENDIR%
@echo ------------initial pdf-latex run
pdflatex -output-directory=%GENDIR% %MAINTEX%
@echo ------------Make index
IF EXIST "%GENDIR%%MAINNAME%.idx" makeindex %GENDIR%%MAINNAME%.idx
@echo ------------2nd pdf-latex run
pdflatex -output-directory=%GENDIR% %MAINTEX%
@echo ------------3rd pdf-latex run
pdflatex -output-directory=%GENDIR% %MAINTEX%
......@@ -87,10 +87,11 @@ void HardwareEmulator::init_simulator( SoftwareSimulatorManager &manager, const
}
Log::info << Log::tag << "Initiated emulator with autopilot: " << software_name << ",\n"
<< "os: " << os_name << ",\n";
Log::info << Log::tag << "Initiated SoftwareSimulator with autopilot: " << software_name << ",\n"
<< "os: " << os_name;
if (test_real)
Log::info << "real autopilot compared,\n";
Log::info << ", native software execution comparison enabled (test_real)";
Log::info << '\n';
}
void HardwareEmulator::resolve_autopilot_os( SoftwareSimulatorManager &manager ) {
......
......@@ -51,7 +51,6 @@ struct PortSimple : public Port {
*/
template<typename PortTypeInfo>
struct PortSimpleEmu : public PortSimple<PortTypeInfo> {
//TODO function name from PortTypeInfo
PortSimpleEmu(const PortInformation& info, HardwareEmulator& emu) : PortSimple<PortTypeInfo>(info.name), computer(emu.computer)
{
emu.resolve(info.main_accessor_function_name, this->emu_function);
......
......@@ -36,8 +36,6 @@ void SoftwareSimulator::init(SoftwareSimulatorManager& manager, const char* conf
Log::err << "Unkown Time Model\n";
}
//TODO time model
if (software_name.size() == 0)
throw_error(Error::hardware_emu_software_load_error("Missing the name of the Software program to load."));
......
......@@ -16,6 +16,10 @@ SoftwareSimulatorManager SoftwareSimulatorManager::instance;
void SoftwareSimulatorManager::init( const char *config) {
if (loaded){
Log::info << Log::tag << "SoftwareSimulatorManager::init() on already loaded manager: skipping.\n";
return;
}
available_threads = std::thread::hardware_concurrency();
available_softwares_string = "";
......@@ -44,6 +48,7 @@ void SoftwareSimulatorManager::init( const char *config) {
}
simulators.clear();
simulator_count = 0;
loaded = true;
}
int SoftwareSimulatorManager::alloc_simulator( const char *config ) {
......@@ -76,16 +81,14 @@ int SoftwareSimulatorManager::alloc_simulator( const char *config ) {
if ( !simulators[i] ) {
simulators[i] = std::move(simulator);
simulator_count++;
Log::info << Log::tag << "Emulator allocated with id " << i << "\n"; //TODO log
return i;
}
}
throw_error("Unknown Error");
throw_error("Unknown Error"); //Should not reach this point
}
void SoftwareSimulatorManager::free_simulator( int id ) {
Log::info << Log::tag << "Emulator " << id << " freed\n";
simulator_count--;
simulators[id].reset();
}
......
......@@ -14,8 +14,9 @@
*/
struct SoftwareSimulatorManager {
static SoftwareSimulatorManager instance;
bool loaded = false;
std::vector<std::unique_ptr<SoftwareSimulator>> simulators;
uint simulator_count = 0;
uint simulator_count;
//std::list<fs::directory_entry> entries;
std::list<FS::File> available_softwares;
......
......@@ -29,7 +29,7 @@ JNI_OnLoad(JavaVM* vm, void* reserved) {
std::cout << "GetEnv: version not supported" << std::endl;
}
JNIEnvironment::instance.init(jni);
std::cout << "JNI_OnLoad() called" << std::endl;
//std::cout << "JNI_OnLoad() called" << std::endl;
//TODO DetachCurrentThread
return JNI_VERSION_1_6;
}
......@@ -53,7 +53,7 @@ JNI_OnUnload(JavaVM* vm, void* reserved) {
std::cout << "GetEnv: version not supported" << std::endl;
}
JNIEnvironment::instance.drop(jni);
std::cout << "JNI_OnUnload() called" << std::endl;
//std::cout << "JNI_OnUnload() called" << std::endl;
//TODO DetachCurrentThread
//vm->DetachCurrentThread();
}
......
......@@ -536,8 +536,8 @@ namespace FS {
Directory Directory::canonical() const
{
// if (!is_absolute())
// throw_system_error("Directory.canonical() only supported on absolute paths.");
//if (!is_absolute())
// throw_system_error("Directory.canonical() only supported on absolute paths.");
char actualpath[PATH_MAX];
if (realpath(path.c_str(), actualpath) != NULL)
......
......@@ -12,7 +12,7 @@
<groupId>montisim</groupId>
<artifactId>hardware_emulator</artifactId>
<version>1.0.3</version>
<version>1.0.4</version>
<properties>
......
......@@ -15,9 +15,10 @@ import de.rwth.monticore.EmbeddedMontiArc.simulators.hardware_emulator.interface
public class SoftwareSimulatorImpl implements SoftwareSimulator {
int id;
int id = -2;
public SoftwareSimulatorImpl(ControllerConfig config) throws Exception {
this.id = CppBridge.allocSimulator(config.get_config_string());
System.out.println("Allocated SoftwareSimulator, id="+this.id);
}
......@@ -43,6 +44,10 @@ public class SoftwareSimulatorImpl implements SoftwareSimulator {
}
@Override
public void free() throws Exception {
CppBridge.freeSimulator(id);
if (this.id < 0)
throw new Exception("Already freed SoftwareSimulator with id: "+this.id);
CppBridge.freeSimulator(this.id);
System.out.println("Freed SoftwareSimulator, id="+this.id);
this.id = -1;
}
}
......@@ -65,6 +65,32 @@ public class ControllerConfig {
return value;
}
}
public static class CacheOption {
private long size; //Cache Size in bytes
private long read_ticks; // Number of CPU cycles for a read action
private long write_ticks; //Number of CPU cycles for a write action
public CacheOption(long size, long read_ticks, long write_ticks){
this.size = size;
this.read_ticks = read_ticks;
this.write_ticks = write_ticks;
}
public String get_config_string(){
return "=" + size + "," + read_ticks + "," + write_ticks + "\n";
}
public long getSize(){
return size;
}
public long getReadSpeedCpuCycles(){
return read_ticks;
}
public long getWriteSpeedCpuCycles(){
return write_ticks;
}
}
private String autopilot;
private EmulatorType emulator_type;
......@@ -156,21 +182,6 @@ public class ControllerConfig {
return this;
}
public static class CacheOption {
private long size; //Cache Size in bytes
private long read_ticks; // Number of CPU cycles for a read action
private long write_ticks; //Number of CPU cycles for a write action
public CacheOption(long size, long read_ticks, long write_ticks){
this.size = size;
this.read_ticks = read_ticks;
this.write_ticks = write_ticks;
}
public String get_config_string(){
return "=" + size + "," + read_ticks + "," + write_ticks + "\n";
}
}
//cache_
public ControllerConfig set_cache_DL1(CacheOption cache) {
......@@ -224,7 +235,7 @@ public class ControllerConfig {
if (emulator_type == EmulatorType.HARDWARE_EMULATOR){
res += os.get_config_entry();
if (test_real){
res += "test_real\n";
res += "test_real";
if (test_real_required){
res += "=required\n";
} else{
......@@ -256,4 +267,44 @@ public class ControllerConfig {
return res;
}
public String getSoftwareName(){
return autopilot;
}
public EmulatorType getSoftwareSimulatorType(){
return emulator_type;
}
public TimeModel getTimeModel(){
return time_model;
}
public Duration getExecutionTime(){
return execution_time;
}
public OS getOS(){
return os;
}
public boolean getTestReal(){
return test_real;
}
public boolean getTestRealRequired(){
return test_real_required;
}
public long getCpuFrequencyHertz(){
return cpu_frequency_hertz;
}
public long getRAMFrequencyHertz(){
return memory_frequency_hertz;
}
public CacheOption getCacheIL1Options(){
return IL1_cache;
}
public CacheOption getCacheDL1Options(){
return DL1_cache;
}
public CacheOption getCacheL2Options(){
return L2_cache;
}
public CacheOption getCacheL3Options(){
return L3_cache;
}
}
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment