Commit 9a16b69a authored by Jean Meurice's avatar Jean Meurice

PowerTrain merge

parents d21b5ce6 af41c827
Pipeline #189833 failed with stages
in 24 minutes and 1 second
...@@ -12,24 +12,12 @@ stages: ...@@ -12,24 +12,12 @@ stages:
- windows - windows
- linux - linux
#masterJobLinux:
# stage: linux
# image: maven:3-jdk-8
# before_script:
# - apt-get update
# - apt-get install -y openjfx
# script:
# - mvn -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn -B clean deploy --settings settings.xml
# - cat target/site/jacoco/index.html
# - mvn package sonar:sonar -s settings.xml
# only:
# - master
masterJobWindowsRpc: masterJobWindowsRpc:
stage: windows stage: windows
script: script:
- cd rpc - cd rpc
- mvn -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn -B clean deploy --settings ..\settings.xml - mvn -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn -B clean deploy --settings ..\settings.xml
- mvn package sonar:sonar --settings ..\settings.xml -DskipTests
tags: tags:
- Windows10 - Windows10
only: only:
...@@ -41,6 +29,7 @@ masterJobWindowsRestful: ...@@ -41,6 +29,7 @@ masterJobWindowsRestful:
- cd restful - cd restful
# TODO: we have to skip tests for now, because most tests require a running instance of zookeeper # TODO: we have to skip tests for now, because most tests require a running instance of zookeeper
- mvn -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn -B clean deploy --settings ..\settings.xml -DskipTests - mvn -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn -B clean deploy --settings ..\settings.xml -DskipTests
- mvn package sonar:sonar --settings ..\settings.xml -DskipTests
tags: tags:
- Windows10 - Windows10
only: only:
...@@ -67,22 +56,32 @@ branchJobWindowsRestful: ...@@ -67,22 +56,32 @@ branchJobWindowsRestful:
except: except:
- master - master
#BranchJobLinux: # run tests
# tags: branchJobLinux:
# - docker tags:
# stage: linux - docker
# image: maven:3-jdk-8 stage: linux
# before_script: script:
# - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - cd docker
# script: - ./build.sh
## - apt-get update - ./run_test.sh
## - apt-get install -y openjfx except:
# - cd restful - master
# - ./run_test.sh
## - docker tag rmi-server:latest $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA # build, test and publish docker images
## - docker tag rmi-server:latest $CI_REGISTRY_IMAGE:latest masterJobLinux:
## - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA tags:
## - docker push $CI_REGISTRY_IMAGE:latest - docker
# # - mvn -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn -B clean install --settings settings.xml stage: linux
# except: before_script:
# - master - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- cd docker
- ./build.sh
- ./run_test.sh
- docker tag rpc-server:latest $CI_REGISTRY_IMAGE:rpc
- docker tag restful-server:latest $CI_REGISTRY_IMAGE:restful
- docker push $CI_REGISTRY_IMAGE:rpc
- docker push $CI_REGISTRY_IMAGE:restful
only:
- master
...@@ -4,6 +4,47 @@ ...@@ -4,6 +4,47 @@
Server repository contains a project binding all modules for running the _simulator_ together with _visualization_ using client/server architecture. The project is based on _SmartFoxServer_ development kit and therefore has to be deployed on running _SmartFoxServer 2X_ server. To provide accessible web application, the _visualization_ project has to be built to _.war_ file and then deployed as a web application on the embedded in SFS _Jetty_ server. Server repository contains a project binding all modules for running the _simulator_ together with _visualization_ using client/server architecture. The project is based on _SmartFoxServer_ development kit and therefore has to be deployed on running _SmartFoxServer 2X_ server. To provide accessible web application, the _visualization_ project has to be built to _.war_ file and then deployed as a web application on the embedded in SFS _Jetty_ server.
# Server 2.0
## Install
```bash
$ mvn clean install -DskipTests -settings=../settings.xml
```
After installation, the executables can be found in restful/target and rpc/target directory.
To use them, please refer to corresponding instructions:
[rpc](https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/simulators/server/blob/master/rpc/README.md)
and
[restful](https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/simulators/server/blob/master/restful/README.md)
## Docker(optional)
Build the docker images:
```bash
$ cd docker
$ ./build.sh
```
Test:
```bash
$ cd docker
$ ./run_test.sh
```
README of [restful](https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/simulators/server/blob/master/restful/README.md)
includes instructions of how to use docker to run this project.
## Just have a look
If you don't want to compile the whole project and just want to have a taste of this project,
you can simply run:
```bash
$ cd docker
$ docker-compose up
```
This will start a pure backend application. Now you can use the
[RESTful API](https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/simulators/server/blob/master/restful/README.md)
to play with the simulation.
API doc can be found at [127.0.0.1:8090/swagger-ui.html](127.0.0.1:8090/swagger-ui.html)
To try it with visualization, download
[visualization project](https://git.rwth-aachen.de/monticore/EmbeddedMontiArc/simulators/visualization),
uncomment the `nginx` part of `docker/docker-compose.yml`, comment or remove the `ports` field of `server`.
# Portable version # Portable version
A portable version for Windows of the SmartFoxServer based simulator together with a portable PostgreSQL database is available as [archive](https://github.com/MontiSim/server/releases/download/portable/distributed-simulator.zip). Running the portable version can be done by running the `start-script.bat` file, provided in the archive. A portable version for Windows of the SmartFoxServer based simulator together with a portable PostgreSQL database is available as [archive](https://github.com/MontiSim/server/releases/download/portable/distributed-simulator.zip). Running the portable version can be done by running the `start-script.bat` file, provided in the archive.
......
FROM maven:3-jdk-8 AS montisim-server-builder
WORKDIR /app
COPY . .
RUN apt-get update && \
apt-get install -y --no-install-recommends metis && \
rm -rf /var/lib/apt/lists/* && \
mvn install -s settings.xml -DskipTests && \
cp rpc/target/rpc-*-jar-with-dependencies.jar rpc/target/rpc.jar && \
find restful/target -regex '.*restful-[0-9.]+\.jar' | \
xargs -i cp {} restful/target/restful.jar
#!/usr/bin/env bash
CURR_DIR=$(pwd)
cd ..
docker build -f "${CURR_DIR}/Dockerfile" -t montisim-server-builder .
cd rpc
./build.sh
cd ../restful
./build.sh
...@@ -9,45 +9,59 @@ version: "3" ...@@ -9,45 +9,59 @@ version: "3"
services: services:
zoo1: zoo1:
image: zookeeper image: zookeeper:3.4
restart: always restart: always
hostname: zoo1 hostname: zoo1
ports:
- 2181:2181
environment: environment:
ZOO_MY_ID: 1 ZOO_MY_ID: 1
ZOO_SERVERS: server.1=0.0.0.0:2888:3888 ZOO_SERVERS: server.1=0.0.0.0:2888:3888
web:
image: montisim-server
ports:
- 5001:5001
depends_on:
- rmi-server
- rpc-server
volumes:
- ../research/web:/app
rmi-server: rmi-server:
image: rmi-server:latest image: registry.git.rwth-aachen.de/monticore/embeddedmontiarc/simulators/rmimodelserver:latest
depends_on: depends_on:
- zoo1 - zoo1
# ports:
# - 10101:10101
restart: always restart: always
# hostname: rmi-server
environment: environment:
ZOO_SERVERS: zoo1 ZOO_SERVERS: zoo1:2181
entrypoint:
- java
- -cp
- rmi-model-server.jar
- -Djava.rmi.server.codebase=file:rmi-model-server.jar
- -Djava.rmi.server.useLocalHostname
- rwth.rmi.model.server.RMIServer
- "10101"
- autopilots
- no_time
- os=linux
rpc-server: rpc-server:
image: rpc-server:latest image: registry.git.rwth-aachen.de/monticore/embeddedmontiarc/simulators/server:rpc
depends_on: depends_on:
- zoo1 - zoo1
# ports:
# - 6000:6000
# - 8787:8787
restart: always restart: always
environment: environment:
ZOO_SERVERS: zoo1 ZOO_SERVERS: zoo1:2181
volumes: # uncomment below to enable multi-sector trajectory re-planning
- ../rpc/target:/app # SIM_SERVER: server
# SIM_PORT: 8090
server:
image: registry.git.rwth-aachen.de/monticore/embeddedmontiarc/simulators/server:restful
depends_on:
- zoo1
ports:
- 8090:8090
restart: always
environment:
ZOO_SERVERS: zoo1:2181
# nginx:
# image: nginx:1.17
# ports:
# - 8090:8090
# restart: always
# volumes:
# - ./nginx.conf:/etc/nginx/nginx.conf:ro
# # edit line below to use visualization
# - /path/to/visualization/src/main/webapp:/app
#user nobody;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
client_max_body_size 50m;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
gzip on;
# montisim-server-restful backend
server {
listen 8090;
server_name 127.0.0.1;
root /app;
add_header Access-Control-Allow-Origin * always;
add_header Access-Control-Allow-Methods '*' always;
location / {
proxy_pass http://server:8090;
}
}
# include servers/*;
}
...@@ -32,8 +32,8 @@ SIMULATOR_DIR="${CURR_DIR}/../rpc" ...@@ -32,8 +32,8 @@ SIMULATOR_DIR="${CURR_DIR}/../rpc"
# prepare docker image for rpc-server # prepare docker image for rpc-server
#cd $SIMULATOR_DIR #cd $SIMULATOR_DIR
#mvn clean install -s ../settings.xml -DskipTests # mvn clean install -s ../settings.xml -DskipTests
#./build.sh #../restful/build.sh
# create network # create network
docker network create simulation-network docker network create simulation-network
...@@ -43,6 +43,7 @@ docker run \ ...@@ -43,6 +43,7 @@ docker run \
--network="simulation-network" \ --network="simulation-network" \
-d \ -d \
--name=zookeeper \ --name=zookeeper \
-e "ZOO_MAX_CLIENT_CNXNS=1024" \
zookeeper:latest zookeeper:latest
# start autopilot server(RMIServer) # start autopilot server(RMIServer)
...@@ -53,7 +54,7 @@ docker run \ ...@@ -53,7 +54,7 @@ docker run \
-e "ZOO_SERVERS=zookeeper" \ -e "ZOO_SERVERS=zookeeper" \
registry.git.rwth-aachen.de/monticore/embeddedmontiarc/simulators/rmimodelserver:latest \ registry.git.rwth-aachen.de/monticore/embeddedmontiarc/simulators/rmimodelserver:latest \
java -cp rmi-model-server.jar -Djava.rmi.server.codebase=file:rmi-model-server.jar \ java -cp rmi-model-server.jar -Djava.rmi.server.codebase=file:rmi-model-server.jar \
-Djava.rmi.server.hostname=localhost rwth.rmi.model.server.RMIServer 10101 autopilots no_time os_linux -Djava.rmi.server.useLocalHostname rwth.rmi.model.server.RMIServer 10101 autopilots no_time os_linux
docker run \ docker run \
--network="simulation-network" \ --network="simulation-network" \
...@@ -62,19 +63,19 @@ docker run \ ...@@ -62,19 +63,19 @@ docker run \
--name=rpc-server \ --name=rpc-server \
rpc-server:latest rpc-server:latest
cd $CURR_DIR cd ${CURR_DIR}
docker build -t server-test-runner -f TestDockerfile .. #docker build -t server-test-runner -f ./docker/TestDockerfile .
docker run --rm --network="simulation-network" -e "ZOO_SERVERS=zookeeper" server-test-runner sh -c "cd restful && mvn test -s ../settings.xml" docker run \
--rm --network="simulation-network" \
# build simulator -e "ZOO_SERVERS=zookeeper" \
#cd $SIMULATOR_DIR montisim-server-builder sh -c "cd restful && mvn test -s ../settings.xml" \
#mvn install -s ../settings.xml -DskipTests 2>&1 | tee result.txt
#cd "${SIMULATOR_DIR}/target" dockerCleanUp
# start rpc-server(remote simulator)
#java -cp "rpc-1.0.0-SNAPSHOT-jar-with-dependencies.jar:." server.rpc.RpcServer 6000 > /dev/null &
#cd $CURR_DIR tail -10 result.txt | grep "BUILD SUCCESS"
#mvn test -s ../settings.xml # code=0 if "BUILD SUCCESS" is found in the end of result.txt
code=$?
echo $code
exit $code
dockerCleanUp
#!/usr/bin/env bash
# This script re-compile all *.fmu files in the current directory using
# current available c++ compiler. This makes the fmu files compatible with
# the host OS.
# For example if you have .fmu files exported from ModelicaEditor running
# in Windows but you need linux version. To make these .fmu files compatible
# with linux, you should:
# 1. import and re-export the Windows version .fmu files using linux ModelicaEditor
# 2. copy this script in the same directory where the re-exported .fmu files are located
# 3. run this script with ./recompile_fmu.sh
# 4. all .fmu files should be re-compiled and compatible with linux
tmp_dir=./unzip_dir
function compile() {
filename="$1"
unzip $filename -d $tmp_dir
cd ${tmp_dir}/sources && ./configure && make
cd ../..
zip -r $filename $tmp_dir
rm -rf $tmp_dir
echo "$filename"
}
for f in ./*.fmu; do
compile "$f"
done
#!/usr/bin/env bash
# This script re-compile all *.fmu files in the current directory using
# current available c++ compiler. This makes the fmu files compatible with
# the host OS.
# For example if you have .fmu files exported from ModelicaEditor running
# in Windows but you need linux version. To make these .fmu files compatible
# with linux, you should:
# 1. import and re-export the Windows version .fmu files using linux ModelicaEditor
# 2. copy this script in the same directory where the re-exported .fmu files are located
# 3. run this script with ./recompile_fmu.sh
# 4. all .fmu files should be re-compiled and compatible with linux
tmp_dir=./unzip_dir
function compile() {
filename="$1"
unzip $filename -d $tmp_dir
cd ${tmp_dir}/sources && ./configure && make
cd ../..
zip -r $filename $tmp_dir
rm -rf $tmp_dir
echo "$filename"
}
for f in ./*.fmu; do
compile "$f"
done
******************************************************************************
MontiCAR Modeling Family, www.se-rwth.de
Copyright (c) 2017, Software Engineering Group at RWTH Aachen,
All rights reserved.
This project is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 3.0 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this project. If not, see <http://www.gnu.org/licenses/>.
*******************************************************************************
This diff is collapsed.
This diff is collapsed.
...@@ -9,9 +9,12 @@ FROM openjdk:8-jdk ...@@ -9,9 +9,12 @@ FROM openjdk:8-jdk
WORKDIR /app WORKDIR /app
RUN apt-get update && apt-get install -y openjfx && apt-get install -y metis && rm -rf /var/lib/apt/lists/* RUN apt-get update && \
apt-get install -y --no-install-recommends metis && \
rm -rf /var/lib/apt/lists/*
COPY ./app.jar ./ COPY --from=montisim-server-builder:latest /app/restful/target/restful.jar ./app.jar
COPY ./app.db ./classes/app.db COPY --from=montisim-server-builder:latest ./app/restful/target/classes/app.db ./classes/app.db
COPY --from=montisim-server-builder:latest ./app/restful/target/classes/default.car ./classes/default.car
CMD java -jar app.jar CMD java -jar app.jar
...@@ -5,11 +5,11 @@ retrieving the results. ...@@ -5,11 +5,11 @@ retrieving the results.
## Install ## Install
Install using maven: Install using maven:
``` ```
$ mvn clean install -settings=../settings.xml $ mvn clean install -settings=../settings.xml -DskipTests
``` ```
Run the server: Run the server:
``` ```
$ java -jar ./target/restful-2.1.0-SNAPSHOT.jar $ java -jar ./target/restful-2.2.0-SNAPSHOT.jar
``` ```
It is also fine to start the server using your IDE, the main class is located at `src/main/java/server/restful/Server.java`. It is also fine to start the server using your IDE, the main class is located at `src/main/java/server/restful/Server.java`.
...@@ -77,7 +77,11 @@ Several options are available in there. ...@@ -77,7 +77,11 @@ Several options are available in there.
## Docker ## Docker
Since the server depends on multiple services, it is a good choice to run them all in docker Since the server depends on multiple services, it is a good choice to run them all in docker
containers. We can start all dependencies with docker at once: containers. First built docker images with `rpc/build.sh` and `restful/build.sh` and then
edit the last line in `dev.yml` to add path to your local visualization files.
You can comment out the nginx service if you don't need visualization,
but in that case, please also uncomment line 49 of `dev.yml`
Finally, start the project with:
``` ```
$ docker-compose -f dev.yml up $ docker-compose -f dev.yml up
``` ```
...@@ -88,12 +92,7 @@ $ docker-compose -f dev.yml up --scale rmi-server=10 rmi-server ...@@ -88,12 +92,7 @@ $ docker-compose -f dev.yml up --scale rmi-server=10 rmi-server
$ docker-compose -f dev.yml up --scale rpc-server=3 rpc-server $ docker-compose -f dev.yml up --scale rpc-server=3 rpc-server
``` ```
To build the docker image, first build the jar with maven and then: ## Developing with Docker
```
$ ./build.sh
```
## Development with Docker
The development always involves debugging, with docker, debugging has to be done via remote debugger. The development always involves debugging, with docker, debugging has to be done via remote debugger.
`dev.yml` already configured the restful server to run in debug mode, the JVM listens for debugging request `dev.yml` already configured the restful server to run in debug mode, the JVM listens for debugging request
at **localhost:8787**. at **localhost:8787**.
...@@ -127,11 +126,6 @@ rpc-server-debugger: ...@@ -127,11 +126,6 @@ rpc-server-debugger:
``` ```
It starts a rpc server that listen on port **8788** for debugging. It starts a rpc server that listen on port **8788** for debugging.
## Test
```
$ ./run_test.sh
```
## Visualization ## Visualization
By default, visualization is not enabled in the server. By default, visualization is not enabled in the server.
To enable visualization, put the static files under `restful/src/main/resources/resources`. To enable visualization, put the static files under `restful/src/main/resources/resources`.
...@@ -141,6 +135,18 @@ copy everything from [webapp directory](https://git.rwth-aachen.de/monticore/Emb ...@@ -141,6 +135,18 @@ copy everything from [webapp directory](https://git.rwth-aachen.de/monticore/Emb
`restful/src/main/resources/resources`. `restful/src/main/resources/resources`.
Start the restful server, the visualization client will be available at [http://127.0.0.1:8090/index.html](http://127.0.0.1:8090/index.html) Start the restful server, the visualization client will be available at [http://127.0.0.1:8090/index.html](http://127.0.0.1:8090/index.html)
It is highly recommended for frontend developers to use reverse proxy to serve
the frontend files. Because otherwise, you need to recompile the restful module
every time you changed something in your code. For example, to use nginx as reverse proxy,
we provided a configuration(`nginx.conf`) for that.
You should
1. update `root /app` to point it to your static files, such as `root /path/to/visualization/src/webapp`
2. update `proxy_pass http://server:8090;` to fit your case. If you started the restful server with `java -jar ...jar` command,
it should be `proxy_pass http://127.0.0.1:8090;`
After you installed nginx, start it with this config file.
If you use docker, you only need to update the last line of `dev.yml` in order to
use nginx.
## Simulation via RESTful API ## Simulation via RESTful API
After the server started, simulation tasks can be configured via the RESTful APIs. After the server started, simulation tasks can be configured via the RESTful APIs.
...@@ -158,3 +164,11 @@ The sequence should be: ...@@ -158,3 +164,11 @@ The sequence should be:
The APIs are defined using OpenAPI2.0 in `openapi.yml`. The APIs are defined using OpenAPI2.0 in `openapi.yml`.
However, the restful server also provides an auto generated API documentation page at However, the restful server also provides an auto generated API documentation page at
[http://127.0.0.1:8090/swagger-ui.html](http://127.0.0.1:8090/swagger-ui.html). [http://127.0.0.1:8090/swagger-ui.html](http://127.0.0.1:8090/swagger-ui.html).
## Tips
### Scenarios and Maps
There are couple of scenarios and maps that you can play with. They are located
in `restful/src/test/resources`. Maps have `.osm` suffix, `.sim` suffix are for scenarios.
### Editing scenario files
`https://www.openstreetmap.org/node/[node id]` will help you with finding source and target coordinates for vehicles easily.
...@@ -7,10 +7,4 @@ ...@@ -7,10 +7,4 @@
# #