Protocol.md 8.91 KB
Newer Older
1
# Protocol {#controller-protocol}
2

Steffen Vogel's avatar
Steffen Vogel committed
3
This page documents the data model/protocol used by VILLAScontroller to control and monitor infrastructure components (IC).
Steffen Vogel's avatar
Steffen Vogel committed
4
The protocol uses AMQP to transport JSON encoded objects which are specified in the following document.
5

6
VILLAScontroller is implemented in Python and using the [Kombo messaging package](https://kombu.readthedocs.io).
7

Steffen Vogel's avatar
Steffen Vogel committed
8
The Git repository is available at: http://git.rwth-aachen.de/acs/public/villas/controller
9

Steffen Vogel's avatar
Steffen Vogel committed
10
11
The puporse of VILLAScontroller is the orchestration of IC in distributed lab setups.
This includes the following tasks:
Steffen Vogel's avatar
Steffen Vogel committed
12

Steffen Vogel's avatar
Steffen Vogel committed
13
14
15
16
- managment: instantiation, deletion & discovery
- monitoring: status?
- control: start, stop, pause, reset, resume, shutdown

Steffen Vogel's avatar
Steffen Vogel committed
17
18
19
20
21
22
23
24
25
26
## Entities {#controller-protocol-entities}

For the purpose of addressing the ICs, we introduce the following categories (1st level) and types (2nd level): 

- `simulator`
  - `dpsim`
  - `generic`
  - `rtlab`
  - `rtds`
  - `dummy`
27
  - `kubernetes` (a [Kubernetes Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/))
Steffen Vogel's avatar
Steffen Vogel committed
28
- `gateway`
29
30
  - `villas-node`
  - `villas-relay`
Steffen Vogel's avatar
Steffen Vogel committed
31
- `manager`
32
33
34
  - `villas-node`
  - `villas-relay`
  - `generic`
Steffen Vogel's avatar
Steffen Vogel committed
35
36
37
  - `kubernetes`

These component types are implemented by the following class hierachy in VILLAScontroller's Python code:
Steffen Vogel's avatar
Steffen Vogel committed
38

Steffen Vogel's avatar
Steffen Vogel committed
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
```mermaid
classDiagram
      Component <-- Manager
      Component <-- Gateway
      Component <-- Simulator
      Simulator <-- DPsimSimulator
      Simulator <-- RTlabSimulator
      Simulator <-- DummySimulator
      Simulator <-- DummySimulator
      Simulator <-- KubernetesJob
      Manager <-- GenericManager
      Manager <-- VILLASnodeManager
      Manager <-- VILLASrelayManager
      Manager <-- KubernetesManager
      Gateway <-- VILLASnodeInterface
      Gateway <-- VILLASrelaySession
```
Steffen Vogel's avatar
Steffen Vogel committed
56

57
## Exchange {#controller-protocol-amqp-exchange}
Steffen Vogel's avatar
Steffen Vogel committed
58

Steffen Vogel's avatar
Steffen Vogel committed
59
All messages processed by VILLAScontroller are sent via an AMQP `headers` exchange.
Steffen Vogel's avatar
Steffen Vogel committed
60

61
The exchange is named `villas`.
Steffen Vogel's avatar
Steffen Vogel committed
62

63
## Routing {#controller-protocol-amqp-routing}
64

65
The following headers are used to identify and route messages to the receipients:
66

67
68
69
70
71
- `realm` describes the entity which is responsible for operating the equipment.
  The realm should be a fully quallified domain name (FQDN) in reverse order.
  - Examples:
    - `de.rwth-aachen.eonerc.acs`
    - `gov.inl`
72

73
74
75
- `category` describes the classes of equipment.
  - Examples:
    - `simulator`
Steffen Vogel's avatar
Steffen Vogel committed
76
	- `manager` (manages ICs, e.g. Kubernetes Jobs, VILLAS WebSocket endpoints)
Steffen Vogel's avatar
Steffen Vogel committed
77
    - `gateway` (e.g. VILLASnode, e.g. VILLASrelay)
78

79
- `type` further defines the type/vendor of a device within its class.
Steffen Vogel's avatar
Steffen Vogel committed
80
  - For `simulator`:
Steffen Vogel's avatar
Steffen Vogel committed
81
82
83
    - `dummy` (_implemented_)
    - `generic` (_implemented_)
    - `dpsim` (_WIP_)
84
85
    - `rtlab` (_planned_)
    - `rscad` (_planned_)
Steffen Vogel's avatar
Steffen Vogel committed
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
  - For `gateway`:
    - `villas-node` (_implemented_)
    - `villas-relay` (_implemented_)
  - For `controller`:
    - `kubernetes`
    - `villas-controller`
  - For `service`:
	- `ems`
	- `custom`?
  - For `equipment`:
	- `chroma-emulator`
	- `chroma-loads`
	- `sma-sunnyboy`
	- `fleps`
	- `sonnenbatterie`
Steffen Vogel's avatar
Steffen Vogel committed
101

102
103
104
105
- `uuid` is a globally unique identifier
  - Examples:
    - `56babc1e-0476-11e8-9375-471525328a77`

Steffen Vogel's avatar
Steffen Vogel committed
106
Example of an application header:
Steffen Vogel's avatar
Steffen Vogel committed
107
108
109

```json
{
110
111
112
113
	"realm" : "de.rwth-aachen.eonerc.acs",
	"category" : "simulator",
	"type" : "rtlab",
	"uuid" : "56babc1e-0476-11e8-9375-471525328a77"
Steffen Vogel's avatar
Steffen Vogel committed
114
115
116
}
```

Steffen Vogel's avatar
Steffen Vogel committed
117
118
119
120
**Important:** Note that at least one of the headers must be provided. Otherwise no component will receive it.

It is also valid to just provide a single header (e.g. `uuid` to address a specific component or `type` to adress a class of components).

Steffen Vogel's avatar
Steffen Vogel committed
121
122
## State Machine

123
124
125
126
127
128
129
130
131
132
133
134
@htmlonly
<div class="mermaid">
stateDiagram-v2
    [*] --> State1
    state State1 {
      [*] --> idle
      idle --> starting: start
      starting --> running
      running --> stopping: stop
      running --> pausing: pause
      pausing --> paused
      paused --> stopping: stop
Iris Marie Köster's avatar
Iris Marie Köster committed
135
      stopping --> idle
136
137
138
139
140
141
142
143
144
145
146
      paused --> resuming: resume
      resuming --> running
    }
    State1 --> resetting: reset
    resetting --> State1
    State1 --> shuttingdown: shutdown
    shuttingdown --> [*]
    State1 --> error: exception
    error --> resetting: reset
</div>
@endhtmlonly
Steffen Vogel's avatar
Steffen Vogel committed
147
148
149
150

## Management

### Discovery
151

Steffen Vogel's avatar
Steffen Vogel committed
152
153
154
In a messaging protocol like AMQP, brokers do not store message contents in a persistant way.
They only temporarily queue messages until they have been delivered.
Therefore, the broker cannot keep track of the state or presence of entitities.
155
156
157
158

This section describes a simple discovery mechanism to work around this limitation.
Each entity must react on `action = ping` messages and respond immediatly with a status response message.

Steffen Vogel's avatar
Steffen Vogel committed
159
160
```json
{
161
	"action" : "ping"
Steffen Vogel's avatar
Steffen Vogel committed
162
163
164
}
```

Steffen Vogel's avatar
Steffen Vogel committed
165
After receiving such a message, each entity shall send a status update message as decribed below.
Steffen Vogel's avatar
Steffen Vogel committed
166

Steffen Vogel's avatar
Steffen Vogel committed
167
### Status Update
168

169
170
171
```json
{
	"status" : {
Steffen Vogel's avatar
Steffen Vogel committed
172
173
		/* Generic state */
		"state" : "running",
Steffen Vogel's avatar
Steffen Vogel committed
174
175
		"version" : "0.1.0",                 /* VILLAScontroller version */
		"uptime" : 123124.0,                 /* in seconds since initialization (float) */
176
177
178
179
180

		"error": {                           /* only present if state == "error" */
			"message": "This is an error message",
			"code": 1234
		},
Steffen Vogel's avatar
Steffen Vogel committed
181
182

		"managed_by": "4af1cf3a-7e80-11eb-a7f4-b700f4a4f18d", /* link to managing IC of category 'manager' */
Steffen Vogel's avatar
Steffen Vogel committed
183
184
185
186
187
188
189
190
191

		/* Custom state (depending on IC type) */
		"villas_node_version": "v0.11.0",
		"kernel": ["Linux", "villas-node-594f8c7bff-wmznc", "5.0.0-29-generic", "#31~18.04.1-Ubuntu SMP Thu Sep 12 18:29:21 UTC 2019", "x86_64"],
		"host": "villas-node-594f8c7bff-wmznc"
	},
	"properties": { /* fixed configuration of IC as provided by user */

		/* Generic properties */
192
193
194
195
		"name" : "OP5600",
		"description" : "some optional description",
		"location" : "OPAL-RT Rack, ACS Real-time Lab, EONERC, RWTH",
		"owner" : "stvogel@eonerc.rwth-aachen.de",
Steffen Vogel's avatar
Steffen Vogel committed
196

Steffen Vogel's avatar
Steffen Vogel committed
197
		"ws_url" : "https://villas-new.k8s.eonerc.rwth-aachen.de/ws/relay/node_1",
Steffen Vogel's avatar
Steffen Vogel committed
198
199
200
201
		"api_url" : "https://villas-new.k8s.eonerc.rwth-aachen.de/ws/relay/api/node_1",

		/* Custom properties (depending on IC type) */
		"opal_hostname": "opal-op5600.acs-lab.eonerc.rwth-aachen.de"
202
203
		"namespace": "villas"
		"job_name": "My_Kubernetes_Simulator"
204
		"pod_names": ["pod_number_1", "pod_number_2"]
205
	},
206
207
208
209
210
211
212
	"schema": {
		"start": {
			/* JSON-Schma describing start parameters. */
		},
		"create": {
			/* JSON-Schma describing properties of new ICs created via create action (targeted at ICs of category=manager). */
		}
Steffen Vogel's avatar
Steffen Vogel committed
213
	},
Steffen Vogel's avatar
Steffen Vogel committed
214
	"when" : 1234567890 /* timestamp in seconds (UTC / Unix epoch / since 1970-01-01) (float) */
Steffen Vogel's avatar
Steffen Vogel committed
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
}
```

Used values for `state`:

- `gone` (issued once after `action = delete`)
- `error`
- `idle`
- `starting`
- `running`
- `pausing`
- `paused`
- `resuming`
- `stopping`
- `resetting`
- `shuttingdown`
- `shutdown`

## Management

Steffen Vogel's avatar
Steffen Vogel committed
235
236
237
### Creation of new components

**Important:** This action type must be directed at a component of category `manager`!
Steffen Vogel's avatar
Steffen Vogel committed
238
239
240
241

```json
{
	"action": "create",
242
	"when": 1234567890,
Steffen Vogel's avatar
Steffen Vogel committed
243
	"parameters": {
Steffen Vogel's avatar
Steffen Vogel committed
244
245
246
247
248
		"name": "DPsim simulator",
		"location": "ACS lab",
		"realm": "de.rwth-aachen.eonerc.acs",
		"category": "simulator",
		"type": "dpsim"
Steffen Vogel's avatar
Steffen Vogel committed
249
250
251
252
		"uuid": "c487a470-6af6-11eb-beee-7fa268050404",
		"properties": {
			"myproperty": 1234
		}
Steffen Vogel's avatar
Steffen Vogel committed
253
	}
254
255
256
}
```

Steffen Vogel's avatar
Steffen Vogel committed
257
258
259
260
261
The `uuid` parameter is optional. If not provided, the managing component will generate a random one.

### Deletion of components

**Important:** This action type must be directed at a component of category `manager`!
Steffen Vogel's avatar
Steffen Vogel committed
262
263
264
265

```json
{
	"action": "delete",
Steffen Vogel's avatar
Steffen Vogel committed
266
267
268
	"parameters": {
		"uuid": "c487a470-6af6-11eb-beee-7fa268050404"
	}
269
	"when": 1234567890,
Steffen Vogel's avatar
Steffen Vogel committed
270
271
272
273
274
275
}
```

## Control

### Reset
276
277
278

```json
{
Steffen Vogel's avatar
Steffen Vogel committed
279
	"action" : "reset",
280
	"when" : 1234567890
281
282
283
}
```

Steffen Vogel's avatar
Steffen Vogel committed
284
### Shutdown
Steffen Vogel's avatar
Steffen Vogel committed
285

286
287
```json
{
Steffen Vogel's avatar
Steffen Vogel committed
288
	"action" : "shutdown",
289
	"when" : 1234567890
290
291
292
}
```

Steffen Vogel's avatar
Steffen Vogel committed
293
### Start
Steffen Vogel's avatar
Steffen Vogel committed
294
295

Start the simulation at a specific point in time.
296
297
298

```json
{
Steffen Vogel's avatar
Steffen Vogel committed
299
300
301
302
303
304
	"action" : "start",
	"parameters" : {
	    "timestep" : 50e-6,
	    "duration" : 10,
	    "domain" : "dp",
	    "downsample" : 1
Steffen Vogel's avatar
Steffen Vogel committed
305
	},
306
	"model" : { // HTTP GET for model download
Steffen Vogel's avatar
Steffen Vogel committed
307
308
309
310
311
		"type": "url-list",
		"url" : [
			"https://villas.k8s.eonerc.rwth-aachen.de/api/v2/files/123421",
			"https://villas.k8s.eonerc.rwth-aachen.de/api/v2/files/123422"
		],
312
		"token": "Xwlsjkdlfksjhdfglskdfg"
Steffen Vogel's avatar
Steffen Vogel committed
313
	},
314
315
316
317
	"results" : { // HTTP POST for results upload
	    "type" : "url"
	    "url" : "https://villas.k8s.eonerc.rwth-aachen.de/api/v2/runs/3452345234/results"
		"token": "Xwlsjkdlfksjhdfglskdfg"
Steffen Vogel's avatar
Steffen Vogel committed
318
	},
319
	"when" : 1234567890
320
321
322
}
```

Steffen Vogel's avatar
Steffen Vogel committed
323
324
325
326
327
328
329
330
331
332
333
334
335
336
Another example for `type = generic`:

```json
{
	"action" : "start",
    "parameters" : {
        "executable" : "ping",
        "argv" : [
            "google.de",
            "-c", "10"
        ],
        "working_directory" : "/tmp/simulationX",
        "shell" : false,
        "environemnt" : [
Steffen Vogel's avatar
Steffen Vogel committed
337
            "DEBUG" : "1"
Steffen Vogel's avatar
Steffen Vogel committed
338
        ]
Steffen Vogel's avatar
Steffen Vogel committed
339
    },
340
	"when" : 1234567890
Steffen Vogel's avatar
Steffen Vogel committed
341
342
343
}
```

Steffen Vogel's avatar
Steffen Vogel committed
344
### Stop
345

Steffen Vogel's avatar
Steffen Vogel committed
346
Stop the simulation at a specific point in time.
347
348
349

```json
{
Steffen Vogel's avatar
Steffen Vogel committed
350
	"action" : "stop",
351
	"when" : 1234567890
352
353
354
}
```

Steffen Vogel's avatar
Steffen Vogel committed
355
### Pause
Steffen Vogel's avatar
Steffen Vogel committed
356
357

Pause the simulation at a specific point in time.
358
359
360

```json
{
Steffen Vogel's avatar
Steffen Vogel committed
361
	"action" : "pause",
362
	"when" : 1234567890
363
364
365
}
```

Steffen Vogel's avatar
Steffen Vogel committed
366
### Resume
Steffen Vogel's avatar
Steffen Vogel committed
367
368

Resume the simulation at a specific point in time.
369
370
371

```json
{
Steffen Vogel's avatar
Steffen Vogel committed
372
	"action" : "resume",
373
	"when" : 1234567890
374
375
}
```