add more sections authored by Sven Michael Lechner's avatar Sven Michael Lechner
Adding new methods and types to the Protobuf RPC layer requires multiple steps:
# Modify protobuf files
First off, you have to modify the protobuf files under `edml/proto`. You can add new fundamental types to `datastructures.proto`. To add new gRPC methods to devices, you can modify `connection.proto`.
# Update Python bindings
```console
python3 -m grpc_tools.protoc -I./edml/proto --python_out=./edml/generated --pyi_out=./edml/generated --grpc_python_out=./edml/generated ./edml/proto/{connection.proto, datastructures.proto}
```
# Embed new method calls into code base
This step requires multiple additions in multiple files. `gRPC` expects something called a `Servicer` to implement the initial hand-off of the `gRPC <-> Python` layer. The method names are the exact same ones that you defined in your protobuf files. They receive three arguments: `self`, `request` and `context`. The code base does not need any of them; they are all unused. Once your method is called, you will have to dispatch the call to your real device implementation. For that, input and output parameters have to be converted from their `gRPC` types to Python types. It's best to take a look at the existing implementation located [here](https://git.rwth-aachen.de/inda_ml/swarm-split-learning/-/blob/main/edml/core/device.py?ref_type=4b5ab71016913511f16dc21bdae33474414f2f76#L449-633).
The method you will dispatch also needs to be added. We add these methods to two places: the abstract base class for devices called [`Device`](https://git.rwth-aachen.de/inda_ml/swarm-split-learning/-/blob/main/edml/core/device.py?ref_type=4b5ab71016913511f16dc21bdae33474414f2f76#L77), and the [concrete implementation](https://git.rwth-aachen.de/inda_ml/swarm-split-learning/-/blob/main/edml/core/device.py?ref_type=4b5ab71016913511f16dc21bdae33474414f2f76#L226). Almost all of these methods follow the same pattern: if the current device is the one that the call was meant to reach, execute it locally. If not, we ask the `gRPC` layer via the `self.request_dispatcher` to forward the RPC call to the correct device.