We use TI-RPC / rpcgen to create RPC routines that execute driver API calls in a daemon process (cricketd) rather than the host application itself. This way the driver has no chance to map garbage into the host application. The statically linked runtime API can be used without any modifications and calls driver API functions that are intercepted using LD_PRELOAD / libcudawrap.so
I only hope the runtime API can be checkpointed without modifications.
-
First RPC tests -
Reverse engineer hidden functions (evil cuGetExportTable) -
Initialize driver using RPC calls -
Figure out how to start a kernel located in a different binary -
Implement remaining RPC calls -
Test checkpointing of isolated host app -
Integrate with cricket proper -
clean up -
Enable remote execution (Have cricketd run on a different node than the host application)