Conversation
Everything is real now
It's not additive to mock and can't really be relied on. Real should just be the absence of mock.
|
This is one big PR to test whether the whole concept would work. It can be split up into multiple PRs if necessary. |
|
When checking with RustNN, I saw that requiring
Apart from that the API seems to work for RustNN (although quite some lifetime annotations were needed. |
56e2e4b to
80a91fa
Compare
c7f2dff to
2c1b9c1
Compare
The autocxx docs recommend using
UniquePtrfor C++ owned data as the easiest option. This allows to use TRT methods directly on the objectsPin<&mut nvinfer1::ITensor>/Pin<&mut nvinfer1::ILayer>can't use a UniquePtr because their destructor is protected. We don't need to run the destructor for those objects since the objects since they are owned by INetworkDefintion.I've added lifetime annotations for weights and the objects created by IBuilder->INetworkDefintion->IXXX . This allows to detect some segfault like this one we currently have in the code base
Regarding API safety, we have two choices regarding the network definition API:
!Send, "!Sync"`- they can not be transferred to other threads, even with a Mutex
- API is safe since single-threaded we are only in one of the TRT functions, so mutable refs are exclusive
&mut network/&networkto all layer and tensor APIs:- we could add
!Sendsafely, since all APIs always require either&mut network/&network. The borrow checker would then validate that network either are have multiple&networkor one exclusive&mut network- the objects can be used from multiple threads if
networkis protected by a mutex or transferred to another thread.TODO:
The mock API will also use the same autocxx API. The idea is that real objects would just use non-null UniquePtr while mock API uses
nullptr(uniqueptr would then just panic if an API is not yet mocked)Fixes #26
Fixes #19