Decouple InferenceSession lifecycle from Predict

[keesschollaart81]

As per #8 I would like to propose some (breaking) changes to your Scorer api/package.

After this PR, the InferenceSession is only created once per YoloScorer instance. Creating a session is relative expensive and when doing a lot of inferences I would like to reuse this session.

I've updated the constructor of the YoloScorer to take two parameters.

• The Model, either the filename or the bytes of the model. Both are not used but fed directly into the constructor of the InferenceSession
• The SessionOptions. I need to be able to provide them to the InferenceSession, in my case I would like to be able to configure the OnnxRuntime to use CUDA. This is an optional parameter, defaulting to new SessionOptions().

The YoloScorer now implements IDisposable as it owns the lifecycle of InferenceSession and SessionOptions which are IDisposable.

I've changed the paths from assets to Assets as paths are case-sensitive on Linux.

Let me know if you agree with these changes. Happy to discuss further changes.

[kkarahainko]

@keesschollaart81 thanks for PR, agree with most of changes, but I see the problem here using var sessionOptions = new SessionOptions(); // Or, fe: SessionOptions.MakeSessionOptionWithCudaProvider(()); MakeSessionOptionWithCudaProvider requires Microsoft.ML.OnnxRuntime.Gpu package installed, does it work on your side? The only option I see for now is Yolov5Net-gpu build.

[keesschollaart81]

Good point. This GPU package does (to my knowledge) not have any .NET assemblies. It makes the native library (.dll or .so) available for the desired OS... So, if you need it, the client app can take this dependency themselves. Right?

[kkarahainko]

@keesschollaart81 Microsoft.ML.OnnxRuntime.Managed delivers .NET wrappers, both Microsoft.ML.OnnxRuntime and Microsoft.ML.OnnxRuntime.Gpu deliver native libs (onnxruntime.dll + some others), CPU and GPU can't be installed together because of the same name of main lib - onnxruntime.dll. This is what I've noticed.

[kkarahainko]

@keesschollaart81 I think it's OK to allow client app to install CPU or GPU package, Scorer project will contain only OnnxRuntime.Managed package.