- The new scheduler will create a dedicated thread pool
- Work will be prioritized based on their weight and new items will have
highest priority. The weight will be determined by the size of the
subscription enqueueable.
- The work queue will be sorted and the weights updated by a dedicated
thread
- There will be a maximum weight value that will 'disable' a work item
until their weight comes down
- Consumer will not know about workers or anything alike anymore.
- Applying a general max work queue size of 400 items, this will reduce
CPU and RAM usage. Can be set by config.
- Adding `CancellationTokenSource` to fix a race condition where the worker could stop
after the consumer had already checked on him, see TriggerProducer,
leaving the consumer waiting for ever GH issue 3885. This cancellation token
works as a flag for this particular case waking up the consumer.
- Adding unit tests
- Deleting `ParallelRunnerController` and `ParallelRunnerWorker`.
Replacing them for `Tasks`. The consumer, `EnqueueableEnumerator` will
directly spin up a new producer Task when he is running low on items.
With the enumerator in blocking mode (backtesting), MoveNext was returning true while Current was being set to null.
This was causing a NullReferenceException in SubscriptionSynchronizer.Sync.
This was causing unnecessary data processing with algorithms using universe selection, so we should expect some improvements in speed and memory usage.
Another less common issue fixed in this PR is unexpected price spikes on symbols with splits.
This fix only affects backtesting, no change in live trading.
Martin Molinero