Commit Graph

16 Commits

Author SHA1 Message Date
Colton Sellers 76a53eb096 Local Object Store Refactor and Fixes (#4880)
* Store temp files in subdirectory

* Fix Dispose case for new temp dir

* Adjust tests for new temp dir

* Dispose unit tests

* Unit test for issue 4811

* Refactor for not using temp files

* Fix storage checks for saving data, plus tests

* Use Base64 for storing keys and decoding them; handles odd key strings

* Don't allow "?" in a key

* Address review

* Deleted test cases

* PersistData handle deletion of files

* Refactor GetFilePath to use Persist()

* Make PathForKey protected
2020-10-30 21:36:23 -03:00
Martin-Molinero 79b9009452 ObjectStore delete will delete file (#4816)
- LocalObjectStore.Delete() will also delete file from the local object
  store path if present, this will avoid the issue where restarting the
  object store will re load the same deleted file. Adding unit test.
  Issue https://github.com/QuantConnect/Lean/issues/4811
2020-10-05 20:19:27 -03:00
Colton Sellers b8674731a5 Feature 2456 custom Python consolidator support (#4637)
* DataConsolidator Wrapper for Python Consolidators

* Regression Unit Test

* Refactor Regression test

* Bad test fix

* pre review

* self review

* Add RegisterIndicator for Python Consolidator

* Python base class for consolidators

* Modify regression algo to register indicator

* unit test - attach event

* Test fix

* Fix test python imports

* Add license header file and null check

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-08-25 17:26:55 -03:00
Martin Molinero e217caef47 Adding null checks for failed initializations
- Adding null checks to handle failed algorithm initializations
2020-07-08 10:18:43 -03:00
Martin Molinero 35f849254e Improvements
- Replace bitwise And operation for .HasFlag call
- Minor change in object store permission error message
2020-06-09 20:55:21 -03:00
Martin Molinero 7a7ad8e7e7 Address review: improve error messaging 2020-06-09 19:29:59 -03:00
Martin Molinero e782ffa4c6 Reuse existing FileAccess permissions 2020-06-09 18:13:12 -03:00
Martin Molinero 5bd11ebd9d Add permission control for Storage
- Adding `Controls.StoragePermissions` to govern permissions for storing into
  the object store. Adding unit tests
2020-06-09 17:24:00 -03:00
Martin Molinero f86af05227 Fix for ObjectStore for QuantBook 2020-04-29 21:03:35 -03:00
Jared aa2f875045 Merge pull request #3962 from QuantConnect/bug-3961-remove-ltrh-unnecessary-call-touniversaltime
Remove ToUniversalTime - Fix LocalObjectStore limit
2020-01-02 13:57:17 -08:00
Martin Molinero 24a3c24f55 Remove ToUniversalTime. Fix LocalObjectStore limit
- Removing unnecessary call to `ToUniversalTime()` in the LTRH
- Fix bug in the `LocalObjectStore` limit check. Adding unit test
2020-01-02 15:33:41 -03:00
Michael Handschuh 39935552a3 Convert ObjectStore ext class to implement IObjectStore for API
Since extension methods don't play well with pythonnet, this change converts
the extensions class into a decorator class. Additionally, this ObjectStore
type is the type that gets exposed via QCAlgorithm so users can access these
methods directly without requiring the use of extension methods.

This approach has many good properties. For one, it doesn't force implementors
of IObjectStore to use a base class. Second, it maintains healthy separation of
API level concerns (such as convenient methods) from the  abstraction level conerns
of IObjectStore. Setting it up in this way ensures ANY implementation of IObjectStore
will still get access to these additional methods. Another thing to note is this
prevents using a base class on QCAlgorithm's public interface. Instead, we have a
specific type that is dedicated to fulfilling API level requirements, which also
provides us flexibility in the event the API needs to be updated. If it were a subclass,
you run the risk of breaking the implementors of the subclass.
2019-12-31 15:55:16 -05:00
Jared 683250ffd3 Flip order of raising event and log to ensure log.
Prevent exception in error handler dropping the log.
2019-12-18 09:55:17 -08:00
Michael Handschuh 5bda58dfd1 Initialize LocalObjectStore
Seed the LocalObjectStore with files read from the local disk on Initialize
2019-12-18 08:42:04 -05:00
Michael Handschuh 999cc275b6 Add IObjectStore.ErrorRaised for persistence errors
Errors raised during persistence aren't able to be handled by user code,
and in fact, are swallowed by the implementation after being logged. By
exposing these errors as events we allow the algorithm to be notified of
such an error and take any step necessary to handle the persistence error.
2019-12-17 22:58:27 -05:00
Michael Handschuh 3de807fd05 Move LocalObjectStore to Lean.Engine
There's no reason for algorithms to have direct access to this implementation.
Moving this into the engine prevents algoriths from directly accessing LocalObjectStore
and instead can only reference it through the IObjectStore abstraction
2019-12-17 22:21:11 -05:00