Colton Sellers
ab6027723f
* Use lean data key as param for request * key -> filePath rename and some cleanup * Refactor * Add Organizations Endpoints * Add some organization api wrapper objects * Address namespace issue * Reorganize Api Test into seperate files using one ApiTestBase * Add Organization tests * Use capitalized "API" test namespace to reduce amount of file changes * Add License to test base * Update /data endpoint functions and response objects * Update ApiDataProvider Logic * Handle deserialization of organization products * Simplify converter * Only throw for equity requests when not subscribed to map/factor files * Add missing header * Make arguement exception * Api adjustments * Add Zip factor and map file providers - Common project will now reference Compression project and not the other way round. - Adding Zip FactorFile and MapFile providers * Refactor FactorFileProvider to use DataProvider to fetch files * Use resulting MinimumDate in construction of FactorFile * Nit FactorFile comments and arrangement * Refactor MapFileProviders to use DataProvider for fetching files * Refactor ZipFileProvider * Clean up * Refactor Backtesting Future/Option chain providers to use dataprovider * Fixes for data/ endpoints and test adjustments * Response objects adjustments/cleanups * ApiDateProvider fixes and testing * Add LocalZipFactorFileTests * Update ApiDataProvider download test to verify stream is not null * Implement posting of agreement summary and signed time * Mark all Api related tests as explicit and document details on running * Clarify default token on ApiTestBase * Adjust summary * Update Api responses for QCC, except org products which are sold in USD * Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests * Fix multiple markets for ZipFactorFile provider * Use Symbol as cache key * Api.cs review * Dispose of factorFileStream after reading * Use zip.EntryFileNames * Address a few reviews * Few more fixes * Address Api Review * Add Job Org id to config * Minor tweaks * Compare with invariant culture * Fixes Option Universe selection * ZipEntryNameSubscriptionDataSourceReader will use IDataProvider * Fix research * Fix null reference exception * Make duplicate log debug Co-authored-by: Martin-Molinero <martin@quantconnect.com>
QuantConnect Research Project
Currently we have two ways to use QuantConnect research notebooks, you can either install and run locally or just use our docker image (Recommended).
The up to date docker image is available at quantconnect/research. You can pull this image with docker pull quantconnect/research.
Using the Docker Image
Starting the Container
The docker image we created can be started using the included .bat/.sh file in this directory (Lean/Research). These scripts take care of all the work required to get the notebook container setup and started for use. Including launching a browser to the notebook lab environment for you.
From a terminal launch the run_docker_notebook.bat/.sh script; there are a few options on how to launch this:
-
Launch with no parameters and answer the questions regarding configuration (Press enter for defaults) ex:
./run_docker_notebook.bat* Enter docker image [default: quantconnect/research:latest]: * Enter absolute path to Data folder [default: ~yourpathtolean~\Lean\Data\]: * Enter absolute path to store notebooks [default: ~yourpathtolean~\Lean\Research\Notebooks]: -
Using the docker.cfg to store args for repeated use; any blank entries will resort to default values! ex:
./run_docker_notebook.bat docker.cfgIMAGE=quantconnect/research:latest DATA_DIR= NOTEBOOK_DIR= -
Inline arguments; anything you don't enter will use the default args! ex:
./run_docker.bat IMAGE=quantconnect/research:latest- Accepted args for inline include all listed in the file docker.cfg
Once the docker image starts, the script will attempt to open your browser to the Jupyter notebook web app, if this fails open your browser and go to localhost:8888
C# Notebook
When using C# for research notebooks it requires that you load our setup script CSX file QuantConnect.csx into your notebook. This will load our QuantConnect libraries into your C# Kernel. In this setup, the file is one directory above the notebooks dir. Be sure to use the following line in your first cell to load in this csx file:
load "../QuantConnect.csx"
After this the environment is ready to use; take a look at our reference notebook KitchenSinkCSharpQuantBookTemplate.ipynb for an example of how to use our QuantBook interface!
Python Notebook
With Python we have a setup script that will automatically load QuantBooks libraries into the Python kernel so there is no need to import them.
You notebook is ready to use; take a look at our reference notebook KitchenSinkQuantBookTemplate.ipynb for an example of how to use our QuantBook interface!
Using the Web Api from Notebook
Both of our setup scripts for Python & C# include a instantiated Api object under the variable name api. Before you can use this api object to interact with the cloud you must edit your config in the root of your Notebook directory. Once this has been done once, it does not need to be done again.
In config.json add the following entries with your respective values
job-user-id: 12345, // Your id here
api-access-token: "token13432", // Your api token here
Once this has been done, you may restart your kernel and begin to use the api variable.
Reference our examples mentioned above for practical uses of this object.
Shutting Down the Notebook Lab
When you are done with the research environment be sure to stop the container with either Docker's dashboard or through the Docker CLI with docker kill LeanResearch.
Build a new image
For most users this will not be necessary, simply use docker pull quantconnect/research to get the latest image.
docker build -t quantconnect/research - < DockerfileJupyter will build a new docker image using the latest version of lean. To build from particular tag of lean a build arg can be provided, for example --build-arg LEAN_TAG=8631.
Running Jupyter Locally
Note: we recommend using the above approach with our Docker container, where the setup and evironment is tested and stable.
Before we enable Jupyter support, follow Lean installation and Python installation to get LEAN running Python algorithms in your machine.
1. Installation:
- Install JupyterLab:
pip install jupyterlab
- Install QuantConnect Python API
pip install quantconnect
- Install pythonnet/clr-loader
pip install clr-loader
2. Run Jupyter:
- Update the
config.jsonfile inLean/Launcher/bin/Debug/folder
"composer-dll-directory": ".",
- Run Jupyter from the command line
cd Lean/Launcher/bin/Debug
jupyter lab