Ronit Jain
c8d13080a8
* added symbolData class to store exchange and token values of tickers * support for India Market (cherry picked from commit 5f629dcb9e72426dc7b560e782cb1fae72861574) * removed local vscode workspace file * support for india market instead of nse and other exchanges * changed hardcoded india values to Market.India * changed zerodha getcash fromm available to net * addded edge case handoing with expections * support for market info (nse, bse) for india market * exchange info for EmitQuoteTick * basic testing with c# * c# algo * c# testing * live orders during market open success * support orderproperties for exchange value for order placement * added exceptional handling cases * fix xml descriptions for the files * fix zerodha-product-type in DefaultOrderProperties * add basic template for india market * fix orderProperites for default values * fix for local data, python implementation not available * fix access modifiers of SymbolData * fix variable name case * fix single check for exchange, call to base * fix styling, redundent code and access modifers * fix non static not callable from static * fix static
LEAN Data Formats
Introduction
From the beginning LEAN strived to use an open, human readible data format - independent of any specific database or file format. From this core philosophy we built LEAN to read its financial data from flat files on disk. Data compression is done in zip format; and all individual files are CSV or JSON.
When there is no activity for a security, the price is omitted to the file. Only new ticks, and price changes are recorded.
File Data Format
Although we strive to make all data formats identical it is often not possible. Below are links to dedicated documentation on the file format of the data in each asset type:
Equity | Forex | Options | Futures | Crypto
Folder Structure
Data files are separated and nested in a few predictable layers:
-
Tick, Second and Minute Financial Data:
/data/securityType/marketName/resolution/ticker/date_tradeType.zip -
Hour, Daily Financial Data:
/data/securityType/marketName/resolution/ticker.zip
The marketName value is used to separate different tradable assets with the same ticker. E.g. EURUSD is traded on multiple brokerages all with slightly different prices.
Core Data Types
LEAN has a few core data types which are represented in all the asset classes we support. Below are links to their implementation in LEAN.
-
TradeBar - TradeBar represents trade ticks of assets consolidated for a period. TradeBar file format is slightly different for high resolution (second, minute) and low resolution (daily, hour).
-
QuoteBar - QuoteBar represents top of book quote data consolidated over a period of time (bid and ask bar).
-
Tick - Tick data represents an individual record of trades ("trade ticks") or quote updates ("quote tick") for an asset. Tick data is instantaneous - it does not have a period.
Data Readers
All data is parsed from disk via Reader() methods. The Reader takes a single line of the file and converts it the appropriate type. i.e. TradeBar.Reader() method is a factory which returns TradeBar objects. When implementing custom data Readers are used
Other Data Formats
Theoretically LEAN can accept data in any format (database, API or flatfile). However practically we currently have reader implementations written for a flat file system.