Commit Graph

202 Commits

Author SHA1 Message Date
Jared 92ff37f1c9 Merge pull request #3402 from gsalaz98/feature-3401-improve-tradingeconomics-downloaders
Improves TradingEconomics Downloaders
2019-07-24 13:35:19 -07:00
Jared 7a2c35435c Merge pull request #3397 from QuantConnect/fxcm-exchange-timezone-mhdb-change
FXCM exchange timezone MHDB change
2019-07-16 15:37:19 -07:00
Gerardo Salazar 920010b242 Add rategate to all requests so that we don't run over rate limits
Improve logging across all TradingEconomics Downloaders

Add retries to HTTP requests due to occassional failure

Added skipping of duplicates, skipping of
fields with no actual value, temp file writing, temp file moving

Write to proper output directory

Update data directory structure

Update GetSource in BaseData implementation for TradingEconomics

Parallelize data processing and writing to disk
2019-07-16 12:02:47 -07:00
Martin Molinero 9f26a22a0b FXCM exchange timezone MHDB change 2019-07-15 14:03:59 -03:00
AlexCatarino 1495345b88 Removes Alternative Data Files
Use `.keep` file as a place holder to keep folder structure.
2019-07-11 00:12:01 +01:00
Mark Uretsky 8564dbb70e Add missing GDAX currency pairs 2019-06-24 21:32:42 +02:00
Jared 66086b8872 Update readme.md 2019-06-23 08:30:19 -07:00
AlexCatarino bb5cec3a49 Address Review and Fix Unit Tests 2019-06-21 13:14:47 +01:00
Jared e12bb68891 Update readme.md 2019-06-20 17:45:00 -07:00
Jared e7b8ba4dee Merge pull request #3268 from AlexCatarino/feature-3200-tradingeconomics
Implements Trading Economics Data
2019-06-20 13:33:51 -07:00
AlexCatarino 66a0c7377d Addresses Reviews
- Changes the destination folder
- Remove Symbol converter
- Refactors Get method to return `Task<string>`
  - Since `HttpRequester` returns `Task<string>`, it makes more sense to pass this result than returning a `IEnumerable<string>`.
2019-06-20 10:35:56 +01:00
AlexCatarino 5b34e2519a Adds TradingEconomicsCalendarIndicatorAlgorithm
Adds C# and Python version of TradingEconomicsCalendarIndicatorAlgorithm to show how TradingEconomics data can be added to an algorithm.
2019-06-20 10:35:56 +01:00
Gerardo Salazar 7ed94c0a35 Update SECDataConverter to not write null values
Update SECReport* to ignore null values
2019-06-19 14:44:48 -07:00
Gerardo Salazar a04d8dc0ab Fix potential null value in field FormerCompanies
Update data to reflect changes

Add missing AAPL data point we originally intended to include
2019-06-19 11:09:47 -07:00
Gerardo Salazar a0506ae290 Update bundled data and include new TWX and AOL SEC data 2019-06-19 09:30:33 -07:00
Gerardo Salazar 5a304a4cb5 Change path of alternative data folder
Delete converter because an existing utility already existed

Moved code from factory to converter class as statics

Add rankandfile as a CIK to ticker mapping source
2019-06-17 16:45:54 -07:00
Gerardo Salazar 2ad1f19f85 Implements Sec Custom Data, ToolBox Application, and Sample Algorithm 2019-06-17 16:45:53 -07:00
Gerardo Salazar 3faabc116a Fix error in symbol-properties-database.csv causing travis build to fail 2019-06-07 08:49:56 -07:00
Gerardo Salazar b0406ba9eb Adds the following symbols to futures database: EI, EMD, ...
EN, EPN, ESK, EVC, EWG, EWN, EXR, F1U, FO, FRC, FSS, GCU, GD, GDK,
GNF, HCL, HG, HH, HP, HRC, HTT, IBV, J7

Add new utility functions in FuturesExpiryUtillityFunctions
2019-06-07 08:49:56 -07:00
Jared 87f03e68ca Merge pull request #3185 from gsalaz98/feature-2991-add-expirations-to-futures-database-part-4
Add Various Symbols to Futures Database (Part 4 / 11)
2019-05-17 07:16:54 -07:00
Gerardo Salazar 44a4451ea3 Adds the following futures to futures database: BTC, ...
BWF, BZ, CB, CJY, CNH, CRB, CSC, CSW, CSX, CU, D1N, DC,
DCB, DY, E6, E7, EAD, ECD, EDP, EH
2019-05-10 17:39:51 -05:00
Stefano Raggi f7e06510d4 Fix missing front month contract in energy futures backtesting
Energy futures (CL, HO, RB, NG) expire in the month before the contract month.

To handle these properly, the following changes have been implemented:
- the AlgoSeekFuturesReader has been updated to create the future symbol with the correct expiry date
- the zip entry names in futures data files now contain the full expiry date (in addition to the contract month)
- the sample data files have been updated to use the new zip entry names
- new unit tests have been added and existing ones updated
2019-04-29 09:40:59 +02:00
Juan José D'Ambrosio a01825992b Add AssetClassification and CompanyProfile fine data 2019-04-24 04:21:46 +00:00
Gerardo Salazar 35b0d84247 Add the following futures to futures database: AGA, AJL, AJS, ...
AJY, AKL, AKZ, ANE, APS, AR0, ARE, AUP, AVZ, AW, AYV, AYX, AZ1, B0, B7H, BCF, BIO, BK, BOO, BR7

Correct wrong entry for symbol ZT in symbol-properties-database

Add missing holidays or early closes for energies and fx in mhdb
2019-04-22 17:16:47 -05:00
Gerardo Salazar 5db98c8f53 Fix symbol ACD's symbol-properties-database entry 2019-04-09 12:38:40 -07:00
Gerardo Salazar 26c2f2e36a Adds The Following Symbols To Futures Database: 6Z, A0D, A0F, A1L, ...
A1M, A1R, A32, A3G, A7E, A7I, A7Q, A8J, A8K, A8O, A91,
A9N, AA6, AA8, ABS, ABT, AC0, ACD, AD0, ADB, AE5
2019-04-04 17:03:57 -07:00
Jared 890ed36d0c Merge pull request #3014 from gsalaz98/feature-2991-add-expirations-to-futures-database-part-1
Add 1S, 22, 6L, 6R, 6M to Futures Database
2019-03-27 17:07:02 -07:00
Jared afd10768a6 Merge pull request #3002 from QuantConnect/bug-subscription-data-reader-time-zones
Fix Time Zone Bug in SubscriptionDataReader
2019-03-27 08:06:17 -07:00
Gerardo Salazar 2152b990bf Add market holidays and early closes to new futures 2019-03-25 09:36:12 -07:00
Gerardo Salazar 5e562c07c0 Use market hours database instead of custom FuturesExpiryCalendar
Added additional test cases for symbols 6L and 6M
2019-03-20 13:45:29 -07:00
Gerardo Salazar 60816ecea5 Fix dataTimeZone for new future 1S 2019-03-18 20:14:56 -07:00
Gerardo Salazar ea6d8ee088 Merge branch 'master' into feature-2991-add-expirations-to-futures-database-part-1 2019-03-18 19:52:47 -07:00
Gerardo Salazar 4d4f27f027 Add 1S, 22, 6L, 6M, 6R to futures database 2019-03-18 17:06:35 -07:00
AlexCatarino f4d5f0e458 Adds RTY, NKD, and YO to Future's Database
Adds Russell 2000 E-Mini (RTY), Nikkei 225 Dollar (NKD) and Sugar 11 CME Globex (YO) to future data: entails update entries in market hours and symbol properties databases.
Updates `FutureExpiryFunction` to handle these contracts expiration date.
2019-03-15 17:22:10 +00:00
Juan José D'Ambrosio a44825bbae Update AU200AUD MHDB 2019-03-14 16:02:03 +00:00
Stefano Raggi d215d0a9f7 Add CfdTimeZonesRegressionAlgorithm 2019-03-14 12:13:14 +01:00
Juan José D'Ambrosio b98d1d7054 Update US200USD MHDB 2019-03-14 09:30:53 +00:00
Juan José D'Ambrosio 5d0449269b Update EU50EUR MHDB 2019-03-14 09:26:18 +00:00
Juan José D'Ambrosio 625b532f64 Update FR40EUR MHDB 2019-03-14 08:51:42 +00:00
Juan José D'Ambrosio 5ab56e56cf Update NL25EUR MHDB 2019-03-14 08:39:14 +00:00
Juan José D'Ambrosio 4ae094256d Update HK33HKD MHDB 2019-03-13 22:06:24 +00:00
Juan José D'Ambrosio 01fb060f1a Update JP225USD MHDB 2019-03-13 20:18:57 +00:00
Juan José D'Ambrosio 8902759a89 Update DE30EUR MHDB 2019-03-13 20:02:26 +00:00
Juan José D'Ambrosio c0ae214d34 Update UK100GBP MHDB 2019-03-13 19:46:12 +00:00
Juan José D'Ambrosio 29ebbc9623 Update SPX500USD MHDB 2019-03-13 19:40:53 +00:00
Juan José D'Ambrosio fde24ee91b Update NAS100USD MHDB 2019-03-13 19:38:07 +00:00
Juan José D'Ambrosio 51dee80789 Update US30USD MHDB 2019-03-13 19:14:01 +00:00
Stefano Raggi e37acc3601 Update sample coarse data files 2019-03-11 22:02:32 +01:00
Gerardo Salazar 463dd1d329 Add documentation for cfd, crypto, forex, future, and option data (#2978)
* Add documentation for cfd, crypto, forex, future, and option data

* Update data documentation to ensure consistency

* Fix titles, removed timezone reference
2019-03-04 16:54:02 -08:00
Douglas Stridsberg b5f4aba711 XAUUSD sample second and tick data files updated 2019-02-27 18:10:14 +00:00