92ff37f1c9
Merge pull request #3402 from gsalaz98/feature-3401-improve-tradingeconomics-downloaders
...
Improves TradingEconomics Downloaders
2019-07-24 13:35:19 -07:00
8056ee814e
Make SEC data converter map file resolver configurable via config.json
2019-07-22 14:59:20 -07:00
3a2d976d0a
Overwrite existing index files in download session
2019-07-22 12:07:29 -07:00
3b5bdd4ab0
Use daily index files to determine what files to download
2019-07-22 10:44:03 -07:00
8c0abe55f8
Address Martin's review
2019-07-18 10:30:52 -07:00
920010b242
Add rategate to all requests so that we don't run over rate limits
...
Improve logging across all TradingEconomics Downloaders
Add retries to HTTP requests due to occassional failure
Added skipping of duplicates, skipping of
fields with no actual value, temp file writing, temp file moving
Write to proper output directory
Update data directory structure
Update GetSource in BaseData implementation for TradingEconomics
Parallelize data processing and writing to disk
2019-07-16 12:02:47 -07:00
fd95aebe2e
Make MapFileResolver configurable by config.json
2019-07-15 17:56:01 -07:00
fdf84d0fb4
Add ConvertHistoricalData method to process backfill data
...
Change method Convert to accept stream instead of file
2019-07-12 13:39:11 -07:00
8eb6ed02c5
Rename raw files
...
remove source for raw file name
2019-07-12 20:05:13 +00:00
77ebcce050
Use Stream and StreamReader to read file instead of copying locally
2019-07-12 09:27:36 -07:00
a11ecbd59f
Copy file locally before attempting to parse it
2019-07-12 08:55:50 -07:00
e3440f4474
Address JayJay's review.
...
Alter temp path to use GUID instead of filename to avoid potential
collisions with failed downloads
2019-07-09 17:47:33 -07:00
b11c14a1e9
Address mchandschuh's review #2
2019-07-09 17:46:15 -07:00
9401481637
Improve psychsignal logging and polish code
2019-07-09 17:46:15 -07:00
652c44726b
Improve logging and fix bug where we would write file to forbidden dir
2019-07-09 17:46:15 -07:00
fde076eb31
Make downloader 15 day check use UtcNow as end date
2019-07-09 17:46:15 -07:00
a9a11fc848
Address Mike's review
2019-07-09 17:46:15 -07:00
4a83793067
Address review
2019-07-09 17:46:15 -07:00
d5e148b05c
Adds Psychsignal data converter and custom data implementation
2019-07-09 17:45:31 -07:00
615793597f
Add exemption for USHolidays in missing SEC archive check
2019-07-08 09:31:19 -07:00
cce9988b63
Update comments to accurately reflect upon the written code
...
Change SECDataConverter CLI argument `--start` to `--date`
2019-07-05 16:58:29 -07:00
052c08a6a8
Use map files to write SEC data to mapped symbol
2019-07-05 16:53:20 -07:00
263de28494
Fix bug where we wrote empty tickers to SEC data folder
...
Updated expected missing data log message to increase clarity
Code polish
2019-07-05 16:53:20 -07:00
5879cb557b
Add whitelist of missing dates to SECDataConverter
2019-07-05 16:53:20 -07:00
ce22687fea
Fix wrong output directory path. Remove unused code
2019-07-05 16:53:20 -07:00
3ba960cd28
Fix destination folder in temporary output
2019-07-05 16:53:20 -07:00
3cb4bf79b9
Fix issue where MapFileResolver null check would always be true
2019-07-05 16:53:20 -07:00
bb62484116
Add support for "FLAWED" tag
2019-07-05 16:53:20 -07:00
8c94f592b9
Fix map file error caused by null entry in SEC's ticker.txt
2019-07-05 16:53:20 -07:00
ac1ada5c09
Add progress logs
2019-07-05 16:53:20 -07:00
4f7439d2bd
Use map files for filtering the tickers
2019-07-05 16:53:20 -07:00
7b6f13f934
Address review: Make gunzip happen in memory instead of using disk
2019-07-05 16:53:20 -07:00
435a60a321
Move file CIK ticker parsing logic to Process method
2019-07-05 16:53:20 -07:00
91f08cf2b3
Address review
2019-07-05 16:53:20 -07:00
b29b8797ad
Make SECConverter to process a single day
2019-07-05 16:53:19 -07:00
6bade144d2
Address review - Rename CLI arg names for SECDataConverter
2019-07-05 16:53:19 -07:00
677904108e
Fix outdated SEC downloader documentation
2019-07-05 16:53:19 -07:00
5cdfb568ab
Improve SEC logging experience. Only serious errors will be outputted now.
...
Split SECDataDownloader and SECDataConverter to two ToolBox applications
2019-07-05 16:53:19 -07:00
4a9f9448c5
Merge pull request #3264 from aytacasan/master
...
IQFeed Downloader Program
2019-07-03 16:53:04 -07:00
062b27e0b0
Bitfinex DataQueueHandler June 2019 Update
...
- Moved IConnectionHandler and DefaultConnectionHandler from ToolBox project to Brokerages project
- Added connectionId to IConnectionHandler.Initialize to support multiple connections
- Added new BitfinexSubscriptionManager to BitfinexBrokerage to handle max 30 channels per websocket
2019-07-02 15:17:06 +02:00
724b11caae
Update to ToolboxArgumentParser and Readme
...
Small updates to indicate to user that API key is required for the IEXDownloader and update to the example displayed based on recent changes
2019-06-26 12:11:35 -07:00
f5380770bc
Migrate IEX Downloader to IEX Cloud API
2019-06-26 11:46:54 -07:00
88e0461a2e
separation of the json parsing using the real thread - newtonsoft is slooow
2019-06-21 20:29:57 +02:00
306b142309
reduction of the locking surface
2019-06-21 20:15:17 +02:00
bb5cec3a49
Address Review and Fix Unit Tests
2019-06-21 13:14:47 +01:00
7b81f1a64d
Implements Estimize Equity Endpoints Alternative Data
...
- Release: Financial releases for the specified company
- Estimate: Financial estimates for the specified company
- Consensus: Consensus of the specified release
For each equity endpoint, we create a `BaseData`, a Downloader and add unit tests.
2019-06-21 13:14:47 +01:00
0675d5be90
Merge pull request #3329 from StefanoRaggi/coinapi-dqh-keepalive-fix
...
CoinApiDataQueueHandler connection handling updates
2019-06-20 13:35:18 -07:00
e7b8ba4dee
Merge pull request #3268 from AlexCatarino/feature-3200-tradingeconomics
...
Implements Trading Economics Data
2019-06-20 13:33:51 -07:00
2eeb718c72
Merge pull request #3325 from gsalaz98/bug-3324-fix-failing-sec-report-deserialization-null-value-formercompanies
...
Fix Handling of Potential Null Values In SEC Report Submissions
2019-06-20 12:09:53 -07:00
ed3d65dce1
Add extra logging for reconnect
2019-06-20 13:36:25 +02:00
Jared