MAINT: Sync and fill benchmarks through latest trading day #2044
Conversation
There was a problem hiding this comment.
Hi @ChrisPappalardo thanks for the PR! I just had one question for you below.
| # combine dt indices and reindex using ffill then bfill | ||
| all_dt = br.index.union(tc.index) | ||
| br = br.reindex(all_dt, method='ffill').fillna(method='bfill') | ||
| tc = tc.reindex(all_dt, method='ffill').fillna(method='bfill') |
There was a problem hiding this comment.
The tradeoff here that I'm concerned about is that we'll end up having benchmark returns for much further back than 5 years (which is the max amount we can get from IEX), except the returns of that benchmark will just be the same b/c we're bfilling.
That's more memory we're using up and also then we have benchmark data that isn't accurate before the 5 year cutoff.
What is it that you're trying to get out of reindexing here?
There was a problem hiding this comment.
I'm trying to sync the two time series.
The Federal Reserve H15 report is not released on a timely basis with market closes, so there will always be missing data on the front-end of the treasury curve time series. At the same time, there are dates missing from both time series relative to each other. Joining the time series, re-indexing, and then filling the missing data solves both issues.
If the concern is filling data back past 5 years on the benchmark, you could add a line to drop all dates older than the most recent start of the two respective time series.
There was a problem hiding this comment.
OK. That makes sense to me and sounds fair. Also just did some quick profiling and mem. usage wasn't what I had thought it would be like.
Can you amend your commit to use a Commit Prefix? I think MAINT: would be appropriate here, with a shorter commit message. Should be good to merge after that 🙂
There was a problem hiding this comment.
@freddiev4 this PR is good to go
freddiev4
changed the title
|
Hmm I thought I had merged this after seeing the email notification... 🤔 Anyways awesome work @ChrisPappalardo. Thanks for the PR 😃 |
It's analytically superior to have a trading system synced to the latest market close than to lag the market waiting for treasury benchmark data.
This PR removes a hard-wired 2-day lag and combines the timeseries of the market and treasury benchmarks before filling data forwards and backwards along the timeseries. This PR could be improved by averaging the treasury curve across points before backfilling the timeseries.
See comments to #2031 for more information.