Benchmark SPY data is miss history prior to 1 year ago

[McSavage]

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: (Windows Version or $ uname --all)
  Microsoft Windows [Version 10.0.15063]

• Python Version: $ python --version
  Python 3.5.4 :: Continuum Analytics, Inc.

• Python Bitness: $ python -c 'import math, sys;print(int(math.log(sys.maxsize + 1, 2) + 1))'
  64

• How did you install Zipline: (pip, conda, or other (please explain))
  $ conda install -c Quantopian zipline

• Python packages: $ pip freeze or $ conda list
  alabaster 0.7.10 py35_0
  alembic 0.7.7 py35_0 Quantopian
  astroid 1.5.3 py35_0
  babel 2.5.0 py35_0
  bcolz 0.12.1 np111py35_0 Quantopian
  bkcharts 0.2 py35_0
  bleach 1.5.0 py35_0
  bottleneck 1.2.1 np111py35_0
  bzip2 1.0.6 vc14_3 [vc14]
  certifi 2016.2.28 py35_0
  chardet 3.0.4 py35_0
  click 6.7 py35_0
  cloudpickle 0.4.0 py35_0
  colorama 0.3.9 py35_0
  contextlib2 0.5.5 py35_0
  cycler 0.10.0 py35_0
  cyordereddict 0.2.2 py35_0 Quantopian
  cython 0.26 py35_0
  cytoolz 0.8.2 py35_0
  datashape 0.5.4 py35_0
  decorator 4.1.2 py35_0
  docutils 0.14 py35_0
  empyrical 0.2.2 py35_0 Quantopian
  entrypoints 0.2.3 py35_0
  flask 0.12.2 py35_0
  flask-cors 3.0.3 py35_0
  hdf5 1.8.17 vc14_0 [vc14]
  html5lib 0.9999999 py35_0
  icu 57.1 vc14_0 [vc14]
  imagesize 0.7.1 py35_0
  inflection 0.3.1 py35_0
  intervaltree 2.1.0 py35_0 Quantopian
  ipykernel 4.6.1 py35_0
  ipython 6.1.0 py35_0
  ipython_genutils 0.2.0 py35_0
  ipywidgets 6.0.0 py35_0
  isort 4.2.15 py35_0
  itsdangerous 0.24 py35_0
  jedi 0.10.2 py35_2
  jinja2 2.9.6 py35_0
  jpeg 9b vc14_0 [vc14]
  jsonschema 2.6.0 py35_0
  jupyter 1.0.0 py35_3
  jupyter_client 5.1.0 py35_0
  jupyter_console 5.2.0 py35_0
  jupyter_core 4.3.0 py35_0
  lazy-object-proxy 1.3.1 py35_0
  libpng 1.6.30 vc14_1 [vc14]
  llvmlite 0.20.0 py35_0
  locket 0.2.0 py35_1
  logbook 0.12.5 py35_0 Quantopian
  lru-dict 1.1.4 py35_0 Quantopian
  mako 1.0.6 py35_0
  markupsafe 1.0 py35_0
  matplotlib 2.0.2 np111py35_0
  mistune 0.7.4 py35_0
  mkl 2017.0.3 0
  more-itertools 3.2.0 py35_0
  mpmath 0.19 py35_1
  multipledispatch 0.4.9 py35_0
  nbconvert 5.2.1 py35_0
  nbformat 4.4.0 py35_0
  networkx 1.11 py35_0
  notebook 5.0.0 py35_0
  numba 0.35.0 np111py35_0
  numexpr 2.6.2 np111py35_0
  numpy 1.11.3 py35_0
  numpydoc 0.7.0 py35_0
  openssl 1.0.2l vc14_0 [vc14]
  pandas 0.18.1 np111py35_0
  pandas-datareader 0.5.0 py35_0
  pandocfilters 1.4.2 py35_0
  partd 0.3.8 py35_0
  path.py 10.3.1 py35_0
  patsy 0.4.1 py35_0
  pickleshare 0.7.4 py35_0
  pip 9.0.1 py35_1
  prompt_toolkit 1.0.15 py35_0
  psutil 5.2.2 py35_0
  pycodestyle 2.3.1 py35_0
  pyface 5.1.0 py35_0
  pyflakes 1.6.0 py35_0
  pygments 2.2.0 py35_0
  pylint 1.7.2 py35_0
  pyparsing 2.2.0 py35_0
  pyqt 5.6.0 py35_2
  pytables 3.3.0 np111py35_0
  python 3.5.4 0
  python-dateutil 2.6.1 py35_0
  pytz 2017.2 py35_0
  pyyaml 3.12 py35_0
  pyzmq 16.0.2 py35_0
  qgrid 0.3.3 py35_0
  qt 5.6.2 vc14_6 [vc14]
  qtawesome 0.4.4 py35_0
  qtconsole 4.3.1 py35_0
  qtpy 1.3.1 py35_0
  quandl 3.1.0 py35_0
  requests 2.14.2 py35_0
  requests-file 1.4.1 py35_0
  requests-ftp 0.3.1 py35_0
  rope 0.9.4 py35_1
  scipy 0.19.0 np111py35_0
  seaborn 0.8 py35_0
  setuptools 36.4.0 py35_0
  simplegeneric 0.8.1 py35_1
  singledispatch 3.4.0.3 py35_0
  sip 4.18 py35_0
  six 1.10.0 py35_1
  snowballstemmer 1.2.1 py35_0
  sortedcontainers 1.5.7 py35_0
  sphinx 1.6.3 py35_0
  sphinx_rtd_theme 0.2.4 py35_0
  sphinxcontrib 1.0 py35_0
  sphinxcontrib-websupport 1.0.1 py35_0
  spyder 3.2.3 py35_0
  sqlalchemy 1.1.13 py35_0
  statsmodels 0.8.0 np111py35_0
  sympy 1.1.1 py35_0
  ta-lib 0.4.9 np111py35_0 Quantopian
  testpath 0.3.1 py35_0
  tk 8.5.18 vc14_0 [vc14]
  toolz 0.8.2 py35_0
  tornado 4.5.2 py35_0
  traitlets 4.3.2 py35_0
  traits 4.6.0 py35_0
  vs2015_runtime 14.0.25420 0
  wcwidth 0.1.7 py35_0
  werkzeug 0.12.2 py35_0
  wheel 0.29.0 py35_0
  widgetsnbextension 3.0.2 py35_0
  win_unicode_console 0.5 py35_0
  wincertstore 0.2 py35_0
  wrapt 1.10.11 py35_0
  zipline 1.1.1 np111py35_0 Quantopian
  zlib 1.2.11 vc14_0 [vc14]

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

I'm not getting SPY (benchmark) history

• What did you expect to happen?
  Ran:
  zipline run -f C:\Users\danmc\OneDrive\Documents\Rational\Rational_Source_Files\buyapple.py --start 2000-1-1 --end 2014-1-1 -o buyapple_out.pickle
• What happened instead?
  Get Error:
  File "C:\Users\danmc\Anaconda3\envs\Quantopian\lib\site-packages\pandas\core\indexing.py", line 1403, in _has_valid_type
  error()
  File "C:\Users\danmc\Anaconda3\envs\Quantopian\lib\site-packages\pandas\core\indexing.py", line 1390, in error
  (key, self.obj._get_axis_name(axis)))
  KeyError: 'the label [2000-01-03 00:00:00+00:00] is not in the [index]'

I see the benchmark data file is updating but the data prior to 1 year ago does not populate:
.....zipline\data\SPY_benchmark.csv

2009-08-11 00:00:00+00:00,0.0
2012-02-02 00:00:00+00:00,0.0
2016-09-19 00:00:00+00:00,-0.1435852161001645
2016-09-20 00:00:00+00:00,4.6858160348550726e-05
.....

2017-09-11 00:00:00+00:00,0.010665909643928995
2017-09-12 00:00:00+00:00,0.003370651257975199
2017-09-13 00:00:00+00:00,0.0004799040191960646
2017-09-14 00:00:00+00:00,-0.00031978254786735327
2017-09-15 00:00:00+00:00,-0.0035987044663920997

Here is how you can reproduce this issue on your machine:

Reproduction Steps

...

What steps have you taken to resolve this already?

I removed zipline from my Anaconda environment -deleted the .zipline folder and then reinstalled zipline

Still get the same problem

Anything else?

...

Sincerely,
dan.mcsavage@gmail.com

[freddiev4]

The reason for this is because Google has now limited users to about 251 days worth of data per request, so you can't run backtests over a year. There is a fix currently being worked on.

There are duplicates of this issue so I'm just going to direct everyone to this issue: #1965. I'll comment there when there is a fix on master