Example script on Windows 7 gives Unicode Error #2036

jengelman · 2017-12-01T18:06:49Z

Environment

OS: Windows 7
Python Version: Anaconda Python 3.6.1
Python Bitness: 64
How did you install Zipline: pip
Python packages:
alabaster (0.7.10)
alembic (0.9.6)
algopy (0.5.5)
anaconda-client (1.6.3)
anaconda-navigator (1.6.2)
anaconda-project (0.6.0)
arch (4.1)
asn1crypto (0.22.0)
astroid (1.4.9)
astropy (1.3.2)
Babel (2.4.0)
backports.shutil-get-terminal-size (1.0.0)
bcolz (0.12.1)
beautifulsoup4 (4.6.0)
bintrees (2.0.7)
bitarray (0.8.1)
blaze (0.10.1)
bleach (2.0.0)
blosc (1.5.1)
bokeh (0.12.5)
boto (2.46.1)
Bottleneck (1.2.1)
cached-property (1.3.1)
certifi (2017.11.5)
cffi (1.10.0)
chardet (3.0.4)
click (6.7)
cloudpickle (0.2.2)
clyent (1.2.2)
colorama (0.3.9)
colorlover (0.2.1)
comtypes (1.1.2)
conda (4.3.30)
contextlib2 (0.5.5)
cookies (2.2.1)
cryptography (1.8.1)
cufflinks (0.12.1)
cycler (0.10.0)
cyordereddict (1.0.0)
Cython (0.27.3)
cytoolz (0.8.2)
dask (0.14.3)
dask-searchcv (0.0.2)
datashape (0.5.4)
decorator (4.1.2)
dill (0.2.7.1)
distributed (1.16.3)
docker (2.6.1)
docker-compose (1.17.1)
docker-pycreds (0.2.1)
dockerpty (0.4.1)
docopt (0.6.2)
docutils (0.13.1)
dtw (1.2)
empyrical (0.3.3)
entrypoints (0.2.3)
enum34 (1.1.6)
et-xmlfile (1.0.1)
fastcache (1.0.2)
fastdtw (0.3.2)
fbprophet (0.1.1)
filterpy (1.0.0)
findatapy (0.5)
finmarketpy (0.11)
Flask (0.12.2)
Flask-Cors (3.0.2)
future (0.16.0)
gdax (1.0.6, c:\anaconda3\lib\site-packages\gdax-1.0.6-py3.6.egg)
gevent (1.2.1)
greenlet (0.4.12)
h5py (2.7.0)
HeapDict (1.0.0)
hide-code (0.4.0)
hmmlearn (0.2.0)
html5lib (0.999999999)
idna (2.6)
imagesize (0.7.1)
imbalanced-learn (0.3.0)
inflection (0.3.1)
intervaltree (2.1.0)
ipykernel (4.6.1)
ipyparallel (6.0.2)
ipython (4.2.0)
ipython-genutils (0.1.0)
ipywidgets (5.1.5)
isort (4.2.5)
itsdangerous (0.24)
jdcal (1.3)
jedi (0.10.2)
Jinja2 (2.10)
joblib (0.11)
jsonschema (2.6.0)
jupyter (1.0.0)
jupyter-client (5.0.1)
jupyter-console (5.1.0)
jupyter-core (4.3.0)
lazy-object-proxy (1.2.2)
lifelines (0.11.2)
line-profiler (2.0)
llvmlite (0.19.0)
locket (0.2.0)
Logbook (1.1.0)
lru-dict (1.1.6)
lxml (3.7.3)
Mako (1.0.7)
manhattan (0.2.11)
MarkupSafe (1.0)
matplotlib (2.1.0)
menuinst (1.4.7)
mistune (0.7.4)
mock (2.0.0)
mockextras (1.0.2)
more-itertools (3.2.0)
mpmath (0.19)
msgpack-python (0.4.8)
multipledispatch (0.4.9)
multiprocess (0.70.5)
multiprocessing-on-dill (3.5.0a4)
munch (2.2.0)
navigator-updater (0.1.0)
nbconvert (5.2.1)
nbformat (4.4.0)
ndg-httpsclient (0.4.3)
networkx (1.11)
nltk (3.2.3)
nose (1.3.7)
nose-parameterized (0.6.0)
notebook (5.0.0)
numba (0.34.0+5.g1762237)
numdifftools (0.9.20)
numexpr (2.6.4)
numpy (1.13.3)
numpydoc (0.6.0)
oauthlib (2.0.6)
odo (0.5.0)
olefile (0.44)
openpyxl (2.4.7)
packaging (16.8)
pandas (0.18.1)
pandas-datareader (0.5.0)
pandoc (1.0.0b2)
pandocfilters (1.4.2)
partd (0.3.8)
path.py (8.1.2)
pathlib2 (2.2.1)
patsy (0.4.1)
pbr (3.1.1)
pdfkit (0.6.1)
pep8 (1.7.0)
pickleshare (0.6)
Pillow (4.2.1)
pip (9.0.1)
plotly (2.2.1)
ply (3.10)
prompt-toolkit (1.0.14)
psutil (5.2.2)
py (1.4.33)
py-earth (0.0.0)
PyAlgoTrade (0.18)
pyasn1 (0.3.7)
pycosat (0.6.2)
pycparser (2.17)
pycrypto (2.6.1)
pycurl (7.43.0)
pydtw (1.0.0)
pyflakes (1.5.0)
pyflux (0.4.15)
pyfolio (0.8.0)
Pygments (2.2.0)
pykalman (0.9.5)
pylint (1.6.4)
pymongo (3.5.1)
PyMySQL (0.7.9)
pyodbc (4.0.16)
pyOpenSSL (17.0.0)
pyparsing (2.2.0)
pyramid-arima (0.6.2)
pyreadline (2.1)
pystan (2.16.0.0)
pytest (3.0.7)
python-dateutil (2.6.1)
python-editor (1.0.3)
pytz (2017.3)
PyWavelets (0.5.2)
pywin32 (220)
PyYAML (3.12)
pyzmq (16.0.2)
qstrader (0.0.1)
QtAwesome (0.4.4)
qtconsole (4.3.0)
QtPy (1.2.1)
Quandl (3.2.1)
QuantLib-Python (1.9)
redis (2.10.5)
requests (2.18.4)
requests-file (1.4.2)
requests-ftp (0.3.1)
requests-oauthlib (0.8.0)
responses (0.8.1)
retrying (1.3.3)
rope-py3k (0.9.4.post1)
scikit-image (0.13.0)
scikit-learn (0.19.0)
scikit-plot (0.3.1)
scipy (1.0.0)
seaborn (0.7.1)
seasonal (0.3.0)
setuptools (38.2.3)
simplegeneric (0.8.1)
singledispatch (3.4.0.3)
six (1.11.0)
sklearn-contrib-lightning (0.4.0)
snowballstemmer (1.2.1)
sortedcollections (0.5.3)
sortedcontainers (1.5.7)
sphinx (1.5.6)
spyder (3.1.4)
SQLAlchemy (1.1.15)
statsmodels (0.8.0, c:\anaconda3\lib\site-packages)
sympy (1.0)
tables (3.4.2)
tblib (1.3.2)
testfixtures (5.3.0)
testpath (0.3.1)
texttable (0.9.1)
tia (0.3.0)
toolz (0.8.2)
tornado (4.5.1)
tqdm (4.19.1.post1)
traitlets (4.1.0)
treelib (1.4.0)
twython (3.6.0)
tzlocal (1.4)
unicodecsv (0.14.1)
urllib3 (1.22)
vispy (0.5.1)
wcwidth (0.1.7)
webencodings (0.5.1)
websocket-client (0.44.0)
Werkzeug (0.12.2)
wheel (0.29.0)
widgetsnbextension (2.0.0)
win-unicode-console (0.5)
wrapt (1.10.10)
xlrd (1.0.0)
XlsxWriter (0.9.6)
xlwings (0.10.4)
xlwt (1.2.0)
zict (0.1.2)

I installed from pip, and successfully ran zipline ingest.
I closed the repo from github, went to the directory, and attempted to run the example script as follows:
zipline run -f zipline/examples/buyapple.py --start 2000-1-1 --end 2014-1-1 -o buyapple_out.pickle

I got the following output:

[2017-12-01 18:02:31.482712] INFO: Loader: Cache at C:\cygwin64\home\jengelman/.zipline\data\SPY_benchmark.csv does not have data from 1990-01-02 00:00:00+00:00 to 2017-11-29 00:00:00+00:00.

[2017-12-01 18:02:31.482712] INFO: Loader: Downloading benchmark data for 'SPY' from 1989-12-29 00:00:00+00:00 to 2017-11-29 00:00:00+00:00
Traceback (most recent call last):
  File "C:\Anaconda3\Scripts\zipline-script.py", line 11, in <module>
    load_entry_point('zipline', 'console_scripts', 'zipline')()
  File "C:\Anaconda3\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Anaconda3\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Anaconda3\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Anaconda3\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "c:\anaconda3\lib\site-packages\zipline-1.1.1+152.g18e4186f-py3.6-win-amd64.egg\zipline\__main__.py", line 101, in _
    return f(*args, **kwargs)
  File "C:\Anaconda3\lib\site-packages\click\decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\anaconda3\lib\site-packages\zipline-1.1.1+152.g18e4186f-py3.6-win-amd64.egg\zipline\__main__.py", line 255, in run
    environ=os.environ,
  File "c:\anaconda3\lib\site-packages\zipline-1.1.1+152.g18e4186f-py3.6-win-amd64.egg\zipline\utils\run_algo.py", line 133, in _run
    env = TradingEnvironment(asset_db_path=connstr, environ=environ)
  File "c:\anaconda3\lib\site-packages\zipline-1.1.1+152.g18e4186f-py3.6-win-amd64.egg\zipline\finance\trading.py", line 99, in __init__
    self.bm_symbol,
  File "c:\anaconda3\lib\site-packages\zipline-1.1.1+152.g18e4186f-py3.6-win-amd64.egg\zipline\data\loader.py", line 166, in load_market_data
    environ,
  File "c:\anaconda3\lib\site-packages\zipline-1.1.1+152.g18e4186f-py3.6-win-amd64.egg\zipline\data\loader.py", line 230, in ensure_benchmark_data
    last_date,
  File "c:\anaconda3\lib\site-packages\zipline-1.1.1+152.g18e4186f-py3.6-win-amd64.egg\zipline\data\benchmarks.py", line 50, in get_benchmark_returns
    last_date
  File "C:\Anaconda3\lib\site-packages\pandas_datareader\data.py", line 137, in DataReader
    session=session).read()
  File "C:\Anaconda3\lib\site-packages\pandas_datareader\base.py", line 181, in read
    params=self._get_params(self.symbols))
  File "C:\Anaconda3\lib\site-packages\pandas_datareader\base.py", line 79, in _read_one_data
    out = self._read_url_as_StringIO(url, params=params)
  File "C:\Anaconda3\lib\site-packages\pandas_datareader\base.py", line 98, in _read_url_as_StringIO
    out.write(bytes_to_str(text))
  File "C:\Anaconda3\lib\site-packages\pandas\compat\__init__.py", line 72, in bytes_to_str
    return b.decode(encoding or 'utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 35301: invalid start byte

I checked the cache, and it has data for the time period mentioned, but running the command again gives the same result. Any help would be appreciated.

The text was updated successfully, but these errors were encountered:

freddiev4 · 2017-12-01T18:23:12Z

Hi @jengelman thanks for opening this. Can you try running this for a smaller time period? Such as 2017-1-1 to 2017-2-1

Aside from that, I believe this is coming from changes to the Google API which we use pandas-datareader to pull data from. I recently opened #2031 to remove the pandas-datareader and Google API dependency, so once that's merged this issue should go away.

jengelman · 2017-12-01T18:27:07Z

@freddiev4 Got almost the exact same error message, including the attempted redownload of the entire benchmark. Only difference was 'position 35199' instead of 'position 35301'.

freddiev4 · 2017-12-01T18:48:51Z

Are you running the latest Zipline master branch? Or at some other commit-ish or version 1.1.1?

freddiev4 · 2017-12-01T18:57:09Z

FWIW, I just ran:

zipline run -f zipline/examples/buyapple.py \
    --start 2017-1-1 \
    --end 2017-2-1 \
    -o buyapple_out.pickle \
    -b quantopian-quandl \

On latest Zipline master, and Zipline 1.1.1, in both a Py35 and Py36 environment (running macOS), and didn't see this issue

jengelman · 2017-12-01T19:10:14Z

Original issue is from Zipline 1.1.1, installed from pip.

I tried uninstalling with pip, building master using python setup.py install, and that command, and got the same error.
Tried rerunning zipline ingest, and got the following (5 times before it gave up):

[2017-12-01 19:06:46.114107] ERROR: zipline.data.bundles.quandl: Exception raised reading Quandl data. Retrying.
Traceback (most recent call last):
  File "C:\Anaconda3\lib\site-packages\zipline-1.1.1+172.g42b32007-py3.6-win-amd64.egg\zipline\data\bundles\quandl.py", line 89, in fetch_data_table
    format_metadata_url(api_key)
  File "C:\Anaconda3\lib\site-packages\pandas\io\parsers.py", line 562, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "C:\Anaconda3\lib\site-packages\pandas\io\parsers.py", line 301, in _read
    compression=kwds.get('compression', None))
  File "C:\Anaconda3\lib\site-packages\pandas\io\common.py", line 308, in get_filepath_or_buffer
    req = _urlopen(str(filepath_or_buffer))
  File "C:\Anaconda3\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Anaconda3\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "C:\Anaconda3\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Anaconda3\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "C:\Anaconda3\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Anaconda3\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
[2017-12-01 19:06:46.116107] INFO: zipline.data.bundles.quandl: Downloading WIKI metadata.
[2017-12-01 19:06:46.249107] ERROR: zipline.data.bundles.quandl: Exception raised reading Quandl data. Retrying.

freddiev4 · 2017-12-01T19:14:33Z

Installing Zipline:
So installing zipline via pip is slightly more involved than the average Python package, which you can read more about in the Install Guide.

If you were to use conda, which is the recommended package manager for installing zipline on Windows, you can install the latest zipline master using conda install -c quantopian/label/ci zipline.

Ingesting Data:
In regards to zipline ingest, the default data bundle is the quandl bundle (was formerly quantopian-quandl, docs should be updated in the next release), in which you need a quandl API key to properly ingest, seen here in the Data Bundles Guide.

You can also run zipline ingest -b quantopian-quandl, which is a mirror of the quandl WIKI dataset that Quantopian provides, with the data in the formats that zipline expects. You don't need to pass the API key arg for that.

Python Versions:
We only build conda packages for Py27 and Py35, and don't yet have support for Py36.

jengelman · 2017-12-01T19:20:10Z

Following the instructions, I tried installing using conda, and got the following conflict:

$ conda install -c Quantopian zipline

Fetching package metadata ...............
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - python 3.6*
  - zipline -> bcolz >=0.12.1,<1 -> numpy 1.10* -> python 2.7*

Does zipline not support python 3.6, or is there a way to change the required version of numpy here?

freddiev4 · 2017-12-01T19:21:18Z

Good question (edited my previous answer above). We don't yet have conda packages built for Python 3.6; if you create a Python 3.5 conda env it should work.

jengelman · 2017-12-01T19:49:44Z

I created a py3.5 conda env, successfully installed zipline from conda, and got the original error "can't decode btye" error again.

edit: tried running both the jupyter notebook and example script directly from python, and got "No module named zipline" for both, so guess installation wasn't successful, despite being able to call zipline from the command line. might be a path issue somewhere?

freddiev4 · 2017-12-01T19:57:24Z

Interesting. So if you've installed via conda, in a Py35 env, and the latest master doesn't work for you and zipline 1.1.1 doesn't work, then I'm thinking #2031 should be the fix. However, I'm still able to run the command I pasted earlier without any issues; so it's a bit strange. Will have to do some more inspection.

If you'd like to test out that #2031 branch, there are Development Guidelines on how to get everything set up.

Otherwise, I'm thinking just wait for that PR to get merged. I don't have access to a Windows machine currently but I also don't think it has anything to do with that b/c there's #2029 (which looks like they're using a non-Windows OS).

freddiev4 · 2017-12-01T20:00:28Z

It's possible there's a path issue, after seeing your edit. Not entirely certain of that.

jengelman · 2017-12-01T20:11:10Z

Built the #2031 branch and running zipline from the command line worked. Tried importing zipline within python, and got this:

No module named 'zipline.utils.calendars._calendar_helpers

freddiev4 · 2018-04-05T14:51:19Z

We recently did a release of zipline. You can see the release notes here Feel free to update to 1.2.0 with either:

pip install -U zipline

or

conda update zipline -c quantopian

If there are any problems, please reopen this or open a new issue 🙂

freddiev4 added Benchmark Windows labels Dec 1, 2017

freddiev4 mentioned this issue Dec 1, 2017

ENH: Use IEX Trading data instead of pandas-datareader #2031

Merged

freddiev4 added the Python 3.6 label Dec 1, 2017

This was referenced Dec 2, 2017

Default examples is not working #2037

Closed

get_banchmark_returns can't handle google finance http response format #2029

Closed

cannot read results.pickle #2030

Closed

freddiev4 closed this as completed Apr 5, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Example script on Windows 7 gives Unicode Error #2036

Example script on Windows 7 gives Unicode Error #2036

jengelman commented Dec 1, 2017

freddiev4 commented Dec 1, 2017

jengelman commented Dec 1, 2017

freddiev4 commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Dec 1, 2017 •

edited

Loading

jengelman commented Dec 1, 2017

freddiev4 commented Dec 1, 2017 •

edited

Loading

jengelman commented Dec 1, 2017

freddiev4 commented Dec 1, 2017 •

edited

Loading

jengelman commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Dec 1, 2017

jengelman commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Apr 5, 2018 •

edited

Loading

Example script on Windows 7 gives Unicode Error #2036

Example script on Windows 7 gives Unicode Error #2036

Comments

jengelman commented Dec 1, 2017

Environment

freddiev4 commented Dec 1, 2017

jengelman commented Dec 1, 2017

freddiev4 commented Dec 1, 2017 • edited Loading

freddiev4 commented Dec 1, 2017 • edited Loading

jengelman commented Dec 1, 2017

freddiev4 commented Dec 1, 2017 • edited Loading

jengelman commented Dec 1, 2017

freddiev4 commented Dec 1, 2017 • edited Loading

jengelman commented Dec 1, 2017 • edited Loading

freddiev4 commented Dec 1, 2017 • edited Loading

freddiev4 commented Dec 1, 2017

jengelman commented Dec 1, 2017 • edited Loading

freddiev4 commented Apr 5, 2018 • edited Loading

freddiev4 commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Dec 1, 2017 •

edited

Loading

jengelman commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Dec 1, 2017 •

edited

Loading

jengelman commented Dec 1, 2017 •

edited

Loading

freddiev4 commented Apr 5, 2018 •

edited

Loading