UnicodeDecodeError on upload

[aspiers]

Unfortunately this is a blocker for me, rendering git-annex's glacier special remote useless - the whole point of trying it was to be able to upload big binary files.

$ glacier --region eu-west-1 archive upload music myiso.exe
Traceback (most recent call last):
  File "/home/adam/bin/glacier", line 730, in <module>
    App().main()
  File "/home/adam/bin/glacier", line 716, in main
    self.args.func()
  File "/home/adam/bin/glacier", line 498, in archive_upload
    file_obj=self.args.file, description=name)
  File "/usr/lib/python2.7/site-packages/boto/glacier/vault.py", line 177, in create_archive_from_file
    writer.write(data)
  File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 219, in write
    self.partitioner.write(data)
  File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 61, in write
    self._send_part()
  File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 75, in _send_part
    self.send_fn(part)
  File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 222, in _upload_part
    self.uploader.upload_part(self.next_part_index, part_data)
  File "/usr/lib/python2.7/site-packages/boto/glacier/writer.py", line 129, in upload_part
    content_range, part_data)
  File "/usr/lib/python2.7/site-packages/boto/glacier/layer1.py", line 1279, in upload_part
    response_headers=response_headers)
  File "/usr/lib/python2.7/site-packages/boto/glacier/layer1.py", line 114, in make_request
    data=data)
  File "/usr/lib/python2.7/site-packages/boto/connection.py", line 1068, in make_request
    retry_handler=retry_handler)
  File "/usr/lib/python2.7/site-packages/boto/connection.py", line 942, in _mexe
    request.body, request.headers)
  File "/usr/lib64/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 827, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 12: ordinal not in range(128)

[basak]

This reminds me of http://bugs.python.org/issue12398 and I think it's likely a bug in boto or python 2.7 or an interaction between the two. I've not seen it in glacier-cli before though. It'll need some deeper investigation I don't have time to do right now, sorry.

[aspiers]

I think you're right - I just found boto/boto#2603

[xsuchy]

I will investigate how glacier-cli works with that mentioned boto change.

[xsuchy]

Note that I'm using #17 path to handle UTF8 filenames.

[aspiers]

Yep I saw that, although in my case the UTF8 is in the content not the filename.

[xsuchy]

I can confirm that with fix from boto/boto#2603 glacier-cli works without problem. So this is not issue in glacier-cli per se and can be closed.

[basak]

Thank you!

[aspiers]

Confirmed it works for me. To be clear, the fix mentioned in boto/boto#2603 is actually boto/boto#2697.