Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python3 JSON Adapter byte string is not serializable #13

Open
Amorik opened this issue Mar 4, 2021 · 0 comments
Open

Python3 JSON Adapter byte string is not serializable #13

Amorik opened this issue Mar 4, 2021 · 0 comments

Comments

@Amorik
Copy link

Amorik commented Mar 4, 2021

Use of the python3 branch and the JSON adapter results in the following error:

Process Process-3:
Traceback (most recent call last):
  File "/opt/eNcore/estreamer/baseproc.py", line 208, in receiveInput
    self.onReceive( item )
  File "/opt/eNcore/estreamer/baseproc.py", line 313, in onReceive
    self.onEvent( item )
  File "/opt/eNcore/estreamer/pipeline.py", line 397, in onEvent
    data = transform( item, self.settings )
  File "/opt/eNcore/estreamer/pipeline.py", line 205, in transform
    output = adapters[ index ].dumps( event['record'] )
  File "/opt/eNcore/estreamer/adapters/json.py", line 30, in dumps
    return json.dumps( data )
  File "/usr/local/lib/python3.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

Im not very familiar with all of the code yet but it appears that the dictionary data being dumped by json.dumps contains values of byte/bytestrings/bytesequences. This was fine in python2 since it handled the encoding/decoding but that not longer happens on its own in the python3 version of JSON.

To resolve i switched from import json to import simplejson as simplejson can handle the bytes conversion. This might not be the right approach for this project, but either way the bytes objects needs to be UTF8 encoded to be JSON compliant/serializable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant