Not able to pickle AnalyzerEngine object. Is there any known issue?

[SushantM]

Hi, I am trying to pickle object and store it in redis, so that during the next invocation, I dont have to construct the object again. Tried same with AnonymizerEngine, its working fine. Getting following error during for AnalyzerEngine.

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 706, in send_packed_command
    sendall(self._sock, item)
  File "/usr/local/lib/python3.9/site-packages/redis/_compat.py", line 9, in sendall
    return sock.sendall(*args, **kwargs)
OSError: [Errno 41] Protocol wrong type for socket

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/susham/Downloads/redis_test.py", line 43, in <module>
    r.set("analyzer_object", pickled_analyzer_object)
  File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 1801, in set
    return self.execute_command('SET', *pieces)
  File "/usr/local/lib/python3.9/site-packages/redis/client.py", line 900, in execute_command
    conn.send_command(*args)
  File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 725, in send_command
    self.send_packed_command(self.pack_command(*args),
  File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 717, in send_packed_command
    raise ConnectionError("Error %s while writing to socket. %s." %
redis.exceptions.ConnectionError: Error 41 while writing to socket. Protocol wrong type for socket.

[omri374]

Hi @SushantM, it might be an issue with pickling the spacy model used by the AnalyzerEngine.
In general we don't support python 3.9 yet. It's worth trying with 3.8 to see if this helps.

[omri374]

BTW, not sure where redis is coming from. Which Presidio analyzer version are you using?

[SushantM]

I tried with python 3.8, pickling did work and I was able to persist the analyzer object in a file on the same machine, and reuse in the next run. Thanks.
Was trying to persist the pickled object in redis, however that is not working.

[omri374]

Not sure why redis persistence didn't work. The main issue is with spacy. All the other parts in Presidio Analyzer should be easily serialized.