Serialized data

[gitonthescene]

Hi, I realize this isn't actively developed, but I thought I'd mention that with growler/k you can serialize K data. Moreover changes have been made to the repl so that if your command is preceded with two spaces, the output is the serialized data. The idea being that you can deserialize the bytes directly to numpy for instance for access to graphing, etc.

Sample session:

>>> p = Popen(('/Users/XXXX/K/ngnk/growler/k/k','/Users/XXXX/K/ngnk/growler/k/repl.k'),stdin=PIPE,stdout=PIPE,close_fds=False)
>>> p.stdout.readline()
b'ngn/k, (c) 2019-2024 ngn, GNU AGPLv3. type \\ for more info\n'
>>> p.stdin.write('  (1+1;`happy)\n'.encode())
15
>>> p.stdin.flush()
>>> p.stdout.read(1)
b' '
>>> ver = ord(p.stdout.read(1))
>>> serialize.KObjFactory.setVersion(ver)
>>> k = serialize.KObjFactory.create(p.stdout)
>>> k
<2, `happy>
>>> p.stdin.write('  (1+1;`happy;(39.7;9 8 2))\n'.encode())
28
>>> p.stdin.flush()
>>> p.stdout.read(1)
b' '
>>> ver = ord(p.stdout.read(1))
>>> serialize.KObjFactory.setVersion(ver)
>>> k = serialize.KObjFactory.create(p.stdout)
>>> k
<2, `happy, <39.7, [9, 8, 2]>>
>>> p.stdin.write(' (1+1;`happy;(39.7;9 8 2))\n'.encode())
27
>>> p.stdin.flush()
>>> p.stdout.readline()
b' (2;`happy;(39.7;9 8 2))\n'
>>> p.stdin.write('(1+1;`happy;(39.7;9 8 2))\n'.encode())
26
>>> p.stdin.flush()
>>> p.stdout.readline()
b' (2\n'
>>> p.stdout.readline()
b' `happy\n'
>>> p.stdout.readline()
b' (39.7;9 8 2))\n'
>>>

I may try to work out my own kernel but I thought I'd offer the heads up.

Regards,
-Doug

[gitonthescene]

FWIW, you can use unbuffered streams to handle this last one better:

>>> p = Popen(('/Users/XXXX/K/ngnk/growler/k/k','/Users/XXXX/K/ngnk/growler/k/repl.k'),stdin=PIPE,stdout=PIPE,close_fds=False,bufsize=0)
>>> p.stdout.readline()
b'ngn/k, (c) 2019-2024 ngn, GNU AGPLv3. type \\ for more info\n'
>>> p.stdin.write('(1+1;`happy;(39.7;9 8 2))\n'.encode())
26
>>> p.stdout.read(1024)
b' (2\n `happy\n (39.7;9 8 2))\n '
>>> 

Non blocking is probably a good idea too:

>>> fd = p.stdout.fileno()
>>> fl = fcntl.fcntl(fd, fcntl.F_GETFL)
>>> fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)
0
>>> p.stdin.write('(1+1;`happy;(39.7;9 8 2))\n'.encode())
26
>>> p.stdout.read(10)
b'(2\n `happy'
>>> p.stdout.read(10)
b'\n (39.7;9 '
>>> p.stdout.read(10)
b'8 2))\n '
>>> p.stdout.read(10)
>>> 

[gitonthescene]

I don't use Jupyter much myself, but it looks like this might be relevant as well. Though simply serializing the data to disk and reading from disk in python should likely work just as well.