New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create a row partitioner #16

Open

xweichu opened this issue Jun 19, 2020 · 1 comment

Collaborator

xweichu commented Jun 19, 2020

Create a row partitioner which partitions the Arrow based on rows. Make each subpartition close to 10MB.

Collaborator Author

xweichu commented Jun 23, 2020

Here is an example of the partitioner:
https://github.com/uccross/skyhookdm-pythonclient/blob/master/client/skyhookdmclient/skyhook.py#L77 , the partitioner function should be extracted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment