Skip to content

Conversation

@gene-peters
Copy link

When pulling result rows from Thrift, the current cursor attempts to load them all into a single variable which it then returns. This can cause major issues with large result sets; I've implemented a cursor class which returns a generator object and iterates over each result row.

Example usage:

from cursor import SSCursor

conn = pyhs2.connect(cursorclass=SSCursor, **kwargs)
cur = conn.cursor()
cur.execute("SELECT * FROM table LIMIT 100")
for result in cur.fetch():
...print result

@davidbgk
Copy link

I second that PR, this is very important when you deal with big data (otherwise your ram will explode).

@ChuckConnell
Copy link

Please see this long discussion #8 on the same topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants