Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Streaming cube and cards directly to file will reduce memory footprint for export scripts. It may improve performance, but I didn't have a large dataset to try to run it against. I didn't fully understand the export decks script, so I didn't try to optimize that one yet.
Future work
For the cubes export, if it is possible to do a snapshot on the first run (export all results, as currently), then do a delta from the modified date on subsequent runs, it would be possible to significantly reduce the time/cost of running this process regularly. That approach would add some complexity to the exports consumer to reconstruct the current cube state by reading applying deltas of cubes changed to the snapshot of initial cubes exported.
If there is another bottleneck (ex. decks export, etc.) can do another pass of optimization.