Skip to content

Commit

Permalink
perf: Faster report exporting logic (frappe#21415)
Browse files Browse the repository at this point in the history
* perf: Faster report exports to Excel

Try exporting report with 100K rows, most likely it fails or takes a really long time.

Root causes:
- visible_idx was a list, lookups are SLOW AF when lists grow to 100k+
- visible_idx check is not required when I am exporting entire report.

Test with 85,000 rows.

|         | Before | After | Improvement |
| ---     | ---    | ---   | ---         |
| Export  | 25.99  | 0.77  | ~33x faster |
  • Loading branch information
ankush committed Jun 17, 2023
1 parent 53b074c commit ae0edd8
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions frappe/desk/query_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,13 @@ def build_xlsx_data(data, visible_idx, include_indentation, ignore_visible_idx=F
datetime.timedelta,
)

if len(visible_idx) == len(data.result):
# It's not possible to have same length and different content.
ignore_visible_idx = True
else:
# Note: converted for faster lookups
visible_idx = set(visible_idx)

result = [[]]
column_widths = []

Expand Down

0 comments on commit ae0edd8

Please sign in to comment.