Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provide a helpful error message on "Row size too large" #234

Merged
merged 2 commits into from
Feb 15, 2024
Merged

Provide a helpful error message on "Row size too large" #234

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
68880da
Provide helpful message on row size too large
kaapstorm Feb 13, 2024
531df4b
Explain the error better, offer a guideline.
kaapstorm Feb 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion commcare_export/env.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import hashlib
import json
import logging
import operator
import sys
import uuid
from typing import Any, Dict, Union, overload

Expand All @@ -12,6 +14,8 @@
from jsonpath_ng import jsonpath
from jsonpath_ng.parser import parse as parse_jsonpath

logger = logging.getLogger(__name__)

JSONPATH_CACHE = {}


Expand Down Expand Up @@ -590,7 +594,20 @@ def lookup(self, key):
def emit_table(self, table_spec):
self.emitted = True
table_spec.rows = self._unwrap_row_vals(table_spec.rows)
self.writer.write_table(table_spec)
try:
self.writer.write_table(table_spec)
except Exception as err:
if (
not logger.isEnabledFor(logging.DEBUG) # not --verbose
and 'Row size too large' in str(err)
):
logging.error(
'Row size too large. You may be trying to export too many '
'columns. A maximum of 200 columns is suggested.'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A maximum of 200 columns is suggested.

I don't know if we should make it a suggestion, as this makes it sound like the user can still go a little above this limit. Given we have a hard-set limit, it might be better to phrase this as something like "A maximum of 200 columns is supported."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should (I hope) explain the error better, and frame the 200-column guideline: 531df4b

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds great, thanks for the rework!

)
sys.exit(1)
else:
raise

def has_emitted_tables(self):
return self.emitted
Expand Down
Loading