Provide a helpful error message on "Row size too large"

[kaapstorm]

Context: SC-3355

Sample output:

2024-02-13 23:57:04,479 commcare_export.writers WARNING  Adding column 'impact_case.column_112 VARCHAR(32) COLLATE "utf8mb4_unicode_ci"'
2024-02-13 23:57:04,509 root         ERROR    Row size too large. You may be trying to export too many columns. A maximum of 200 columns is suggested.

Full stacktrace when called with --verbose:

2024-02-13 23:31:52,206 commcare_export.writers WARNING  Adding column 'impact_case.column_111 VARCHAR(32) COLLATE "utf8mb4_unicode_ci"'
2024-02-13 23:31:52,244 commcare_export.writers WARNING  Adding column 'impact_case.column_112 VARCHAR(32) COLLATE "utf8mb4_unicode_ci"'
Traceback (most recent call last):
  File "commcare_export/env.py", line 136, in emit_table
    return self.left.emit_table(table_spec)
  File "commcare_export/env.py", line 79, in emit_table
    raise CannotEmit()
commcare_export.env.CannotEmit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "commcare_export/env.py", line 136, in emit_table
    return self.left.emit_table(table_spec)
  File "commcare_export/env.py", line 138, in emit_table
    return self.right.emit_table(table_spec)
  File "commcare_export/env.py", line 79, in emit_table
    raise CannotEmit()
commcare_export.env.CannotEmit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "commcare_export/env.py", line 136, in emit_table
    return self.left.emit_table(table_spec)
  File "commcare_export/env.py", line 138, in emit_table
    return self.right.emit_table(table_spec)
  File "commcare_export/env.py", line 79, in emit_table
    raise CannotEmit()
commcare_export.env.CannotEmit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
    self.dialect.do_execute(
  File "venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
  File "venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
    result = self._query(query)
  File "venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result
    result.read()
  File "venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read
    first_packet = self.connection._read_packet()
  File "venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet
    packet.raise_for_error()
  File "venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.OperationalError: (1118, 'Row size too large. The maximum row size for the used table type, not counting BLOBs, is 8126. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "venv/bin/commcare-export", line 33, in <module>
    sys.exit(load_entry_point('commcare-export', 'console_scripts', 'commcare-export')())
  File "commcare_export/cli.py", line 479, in entry_point
    main(sys.argv[1:])
  File "commcare_export/cli.py", line 217, in main
    exit(main_with_args(args))
  File "commcare_export/cli.py", line 464, in main_with_args
    exit_status = evaluate_query(env, query)
  File "commcare_export/cli.py", line 369, in evaluate_query
    lazy_result = query.eval(env)
  File "commcare_export/minilinq.py", line 149, in eval
    return self.body.eval(env.bind(self.name, self.value.eval(env)))
  File "commcare_export/minilinq.py", line 522, in eval
    env.emit_table(
  File "commcare_export/env.py", line 138, in emit_table
    return self.right.emit_table(table_spec)
  File "commcare_export/env.py", line 593, in emit_table
    self.writer.write_table(table_spec)
  File "commcare_export/writers.py", line 616, in write_table
    table = self.make_table_compatible(table, row_dict, data_type_dict)
  File "commcare_export/writers.py", line 518, in make_table_compatible
    op.add_column(
  File "<string>", line 3, in add_column
  File "venv/lib/python3.10/site-packages/alembic/operations/ops.py", line 2142, in add_column
    return operations.invoke(op)
  File "venv/lib/python3.10/site-packages/alembic/operations/base.py", line 445, in invoke
    return fn(self, operation)
  File "venv/lib/python3.10/site-packages/alembic/operations/toimpl.py", line 171, in add_column
    operations.impl.add_column(table_name, column, schema=schema, **kw)
  File "venv/lib/python3.10/site-packages/alembic/ddl/impl.py", line 334, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "venv/lib/python3.10/site-packages/alembic/ddl/impl.py", line 207, in _exec
    return conn.execute(construct, multiparams)
  File "venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1385, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "venv/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
    return connection._execute_ddl(
  File "venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1477, in _execute_ddl
    ret = self._execute_context(
  File "venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
    self._handle_dbapi_exception(
  File "venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapi_exception
    util.raise_(
  File "venv/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
    self.dialect.do_execute(
  File "venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
  File "venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
    result = self._query(query)
  File "venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result
    result.read()
  File "venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read
    first_packet = self.connection._read_packet()
  File "venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet
    packet.raise_for_error()
  File "venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1118, 'Row size too large. The maximum row size for the used table type, not counting BLOBs, is 8126. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs')
[SQL: ALTER TABLE impact_case ADD COLUMN column_112 VARCHAR(32) COLLATE utf8mb4_unicode_ci]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

[zandre-eng]

Looks good, just one non-blocking comment.

[zandre-eng]

A maximum of 200 columns is suggested.

I don't know if we should make it a suggestion, as this makes it sound like the user can still go a little above this limit. Given we have a hard-set limit, it might be better to phrase this as something like "A maximum of 200 columns is supported."

[kaapstorm]

This should (I hope) explain the error better, and frame the 200-column guideline: 531df4b

[zandre-eng]

It sounds great, thanks for the rework!