✨Feedback from user survey: System input robustness

[joeyylow]

Cc:

Hi

Pain

The system is currently unable to accept input with non-conventional charset/ encoding. Some examples include emojis (🙃) and characters in foreign languages (र).

Suggested Solution

Allow the system to read and process files with non-conventional charset/ encoding without returning an error.

Tasks

• Research on the more frequently used non-conventional charset/ encoding that could appear in assignments.
• Ensure that the system is able to read and process them.

Acceptance Criteria

Non-conventional charset/ encoding can be read and processed.

More Information

The feedback provided originates directly from the user feedback surveys and may include requirements and suggestions that lack clarity. The SSID team should evaluate and decide whether the team should address the identified issues or reject suggestions that do not provide clear and actionable insights.

Screenshots

Thanks!

[sibinhho99]

Currently emojis and special characters are scrubbed from files as of
https://github.com/WING-NUS/SSID/pull/244/files

The ranges to exclude can be found in the code diff above, though I don't remember the source