Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test friend account and Unicode in Submission.student_uniqname #13

Open
ssciolla opened this issue Jun 30, 2020 · 4 comments
Open

Test friend account and Unicode in Submission.student_uniqname #13

ssciolla opened this issue Jun 30, 2020 · 4 comments
Labels
api Involves changes to/discussion of API usage

Comments

@ssciolla
Copy link
Contributor

ssciolla commented Jun 30, 2020

Raised by @pushyamig while reviewing PR #12

In the previous iteration of the project, we have had issues sending scores with emails and Unicode in the string for the students' uniqname. While this problem may be obviated by the new batch M-Pathways endpoint -- since the strings are passed as JSON, rather than in the URL -- we still need to test to ensure this works properly given these cases. If there are issues, tests and fixtures should be created to ensure the code properly handles these strings.

@ssciolla ssciolla self-assigned this Jun 30, 2020
@ssciolla ssciolla added the api Involves changes to/discussion of API usage label Jun 30, 2020
@ssciolla
Copy link
Contributor Author

ssciolla commented Aug 4, 2020

After doing some testing, I determined that the database and process can handle most Unicode characters, but the database appears to struggle with the four-byte UTF8 characters. Colleagues have been working on handling this in other applications (see tl-its-umich-edu/my-learning-analytics#992), and django_mysql describes the issue and how to handle it here: https://django-mysql.readthedocs.io/en/latest/checks.html#django-mysql-w003-utf8mb4

It seems like, if this doesn't cause other issues, it would be worth resolving now.

@ssciolla
Copy link
Contributor Author

ssciolla commented Aug 6, 2020

Regarding friend accounts and special characters, there are almost no cases of non-ASCII characters in Canvas unique names (or login_ids). The only one that I did find was a non-breaking space, which doesn't require utf8mb4 (I believe it's part of the latin1 supplement) and was only encountered once. Provided there is not leading or trailing whitespace (see issue #36), our code and the M-Pathways endpoint should be able to handle non-four-byte Unicode characters. (The M-Pathways endpoint struggled to handle those characters, as did the docker-mysql database; see the above comment).

@ssciolla
Copy link
Contributor Author

ssciolla commented Aug 7, 2020

The utf8m4 changes that are needed are slightly out of scope for this issue, as the M-Pathways endpoint can't handle the four-byte Unicode characters anyway (nor does it seem like we need to, based on my analysis). I'll open a separate issue.

@ssciolla
Copy link
Contributor Author

PRs #38 and #39 should handle the issues regarding friend accounts and Unicode that I have identified in our application. Going forward, we'll need to investigate and monitor how the application and M-Pathways perform with respect to friend accounts. I am going to leave this issue open as a reminder to keep an eye on this. If all concerns are dissipated in the future, we can resolve.

@ssciolla ssciolla removed their assignment Feb 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Involves changes to/discussion of API usage
Projects
None yet
Development

No branches or pull requests

1 participant