Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix response handling #331

Merged
merged 4 commits into from
Jan 12, 2025
Merged

Fix response handling #331

merged 4 commits into from
Jan 12, 2025

Conversation

biskweet
Copy link
Contributor

@biskweet biskweet commented Jan 7, 2025

Following

Here is the fix for the piratebay search engine. A gist of the code is available here for testing.

Recalling the problem:

API apibay.org returns weird JSON that causes the piratebay search engine to crash when handling its response. If some search results contain " (quotation marks) characters, the server escapes them by replacing " with " HTML entities in order to still provide a syntactically valid JSON response. While this is not incorrect, it would be best if apibay.org returned properly escaped quotes, i.e. using backslashes.

When handling the response data, functions retrieve_url and htmlentitydecode blindly unescape all entities thereby corrupting previously valid JSON. As a consequence, json.loads crashes. For example:

{
  "title": "Ubuntu 22.04.5 LTS ("Jammy Jellyfish")"
}

becomes

{
  "title": "Ubuntu 22.04.5 LTS ("Jammy Jellyfish")"
}

Solution proposed

We no logner use the retrieve_url function -- instead, I created a dedicated retrieve_url function (which is almost a copy-paste of the orignal) that fixes the problem by manually escaping quotes before escaping the rest of the data.

Copy link
Member

@Chocobo1 Chocobo1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take a look at the CI. There are errors present.
Also ignore the "line too long" error. I'll turn it off.

nova3/engines/piratebay.py Show resolved Hide resolved
nova3/engines/piratebay.py Outdated Show resolved Hide resolved
nova3/engines/piratebay.py Outdated Show resolved Hide resolved
@biskweet biskweet requested a review from Chocobo1 January 12, 2025 01:15
nova3/engines/piratebay.py Outdated Show resolved Hide resolved
nova3/engines/piratebay.py Outdated Show resolved Hide resolved
@Chocobo1 Chocobo1 changed the title Fix apibay.org search engine API response handling Fix response handling Jan 12, 2025
@Chocobo1 Chocobo1 merged commit 37e6741 into qbittorrent:master Jan 12, 2025
8 checks passed
@Chocobo1
Copy link
Member

@biskweet
Thank you!

@biskweet biskweet deleted the fix_escaping branch January 12, 2025 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants