Skip to content

Conversation

Kavindya-Kariyawasam
Copy link

This PR re-enables Codementor site detection by updating [data.json] and removing it from [removed-sites.md]. Codementor now returns 200 for claimed usernames and 404 for unclaimed ones. However, Sherlock currently does not detect the profile even when it exists (ex: "blue"). Please review the detection logic for this site.

Copy link
Contributor

github-actions bot commented Oct 4, 2025

Automatic validation of changes

Target F+ Check F- Check
Codementor ✔️   Pass ❌   Fail

Failures were detected on at least one updated target. Commits containing accuracy failures will often not be merged (unless a rationale is provided, such as false negatives due to regional differences).

@akh7177
Copy link
Contributor

akh7177 commented Oct 4, 2025

Hello @Kavindya-Kariyawasam,

Was Codementor the only site changed in data.json? There seems to be quite a large diff. Could you verify it once please ?

@ppfeister ppfeister added the question Further information is requested label Oct 4, 2025
Copy link
Contributor

github-actions bot commented Oct 5, 2025

Automatic validation of changes

Target F+ Check F- Check
Codementor ❌   Fail ✔️   Pass

Failures were detected on at least one updated target. Commits containing accuracy failures will often not be merged (unless a rationale is provided, such as false negatives due to regional differences).

1 similar comment
Copy link
Contributor

github-actions bot commented Oct 5, 2025

Automatic validation of changes

Target F+ Check F- Check
Codementor ❌   Fail ✔️   Pass

Failures were detected on at least one updated target. Commits containing accuracy failures will often not be merged (unless a rationale is provided, such as false negatives due to regional differences).

@Kavindya-Kariyawasam
Copy link
Author

Hi @akh7177 , thanks a lot for pointing that out!
It looks like my Prettier extension auto-formatted some parts of the files unintentionally, I am sorry about that.
I've cleaned up the changes now and pushed an updated commit. Appreciate your help!

@ppfeister
Copy link
Member

ppfeister commented Oct 5, 2025

For when the F+ is resolved, there's no need for a dedicated test --- all sites are tested by the validate_targets test module (actually, the same one that's feeding the GitHub Actions comment)

poetry run pytest -q --tb no -rA -m validate_targets --chunked-sites "Codementor"

The -q --tb no -rA are optional of course, just improving output readability.

Skipping --chunked-sites "Codementor" will trigger the test against all sites (-n 20 recommended then, to use additional workers).

Copy link
Contributor

github-actions bot commented Oct 5, 2025

Automatic validation of changes

Target F+ Check F- Check
Codementor ✔️   Pass ❌   Fail

Failures were detected on at least one updated target. Commits containing accuracy failures will often not be merged (unless a rationale is provided, such as false negatives due to regional differences).

@Kavindya-Kariyawasam
Copy link
Author

Hi @ppfeister ,
I've removed the dedicated Codementor test script as requested and committed the change. I ran the project’s automated validation tests.
The results show that the false positive issue is resolved, but a false negative remains - Sherlock does not detect existing Codementor accounts even though the site returns 200 for claimed and 404 for unclaimed usernames.
Please let me know if you'd like me to make any further changes. Thank you for your guidance!

@akh7177
Copy link
Contributor

akh7177 commented Oct 9, 2025

Hi @ppfeister , I've removed the dedicated Codementor test script as requested and committed the change. I ran the project’s automated validation tests. The results show that the false positive issue is resolved, but a false negative remains - Sherlock does not detect existing Codementor accounts even though the site returns 200 for claimed and 404 for unclaimed usernames. Please let me know if you'd like me to make any further changes. Thank you for your guidance!

Hey @Kavindya-Kariyawasam , adding a GET request method seems to solve the issue. Kindly add the below line and lemme know if it fixes it
"request_method": "GET"

Copy link
Contributor

github-actions bot commented Oct 9, 2025

Automatic validation of changes

Target F+ Check F- Check
Codementor ❌   Fail ✔️   Pass

Failures were detected on at least one updated target. Commits containing accuracy failures will often not be merged (unless a rationale is provided, such as false negatives due to regional differences).

@akh7177
Copy link
Contributor

akh7177 commented Oct 9, 2025

@Kavindya-Kariyawasam Hm, that's wierd. Can you see check if its working locally? Or is it returning F+/F-?

Copy link
Contributor

github-actions bot commented Oct 9, 2025

Automatic validation of changes

Target F+ Check F- Check
Codementor ❌   Fail ✔️   Pass

Failures were detected on at least one updated target. Commits containing accuracy failures will often not be merged (unless a rationale is provided, such as false negatives due to regional differences).

@Kavindya-Kariyawasam
Copy link
Author

@akh7177 I reran the Codementor chunk locally with
poetry run pytest -q --tb no -rA -m validate_targets --chunked-sites "Codementor"
and both the false-positive and false-negative checks pass (2 passed, 25 deselected in 3.25s).
I'll check again.

@Kavindya-Kariyawasam
Copy link
Author

@akh7177
@ppfeister

poetry run pytest -q --tb no -rA -m validate_targets -n 20 --chunked-sites "Codementor"

This reports "2 passed in 4.17s". I also confirmed Codementor still returns 404 on GET for random usernames and 200 for "@blue". The previous CI failure looks like a transient false positive from Codementor; could you kick off another workflow run? Let me know if anything else pops up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

question Further information is requested

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants