Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add lint/check to avoid missing entries in requirements.txt/pyproject.toml #2362

Open
mr-tz opened this issue Sep 13, 2024 · 10 comments · May be fixed by #2599
Open

add lint/check to avoid missing entries in requirements.txt/pyproject.toml #2362

mr-tz opened this issue Sep 13, 2024 · 10 comments · May be fixed by #2599
Labels
CI Continuous Integration configuration

Comments

@mr-tz
Copy link
Collaborator

mr-tz commented Sep 13, 2024

as encountered in #2359

@mr-tz mr-tz added the CI Continuous Integration configuration label Sep 13, 2024
@williballenthin
Copy link
Collaborator

williballenthin commented Sep 13, 2024

in CI we install deps from requirements.txt and then install from pyproject. but the second step should be a no-op, except to install capa. so maybe we can inspect the pip output to assert there are no extra packages being installed. or we could do pip freeze before/after and ensure the diff is a single line.

these aren't lints that we can run standalone but would help us catch things along the way

@dhruvak001
Copy link
Contributor

we can inspect the pip output to assert there are no extra packages being installed. or we could do pip freeze before/after and ensure the diff is a single line.

@williballenthin we have to make sure there is no extra dependencies as quoted or just check if we are not missing any entries from requirements.txt/pyproject.toml as stated in issue name.

@williballenthin
Copy link
Collaborator

i'm not sure how to check for extra dependencies but that would be really nice. i'd be interested in any suggestions you have.

@dhruvak001
Copy link
Contributor

@williballenthin We can run 'pip freeze > before.txt' before '- name: Install capa with build requirements' in build.yml and 'pip freeze > after.txt' this run. Then in the next run we can check the difference in the txt for extra dependency. And if we found one what needs to be done? Exit the CI Job?

@williballenthin
Copy link
Collaborator

yeah we would exit with nonzero status.

this is a good idea.

i also wonder about finding stale dependencies not referenced in code anymore, but i'm not quite sure how to do that right now.

@dhruvak001
Copy link
Contributor

@williballenthin we can use deptry or vulture library for this. They keep track of all written in the requirements and imported dependency in the codebase.

@williballenthin
Copy link
Collaborator

perfect, we already have deptry set up

@williballenthin
Copy link
Collaborator

would you please confirm it's configured to do this sort of check on our codebase?

@dhruvak001
Copy link
Contributor

Its already available and doing the work we require.
- name: Check imports against dependencies run: pre-commit run deptry --hook-stage manual

Line 63 in test.yml mentions this.

Shall i make changes for extra dependency as discussed.

We can run 'pip freeze > before.txt' before '- name: Install capa with build requirements' in build.yml and 'pip freeze > after.txt' this run. Then in the next run we can check the difference in the txt for extra dependency. And if we found one what needs to be done? Exit the CI Job?

Or we can close the issue as the qouted part is already there.

i also wonder about finding stale dependencies not referenced in code anymore, but i'm not quite sure how to do that right now.

@williballenthin
Copy link
Collaborator

yes please go ahead and add the check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration configuration
Projects
None yet
3 participants