Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP correct CI errors relating to partial_fit #58

Merged
merged 9 commits into from
Aug 28, 2023
Merged

Conversation

PSSF23
Copy link
Member

@PSSF23 PSSF23 commented Aug 24, 2023

Reference Issues/PRs

Fixes #57

What does this implement/fix? Explain your changes.

Any other comments?

@github-actions
Copy link

github-actions bot commented Aug 24, 2023

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: b22d33a. Link to the linter CI: here

@adam2392
Copy link
Collaborator

adam2392 commented Aug 25, 2023

For the segfault that occurs with honest forest, checkout code here neurodata/treeple#118

and run: pytest ./sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator

@PSSF23
Copy link
Member Author

PSSF23 commented Aug 26, 2023

I could not run the test because it always give this import error or check_build, even when sktree (installed through main) runs properly in notebooks and scripts.

ImportError while importing test module '/Users/pssf23/GitHub/scikit-tree/sktree/tests/test_honest_forest.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
scikit-tree/sktree/_lib/sklearn/__check_build/__init__.py:45: in <module>
    from ._check_build import check_build  # noqa
E   ModuleNotFoundError: No module named 'sktree._lib.sklearn.__check_build._check_build'

@adam2392
Copy link
Collaborator

Oh maybe after building using meson. Run the pip install editable line on the readme?

Otw you can't run pytest directly.

Copy link
Member Author

@PSSF23 PSSF23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous commit reduced the only failing CI to macOS pylatest_conda_mkl_no_openmp, but it shows Segmentation Fault again. I really don't understand why all others passed.

This time I removed the parallel preference and see if this has any effects.

Edit: parallel change seems irrelevant. But I can finally recreate the error on my local machine. It probably has something to do with how initial_roots works, so I'll try to optimize the object.

Copy link
Member Author

@PSSF23 PSSF23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adam2392 I have modified the initial_roots and how it's stored in tree builders. Can you test whether Honest Forest work now on your end? I'm so confused by this bug. It seems that I cannot satisfy all system configurations at once (before it was mac, right now it's windows), but at least there should be improvements.

@adam2392
Copy link
Collaborator

@adam2392 I have modified the initial_roots and how it's stored in tree builders. Can you test whether Honest Forest work now on your end? I'm so confused by this bug. It seems that I cannot satisfy all system configurations at once (before it was mac, right now it's windows), but at least there should be improvements.

It seems there is still a segfault in neurodata/treeple#118. If you want to try it out:

./spin build -j 6 --clean
pip install --no-build-isolation --editable .
pytest ./sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator

@adam2392
Copy link
Collaborator

I get:

(sktree) adam2392@adams-mbp-3 scikit-tree % pytest ./sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator
=============================================== test session starts ================================================
platform darwin -- Python 3.9.15, pytest-7.4.0, pluggy-1.0.0 -- /Users/adam2392/miniforge3/envs/sktree/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/adam2392/Documents/scikit-tree
configfile: pyproject.toml
plugins: parallel-0.1.1, cov-4.1.0, typeguard-2.13.3
collected 56 items                                                                                                 

sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator[HonestForestClassifier(n_estimators=10,random_state=0)-check_no_attributes_set_in_init] PASSED [  1%]
sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator[HonestForestClassifier(n_estimators=10,random_state=0)-check_estimators_dtypes] PASSED [  3%]
sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimator[HonestForestClassifier(n_estimators=10,random_state=0)-check_fit_score_takes_y] Fatal Python error: Segmentation fault

Current thread 0x00000001f8815e00 (most recent call first):
  File "/Users/adam2392/Documents/scikit-tree/sktree/_lib/./sklearn/tree/_classes.py", line 1377 in partial_fit
  File "/Users/adam2392/Documents/scikit-tree/sktree/tree/_honest_tree.py", line 426 in partial_fit
  File "/Users/adam2392/Documents/scikit-tree/sktree/_lib/./sklearn/ensemble/_forest.py", line 252 in _parallel_update_trees
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/sklearn/utils/parallel.py", line 127 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/joblib/parallel.py", line 1792 in _get_sequential_output
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/joblib/parallel.py", line 1863 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/sklearn/utils/parallel.py", line 65 in __call__
  File "/Users/adam2392/Documents/scikit-tree/sktree/_lib/./sklearn/ensemble/_forest.py", line 1332 in partial_fit
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/sklearn/utils/estimator_checks.py", line 1816 in check_fit_score_takes_y
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/sklearn/utils/_testing.py", line 180 in wrapper
  File "/Users/adam2392/Documents/scikit-tree/sktree/tests/test_honest_forest.py", line 192 in test_sklearn_compatible_estimator
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/python.py", line 1788 in runtest
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 341 in from_call
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/main.py", line 324 in _main
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/main.py", line 270 in wrap_session
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/config/__init__.py", line 166 in main
  File "/Users/adam2392/miniforge3/envs/sktree/lib/python3.9/site-packages/_pytest/config/__init__.py", line 189 in console_main
  File "/Users/adam2392/miniforge3/envs/sktree/bin/pytest", line 8 in <module>
zsh: segmentation fault  pytest ./sktree/tests/test_honest_forest.py::test_sklearn_compatible_estimato

Copy link
Member Author

@PSSF23 PSSF23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adam2392 adam2392 merged commit 3847773 into submodulev3 Aug 28, 2023
16 of 17 checks passed
@adam2392
Copy link
Collaborator

Okay merging in since this seems to fix the CIs. Thanks @PSSF23 !

@adam2392 adam2392 deleted the resize_fix branch August 28, 2023 18:22
adam2392 added a commit that referenced this pull request Sep 8, 2023
<!--
Thanks for contributing a pull request! Please ensure you have taken a
look at
the contribution guidelines:
https://github.com/scikit-learn/scikit-learn/blob/main/CONTRIBUTING.md
-->

#### Reference Issues/PRs
<!--
Example: Fixes scikit-learn#1234. See also scikit-learn#3456.
Please use keywords (e.g., Fixes) to create link to the issues or pull
requests
you resolved, so that they will automatically be closed when your pull
request
is merged. See
https://github.com/blog/1506-closing-issues-via-pull-requests
-->
Fixes #57 

#### What does this implement/fix? Explain your changes.


#### Any other comments?


<!--
Please be aware that we are a loose team of volunteers so patience is
necessary; assistance handling other issues is very welcome. We value
all user contributions, no matter how minor they are. If we are slow to
review, either the pull request needs some benchmarking, tinkering,
convincing, etc. or more likely the reviewers are simply busy. In either
case, we ask for your understanding during the review process.
For more information, see our FAQ on this topic:

http://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.

Thanks for contributing!
-->

---------

Co-authored-by: Adam Li <adam2392@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Segmentation faults during partial_fit for check_fit_score_takes_y unit-test in scikit-learn
2 participants