Skip to content

SLEP needed: fit_transform does something other than fit(X).transform(X) #12

Open
@amueller

Description

@amueller
Member

This is required for stacking and leave-one-out target encoders if we want a nice design. We already kind of do this in some places but don't have a coherent contract. I think a slep would be nice.

Activity

raamana

raamana commented on Feb 7, 2019

@raamana

A FitTransformMixin common to all estimators would solve it perhaps?

amueller

amueller commented on Feb 7, 2019

@amueller
MemberAuthor

How so? The point is whether we can adjust the contract / user expectations. Either we need to redefine the contract or we need to provide an interface.
The implementation is the easy part (that's probably true for all the SLEPs)

jnothman

jnothman commented on Feb 7, 2019

@jnothman
Member
amueller

amueller commented on Feb 17, 2020

@amueller
MemberAuthor

I just realized that this is the place where I should have put scikit-learn/scikit-learn#15553

Basically what MLR does is that fit returns the training-time version of transform which basically solves the problem without introducing a new verb/method.

However, method chaining is so idiomatic in sklearn that I don't really see how to get around this. We could also "just" introduce a different verb/method say forward that does training time transformation and returns X and y that's used in meta-estimators (mostly pipeline).

That would also solve #15.

amueller

amueller commented on Feb 17, 2020

@amueller
MemberAuthor

actually SLEP 001 (#2) basically implements something like this, only SLEP 001 also has a test-time version of it, which I think is a bit weird, or rather it conflates two issues: distinguishing training vs test time, and returning a complex object / tuple.

amueller

amueller commented on Dec 7, 2020

@amueller
MemberAuthor

There's a relevant discussion happening here:
#15 (comment)

Maybe it's fair to say that there are three extensions to transform we're considering:

  1. doing something that destroys the sample-wise alignment of input and output
  2. doing something different on training and test data
  3. returning something other than the transformed data (i.e. targets etc).

We're doing all three in SLEP 5, but with a focus on doing non-sample-aligned transformations, and only allowing somewhat trivial versions of 2 and 3.

We need some more elaborate version of 2) for the target encoder (that I've brought up many times now lol): scikit-learn/scikit-learn#17323

SLEP 1 tries to do all of the together, I think which might not be necessary (we couldn't come up with good examples).

We could achieve stacking and target transformers and more straight-forward handling in the neighbor graphs by:

  • "just allowing" fit_transform != transform. I haven't thought deeply about the consequences yet. What are the issues that will come up?
  • piggy-backing on resampling as @jnothman suggests above and re-use fit_resample to do the training-set transformation. In that case I still think it's weird to call the method resample. In that case we'd also have the potential to modify y during training.
  • add another verb, and enforce sample-alignment (and maybe or maybe not also return y).

cc @GaelVaroquaux who would like to move this forward :)

amueller

amueller commented on Dec 8, 2020

@amueller
MemberAuthor

Good point that @GaelVaroquaux brought up that was probably also in the other thread:
we don't want to allow fit_resample in a ColumnTransformer but we want to allow the target encoder.

jnothman

jnothman commented on Dec 10, 2020

@jnothman
Member
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @jnothman@amueller@raamana

        Issue actions

          SLEP needed: fit_transform does something other than fit(X).transform(X) · Issue #12 · scikit-learn/enhancement_proposals