Real space have different default transformations depending on how it is initialised

[SRFU-NN]

Real dimension have different default transformations, depending on how they are defined:

This shouldn't be the case, but what should be the default?

[sqbl]

This is suboptimal. For now, I shall refrain from asking why it does so...

I'm just throwing this one in: "Shouldn't it always 'normalize'?"

[SRFU-NN]

The Optimizer only does this if the base estimator is a Gaussian Process Regressor (line 290 in optimizer.py). To quote the docstring of normalize_dimensions:

"Create a Space where all dimensions are normalized to unit range. This is particularly useful for Gaussian process based regressors [...]"

The easy fix is to change the default for Space to "normalize". But that would entail changing the default behavior for all regressors.

[sqbl]

I assume that 98% of our use cases are GP's, I feel that it would be fair.
BUT... If you take 'direct_space' from above and shove into an optimizer, it will change the transformation into normalize. So, you COULD argue that it is intended (or at least not fully unintended)

[SRFU-NN]

The next question is then: Do we need the option for identity transform? When is it an advantage? Or could we simplify the code by removing it?

[sqbl]

I know that I'm argue back and forth here... But we need the indentity transform for random forests type regressors, right? (although, we don't use them much).
So as I see it, the confusion and the "problem" to users is that a Space is not necessarily the same as a opt.space and that a Space can be turned into a opt.space with changes that are not apparent to the user. Or that a freshly instantiated Space does not behave as a opt.space which you might have gotten from an ongoing optimization.(?)
Asking ignorantly, is it trivial to go from "Indentity" to normalize (I assume yes)? Is it equally trivial to go the other way?

[SRFU-NN]

" But we need the indentity transform for random forests type regressors, right? "

Would they suffer from being transformed? Is the result of a random forest regressor on [(0,1), (0,100)] significantly different from the same on [(0,1),(0,1)], disregarding the scaling? Is distance used in random forest regressors? (yes, that is the same question three times, I think).

"So as I see it, the confusion and the "problem" to users is that a Space is not necessarily the same as a opt.space and that a Space can be turned into a opt.space with changes that are not apparent to the user. Or that a freshly instantiated Space does not behave as a opt.space which you might have gotten from an ongoing optimization."

I would expect the implicitly defined Space from Optimizer to be identical to the explicitly defined one. They don't have to be if there is a good reason for the discrepancy, then we just have to write that clearly.

"is it trivial to go from "Indentity" to normalize (I assume yes)"

Yes, the normalize_dimensions function does that.

"Is it equally trivial to go the other way?"

Such a function It would have to handle Categorical as well. They are one-hot by default but can be identity transformed. I would prefer not dealing with that complexity, since I assume we would normally want to keep them as one-hot.

[sqbl]

All good points.
Lat's make absolutely sure that there are no obvious "exclusive" use-case for Identity (i.e. a case that breaks in the absence of identity). And then do a deprecation warning(?)
We normalize normalize, then

[SRFU-NN]

@dk-teknologisk-mon @RuneChristensen-NN, if we have time, I would like this to be discussed Friday. I need the conclusion before I can finish my documentation on Space.

[SRFU-NN]

@RuneChristensen-NN and I discussed it this morning. Rune pointed out that other uses of Space, e.g. DoE, would make more sense with non-transformed spaces, and that the non-transformed is what you expect.

To I would suggest adding a transform parameter to Space.__init__() that is False by default. And then have Optimizer set that to True if it gets a list of lists as dimensions, and if the base estimator is a Gaussian Process Regressor.

That way, Space gives you the simple option by default, Optimizer gives you what you most likely want by default, and you can define your own spiffy Space and give that to Optimizer if you want something specific.

This means that Optimizer(dimensions=[[1.0,10.0],[1.0,10.0]]) and Optimizer(dimensions=Space([[1.0,10.0],[1.0,10.0]])) yields different results. I think this is an acceptable compromise between the two needs.

[SRFU-NN]

As discussed on the nerdy meeting today, we keep the current behavior, since people probably want to transform their space when doing Gaussian Process Regression, while they probably don't for other uses.

I will try and document it on Optimizer in the pull request I will make for documentation of Space.