If the acquisition funcition is GP_hedge, do we know how often it uses the different possible acquisition functions?

[SRFU-NN]

If acq_func = "gp_hedge" and xi is set too low, there is a tendency for consecutive suggested points to be very close.

Playing around, this behaviour seems to be typical of acq_func = "PI". I wonder if what really happens is that LCB is always evaluated to be bad, and EI becomes bad if xi is slow, so PI dominates?

Should we remove PI as an option when acq_func = "gp_hedge"? It doesn't seem to give very good results.

Note that I have only tested this on a modified Branin-Hoo score function, so there has NOT been done enough work to come to a decision.

[SRFU-NN]

Also, doesn't the choice we do in gp_hedge also favours exploitation?

As I read the code, it randomly choses which acquisition function to use, with the probabilities depending on the NEW surrogate functions predictions for the OLD suggested next points. But an exploring acquisition function will suggest a point that has a low expected result. So if the last choice did not go with the exploring acquisition function, the new surrogate model will predict that it as a bad point, and if the last choice did go with the exploring acquisition function, it will only have an advantage in the unlikely event that it was much better than expected.

Let me know if that doesn't make sense, I have a feeling I did not explain it as well as I wanted to.