Implement an outcome_constraint in Ax

[aleczhanshi]

Hi there, I was trying to implement an outcome constraint in Ax and BO with SimpleExperiment. I came across the code example below:

exp = SimpleExperiment(
    name="test_branin",
    search_space=hartmann_search_space,
    evaluation_function=hartmann_evaluation_function,
    objective_name="hartmann6",
    minimize=True,
    outcome_constraints=[
        OutcomeConstraint(
            metric=L2NormMetric(
                name="l2norm", param_names=[f"x{i}" for i in range(6)], noise_sd=0.2
            ),
            op=ComparisonOp.LEQ,
            bound=1.25,
            relative=False,
        )
    ],
)

In particular, my outcome constraint is x1*x2+16*x1+16*x2<=64 where x1 and x2 are integer parameters, and it's not clear to me how to implement this specific constraint here and what my "metric" should me.

Any help is appreciated, thanks!

[Balandat]

I assume your x are your parameters? The outcome constraint is typically used to model black-box constraint, i.e. on outcomes whose functional form is unknown and needs to be modeled as the objective does.

If you know the constraints then it’s typically preferred to include them as parameter constraints directly on the parameters. Now this is a nonlinear constraint, so that is unfortunately not straightforward to do in Ax.

Do you have other parameters as well? If not, then the size of feasible set In your case is quite small and could easily be enumerated: {(4, 0), (3,,1), (3,0), ...

A general problem with this kind of non-convex constraint set is that getting a reasonable model might be challenging, but it would be worth a try.

[aleczhanshi]

@Balandat Thanks for the quick response! Yeah I'm implementing it as an outcome constraint due to the nonlinearity. It is understood that outcome constraints are typically used for black-box metrics, but I assume it could also be closed-form right? I want to try this out to see if it helps to have those non-linear constraints as outcome constraints.

[lena-kashtelyan]

The constraint-in-question (x1x2+16x1+16*x2<=64) would still need to be a parameter constraint, simply because of the way the parameter vs. outcome constraints are applied in Ax (outcome constraints apply specifically to constraints on metric outcomes, and changing this would require a series of structural changes to Ax). Non-linear constraints are not available as outcome constraints either, I just wanted to emphasize that a parameter constraint cannot be forced to be a synthetic outcome constraint in either case.

Here is the duplicate issue for non-linear parameter constraints: #153 we already have open.