modified the test to have a unique answer

[IssamT]

cc @odow

[mlubin]

How does this relate to #127?

[IssamT]

#127 and #130 suggest that cd could be scaled by any positive value.

Depending on how you define the infeasibility certificate , one may argue that it can be scaled by any value (not only positive one): Since this one constraint 2x+y <= -1 can't be feasible while respecting the non negativity constraints, then multiplying both sides by any constant would still lead to the same result.

But in the following, we need a linear combination of 2 constraints to certify infeasibility

2x-3y <= -7
       y <= 2

the vector [1 , 3] is an infeasibility certificate. Here again, any vector [x, 3x] is also an infeasibility certificate, but it makes sense to test that the second component of the vector is 3 times the first component.

[mlubin]

Could you also check the dual solution on the bounds so that we can close #127? [1, 3] is not the full infeasibility certificate, you also need the nonnegativity on x.

[blegat]

@IssamT I don't agree. The ratio does not have to be 3.
The dual of

min x
 s.t. 2x-3y <= -7
       y <= 2
       x,y >= 0

is

max -7a + 2b
 s.t. 2a <= 1
     -3a + b <= 0
       a, b <= 0

Therefore, an infeasibility ray of the primal/unbounded ray of the dual is a vector such that

-7a + 2b > 0
 s.t. 2a <= 0
     -3a + b <= 0
       a, b <= 0

The dual of x >= 0 is 2a and the dual of y>=0 is -3a + b. Here you assume that the dual of y>=0 is 0 to deduce that b/a=3 which is not necessarily the case.
We can add a new test with this problem of 2 constraints but it is probably a good idea to keep linear8 as it is with only 1 constraint since it is easier to debug for someone implementing a solver wrapper.

Moreover, it can only be scaled by any positive constant since a, b should be nonpositive.

[IssamT]

Maybe we are just not using the same convention, or maybe I am wrong. From my understanding when I use SingleVariable I consider that the MOI constraint is defining a variable bound in the solver and not a linear constraint. to have y>=0 as a constraint in the solver, ScalarAffineFunction need to be used instead (see #58). Therefore, when y>=0 is defined as a bound in the solver, the dual dimension is 2 which is the number of constraints other than the non-negativity constraints

[mlubin]

@IssamT, it doesn't matter whether the constraint is treated as a bound or a linear constraint, it has a dual multiplier either way. Dual multipliers on bounds are usually called reduced costs.

[IssamT]

Isn't that already a transformation when you consider that reduced costs are the dual multipliers of the bound "constraints" ? When I ask the solver for infeasibility ray, It only returns a vector of 2 elements [0.33 1]. If the problem was feasible, the dual solution vector will also contain only 2 elements.

So maybe what you are saying is that MOI should always get the answer as if those were constraints and not bounds (since we call them MOI.Constraint). if the solver is using them as bounds we should get the reduced costs, add them to the dual vector of the solver to get the dual vector for MOI. Am I wrong?

[blegat]

No that is right. From the MOI viewpoint, these 2 constraints should behave the same way. The wrapper needs to do the necessary transformation to make it so.

[IssamT]

I can't find references agreeing on a "standard" definition of infeasibility certificate. So let's just agree on some definition and use it. First off we agree that whatever definition we use it should be a clear proof that the problem is infeasible. We also probably agree that an infeasibility certificate may not be unique. One way to obtain it is to use the algorithm based on Farkas lemma in which case the obtained infeasibility certificate is sometimes called Farkas certificate.

Definition 1) A set of multipliers for each constraint (excluding bounds) such that the linear combination of all constraints using those multipliers is not feasible with respect to variable bounds.

with this definition, using the primal I posted in this PR

a * (2x - 3y <= -7)
+
b * (y <= 2)

gives an infeasibility certificate if and only if b/a = 3
one possible infeasibility certificate is [a=1, b=3]

such infeasibility certificate can be obtained by requesting the infeasibility ray from the solver.

Definition 2) A set of multipliers for each constraint (including bounds) such that the linear combination of all constraints using those multipliers has the form V <= W with ( W > V , both are constants ) or W >= V with ( V < W ).

a * (2x - 3y <= -7)
+
b * (y <= 2)
+
c * (x >= 0)
+
d * (y >= 0)

gives an infeasibility certificate if and only if (b - d)/a = 3
A possible infeasibility certificate is [a=1, b=3, c=2, d=0]

such infeasibility certificate can be obtained through the infeasibility ray when bounds are passed as constraints. It can be also obtrained from the infeasibility ray without bounds passed as constraints through the following transformation:

• Step 1: using infeasibility ray without bounds passed as constraints, to get the multipliers for constraints other than bounds.
• Step 2: computing the linear combination as in Definition 1 then use the variable coefficients in the resulting constraint to define the multipliers for bound constraints.

Example using the same primal.

Step 1:

with [a=1, b=3]

a * (2x - 3y <= -7)
+
b * (y <= 2)

= 2 x + 0 y <= -7

Step 2:

c = 2

d = 0

Definition 3): The same as Definition 2) but forces arbitrarily the sense of the resulting constraint.

From remarks of @mlubin and @blegat, I guess that Definitions 2 and 3 are probably more suited to MOI which does not make a difference between bounds and constraints.

[blegat]

I would vote for Definition 3. Note that it matches the definition of an unbounded ray of the dual with the dual defined here. Therefore the sense is not arbitrary, it follows from the convention we have taken for defining the dual program.

[IssamT]

That makes sense @blegat. And if we use Definition 3, then we can keep the original test with the modifications in #127 .

[blegat]

Note that the reason we call it infeasibility certificate while it really is an unbounded ray of the dual is discussed here, here and here. In summary, most solver actually find an unbounded ray of the primal/dual and stop without checking for the existence of a feasible point. Therefore they only certify the infeasibility of the dual/primal and not really the unboundedness of the primal/dual. This why we talk about infeasibility certificate and not about unbounded ray in MOI. An unbounded ray should be composed of two elements, a feasible point and a feasible ray that makes the objective grow unbounded. Most solver only find the second element and call it an unbounded ray which is misleading.

[blegat]

@IssamT It might still be good to add your test as a new test if you modify it to take the dual variables of the bounds into account.

[IssamT]

OK. I'll wait until we merge #127, to avoid conflicts.

[blegat]

You can add it as a separate test i.e. linear12test so there should not be too much conflict.
Moreover, #127 probably won't be merged soon since LQOI is not ready yet.

[mlubin]

Yes, definition 3 seems appropriate. The rays must be consistent with the discussion at http://www.juliaopt.org/MathOptInterface.jl/latest/apimanual.html#Duals-1.