Improve docs of Frank Wolfe.

src/solvers/FrankWolfe.jl

@@ -7,13 +7,13 @@ It comes in two forms, depending on the realisation of the `subproblem`.
 
 # Fields
 
-* `p` – the current iterate, i.e. a point on the manifold
-* `X` – the current gradient ``\operatorname{grad} F(p)``, i.e. a tangent vector to `p`.
+* `p`           – the current iterate, i.e. a point on the manifold
+* `X`           – the current gradient ``\operatorname{grad} F(p)``, i.e. a tangent vector to `p`.
 * `inverse_retraction_method` – (`default_inverse_retraction_method(M, typeof(p))`) an inverse retraction method to use within Frank Wolfe.
-* `sub_problem` – an [`AbstractManoptProblem`](@ref) problem for the subsolver
-* `sub_state` – an [`AbstractManoptSolverState`](@ref) for the subsolver
-* `stop` – ([`StopAfterIteration`](@ref)`(200) | `[`StopWhenGradientNormLess`](@ref)`(1.0e-6)`) a [`StoppingCriterion`](@ref)
-* `stepsize` - ([`DecreasingStepsize`](@ref)`(; length=2.0, shift=2)`) ``s_k`` which by default is set to ``s_k = \frac{2}{k+2}``.
+* `sub_problem` – an [`AbstractManoptProblem`](@ref) problem or a function `(M, p, X) -> q` or `(M, q, p, X)` for the a closed form solution of the sub problem
+* `sub_state`   – an [`AbstractManoptSolverState`](@ref) for the subsolver or an [`AbstractEvaluationType`](@ref) in case the sub problem is provided as a function
+* `stop`        – ([`StopAfterIteration`](@ref)`(200) | `[`StopWhenGradientNormLess`](@ref)`(1.0e-6)`) a [`StoppingCriterion`](@ref)
+* `stepsize`    - ([`DecreasingStepsize`](@ref)`(; length=2.0, shift=2)`) ``s_k`` which by default is set to ``s_k = \frac{2}{k+2}``.
 * `retraction_method` – (`default_retraction_method(M, typeof(p))`) a retraction to use within Frank-Wolfe
 
 For the subtask, we need a method to solve
@@ -24,7 +24,7 @@ For the subtask, we need a method to solve
 
 # Constructor
 
-    FrankWolfeState(M, p, X, sub_problem, sub_task)
+    FrankWolfeState(M, p, X, sub_problem, sub_state)
 
 where the remaining fields from above are keyword arguments with their defaults already given in brackets.
 """
@@ -146,24 +146,41 @@ use a retraction and its inverse.
 
 # Input
 
-* `M` – a manifold ``\mathcal M``
-* `f` – a cost function ``f: \mathcal M→ℝ`` to find a minimizer ``p^*`` for
+* `M`      – a manifold ``\mathcal M``
+* `f`      – a cost function ``f: \mathcal M→ℝ`` to find a minimizer ``p^*`` for
 * `grad_f` – the gradient ``\operatorname{grad}f: \mathcal M → T\mathcal M`` of f
-  - as a function `(M, p) -> X` or a function `(M, X, p) -> X`
-* `p` – an initial value ``p ∈ \mathcal C``, note that it really has to be a feasible point
+  - as a function `(M, p) -> X` or a function `(M, X, p) -> X` working in place of `X`.
+* `p`      – an initial value ``p ∈ \mathcal C``, note that it really has to be a feasible point
 
 Alternatively to `f` and `grad_f` you can prodive
 the [`AbstractManifoldGradientObjective`](@ref) `gradient_objective` directly.
 
 ## Keyword Arguments
 
-* `evaluation` ([`AllocatingEvaluation`](@ref)) whether `grad_F` is an inplace or allocating (default) funtion
-* `initial_vector` – (`zero_vectoir(M,p)`) how to initialize the inner gradient tangent vector
+* `evaluation`         - ([`AllocatingEvaluation`](@ref)) whether `grad_f` is an inplace or allocating (default) funtion
+* `initial_vector`     – (`zero_vectoir(M,p)`) how to initialize the inner gradient tangent vector
 * `stopping_criterion` – ([`StopAfterIteration`](@ref)`(500) | `[`StopWhenGradientNormLess`](@ref)`(1.0e-6)`) a stopping criterion
-* `retraction_method` – (`default_retraction_method(M, typeof(p))`) a type of retraction
-* `stepsize` ([`DecreasingStepsize`](@ref)`(; length=2.0, shift=2)`
+* `retraction_method`  – (`default_retraction_method(M, typeof(p))`) a type of retraction
+* `stepsize`           -([`DecreasingStepsize`](@ref)`(; length=2.0, shift=2)`
   a [`Stepsize`](@ref) to use; but it has to be always less than 1. The default is the one proposed by Frank & Wolfe:
   ``s_k = \frac{2}{k+2}``.
+* `sub_cost`           - ([`FrankWolfeCost`](@ref)`(p, initiel_vector)`) – the cost of the Frank-Wolfe sub problem
+  which by default uses the current iterate and (sub)gradient of the current iteration to define a default cost,
+  this is used to define the default `sub_objective`. It is ignored, if you set that or the `sub_problem` directly
+* `sub_grad`           - ([`FrankWolfeGradient`](@ref)`(p, initial_vector)`) – the gradient of the Frank-Wolfe sub problem
+  which by default uses the current iterate and (sub)gradient of the current iteration to define a default gradient
+  this is used to define the default `sub_objective`. It is ignored, if you set that or the `sub_problem` directly
+* `sub_objective`      - ([`ManifoldGradientObjective`](@ref)`(sub_cost, sub_gradient)`) – the objective for the Frank-Wolfe sub problem
+  this is used to define the default `sub_problem`. It is ignored, if you set the `sub_problem` manually
+* `sub_problem`        - ([`DefaultManoptProblem`](@ref)`(M, sub_objective)`) – the Frank-Wolfe sub problem to solve.
+  This can be given in three forms
+   1. as an [`AbstractManoptProblem`](@ref), then the `sub_state` specifies the solver to use
+   2. as a closed form solution, e.g. a function, evaluating with new allocations, that is a function `(M, p, X) -> q` that solves the sub problem on `M` given the current iterate `p` and (sub)gradient `X`.
+   3. as a closed form solution, e.g. a function, evaluating in place, that is a function `(M, q, p, X) -> q` working in place of `q`, with the parameters as in the last point
+  For points 2 and 3 the `sub_state` has to be set to the corresponding [`AbstractEvaluationType`](@ref), [`AllocatingEvaluation`](@ref) and [`InplaceEvaluation`](@ref), respectively
+* `sub_state`          - (`evaluation` if `sub_problem` is a function, a decorated [`GradientDescentState`](@ref) otherwise)
+  for a function, the evaluation is inherited from the Frank-Wolfe `evaluation` keyword.
+* `sub_kwargs`         - (`[]`) – keyword arguments to decorate the `sub_state` default state in case the sub_problem is not a function
 
 All other keyword arguments are passed to [`decorate_state!`](@ref) for decorators or
 [`decorate_objective!`](@ref), respectively.