Optimisation (various optimisation problems solution using CVXPy, JAX, etc):

1. Dynamic of spaceship controls:

a) Find the position, speed and forse in evert timestamp such that the total amount of consumed fuel is minimal (CVXPy is used)

b) Find the minimal time that is sufficient to land the spaceship

2. Nonnegative matrix factorization for image processing:

(NMF has become so popular is because of its ability to automatically extract sparse and easily interpretable factors.)

a) implement with CVXPy the alternating optimization method

b) plot the convergence line (objective function value vs. iteration idx) and confirm that the method converegs

3. Steepest Gradient Descent

a) Solve the problem with the steepest gradient descent method

b) Simple binary classification problem, where optimization problem is solved using GradientDescent

c) comparison with with the results if Armijo rule to adapt the step size is used

4. Convergence comparison for different methods

a) comparison of preformance of SGD, GD, ADAM, Nesterov Momentum

b) comparison of gradient descent, heavy ball method and accelerated gradient methods, step size selection

5. Optimisation problems solution

a) convex, strongly convex functions, set

b) KKT conditions

c) Newton method

d) minimization problem of quadratic function

e) linear least squares problem for random matrix A (GD, HeavyBall, accelerated GD)

f) solution with CVXPy