Generic optimal control notations

[azev77]

It would be nice to help users from other fields:
1: quickly see if some of the types of problems solved by the package can solve problems they're interested in
2: easily see how to write down their problem in a way solvable by the package

I had in mind something like:

I think this formulation is recognizable to users from most disciplines?
{Modulo small things like min/max...}
If not what are other formulations?

[pulsipher]

This is is good start and I certainly think providing templates is a worthy contribution. Other considerations would include alternate objective forms and the use of explicit differential and algebraic constraints instead of explicit ODEs. The key will be finding the right balance between generality and not being too abstract to be helpful. I probably won't have time for this in the immediate future. Pull requests welcome :)

[azev77]

Good content on notation: https://twitter.com/Apoorva__Lal/status/1438554469210279936?s=20&t=OQFXvDzxbm2HA9Ivobc57A

[pulsipher]

Nice find! This certainly helps. Developing some optimal control specific tutorials is definitely on the todo list. Unfortunately, mine is pretty backlogged for the next few months.

[DarioSlaifsteinSk]

May I suggest the Universal Modeling Framework from W.B. Powell??
The idea is to bridge between different fields (optimal control, reinforcement learning, operations research and many more) with common nomenclature.
The original paper is Powell (2019) and is later presented in depth in his last two books. Here is a summary:
Having stated its motivation, we will define the elements of the models to be used. The core elements of a sequential decision problem are:

• State variables $S_t$. The state variable contains everything we know from our system, only the necessary information necessary to make a decision. They might be physical states $R_t$, non-physical states with perfect information $I_t$, or beliefs/approximations of uncertain quantities $B_t$.
  $S_t=[R_t, I_t, B_t]^T$

• Decision variables $x_t$. The decision variables are the available quantities we can control/move to produce the desired output/performance of our system/problem. They are subject to constraints and thus are inside a domain of possible actions as $x_t \in \mathcal{X_t}$. They are produced by an algorithm or method we call a policy which we present as $x_t = X^{\pi}(S_t)$.

• Exogenous information $W_{t+1}$. This is the information we receive after we make our decision; this information was previously unknown.

• The transition function $S^M(S_t, x_t, W_{t+1})$. This function represents the set of equations that explain the evolution of our system.
  $S_{t+1} = S^M(S_t, x_t, W_{t+1})$

• The objective function. This function can have different forms. Generally, it could be defined as the function that accounts for the contribution (cost, reward, income, etc.) that is done in each period (point reward) or over several periods (cumulative reward). Generally, it is called $C(S_t, x_t)$.