Optimal Control

The aim of this project is to explore and compare the different methods of solving optimal control problems.

This project will focus on apply different optimal control methods on different types of systems with the goal.

The optimal control methods implemented in this projects are categorised based on how much information about the system is available as Planning Methods which require complete information on the system, Model-Based Learning which require some information on the system, and Model-Free Learning which requires almost no information on the system.

The systems modelled in this project are also categorised based on how much information about the system is available as Deterministic systems, and Stochastic Systems.

The Objective

The objective of this project is to determine the most effective optimal control method for each system

Table of Contents

• Introduction to Optimal Control Problems
  • Dynamical Systems
    • Properties of dynamical systems
    • Challenges that may arise when modelling a dynamical system
    • Types of dynamical systems
  • Control Theory
    • Basic terminology of control theory
    • Challenges that may arise when controlling a system
    • Methods of control
    • Markov Decision Processes
  • Optimization Problems
    • Types of optimization problems
    • Methods of optimization
    • Optimization in Control Theory
• Optimal Control Methods
  • Analytical/ Planning Methods
    • Variational Calculus
    • Minimum Principle
    • Hamilton-Jacobi-Bellman Equation
  • Learning Methods
    • Dynamic Programming
    • Model-Based Reinforcemnt Learning
    • Model-Free Reinforment Learning
• Implementation of Solutions
  • Case Study 1: Trajectory Optimization
    • System Design
    • Control Implementation
  • Case Study 2: Adaptive Control
    • System Design
    • Control Implementation
  • Case Study 3: Stochastic Control
    • System Design
    • Control Implementation

Code Design

The code for this project will be written in Python with Jupyter Notebook and will be divided into the following sections:

• Optimal Control Methods: This section will contain the implementation of the different optimal control methods.
• System Models: This section will contain the implementation of the different system models.
• Case Studies: This section will contain the implementation of the different case studies.
• Tests: This section will contain the code to test each System Model, and Optimal Control Method
• Tools: This section will contain scripts to automate tasks in the project.

The code is tested with Python 3.12.1 with the following packages:

Package    Version
---------- -------
numpy      1.26.4
pip        24.0
torch      2.3.1

References

Model-based Reinforcement Learning: A Survey.(https://arxiv.org/pdf/2006.16712)