Optimal Control for Dynamic Systems
Implementation of iLQR and DDP algorithms for nonlinear trajectory optimization.
Developed and analyzed optimal control algorithms for nonlinear dynamic systems, focusing on iterative Linear Quadratic Regulator (iLQR) and Differential Dynamic Programming (DDP) methods.
This project explored how dynamic programming and second-order expansions can efficiently solve continuous-time control problems with nonlinearity and constraints.
High-level structure of the iterative LQR algorithm with forward rollout and backward pass optimization.
🧩 Problem Setup
- System models: unicycle, cart–pole, and double integrator.
- Objective: minimize quadratic cost function
[ J = \sum_{t=0}^{T} (x_t^T Q x_t + u_t^T R u_t) ] - Approach: linearize dynamics along nominal trajectories, use backward Riccati recursion to compute feedback gains.
Left: optimized cart-pole swing-up trajectory. Right: convergence of cost-to-go function under iLQR iterations.
⚙️ Key Features
- Implemented iLQR and DDP in Python (NumPy) and MATLAB.
- Verified convergence and control stability across multiple nonlinear systems.
- Compared gradient-based optimization with dynamic programming–based methods.
- Built visualizations for trajectory evolution and value function contours.
🧠Insights
- The iLQR backward pass improves numerical stability compared to naive gradient descent.
- Warm-starting DDP with iLQR solutions reduces iterations by ~40%.
- Highlighted the trade-off between accuracy and real-time feasibility for embedded control.
🧰 Tools
Python · NumPy · Matplotlib · SciPy · MATLAB
Results:
Achieved real-time trajectory optimization (<50 ms per iteration) for the unicycle model; successfully validated DDP convergence properties on the cart–pole system.