P.A. Tochilin. On the construction of a piecewise affine value function in an infinite-horizon optimal control problem ... P. 223-238

The paper is devoted to the approximate solution of an infinite-horizon optimal control problem for a nonlinear system of differential equations with an integral cost functional. We use the technique of piecewise linearization (“hybridization”) of the original nonlinear system followed by the analysis of the resulting switched system. Then the methods of piecewise affine value and control functions, the method of dynamic programming, and the comparison principle are applied. Two cases are considered sequentially: with continuous piecewise affine value and control functions and with functions admitting discontinuities. In the latter case, it is possible to increase the effectiveness of the proposed approach by allowing gaps. Theorems on sufficient conditions for the solvability of the control problem are formulated and proved. The theorems also provide upper estimates of the minimized functional. Computationally simple algorithms are derived for the construction of estimates of the value function for this problem and of the corresponding feedback control. The operation of the proposed algorithm is demonstrated for a problem of control of a wheeled robot on the plane.

Keywords: nonlinear dynamics, linearization, switched system, optimal control, dynamic programming, piecewise affine value function

Received October 20, 2019

Revised January 22, 2019

Accepted January 27, 2020

Funding Agency: This work was supported by the Russian Foundation for Basic Research (projects no. 19–01–00613a and no. 16–29–04191ofi_m).

Pavel Aleksandrovich Tochilin, Cand. Sci. (Phys.-Math.), Lomonosov Moscow State University, Faculty of Computational Mathematics and Cybernetics, Moscow, 119991 Russia, e-mail: tochilin@cs.msu.ru

REFERENCES

1.   Kurzhanski A.B. Comparison principle for equations of the Hamilton-Jacobi type in control theory. Proc. Steklov Inst. Math., 2006, vol. 253, no. 1, pp. S185–S195. doi: 10.1134/S0081543806050130 

2.   Kurzhanski A.B., Varaiya P. Dynamics and control of trajectory tubes. Basel: Birkhauser, 2014, 445 p. ISBN: 978-3-319-10277-1 .

3.   Habets L.C.G.J.M., Collins P.J., van Schuppen J.H. Reachability and control synthesis for piecewise-affine hybrid systems on simplices. IEEE Trans Automatic Control, 2006, vol. 51, no. 6, pp. 938–948. doi: 10.1109/TAC.2006.876952 

4.   Girard A., Martin S. Synthesis of constrained nonlinear systems using hybridization and robust controllers on simplices. IEEE Trans Automatic Control, 2012, vol. 57, no. 4, pp. 1046–1051. doi: 10.1109/TAC.2011.2168874 

5.   Bardi M., Capuzzo-Dolcetta I. Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Boston: Birkhauser, 2008, 570 p. ISBN: 0817647546 .

6.   Subbotin A.I. Generalized solutions of first-order PDEs. The dynamical optimization perspective. Basel: Birkhauser, 1995, 314 p. doi: 10.1007/978-1-4612-0847-1 . Translated to Russian under the title Obobshchennye resheniya uravnenii v chastnykh proizvodnykh pervogo poryadka: Perspektivy dinamicheskoi optimizatsii. Izhevsk: Inst. Komp’yuter. Issled. Publ., 2003, 336 p.

7.   Fleming W.H., Soner H.M. Controlled Markov processes and viscosity solutions. N Y: Springer, 2006. 429 p. ISBN: 978-0-387-31071-8 .

8.   Kurzhanski A.B., Tochilin P.A. Weakly invariant sets of hybrid systems. Diff. Eq., 2008, vol. 44, no. 11, pp. 1585–1594. doi: 10.1134/S0012266108110104 

9.   Asarin E., Dang T., Girard A. Hybridization methods for the analysis of nonlinear systems. Acta Informatica, 2007, vol. 43, no. 7, pp. 451–476. doi: 10.1007/s00236-006-0035-7 

10.   Tochilin P.A. On the construction of nonconvex approximations to reach sets of piecewise linear systems. Diff. Eq., 2015, vol. 51, no. 11, pp. 1503–1515. doi: 10.1134/S0012266115110117 

11.   Mayantsev K.S., Tochilin P.A. The feedback control problem for switched system with uncertainties. IFAC Proceedings Volumes, 2017, vol. 50, no. 1, pp. 2187–2192. doi: 10.1016/j.ifacol.2017.08.279 

12.   Filippov A.F. Differential equations with discontinuous righthand sides. Dordrecht: Springer, 1988, 304 p. doi: 10.1007/978-94-015-7793-9 . Original Russian text published in Filippov A.F. Differentsial’nye uravneniya s razryvnoi pravoi chast’yu. Moscow: Nauka Publ., 1985, 225 c.

13.   Pshenichnyi B.N. Vypuklyi analiz i ekstremal’nye zadachi [Convex analysis and extremal problems]. Moscow: Nauka Publ., 1980, 319 p.

Cite this article as: P.A. Tochilin. On the construction of a piecewise affine value function in an infinite-horizon optimal control problem, Trudy Instituta Matematiki i Mekhaniki URO RAN, 2020, vol. 26, no. 1, pp. 223–238.