The Zero-Sum Discrete-Time Feedback Linear Quadratic Dynamic Game: From Two-Player Case to Multi-Player Case
DOI:
https://doi.org/10.12928/bamme.v4i1.9351Keywords:
linear quadratic dynamic game, Feedback information structure, Discrete-time, Multi-playerAbstract
The two-person zero-sum discrete-time feedback linear quadratic dynamic game is considered. A method that provides a saddle-point for the zero-sum discrete dynamic game is developed to derive a necessary and sufficient condition under which the game has a feedback saddle-point solution. Existence solutions, which are described in terms of a sequence of nonnegative definite algebraic Riccati matrices, are constructed. Next, a generalization of such a game to a multi-player case is studied. Using the results in a two-person case, the characterization of a feedback saddle-point solution for the multi-player game is derived.
References
Başar, T., & Bernhard, P. (2008). H-infinity Optimal Control and Related Minimax Design Problems: a Dynamic Game Approach. Springer Science & Business Media.
Başar, T., & Olsder, G. J. (1998). Dynamic noncooperative game theory. Society for Industrial and Applied Mathematics.
Couto, M. C., & Pal, S. (2023). Introspection dynamics in asymmetric multiplayer games. Dynamic Games and Applications, 13(4), 1256-1285, https://doi.org/10.1007/s13235-023-00525-8
Delfour, M. C. (2007). Linear quadratic differential games: Saddle point and Riccati differential equation. SIAM journal on control and optimization, 46(2), 750-774.
Engwerda, J. (2005). LQ dynamic optimization and differential games. John Wiley & Sons.
Garcia, E., Casbeer, D. W., Pachter, M., Curtis, J. W., & Doucette, E. (2020, July). A two-team linear quadratic differential game of defending a target. In 2020 American Control Conference (ACC) (pp. 1665-1670). IEEE.
Gokhale, C. S., & Traulsen, A. (2014). Evolutionary multi-player games. Dynamic Games and Applications, 4, 468-488.
Jank, G. and Kun, G. Optimal Control of Disturbed Linear-Quadratic Differential Games, European Journal of Control, Volume 8, Issue 2, 2002, Pages 152-162
Kebriaei, H., & Iannelli, L. (2017). Discrete-time robust hierarchical linear-quadratic dynamic games. IEEE Transactions on Automatic Control, 63(3), 902-909.
Köpf, F., Inga, J., Rothfuß, S., Flad, M., & Hohmann, S. (2017). Inverse reinforcement learning for identification in linear-quadratic dynamic games. IFAC-PapersOnLine, 50(1), 14902-14908.
Lukes, D. L., & Russell, D. L. (1971). A global theory for linear-quadratic differential games. Journal of Mathematical Analysis and Applications, 33(1), 96–123.
Mahajan, A., Samvelyan, M., Gupta, T., Ellis, B., Sun, M., Rocktäschel, T., & Whiteson, S. (2022). Generalization in cooperative multi-agent systems. arXiv preprint arXiv:2202.00104.
Mazumdar, E., Ratliff, L. J., Jordan, M. I., & Sastry, S. S. (2019). Policy-gradient algorithms have no guarantees of convergence in linear quadratic games—arXiv preprint arXiv:1907.03712.
Moon, J., & Başar, T. (2016). Linear quadratic risk-sensitive and robust mean field games. IEEE Transactions on Automatic Control, 62(3), 1062-1077.
Musthofa, M. W., Salmah, , Engwerda, J. C., & Suparwanto, A. (2013). Feedback saddle point equilibria for soft-constrained zero-sum linear quadratic descriptor differential game. Archives of Control Sciences, 23(4), 473-493.
Musthofa, M. W., Engwerda, J., & Suparwanto, A. (2016). Robust Optimal Control Design Using a Differential Game Approach for Open-Loop Linear Quadratic Descriptor Systems. Journal of Optimization Theory and Applications, 168(3), 1046-1064.
Musthofa, M. W. (2021). The Open-Loop Zero-Sum Linear Quadratic Index One Discrete-Time Soft-Constrained Descriptor Dynamic Games. Journal of Mathematical Control Science and Applications, 7(1), 1 – 10.
Omidshafiei, S., Tuyls, K., Czarnecki, W. M., Santos, F. C., Rowland, M., Connor, J., & Munos, R. (2020). Navigating the landscape of multi-player games. Nature communications, 11(1), 5603.
Pachter, M., & Pham, K. D. (2010). Discrete-time linear-quadratic dynamic games. Journal of Optimization Theory and Applications, 146, 151-179.
Ratliff, L. J., Coogan, S., Calderone, D., & Sastry, S. S. (2012, October). Pricing in linear-quadratic dynamic games. In 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton) (pp. 1798-1805). IEEE.
Van Den Broek, W. A., Engwerda, J. C., & Schumacher, J. M. (2003). Robust equilibria in indefinite linear-quadratic differential games. Journal of Optimization Theory and Applications, pp. 119, 565–595.
Wu, J. (2023). Learning Zero-Sum Linear Quadratic Games with Improved Sample Complexity and Last-Iterate Convergence (Master's thesis).
Xu, H., & Mukaidani, H. (2003). The linear quadratic dynamic game for discrete-time descriptor systems. International Game Theory Review, 5(04), 361-374.
Xu, R. and Wu, T., Risk-sensitive large-population linear-quadratic-Gaussian games with major and minor agents, Asian J Control (2023), 1–13, DOI 10.1002/asjc.3106.
Yu, C., Li, Y., Li, S., & Chen, J. (2022). Inverse linear quadratic dynamic games using partial state observations. Automatica, 145, 110534.
uz Zaman, M. A., Miehling, E., & Başar, T. (2023). Reinforcement learning for non-stationary discrete-time linear–quadratic mean-field games in multiple populations. Dynamic Games and Applications, 13(1), 118-164.
Zhang, K., Yang, Z., & Başar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of reinforcement learning and control, 321-384.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Muhammad Wakhid Musthofa, Jacob Engwerda
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
