Online Linear Quadratic Control

Alon Cohen; Avinatan Hassidim; Tomer Koren; Nevena Lazic; Yishay Mansour; Kunal Talwar

Online Linear Quadratic Control

Alon Cohen
, Avinatan Hassidim
, Tomer Koren
, Nevena Lazic
, Yishay Mansour
, Kunal Talwar

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

22 Scopus citations

Abstract

We study the problem of controlling linear time- invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee O(Vf) regret under mild assumptions, where T is the time horizon. Our algorithms rely on a novel SDP relaxation for the steady-state distribution of the system. Crucially, and in contrast to previously proposed relaxations, the feasible solutions of our SDP all correspond to "strongly stable" policies that mix exponentially fast to a steady state.

Original language	English
Title of host publication	35th International Conference on Machine Learning, ICML 2018
Editors	Andreas Krause, Jennifer Dy
Publisher	International Machine Learning Society (IMLS)
Pages	1667-1681
Number of pages	15
ISBN (Electronic)	9781510867963
State	Published - 2018
Event	35th International Conference on Machine Learning, ICML 2018 - Stockholm, Sweden Duration: 10 Jul 2018 → 15 Jul 2018

Publication series

Name	35th International Conference on Machine Learning, ICML 2018
Volume	3

Conference

Conference	35th International Conference on Machine Learning, ICML 2018
Country/Territory	Sweden
City	Stockholm
Period	10/07/18 → 15/07/18

Bibliographical note

Publisher Copyright:
© 2018 35th International Conference on Machine Learning, ICML 2018. All rights reserved.

Cite this

@inproceedings{493b3c59afdc40d2bc5f7f345b46a253,

title = "Online Linear Quadratic Control",

abstract = "We study the problem of controlling linear time- invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee O(Vf) regret under mild assumptions, where T is the time horizon. Our algorithms rely on a novel SDP relaxation for the steady-state distribution of the system. Crucially, and in contrast to previously proposed relaxations, the feasible solutions of our SDP all correspond to {"}strongly stable{"} policies that mix exponentially fast to a steady state.",

author = "Alon Cohen and Avinatan Hassidim and Tomer Koren and Nevena Lazic and Yishay Mansour and Kunal Talwar",

note = "Publisher Copyright: {\textcopyright} 2018 35th International Conference on Machine Learning, ICML 2018. All rights reserved.; 35th International Conference on Machine Learning, ICML 2018 ; Conference date: 10-07-2018 Through 15-07-2018",

year = "2018",

language = "אנגלית",

series = "35th International Conference on Machine Learning, ICML 2018",

publisher = "International Machine Learning Society (IMLS)",

pages = "1667--1681",

editor = "Andreas Krause and Jennifer Dy",

booktitle = "35th International Conference on Machine Learning, ICML 2018",

}

Cohen, A, Hassidim, A, Koren, T, Lazic, N, Mansour, Y & Talwar, K 2018, Online Linear Quadratic Control. in A Krause & J Dy (eds), 35th International Conference on Machine Learning, ICML 2018. 35th International Conference on Machine Learning, ICML 2018, vol. 3, International Machine Learning Society (IMLS), pp. 1667-1681, 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 10/07/18.

Online Linear Quadratic Control. / Cohen, Alon; Hassidim, Avinatan; Koren, Tomer et al.
35th International Conference on Machine Learning, ICML 2018. ed. / Andreas Krause; Jennifer Dy. International Machine Learning Society (IMLS), 2018. p. 1667-1681 (35th International Conference on Machine Learning, ICML 2018; Vol. 3).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Online Linear Quadratic Control

AU - Cohen, Alon

AU - Hassidim, Avinatan

AU - Koren, Tomer

AU - Lazic, Nevena

AU - Mansour, Yishay

AU - Talwar, Kunal

PY - 2018

Y1 - 2018

N2 - We study the problem of controlling linear time- invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee O(Vf) regret under mild assumptions, where T is the time horizon. Our algorithms rely on a novel SDP relaxation for the steady-state distribution of the system. Crucially, and in contrast to previously proposed relaxations, the feasible solutions of our SDP all correspond to "strongly stable" policies that mix exponentially fast to a steady state.

AB - We study the problem of controlling linear time- invariant systems with known noisy dynamics and adversarially chosen quadratic losses. We present the first efficient online learning algorithms in this setting that guarantee O(Vf) regret under mild assumptions, where T is the time horizon. Our algorithms rely on a novel SDP relaxation for the steady-state distribution of the system. Crucially, and in contrast to previously proposed relaxations, the feasible solutions of our SDP all correspond to "strongly stable" policies that mix exponentially fast to a steady state.

UR - https://www.scopus.com/pages/publications/85057286229

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85057286229

T3 - 35th International Conference on Machine Learning, ICML 2018

SP - 1667

EP - 1681

BT - 35th International Conference on Machine Learning, ICML 2018

A2 - Krause, Andreas

A2 - Dy, Jennifer

PB - International Machine Learning Society (IMLS)

T2 - 35th International Conference on Machine Learning, ICML 2018

Y2 - 10 July 2018 through 15 July 2018

ER -

Online Linear Quadratic Control

Abstract

Publication series

Conference

Bibliographical note

Other files and links

Fingerprint

Cite this