Synthesizing Control for a System with Black Box Environment, Based on Deep Learning

Simon Iosti; Doron Peled; Khen Aharon; Saddek Bensalem; Yoav Goldberg

doi:10.1007/978-3-030-61470-6_27

Synthesizing Control for a System with Black Box Environment, Based on Deep Learning

Simon Iosti, Doron Peled, Khen Aharon, Saddek Bensalem, Yoav Goldberg

Department of Computer Science

Université Grenoble Alpes

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

We study the synthesis of control for a system that interacts with a black-box environment, based on deep learning. The goal is to minimize the number of interaction failures. The current state of the environment is unavailable to the controller, hence its operation depends on a limited view of the history. We suggest a reinforcement learning framework of training a Recurrent Neural Network (RNN) to control such a system. We experiment with various parameters: loss function, exploration/exploitation ratio, and size of lookahead. We designed examples that capture various potential control difficulties. We present experiments performed with the toolkit DyNet.

Original language	English
Title of host publication	Leveraging Applications of Formal Methods, Verification and Validation
Subtitle of host publication	Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings
Editors	Tiziana Margaria, Bernhard Steffen
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	457-472
Number of pages	16
ISBN (Print)	9783030614690
DOIs	https://doi.org/10.1007/978-3-030-61470-6_27
State	Published - 2020
Event	9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020 - Rhodes, Greece Duration: 20 Oct 2020 → 30 Oct 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12477 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020
Country/Territory	Greece
City	Rhodes
Period	20/10/20 → 30/10/20

Bibliographical note

Publisher Copyright:
© 2020, Springer Nature Switzerland AG.

Funding

S. Iosti and S. Bensalem—The research performed by these authors was partially funded by H2020-ECSEL grants CPS4EU 2018-IA call - Grant Agreement number 826276. D. Peled and K. Aharon—The research performed by these authors was partially funded by ISF grants “Runtime Measuring and Checking of Cyber Physical Systems” (ISF award 2239/15) and “Efficient Runtime Verification for Systems with Lots of Data and its Applications” (ISF award 1464/18).

Funders	Funder number
H2020-ECSEL
Horizon 2020 Framework Programme	826276
Israel Science Foundation	1464/18, 2239/15

Access to Document

10.1007/978-3-030-61470-6_27

Cite this

Iosti, S., Peled, D., Aharon, K., Bensalem, S., & Goldberg, Y. (2020). Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. In T. Margaria, & B. Steffen (Eds.), Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings (pp. 457-472). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12477 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-61470-6_27

Iosti, Simon ; Peled, Doron ; Aharon, Khen et al. / Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings. editor / Tiziana Margaria ; Bernhard Steffen. Springer Science and Business Media Deutschland GmbH, 2020. pp. 457-472 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2b552a4eb9ce41a39c30332987530834,

title = "Synthesizing Control for a System with Black Box Environment, Based on Deep Learning",

abstract = "We study the synthesis of control for a system that interacts with a black-box environment, based on deep learning. The goal is to minimize the number of interaction failures. The current state of the environment is unavailable to the controller, hence its operation depends on a limited view of the history. We suggest a reinforcement learning framework of training a Recurrent Neural Network (RNN) to control such a system. We experiment with various parameters: loss function, exploration/exploitation ratio, and size of lookahead. We designed examples that capture various potential control difficulties. We present experiments performed with the toolkit DyNet.",

author = "Simon Iosti and Doron Peled and Khen Aharon and Saddek Bensalem and Yoav Goldberg",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020 ; Conference date: 20-10-2020 Through 30-10-2020",

year = "2020",

doi = "10.1007/978-3-030-61470-6_27",

language = "אנגלית",

isbn = "9783030614690",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "457--472",

editor = "Tiziana Margaria and Bernhard Steffen",

booktitle = "Leveraging Applications of Formal Methods, Verification and Validation",

address = "גרמניה",

}

Iosti, S, Peled, D, Aharon, K, Bensalem, S & Goldberg, Y 2020, Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. in T Margaria & B Steffen (eds), Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12477 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 457-472, 9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020, Rhodes, Greece, 20/10/20. https://doi.org/10.1007/978-3-030-61470-6_27

Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. / Iosti, Simon; Peled, Doron; Aharon, Khen et al.
Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings. ed. / Tiziana Margaria; Bernhard Steffen. Springer Science and Business Media Deutschland GmbH, 2020. p. 457-472 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12477 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Synthesizing Control for a System with Black Box Environment, Based on Deep Learning

AU - Iosti, Simon

AU - Peled, Doron

AU - Aharon, Khen

AU - Bensalem, Saddek

AU - Goldberg, Yoav

PY - 2020

Y1 - 2020

N2 - We study the synthesis of control for a system that interacts with a black-box environment, based on deep learning. The goal is to minimize the number of interaction failures. The current state of the environment is unavailable to the controller, hence its operation depends on a limited view of the history. We suggest a reinforcement learning framework of training a Recurrent Neural Network (RNN) to control such a system. We experiment with various parameters: loss function, exploration/exploitation ratio, and size of lookahead. We designed examples that capture various potential control difficulties. We present experiments performed with the toolkit DyNet.

AB - We study the synthesis of control for a system that interacts with a black-box environment, based on deep learning. The goal is to minimize the number of interaction failures. The current state of the environment is unavailable to the controller, hence its operation depends on a limited view of the history. We suggest a reinforcement learning framework of training a Recurrent Neural Network (RNN) to control such a system. We experiment with various parameters: loss function, exploration/exploitation ratio, and size of lookahead. We designed examples that capture various potential control difficulties. We present experiments performed with the toolkit DyNet.

UR - http://www.scopus.com/inward/record.url?scp=85096593874&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-61470-6_27

DO - 10.1007/978-3-030-61470-6_27

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85096593874

SN - 9783030614690

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 457

EP - 472

BT - Leveraging Applications of Formal Methods, Verification and Validation

A2 - Margaria, Tiziana

A2 - Steffen, Bernhard

PB - Springer Science and Business Media Deutschland GmbH

T2 - 9th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation, ISoLA 2020

Y2 - 20 October 2020 through 30 October 2020

ER -

Iosti S, Peled D, Aharon K, Bensalem S, Goldberg Y. Synthesizing Control for a System with Black Box Environment, Based on Deep Learning. In Margaria T, Steffen B, editors, Leveraging Applications of Formal Methods, Verification and Validation: Engineering Principles - 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 457-472. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-61470-6_27

Synthesizing Control for a System with Black Box Environment, Based on Deep Learning

Abstract

Publication series

Conference

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this