A Reinforcement-Learning Style Algorithm for Black Box Automata

Itay Cohen, Roi Fogler, Doron Peled

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The analysis of hardware and software systems is often applied to a model of a system rather than to the system itself. Obtaining a faithful model for a system may sometimes be a complex task. For learning the regular (finite automata) structure of a black box system, Angluin's $L^{*}$ algorithm and its successors employ membership and equivalence queries. The regular positive-negative inference (RPNI) family of algorithms use a less powerful capability of collecting observations for learning, with no control on selecting the inputs. We suggest and study here an alternative approach for learning, which is based on calculating utility values, obtained as a discounted sum of rewards, in the style of reinforcement learning. The utility values are used to classify the observed input prefixes into different states, and then to construct the learned automaton structure. We show cases where this classification is not enough to separate the prefixes, and subsequently remedy the situation by exploring deeper than the current prefix: checking the consistency between descendants of the current prefix that are reached with the same sequence of inputs. We show the connection of this algorithm with the RPNI algorithm and compare between these two approaches experimentally.

Original languageEnglish
Title of host publication2022 20th ACM-IEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350331905
DOIs
StatePublished - 2022
Event20th ACM-IEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2022 - Shanghai, China
Duration: 13 Oct 202214 Oct 2022

Publication series

Name2022 20th ACM-IEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2022

Conference

Conference20th ACM-IEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2022
Country/TerritoryChina
CityShanghai
Period13/10/2214/10/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Fingerprint

Dive into the research topics of 'A Reinforcement-Learning Style Algorithm for Black Box Automata'. Together they form a unique fingerprint.

Cite this