End-to-End AI Generated Runtime Verification from Natural Language Specification

Itay Cohen; Doron Peled

doi:10.1007/978-3-031-73741-1_23

End-to-End AI Generated Runtime Verification from Natural Language Specification

Itay Cohen, Doron Peled

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We demonstrate how LLMs can be harnessed to synthesize runtime verification monitors from natural language specifications. We instruct the LLM through prompts to assemble the monitors through a sequence of stages. To start with, we employ the learned insights LLMs possess regarding linear temporal logic for translating natural language specifications into propositional past time LTL formulas. Next, we utilize a sequence of prompts to synthesize from the LTL specification a runtime verification monitor for the given formula. Part of these prompts enable the LLM to validate its own outputs, thereby significantly improving the probability of obtaining a correct monitor.

Original language	English
Title of host publication	Bridging the Gap Between AI and Reality - 1st International Conference, AISoLA 2023, Selected Papers
Editors	Bernhard Steffen
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	362-384
Number of pages	23
ISBN (Print)	9783031737404
DOIs	https://doi.org/10.1007/978-3-031-73741-1_23
State	Published - 2025
Event	1st International Symposium on Leveraging Applications of Formal Methods, AISoLA 2023 - Crete, Greece Duration: 23 Oct 2023 → 28 Oct 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14129 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	1st International Symposium on Leveraging Applications of Formal Methods, AISoLA 2023
Country/Territory	Greece
City	Crete
Period	23/10/23 → 28/10/23

Bibliographical note

Publisher Copyright:
© The Author(s) 2025.

Access to Document

10.1007/978-3-031-73741-1_23

Cite this

Cohen, I., & Peled, D. (2025). End-to-End AI Generated Runtime Verification from Natural Language Specification. In B. Steffen (Ed.), Bridging the Gap Between AI and Reality - 1st International Conference, AISoLA 2023, Selected Papers (pp. 362-384). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14129 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-73741-1_23

Cohen, Itay ; Peled, Doron. / End-to-End AI Generated Runtime Verification from Natural Language Specification. Bridging the Gap Between AI and Reality - 1st International Conference, AISoLA 2023, Selected Papers. editor / Bernhard Steffen. Springer Science and Business Media Deutschland GmbH, 2025. pp. 362-384 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{549ffa3ead9c447a92f1a04bf0e18d8f,

title = "End-to-End AI Generated Runtime Verification from Natural Language Specification",

abstract = "We demonstrate how LLMs can be harnessed to synthesize runtime verification monitors from natural language specifications. We instruct the LLM through prompts to assemble the monitors through a sequence of stages. To start with, we employ the learned insights LLMs possess regarding linear temporal logic for translating natural language specifications into propositional past time LTL formulas. Next, we utilize a sequence of prompts to synthesize from the LTL specification a runtime verification monitor for the given formula. Part of these prompts enable the LLM to validate its own outputs, thereby significantly improving the probability of obtaining a correct monitor.",

author = "Itay Cohen and Doron Peled",

note = "Publisher Copyright: {\textcopyright} The Author(s) 2025.; 1st International Symposium on Leveraging Applications of Formal Methods, AISoLA 2023 ; Conference date: 23-10-2023 Through 28-10-2023",

year = "2025",

doi = "10.1007/978-3-031-73741-1_23",

language = "אנגלית",

isbn = "9783031737404",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "362--384",

editor = "Bernhard Steffen",

booktitle = "Bridging the Gap Between AI and Reality - 1st International Conference, AISoLA 2023, Selected Papers",

address = "גרמניה",

}

Cohen, I & Peled, D 2025, End-to-End AI Generated Runtime Verification from Natural Language Specification. in B Steffen (ed.), Bridging the Gap Between AI and Reality - 1st International Conference, AISoLA 2023, Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14129 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 362-384, 1st International Symposium on Leveraging Applications of Formal Methods, AISoLA 2023, Crete, Greece, 23/10/23. https://doi.org/10.1007/978-3-031-73741-1_23

End-to-End AI Generated Runtime Verification from Natural Language Specification. / Cohen, Itay; Peled, Doron.
Bridging the Gap Between AI and Reality - 1st International Conference, AISoLA 2023, Selected Papers. ed. / Bernhard Steffen. Springer Science and Business Media Deutschland GmbH, 2025. p. 362-384 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14129 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - End-to-End AI Generated Runtime Verification from Natural Language Specification

AU - Cohen, Itay

AU - Peled, Doron

PY - 2025

Y1 - 2025

N2 - We demonstrate how LLMs can be harnessed to synthesize runtime verification monitors from natural language specifications. We instruct the LLM through prompts to assemble the monitors through a sequence of stages. To start with, we employ the learned insights LLMs possess regarding linear temporal logic for translating natural language specifications into propositional past time LTL formulas. Next, we utilize a sequence of prompts to synthesize from the LTL specification a runtime verification monitor for the given formula. Part of these prompts enable the LLM to validate its own outputs, thereby significantly improving the probability of obtaining a correct monitor.

AB - We demonstrate how LLMs can be harnessed to synthesize runtime verification monitors from natural language specifications. We instruct the LLM through prompts to assemble the monitors through a sequence of stages. To start with, we employ the learned insights LLMs possess regarding linear temporal logic for translating natural language specifications into propositional past time LTL formulas. Next, we utilize a sequence of prompts to synthesize from the LTL specification a runtime verification monitor for the given formula. Part of these prompts enable the LLM to validate its own outputs, thereby significantly improving the probability of obtaining a correct monitor.

UR - http://www.scopus.com/inward/record.url?scp=85208633556&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-73741-1_23

DO - 10.1007/978-3-031-73741-1_23

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85208633556

SN - 9783031737404

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 362

EP - 384

BT - Bridging the Gap Between AI and Reality - 1st International Conference, AISoLA 2023, Selected Papers

A2 - Steffen, Bernhard

PB - Springer Science and Business Media Deutschland GmbH

T2 - 1st International Symposium on Leveraging Applications of Formal Methods, AISoLA 2023

Y2 - 23 October 2023 through 28 October 2023

ER -

Cohen I, Peled D. End-to-End AI Generated Runtime Verification from Natural Language Specification. In Steffen B, editor, Bridging the Gap Between AI and Reality - 1st International Conference, AISoLA 2023, Selected Papers. Springer Science and Business Media Deutschland GmbH. 2025. p. 362-384. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-73741-1_23