From finite-system entropy to entropy rate for a hidden Markov process

Or Zuk; Eytan Domany; Ido Kanter; Michael Aizenman

doi:10.1109/LSP.2006.874466

From finite-system entropy to entropy rate for a hidden Markov process

Or Zuk, Eytan Domany, Ido Kanter, Michael Aizenman

Department of Physics

Research output: Contribution to journal › Article › peer-review

13 Scopus citations

Abstract

A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable ε. The coefficients of the expansion around the noiseless (ε = 0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.

Original language	English
Pages (from-to)	517-520
Number of pages	4
Journal	IEEE Signal Processing Letters
Volume	13
Issue number	9
DOIs	https://doi.org/10.1109/LSP.2006.874466
State	Published - Sep 2006

Bibliographical note

Funding Information:
Manuscript received January 4, 2006; revised January 23, 2006. The work of M. Aizenman was supported in part by the Einstein Center for Theoretical Physics and in part by the Minerva Center for Nonlinear Physics. The work of I. Kanter at the Weizmann Institute was supported by the Einstein Center for Theoretical Physics. E. Domany and O. Zuk were supported in part by the Minerva Foundation and in part by the European Community’s Human Potential Programme under Contract HPRN-CT-2002-00319, STIPCO. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Cihan Tepedelenlioglu.

Funding

Manuscript received January 4, 2006; revised January 23, 2006. The work of M. Aizenman was supported in part by the Einstein Center for Theoretical Physics and in part by the Minerva Center for Nonlinear Physics. The work of I. Kanter at the Weizmann Institute was supported by the Einstein Center for Theoretical Physics. E. Domany and O. Zuk were supported in part by the Minerva Foundation and in part by the European Community’s Human Potential Programme under Contract HPRN-CT-2002-00319, STIPCO. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Cihan Tepedelenlioglu.

Funders	Funder number
European Community’s Human Potential Programme	HPRN-CT-2002-00319
Minerva Center for Nonlinear Physics
Center for Theoretical Biological Physics
Minerva Foundation

Keywords

Entropy
Hidden markov process (HMP)
Taylor series

Access to Document

10.1109/LSP.2006.874466

Cite this

@article{ce146915a89845968717551ca601fe1c,

title = "From finite-system entropy to entropy rate for a hidden Markov process",

abstract = "A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable ε. The coefficients of the expansion around the noiseless (ε = 0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.",

keywords = "Entropy, Hidden markov process (HMP), Taylor series",

author = "Or Zuk and Eytan Domany and Ido Kanter and Michael Aizenman",

note = "Funding Information: Manuscript received January 4, 2006; revised January 23, 2006. The work of M. Aizenman was supported in part by the Einstein Center for Theoretical Physics and in part by the Minerva Center for Nonlinear Physics. The work of I. Kanter at the Weizmann Institute was supported by the Einstein Center for Theoretical Physics. E. Domany and O. Zuk were supported in part by the Minerva Foundation and in part by the European Community{\textquoteright}s Human Potential Programme under Contract HPRN-CT-2002-00319, STIPCO. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Cihan Tepedelenlioglu.",

year = "2006",

month = sep,

doi = "10.1109/LSP.2006.874466",

language = "אנגלית",

volume = "13",

pages = "517--520",

journal = "IEEE Signal Processing Letters",

issn = "1070-9908",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "9",

}

TY - JOUR

T1 - From finite-system entropy to entropy rate for a hidden Markov process

AU - Zuk, Or

AU - Domany, Eytan

AU - Kanter, Ido

AU - Aizenman, Michael

N1 - Funding Information: Manuscript received January 4, 2006; revised January 23, 2006. The work of M. Aizenman was supported in part by the Einstein Center for Theoretical Physics and in part by the Minerva Center for Nonlinear Physics. The work of I. Kanter at the Weizmann Institute was supported by the Einstein Center for Theoretical Physics. E. Domany and O. Zuk were supported in part by the Minerva Foundation and in part by the European Community’s Human Potential Programme under Contract HPRN-CT-2002-00319, STIPCO. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Cihan Tepedelenlioglu.

PY - 2006/9

Y1 - 2006/9

N2 - A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable ε. The coefficients of the expansion around the noiseless (ε = 0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.

AB - A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable ε. The coefficients of the expansion around the noiseless (ε = 0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.

KW - Entropy

KW - Hidden markov process (HMP)

KW - Taylor series

UR - http://www.scopus.com/inward/record.url?scp=33747783917&partnerID=8YFLogxK

U2 - 10.1109/LSP.2006.874466

DO - 10.1109/LSP.2006.874466

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:33747783917

SN - 1070-9908

VL - 13

SP - 517

EP - 520

JO - IEEE Signal Processing Letters

JF - IEEE Signal Processing Letters

IS - 9

ER -

From finite-system entropy to entropy rate for a hidden Markov process

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this