TY - JOUR

T1 - Discrete universal filtering through incremental parsing

AU - Ordentlich, Erik

AU - Weissman, Tsachy

AU - Weinberger, Marcelo J.

AU - Somekh-Baruch, Anelia

AU - Merhav, Neri

PY - 2004

Y1 - 2004

N2 - In the discrete filtering problem, a data sequence over a finite alphabet is assumed to be corrupted by a discrete memoryless channel. The goal is to reconstruct the clean sequence, with as high a fidelity as possible, by way of causal processing of the noisy sequence alone, with the reconstruction at time t depending only on noisy observations occurring no later than t. We study a universal version of this problem in which no assumptions are made about the distribution of the clean data, which may even be non-stochastic. Using techniques from universal data compression, in particular, the incremental parsing rule of LZ78, we derive practical and efficient algorithms for the universal filtering of discrete sources. A finite-memory filter of order k has the property that the reconstruction at any time t is a time-invariant function only of noisy observations occurring between times t - k and t, inclusive. We show that our universal filtering algorithms perform essentially as well, in an expected sense (with respect to the noise process), as the best finite-memory filter of any fixed order, determined with full knowledge of the actual clean data sequence, for all such data sequences. We also consider more general finite-state filters and show that any such filter is arbitrarily well approximated by a finite-memory filter of growing order, thereby establishing the universality of the proposed algorithms with respect to this larger class. This result can be viewed as the filtering analogue of the well known optimality of LZ78 relative to the class of finite-state compressors.

AB - In the discrete filtering problem, a data sequence over a finite alphabet is assumed to be corrupted by a discrete memoryless channel. The goal is to reconstruct the clean sequence, with as high a fidelity as possible, by way of causal processing of the noisy sequence alone, with the reconstruction at time t depending only on noisy observations occurring no later than t. We study a universal version of this problem in which no assumptions are made about the distribution of the clean data, which may even be non-stochastic. Using techniques from universal data compression, in particular, the incremental parsing rule of LZ78, we derive practical and efficient algorithms for the universal filtering of discrete sources. A finite-memory filter of order k has the property that the reconstruction at any time t is a time-invariant function only of noisy observations occurring between times t - k and t, inclusive. We show that our universal filtering algorithms perform essentially as well, in an expected sense (with respect to the noise process), as the best finite-memory filter of any fixed order, determined with full knowledge of the actual clean data sequence, for all such data sequences. We also consider more general finite-state filters and show that any such filter is arbitrarily well approximated by a finite-memory filter of growing order, thereby establishing the universality of the proposed algorithms with respect to this larger class. This result can be viewed as the filtering analogue of the well known optimality of LZ78 relative to the class of finite-state compressors.

UR - http://www.scopus.com/inward/record.url?scp=2642510939&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.conferencearticle???

AN - SCOPUS:2642510939

SN - 1068-0314

SP - 352

EP - 361

JO - Proceedings of the Data Compression Conference

JF - Proceedings of the Data Compression Conference

T2 - Proceedings - DCC 2004 Data Compression Conference

Y2 - 23 March 2004 through 25 March 2004

ER -