Lfinite Sample Bounds on the Performance of Weighted Linear Least Squares in Sub-Gaussian Correlated Noise

Michael Krikheli; Amir Leshem

doi:10.1109/dsw.2019.8755574

Lfinite Sample Bounds on the Performance of Weighted Linear Least Squares in Sub-Gaussian Correlated Noise

Michael Krikheli, Amir Leshem

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In this paper we provide finite sample bounds on the performance of the weighted linear least squares estimator in sub-Gaussian martingale difference correlated noise. In contrast to standard performance analysis which uses bounds on the mean square error together with asymptotic normality, our bounds are based on concentration of measure. We extend previous results by analyzing the weighted least squares estimator and provide novel results in the case of correlated noise and heteroscedasticity. Using these bounds we obtain accurate bounds on the tail of the estimator. We show fast exponential convergence of the L^∞ probability of error. We analyze the fixed design setting. We use the results to analyze the performance of the weighted least squares estimator for the important problem of system identification. We show how to extend the results to different norms and state a theorem for the L² norm.

Original language	English
Title of host publication	2019 IEEE Data Science Workshop, DSW 2019 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	110-114
Number of pages	5
ISBN (Electronic)	9781728107080
DOIs	https://doi.org/10.1109/dsw.2019.8755574
State	Published - Jun 2019
Event	2019 IEEE Data Science Workshop, DSW 2019 - Minneapolis, United States Duration: 2 Jun 2019 → 5 Jun 2019

Publication series

Name	2019 IEEE Data Science Workshop, DSW 2019 - Proceedings

Conference

Conference	2019 IEEE Data Science Workshop, DSW 2019
Country/Territory	United States
City	Minneapolis
Period	2/06/19 → 5/06/19

Bibliographical note

Publisher Copyright:
© 2019 IEEE.

Keywords

Estimation
concentration bounds
confidence bounds
finite sample
large deviations
martingale difference sequence
non Gaussian
system identification
weighted least squares

Access to Document

10.1109/dsw.2019.8755574

Cite this

Krikheli, M., & Leshem, A. (2019). Lfinite Sample Bounds on the Performance of Weighted Linear Least Squares in Sub-Gaussian Correlated Noise. In 2019 IEEE Data Science Workshop, DSW 2019 - Proceedings (pp. 110-114). Article 8755574 (2019 IEEE Data Science Workshop, DSW 2019 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/dsw.2019.8755574

@inproceedings{b945d6e793574ea88faca2649c8fb951,

title = "Lfinite Sample Bounds on the Performance of Weighted Linear Least Squares in Sub-Gaussian Correlated Noise",

abstract = "In this paper we provide finite sample bounds on the performance of the weighted linear least squares estimator in sub-Gaussian martingale difference correlated noise. In contrast to standard performance analysis which uses bounds on the mean square error together with asymptotic normality, our bounds are based on concentration of measure. We extend previous results by analyzing the weighted least squares estimator and provide novel results in the case of correlated noise and heteroscedasticity. Using these bounds we obtain accurate bounds on the tail of the estimator. We show fast exponential convergence of the L∞ probability of error. We analyze the fixed design setting. We use the results to analyze the performance of the weighted least squares estimator for the important problem of system identification. We show how to extend the results to different norms and state a theorem for the L2 norm.",

keywords = "Estimation, concentration bounds, confidence bounds, finite sample, large deviations, martingale difference sequence, non Gaussian, system identification, weighted least squares",

author = "Michael Krikheli and Amir Leshem",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 IEEE Data Science Workshop, DSW 2019 ; Conference date: 02-06-2019 Through 05-06-2019",

year = "2019",

month = jun,

doi = "10.1109/dsw.2019.8755574",

language = "אנגלית",

series = "2019 IEEE Data Science Workshop, DSW 2019 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "110--114",

booktitle = "2019 IEEE Data Science Workshop, DSW 2019 - Proceedings",

address = "ארצות הברית",

}

Krikheli, M & Leshem, A 2019, Lfinite Sample Bounds on the Performance of Weighted Linear Least Squares in Sub-Gaussian Correlated Noise. in 2019 IEEE Data Science Workshop, DSW 2019 - Proceedings., 8755574, 2019 IEEE Data Science Workshop, DSW 2019 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 110-114, 2019 IEEE Data Science Workshop, DSW 2019, Minneapolis, United States, 2/06/19. https://doi.org/10.1109/dsw.2019.8755574

Lfinite Sample Bounds on the Performance of Weighted Linear Least Squares in Sub-Gaussian Correlated Noise. / Krikheli, Michael; Leshem, Amir.
2019 IEEE Data Science Workshop, DSW 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. p. 110-114 8755574 (2019 IEEE Data Science Workshop, DSW 2019 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Lfinite Sample Bounds on the Performance of Weighted Linear Least Squares in Sub-Gaussian Correlated Noise

AU - Krikheli, Michael

AU - Leshem, Amir

PY - 2019/6

Y1 - 2019/6

N2 - In this paper we provide finite sample bounds on the performance of the weighted linear least squares estimator in sub-Gaussian martingale difference correlated noise. In contrast to standard performance analysis which uses bounds on the mean square error together with asymptotic normality, our bounds are based on concentration of measure. We extend previous results by analyzing the weighted least squares estimator and provide novel results in the case of correlated noise and heteroscedasticity. Using these bounds we obtain accurate bounds on the tail of the estimator. We show fast exponential convergence of the L∞ probability of error. We analyze the fixed design setting. We use the results to analyze the performance of the weighted least squares estimator for the important problem of system identification. We show how to extend the results to different norms and state a theorem for the L2 norm.

AB - In this paper we provide finite sample bounds on the performance of the weighted linear least squares estimator in sub-Gaussian martingale difference correlated noise. In contrast to standard performance analysis which uses bounds on the mean square error together with asymptotic normality, our bounds are based on concentration of measure. We extend previous results by analyzing the weighted least squares estimator and provide novel results in the case of correlated noise and heteroscedasticity. Using these bounds we obtain accurate bounds on the tail of the estimator. We show fast exponential convergence of the L∞ probability of error. We analyze the fixed design setting. We use the results to analyze the performance of the weighted least squares estimator for the important problem of system identification. We show how to extend the results to different norms and state a theorem for the L2 norm.

KW - Estimation

KW - concentration bounds

KW - confidence bounds

KW - finite sample

KW - large deviations

KW - martingale difference sequence

KW - non Gaussian

KW - system identification

KW - weighted least squares

UR - http://www.scopus.com/inward/record.url?scp=85069453439&partnerID=8YFLogxK

U2 - 10.1109/dsw.2019.8755574

DO - 10.1109/dsw.2019.8755574

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85069453439

T3 - 2019 IEEE Data Science Workshop, DSW 2019 - Proceedings

SP - 110

EP - 114

BT - 2019 IEEE Data Science Workshop, DSW 2019 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE Data Science Workshop, DSW 2019

Y2 - 2 June 2019 through 5 June 2019

ER -

Lfinite Sample Bounds on the Performance of Weighted Linear Least Squares in Sub-Gaussian Correlated Noise

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this