Capturing between-tasks covariance and similarities using multivariate linear mixed models

Aviv Navon, Saharon Rosset

Research output: Contribution to journalArticlepeer-review


We consider the problem of predicting several response variables using the same set of explanatory variables. This setting naturally induces a group structure over the coefficient matrix, in which every explanatory variable corresponds to a set of related coefficients. Most of the existing methods that utilize this group formation assume that the similarities between related coefficients arise solely through a joint sparsity struc-ture. In this paper, we propose a procedure for constructing multivariate regression models, that directly capture and model the within-group simi-larities, by employing a multivariate linear mixed model formulation, with a joint estimation of covariance matrices for coefficients and errors via penalized likelihood. Our approach, which we term MrRCE for Multivariate random Regression with Covariance Estimation, encourages structured sim-ilarity in parameters, in which coefficients for the same variable in related tasks share the same sign and similar magnitude. We illustrate the benefits of our approach in synthetic and real examples, and show that the proposed method outperforms natural competitors and alternative estimators under several model settings.

Original languageEnglish
Pages (from-to)3821-3844
Number of pages24
JournalElectronic Journal of Statistics
Issue number2
StatePublished - 2020

Bibliographical note

Publisher Copyright:
© 2020, Institute of Mathematical Statistics. All rights reserved.


This work was done while the author was at Tel Aviv University, Israel. This research was partially supported by Israeli Science Foundation grant 1804/16. supported by Israeli Science Foundation grant

FundersFunder number
Israeli Science Foundation
Israel Science Foundation1804/16
Tel Aviv University


    • Covariance selection
    • EM algorithm
    • Multivariate regression
    • Penalized likelihood
    • Regularization methods
    • Sparse precision matrix


    Dive into the research topics of 'Capturing between-tasks covariance and similarities using multivariate linear mixed models'. Together they form a unique fingerprint.

    Cite this