Generalization bounds and consistency for latent structural probit and ramp loss

David McAllester; Joseph Keshet

Generalization bounds and consistency for latent structural probit and ramp loss

David McAllester, Joseph Keshet

Toyota Technological Institute at Chicago

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

45 Scopus citations

Abstract

We consider latent structural versions of probit loss and ramp loss. We show that these surrogate loss functions are consistent in the strong sense that for any feature map (finite or infinite dimensional) they yield predictors approaching the infimum task loss achievable by any linear predictor over the given features. We also give finite sample generalization bounds (convergence rates) for these loss functions. These bounds suggest that probit loss converges more rapidly. However, ramp loss is more easily optimized on a given sample.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 24
Subtitle of host publication	25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011
Publisher	Neural Information Processing Systems
ISBN (Print)	9781618395993
State	Published - 2011
Externally published	Yes
Event	25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011 - Granada, Spain Duration: 12 Dec 2011 → 14 Dec 2011

Publication series

Name	Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011

Conference

Conference	25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011
Country/Territory	Spain
City	Granada
Period	12/12/11 → 14/12/11

Cite this

McAllester, D., & Keshet, J. (2011). Generalization bounds and consistency for latent structural probit and ramp loss. In Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011 (Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011). Neural Information Processing Systems.

McAllester, David ; Keshet, Joseph. / Generalization bounds and consistency for latent structural probit and ramp loss. Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011. Neural Information Processing Systems, 2011. (Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011).

@inproceedings{599523670867408e8c7e9919df6ce7ed,

title = "Generalization bounds and consistency for latent structural probit and ramp loss",

abstract = "We consider latent structural versions of probit loss and ramp loss. We show that these surrogate loss functions are consistent in the strong sense that for any feature map (finite or infinite dimensional) they yield predictors approaching the infimum task loss achievable by any linear predictor over the given features. We also give finite sample generalization bounds (convergence rates) for these loss functions. These bounds suggest that probit loss converges more rapidly. However, ramp loss is more easily optimized on a given sample.",

author = "David McAllester and Joseph Keshet",

year = "2011",

language = "אנגלית",

isbn = "9781618395993",

series = "Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011",

publisher = "Neural Information Processing Systems",

booktitle = "Advances in Neural Information Processing Systems 24",

note = "25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011 ; Conference date: 12-12-2011 Through 14-12-2011",

}

McAllester, D & Keshet, J 2011, Generalization bounds and consistency for latent structural probit and ramp loss. in Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011. Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011, Neural Information Processing Systems, 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011, Granada, Spain, 12/12/11.

Generalization bounds and consistency for latent structural probit and ramp loss. / McAllester, David; Keshet, Joseph.
Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011. Neural Information Processing Systems, 2011. (Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Generalization bounds and consistency for latent structural probit and ramp loss

AU - McAllester, David

AU - Keshet, Joseph

PY - 2011

Y1 - 2011

N2 - We consider latent structural versions of probit loss and ramp loss. We show that these surrogate loss functions are consistent in the strong sense that for any feature map (finite or infinite dimensional) they yield predictors approaching the infimum task loss achievable by any linear predictor over the given features. We also give finite sample generalization bounds (convergence rates) for these loss functions. These bounds suggest that probit loss converges more rapidly. However, ramp loss is more easily optimized on a given sample.

AB - We consider latent structural versions of probit loss and ramp loss. We show that these surrogate loss functions are consistent in the strong sense that for any feature map (finite or infinite dimensional) they yield predictors approaching the infimum task loss achievable by any linear predictor over the given features. We also give finite sample generalization bounds (convergence rates) for these loss functions. These bounds suggest that probit loss converges more rapidly. However, ramp loss is more easily optimized on a given sample.

UR - http://www.scopus.com/inward/record.url?scp=85162419778&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85162419778

SN - 9781618395993

T3 - Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011

BT - Advances in Neural Information Processing Systems 24

PB - Neural Information Processing Systems

T2 - 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011

Y2 - 12 December 2011 through 14 December 2011

ER -

McAllester D, Keshet J. Generalization bounds and consistency for latent structural probit and ramp loss. In Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011. Neural Information Processing Systems. 2011. (Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011).

Generalization bounds and consistency for latent structural probit and ramp loss

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this