Self-similar epochs: Value in arrangement

Eliav Buchnik, Edith Cohen, Avinatan Hassidim, Yossi Matias

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Optimization of machine learning models is commonly performed through stochastic gradient updates on randomly ordered training examples. This practice means that each fraction of an epoch comprises an independent random sample of the training data that may not preserve informative structure present in the full data. We hypothesize that the training can be more effective with self-similar arrangements that potentially allow each epoch to provide benefits of multiple ones. We study this for "matrix factorization" - the common task of learning metric embeddings of entities such as queries, videos, or words from example pairwise associations. We construct arrangements that preserve the weighted Jaccard similarities of rows and columns and experimentally observe training acceleration of 3%-37% on synthetic and recommendation datasets. Principled arrangements of training examples emerge as a novel and potentially powerful enhancement to SGD that merits further exploration.

Original languageEnglish
Title of host publication36th International Conference on Machine Learning, ICML 2019
PublisherInternational Machine Learning Society (IMLS)
Number of pages13
ISBN (Electronic)9781510886988
StatePublished - 2019
Externally publishedYes
Event36th International Conference on Machine Learning, ICML 2019 - Long Beach, United States
Duration: 9 Jun 201915 Jun 2019

Publication series

Name36th International Conference on Machine Learning, ICML 2019


Conference36th International Conference on Machine Learning, ICML 2019
Country/TerritoryUnited States
CityLong Beach

Bibliographical note

Funding Information:
We are grateful to the anonymous ICML 2019 reviewers for many helpful comments that allowed us to improve the presentation. This research is partially supported by the Israel Science Foundation (Grant No. 1841/14).

Publisher Copyright:
© 36th International Conference on Machine Learning, ICML 2019. All rights reserved.


Dive into the research topics of 'Self-similar epochs: Value in arrangement'. Together they form a unique fingerprint.

Cite this