Collaborative Image Understanding

Koby Bibas, Oren Sar Shalom, Dietmar Jannach

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations


Automatically understanding the contents of an image is a highly relevant problem in practice. In e-commerce and social media settings, for example, a common problem is to automatically categorize user-provided pictures. Nowadays, a standard approach is to fine-tune pre-trained image models with application-specific data. Besides images, organizations however often also collect collaborative signals in the context of their application, in particular how users interacted with the provided online content, e.g., in forms of viewing, rating, or tagging. Such signals are commonly used for item recommendation, typically by deriving latent user and item representations from the data. In this work, we show that such collaborative information can be leveraged to improve the classification process of new images. Specifically, we propose a multitask learning framework, where the auxiliary task is to reconstruct collaborative latent item representations. A series of experiments on datasets from e-commerce and social media demonstrates that considering collaborative signals helps to significantly improve the performance of the main task of image classification by up to 9.1%.

Original languageEnglish
Title of host publicationCIKM 2022 - Proceedings of the 31st ACM International Conference on Information and Knowledge Management
PublisherAssociation for Computing Machinery
Number of pages11
ISBN (Electronic)9781450392365
StatePublished - 17 Oct 2022
Externally publishedYes
Event31st ACM International Conference on Information and Knowledge Management, CIKM 2022 - Atlanta, United States
Duration: 17 Oct 202221 Oct 2022

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings


Conference31st ACM International Conference on Information and Knowledge Management, CIKM 2022
Country/TerritoryUnited States

Bibliographical note

Publisher Copyright:
© 2022 ACM.


  • image classification
  • multi-task learning
  • recommendation system


Dive into the research topics of 'Collaborative Image Understanding'. Together they form a unique fingerprint.

Cite this