Generating Factually Consistent Sport Highlights Narrations

Noah Sarfati, Ido Yerushalmy, Michael Chertok, Yosi Keller

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Sports highlights are an important form of media for fans worldwide, as they provide short videos that capture key moments from games, often accompanied by the original commentaries of the game's announcers. However, traditional forms of presenting sports highlights have limitations in conveying the complexity and nuance of the game. In recent years, the use of Large Language Models (LLMs) for natural language generation has emerged and is a promising approach for generating narratives that can provide a more compelling and accessible viewing experience. In this paper, we propose an end-to-end solution to enhance the experience of watching sports highlights by automatically generating factually consistent narrations using LLMs and crowd noise extraction. Our solution involves several steps, including extracting the source of information from the live broadcast using a transcription model, prompt engineering, and comparing out-of-the-box models for consistency evaluation. We also propose a new dataset annotated on generated narratives from 143 Premier League plays and fine-tune a Natural Language Inference (NLI) model on it, achieving 92% precision. Furthermore, we extract crowd noise from the original video to create a more immersive and realistic viewing experience for sports fans by adapting speech enhancement SOTA models on a brand new dataset created from 155 Ligue 1 games.

Original languageEnglish
Title of host publicationMMSports 2023 - Proceedings of the 6th International Workshop on Multimedia Content Analysis in Sports, Co-located with
Subtitle of host publicationMM 2023
PublisherAssociation for Computing Machinery, Inc
Pages15-22
Number of pages8
ISBN (Electronic)9798400702693
DOIs
StatePublished - 29 Oct 2023
Externally publishedYes
Event6th ACM International Workshop on Multimedia Content Analysis in Sports, MMSports 2023, co-located with ACM Multimedia 2023 - Ottawa, Canada
Duration: 29 Oct 2023 → …

Publication series

NameMMSports 2023 - Proceedings of the 6th International Workshop on Multimedia Content Analysis in Sports, Co-located with: MM 2023

Conference

Conference6th ACM International Workshop on Multimedia Content Analysis in Sports, MMSports 2023, co-located with ACM Multimedia 2023
Country/TerritoryCanada
CityOttawa
Period29/10/23 → …

Bibliographical note

Publisher Copyright:
© 2023 ACM.

Keywords

  • factual consistency evaluation
  • hallucinations
  • large language models (llms)
  • natural language inference
  • prompt engineering
  • speech enhancing

Fingerprint

Dive into the research topics of 'Generating Factually Consistent Sport Highlights Narrations'. Together they form a unique fingerprint.

Cite this