Subjectivity Detection in English News using Large Language Models

Mohammad Shokri; Vivek Sharma; Elena Filatova; Shweta Jain; Sarah Ita Levitan

Subjectivity Detection in English News using Large Language Models

Mohammad Shokri
, Vivek Sharma
, Elena Filatova
, Shweta Jain
, Sarah Ita Levitan

City University of New York

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

16 Scopus citations

Abstract

Trust in media has reached a historical low as consumers increasingly doubt the credibility of the news they encounter. This growing skepticism is exacerbated by the prevalence of opinion-driven articles, which can influence readers’ beliefs to align with the authors’ viewpoints. In response to this trend, this study examines the expression of opinions in news by detecting subjective and objective language. We conduct an analysis of the subjectivity present in various news datasets and evaluate how different language models detect subjectivity and generalize to out-of-distribution data. We also investigate the use of in-context learning (ICL) within large language models (LLMs) and propose a straightforward prompting method that outperforms standard ICL and chain-of-thought (CoT) prompts.

Original language	English
Title of host publication	WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop
Editors	Orphee De Clercq, Valentin Barriere, Jeremy Barnes, Roman Klinger, Joao Sedoc, Shabnam Tafreshi
Publisher	Association for Computational Linguistics (ACL)
Pages	215-226
Number of pages	12
ISBN (Electronic)	9798891761568
State	Published - 2024
Externally published	Yes
Event	14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, WASSA 2024 - Bangkok, Thailand Duration: 15 Aug 2024 → …

Publication series

Name	WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop

Conference

Conference	14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, WASSA 2024
Country/Territory	Thailand
City	Bangkok
Period	15/08/24 → …

Bibliographical note

Publisher Copyright:
© 2024 Association for Computational Linguistics.

Cite this

Shokri, M., Sharma, V., Filatova, E., Jain, S., & Levitan, S. I. (2024). Subjectivity Detection in English News using Large Language Models. In O. De Clercq, V. Barriere, J. Barnes, R. Klinger, J. Sedoc, & S. Tafreshi (Eds.), WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop (pp. 215-226). (WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop). Association for Computational Linguistics (ACL).

Shokri, Mohammad ; Sharma, Vivek ; Filatova, Elena et al. / Subjectivity Detection in English News using Large Language Models. WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop. editor / Orphee De Clercq ; Valentin Barriere ; Jeremy Barnes ; Roman Klinger ; Joao Sedoc ; Shabnam Tafreshi. Association for Computational Linguistics (ACL), 2024. pp. 215-226 (WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop).

@inproceedings{b02f890d3dd349b29627beb2f6733de1,

title = "Subjectivity Detection in English News using Large Language Models",

abstract = "Trust in media has reached a historical low as consumers increasingly doubt the credibility of the news they encounter. This growing skepticism is exacerbated by the prevalence of opinion-driven articles, which can influence readers{\textquoteright} beliefs to align with the authors{\textquoteright} viewpoints. In response to this trend, this study examines the expression of opinions in news by detecting subjective and objective language. We conduct an analysis of the subjectivity present in various news datasets and evaluate how different language models detect subjectivity and generalize to out-of-distribution data. We also investigate the use of in-context learning (ICL) within large language models (LLMs) and propose a straightforward prompting method that outperforms standard ICL and chain-of-thought (CoT) prompts.",

author = "Mohammad Shokri and Vivek Sharma and Elena Filatova and Shweta Jain and Levitan, \{Sarah Ita\}",

note = "Publisher Copyright: {\textcopyright} 2024 Association for Computational Linguistics.; 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, WASSA 2024 ; Conference date: 15-08-2024",

year = "2024",

language = "אנגלית",

series = "WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "215--226",

editor = "\{De Clercq\}, Orphee and Valentin Barriere and Jeremy Barnes and Roman Klinger and Joao Sedoc and Shabnam Tafreshi",

booktitle = "WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop",

address = "ארצות הברית",

}

Shokri, M, Sharma, V, Filatova, E, Jain, S & Levitan, SI 2024, Subjectivity Detection in English News using Large Language Models. in O De Clercq, V Barriere, J Barnes, R Klinger, J Sedoc & S Tafreshi (eds), WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop. WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop, Association for Computational Linguistics (ACL), pp. 215-226, 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, WASSA 2024, Bangkok, Thailand, 15/08/24.

Subjectivity Detection in English News using Large Language Models. / Shokri, Mohammad; Sharma, Vivek; Filatova, Elena et al.
WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop. ed. / Orphee De Clercq; Valentin Barriere; Jeremy Barnes; Roman Klinger; Joao Sedoc; Shabnam Tafreshi. Association for Computational Linguistics (ACL), 2024. p. 215-226 (WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Subjectivity Detection in English News using Large Language Models

AU - Shokri, Mohammad

AU - Sharma, Vivek

AU - Filatova, Elena

AU - Jain, Shweta

AU - Levitan, Sarah Ita

PY - 2024

Y1 - 2024

N2 - Trust in media has reached a historical low as consumers increasingly doubt the credibility of the news they encounter. This growing skepticism is exacerbated by the prevalence of opinion-driven articles, which can influence readers’ beliefs to align with the authors’ viewpoints. In response to this trend, this study examines the expression of opinions in news by detecting subjective and objective language. We conduct an analysis of the subjectivity present in various news datasets and evaluate how different language models detect subjectivity and generalize to out-of-distribution data. We also investigate the use of in-context learning (ICL) within large language models (LLMs) and propose a straightforward prompting method that outperforms standard ICL and chain-of-thought (CoT) prompts.

AB - Trust in media has reached a historical low as consumers increasingly doubt the credibility of the news they encounter. This growing skepticism is exacerbated by the prevalence of opinion-driven articles, which can influence readers’ beliefs to align with the authors’ viewpoints. In response to this trend, this study examines the expression of opinions in news by detecting subjective and objective language. We conduct an analysis of the subjectivity present in various news datasets and evaluate how different language models detect subjectivity and generalize to out-of-distribution data. We also investigate the use of in-context learning (ICL) within large language models (LLMs) and propose a straightforward prompting method that outperforms standard ICL and chain-of-thought (CoT) prompts.

UR - https://www.scopus.com/pages/publications/85204886981

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85204886981

T3 - WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop

SP - 215

EP - 226

BT - WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop

A2 - De Clercq, Orphee

A2 - Barriere, Valentin

A2 - Barnes, Jeremy

A2 - Klinger, Roman

A2 - Sedoc, Joao

A2 - Tafreshi, Shabnam

PB - Association for Computational Linguistics (ACL)

T2 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, WASSA 2024

Y2 - 15 August 2024

ER -

Shokri M, Sharma V, Filatova E, Jain S, Levitan SI. Subjectivity Detection in English News using Large Language Models. In De Clercq O, Barriere V, Barnes J, Klinger R, Sedoc J, Tafreshi S, editors, WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2024. p. 215-226. (WASSA 2024 - 14th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, Proceedings of the Workshop).

Subjectivity Detection in English News using Large Language Models

Abstract

Publication series

Conference

Bibliographical note

Other files and links

Fingerprint

Cite this