JCT at SemEval-2020 Task 12: Offensive Language Detection in Tweets using Preprocessing Methods, Character and Word N-grams

Moshe Uzan, Yaakov HaCohen-Kerner

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

In this paper, we describe our submissions to SemEval-2020 contest. We tackled subtask 12 - “Multilingual Offensive Language Identification in Social Media”. We developed different models for four languages: Arabic, Danish, Greek, and Turkish. We applied three supervised machine learning methods using various combinations of character and word n-gram features. In addition, we applied various combinations of basic preprocessing methods. Our best submission was a model we built for offensive language identification in Danish using Random Forest. This model was ranked at the 6th position out of 39 submissions. Our result is lower by only 0.0025 than the result of the team that won the 4th place using entirely non-neural methods. Our experiments indicate that char ngram features are more helpful than word ngram features. This phenomenon probably occurs because tweets are more characterized by characters than by words, tweets are short, and contain various special sequences of characters, e.g., hashtags, shortcuts, slang words, and typos.

Original languageEnglish
Title of host publication14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings
EditorsAurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
PublisherInternational Committee for Computational Linguistics
Pages2017-2022
Number of pages6
ISBN (Electronic)9781952148316
StatePublished - 2020
Event14th International Workshops on Semantic Evaluation, SemEval 2020 - Barcelona, Spain
Duration: 12 Dec 202013 Dec 2020

Publication series

Name14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings

Conference

Conference14th International Workshops on Semantic Evaluation, SemEval 2020
Country/TerritorySpain
CityBarcelona
Period12/12/2013/12/20

Bibliographical note

Publisher Copyright:
© 2020 14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings. All rights reserved.

Fingerprint

Dive into the research topics of 'JCT at SemEval-2020 Task 12: Offensive Language Detection in Tweets using Preprocessing Methods, Character and Word N-grams'. Together they form a unique fingerprint.

Cite this