A hybrid mixed methods design of qualitative enhancement and reciprocal feedback loop for augmented text classification

Keeping the ‘human-in-the-loop’ in automated text classification can improve its inference quality by supporting human sense-making that goes beyond current machine-learning algorithms. Hence, this methodological article presents a novel mixed-methods design that aims to enhance human sense-making and improve the inference quality of augmented text classification. It is a three-phase hybrid model: a preliminary qualitative phase, a core quantitative phase (i.e., the automated text classification), and a reciprocal feedback loop of a follow-up quantitative evaluation phase. This Hybrid mixed-methods design with a Reciprocal Feedback Loop is specified and then illustrated with a study of automated classification of illicit drug transaction messages in a Darknet forum. The article also discusses the conditions under which this design can improve the inference quality, and the benefit of reciprocal human–machine learning.