Facing Change: Using Automated Facial Expression Analysis to Examine Emotional Flexibility in the Treatment of Depression

Objective: Depression involves deficits in emotional flexibility. To date, the varied and dynamic nature of emotional processes during therapy has mostly been measured at discrete time intervals using clients’ subjective reports. Because emotions tend to fluctuate and change from moment to moment, the understanding of emotional processes in the treatment of depression depends to a great extent on the existence of sensitive, continuous, and objectively codified measures of emotional expression. In this observational study, we used computerized measures to analyze high-resolution time-series facial expression data as well as self-reports to examine the association between emotional flexibility and depressive symptoms at the client as well as at the session levels. Method: Video recordings from 283 therapy sessions of 58 clients who underwent 16 sessions of manualized psychodynamic psychotherapy for depression were analyzed. Data was collected as part of routine practice in a university clinic that provides treatments to the community. Emotional flexibility was measured in each session using an automated facial expression emotion recognition system. The clients’ depression level was assessed at the beginning of each session using the Beck Depression Inventory-II (Beck et al., 1996). Results: Higher emotional flexibility was associated with lower depressive symptoms at the treatment as well as at the session levels. Conclusion: These findings highlight the centrality of emotional flexibility both as a trait-like as well as a state-like characteristic of depression. The results also demonstrate the usefulness of computerized measures to capture key emotional processes in the treatment of depression at a high scale and specificity.