A Language Model identifies population-level features of the T cell Receptor via self-supervised learning

T cells are at the core of human health. Their unique ability to produce an overwhelming repertoire of receptors which they use to interact with threats and to assist with healthy functions has made them an interesting subject for Data Science. One such function, involved with multiple conditions including cancer, response to viral threats, autoimmune disease and more, is the ability to produce what are termed Public clones. Those Public Clones are T cells that are shared between individuals. Some of those clones are even shared between a high percentage of all observed samples. Yet, the reason for this sharing, as well as the DNA sequence that might characterize them is still unknown. Here, using a BERT-based language model, we show that a latent space built by self supervised learning provides distinct areas for Public and Private sequences. We continue to show that these embeddings could be successfully used for binary classification to tell apart Public and Private sequences.