This article aims to demonstrate the virtues of the contextual approach to political theory through an examination of a well-known question: should liberal states tolerate illiberal groups? This question will be analysed via the illustrative case of the Israeli Ultra-Orthodox Jews' request to win an exemption from mandatory military conscription. We aim to demonstrate how a careful examination of the Ultra-Orthodox culture, and especially some core texts with regard to military service, will reveal significant insights, as the Ultra-Orthodox formative texts demonstrate a long-standing dispute regarding military service. This 'internal' dispute would not have been available to the researcher of toleration via a purely analytical approach. Several distinctive advantages follow the usage of the contextual approach, among them a better acquaintance with the group, the ability to structure adequately the tolerating approach and the likelihood that the chosen tolerating approach will win more legitimacy from both the general public and the illiberal group itself.