Sensory modeling: Understanding computation in sensory systems through image-computable models

A primary goal of neuroimaging is to relate fMRI measurements to neural tuning properties. Over the years, increasingly complex models, such as deep neural networks, have been proposed, attempting to extract as much information as possible from fMRI data. A complementary approach consists of modeling the neural system and using those models to fit the fMRI data. This approach is driven by insights into what the sensory system computes, rather than engineering an analysis pipeline that extracts information from fMRI data. Canonical models such as the steerable pyramid and Gabor filter banks have been applied to the visual cortex and have yielded important insights into the function and organization of both primary and higher order sensory brain regions. In this chapter, we describe the elements of the most commonly used image-computable models for visual neuroscience and discuss recent work applying these models to fMRI data. Finally, we discuss future potential extensions of the current models to other stimulus dimensions, modalities, and brain systems.