MIAT: A novel attribute selection approach to better predict upper gastrointestinal cancer

The use of data mining has led to many significant medical discoveries. However, many challenges still exist in using these methods for knowledge discovery within this field given that the large amounts of data medical practitioners collect often creates a curse of dimensionality. To address this challenge, attribute selection approaches have been developed. However, current approaches typically put equal weight on all values within that attribute. At times, and especially within medical domains, we claim that these approaches might miss attributes where only a small subset of attribute values contain a strong indication for one of the target values and thus should still be selected. To quantify this approach, we present MIAT, an algorithm that defines Minority Interesting Attribute Thresholds to find these important attribute values. As we developed MIAT to help better diagnose upper gastrointestinal cancer, we present how we use the attributes selected through this approach to build a predictive model for this cancer. To demonstrate MIAT's generality, we also applied it to a canonical Hungarian Heart Disease Dataset. In both datasets we found that MIAT yields significantly better accuracy and sensitivity over traditional attribute selection approaches.