Boosting optimal logical patterns using noisy data

Noam Goldberg, Chung Chieh Shan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

9 Scopus citations


We consider the supervised learning of a binary classifier from noisy observations. We use smooth boosting to linearly combine abstaining hypotheses, each of which maps a subcube of the attribute space to one of the two classes. We introduce a new branch-and-bound weak learner to maximize the agreement rate of each hypothesis. Dobkin et al. give an algorithm for maximizing agreement with real-valued attributes [9]. Our algorithm improves on the time complexity of Dobkin et al.'s as long as the data can be binarized so that the number of binary attributes is o(log of the number of observations x number of real-valued attributes). Furthermore, we have fine-tuned our branch-and-bound algorithm with a queuing discipline and optimality gap to make it fast in practice. Finally, since logical patterns in Hammer et al.'s Logical Analysis of Data (LAD) framework [8, 6] are equivalent to abstaining monomial hypotheses, any boosting algorithm can be combined with our proposed weak learner to construct LAD models. On various data sets, our method outperforms state-of-the-art methods that use suboptimal or heuristic weak learners, such as SLIPPER. It is competitive with other optimizing classifiers that combine monomials, such as LAD. Compared to LAD, our method eliminates many free parameters that restrict the hypothesis space and require extensive fine-tuning by cross-validation.

Original languageEnglish
Title of host publicationProceedings of the 7th SIAM International Conference on Data Mining
PublisherSociety for Industrial and Applied Mathematics Publications
Number of pages9
ISBN (Print)9780898716306
StatePublished - 2007
Externally publishedYes
Event7th SIAM International Conference on Data Mining - Minneapolis, MN, United States
Duration: 26 Apr 200728 Apr 2007

Publication series

NameProceedings of the 7th SIAM International Conference on Data Mining


Conference7th SIAM International Conference on Data Mining
Country/TerritoryUnited States
CityMinneapolis, MN


Dive into the research topics of 'Boosting optimal logical patterns using noisy data'. Together they form a unique fingerprint.

Cite this