Search engines are perceived as a reliable source for general information needs. However, finding the answer to medical questions using search engines can be challenging for an ordinary user. Content can be biased and results may present different opinions. In addition, interpreting medically related content can be difficult for users with no medical background. All of these can lead users to incorrect conclusions regarding health related questions. In this work we address this problem from two perspectives. First, to gain insight on users' ability to correctly answer medical questions using search engines, we conduct a comprehensive user study. We show that for questions regarding medical treatment effectiveness, participants struggle to find the correct answer and are prone to overestimating treatment effectiveness. We analyze participants' demographic traits according to age and education level and show that this problem persists in all demographic groups. We then propose a semi-automatic machine learning approach to find the correct answer to queries on medical treatment effectiveness as it is viewed by the medical community. The model relies on the opinions presented in medical papers related to the queries, as well as features representing their impact. We show that, compared to human behaviour, our method is less prone to bias. We compare various configurations of our inference model and a baseline method that determines treatment effectiveness based solely on the opinion of medical papers. The results bolster our confidence that our approach can pave the way to developing automatic bias-free tools that can help mediate complex health related content to users.
|Title of host publication||SIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval|
|Publisher||Association for Computing Machinery, Inc|
|Number of pages||10|
|State||Published - 11 Jul 2021|
|Event||44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021 - Virtual, Online, Canada|
Duration: 11 Jul 2021 → 15 Jul 2021
|Name||SIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval|
|Conference||44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021|
|Period||11/07/21 → 15/07/21|
Bibliographical noteFunding Information:
We thank Ryen White and Ahmed Hassan for sharing their data with us. This work was supported in part by the NSF IIS (Grant No. 1553568) and the Israel Innovation Authority (Grant No. 70069).
© 2021 ACM.
- health search
- machine learning