Using 5WH Competency Questions for Validating Information Retrieval from Al-Quran: A Methodological Approach

Abstract

Various methods are used to validate information retrieved (IR) from unstructured text. Measuring the effectiveness of IR systems is an important focus of IR research, based on sample retrieval for testing combined with evaluation measures. Evaluation measures include both quantitative and qualitative aspects. Quantitative measures assess precision, recall, and benchmark scores, while qualitative measures focus on relevance, user satisfaction, and usability. As the quality of information retrieved is of importance to humans, the use of competency questions (CQs) has been introduced, particularly, in the case of information retrieved from Al-Quran - to validate the output. As the design of CQs usually relies on expert support, which can limit usability for general users seeking guidance, this research proposes using basic fact-finding 5WH questions. These are fact finding why, what, how, where, when and who questions. It is proposed to use these as CQs to test the relevance of Verses/Ayats retrieved on the basis of common text-patterns found in Al-Quran. We propose this methodological approach using 5WH CQs to test the competency of retrieved information, focusing on popular concepts like Salat, Zakat, and the Sun. It is found that sample Ayats retrieved on the basis of text-patterns found answer fundamental questions such as WHY? For example, why pray Salat? or why was the Sun created? Results show that this method evaluates the relevance and usability of IR systems. Further work based on repeating text-patterns in the Al-Quran is recommended to test whether this evaluation approach can help ontology learning.