Human Evaluation of Yorùbá Noun Ontology

Abstract

African Scholars have engaged in Ontological annotation   to present explicitly the content of concepts in a domain by using formal machine-readable labels which must conform to an agreed standard in order to achieve interoperability of Natural Language Processing (NLP) systems. As good as this development is, there is the need to evaluate the contents of the concepts labelled for semantic web activities. This paper therefore presents an evaluation report carried out in building a hybrid semantic web engineering system for Yorùbá nouns. Human evaluation method through the statistic evaluation tools was employed. To achieve the desired objective of proper human evaluation, two different technical workshops were held on the 13^th and 20^th April 2018 at the University of Ibadan, bringing twenty (20) Linguistics Scholars together to examine and validate the extracted lexical entries before annotation. After thorough scrutiny, each of the scholars evaluated the entries with the use of questionnaire. The SPSS data analysis software was used to analyse the responses to different question statements based on the five (5) variables of: Accuracy, Completeness, Semantic error freeness, Orthographic error freeness and Ambiguity freeness rating. Eight (8) questions were also used to measure the experts’ perception on the usefulness of the model. The t-test on the experts’ perception of the five variables indicated that none of the respondents has poor perception of the software. The study therefore concludes that thorough evaluation procedure is essential to benchmark the internal lexical contents for building ontology models.

Keywords: Yorùbá noun, Ontological annotation, Evaluation, Natural language processing