Evaluation in (XML) Information Retrieval: Expected Precision-Recall
with User Modelling (EPRUM)
Source:
SIGIR 2006: Proceedings of the 29th Annual International ACM SIGIR
Conference on Research and Development in Information Retrieval, p.260–267 (2006)
Abstract:
Standard Information Retrieval (IR) metrics assume a simple model
where documents are understood as independent units. Such an assumption
is not adapted to new paradigms like XML or Web IR where retrievable
informations are parts of documents or sets of related documents.
Moreover, classical hypotheses assumes that the user ignores the
structural or logical context of document elements and hence the
possibility of navigation between units. EPRUM is a generalisation
of Precision-Recall (PR) that aims at allowing the user to navigate
or browse in the corpus structure. Like the Cumulated Gain metrics,
it is able to handle continuous valued relevance. We apply and compare
EPRUM in the context of XML Retrieval– a very active field
for evaluation metrics. We also explain how EPRUM can be used in
other IR paradigms.
