Assessments
The relevance assessment are provided in standard TREC qrel format.
Result Set for training data
Relevance assessments for the 5 training queries were formed based on pooled sets generated using the Vector Space
Model and Okapi BM25. Pooled sets were created by taking the top 30 ranked documents returned by the two retrieval models
with duplicates removed. Relevance assessment has been performed by medical professionals.
Relevance is provided on a 2-point-scale: Non relevant (0); Relevant (1).
Result Set for Test Queries
The result set for Test Queries has been generated by pooling the top ranked runs submitted by task participants, and
having nursing professionals and lay people conduct manual relevance assessment on the pooled sets.
Relevance assessment was completed on a 4-point-scale, which is also mapped to a 2-point-scale.
ASSESSMENTS WILL BE PROVIDED AFTER SIGNING THE END-USER AGREEMENT.