The CLEFeHealth Lab
This CLEF evaluation lab is a benchmarking activity aiming to develop processing methods and resources to enrich difficult-to-understand health text as well as their evaluation setting. It consists of an evaluation campaign and one-day workshop in CLEF2013 organised in Valencia, Spain on 23-26 September, 2013. The full programme is available here.
Focus
Both laypeople and clinicians have problems in understanding the jargon of eHealth documents. This lab aims to develop processing methods and resources to enrich difficult-to-understand health text as well as their evaluation setting that includes not only statistical metrics of correctness but also end-user engagement. The lab also provides mentoring to graduate students working on related fields.
Task Summary
Discharge summaries transfer summarised information from a ward/clinic to another. They describe the course of treatment, the status at release, and care plans. Both nurses and patients are likely to have difficulties in understanding their content, because of their compressed language full of medical jargon, nonstandard abbreviations, and ward-specific idioms. To support the continuum of care, the goal of the CLEF eHealth Evaluation Lab is to develop methods and resources that make discharge documents easier to understand from nurses and patients' perspective and address their differing queries and information needs when searching further details on matters mentioned in the discharge summaries. This could include extending abbreviations, generalising from trade names to more generic descriptions of medicine, attaching further definitions to difficult phrasings, and having user-centric web search engines available.
In 2013, the shared task contained three related sub-tasks:
- Task 1: identification of disorders from clinical reports and mapping of the SNOMED CT disorders to UMLS codes,
- Task 2: mapping abbreviations and acronyms in clinical reports to UMLS codes,
- Task 3: information retrieval to address questions patients may have when reading clinical reports.
For more details on the CLEFeHealth Lab, go to the 2013 CLEFehealth website.
The CLEFeHealth Task 3 - User-centred health information retrieval Task
Task 3 aims at evaluating information retrieval to address questions patients may have when reading clinical reports. It is a standard TREC-style information retrieval (IR) task using (a) a 2012 crawl of approximately one million medical documents made available by the EU-FP7 Khresmoi project in plain text form and (b) general public queries that individuals may realistically pose based on the content of their discharge summaries. Queries were generated from discharge summaries used in Tasks 1 and 2. The goal of Task 3 is to retrieve the relevant documents for the user queries.