Zur Hauptnavigation wechseln Zur Suche wechseln Zum Hauptinhalt wechseln

Using Small Language Models to Measure Nursing Key Performance Indicators From Patient Stories

  • Federico Cruciani*
  • , Birgit Schonfelder
  • , Michael Milliken*
  • , Ian Cleland*
  • , Chris Nugent*
  • , Hanna Mayer
  • , Tanya McCance
  • *Korrespondierende/r Autor/-in für diese Arbeit

Publikation: Beitrag in Buch/Bericht/Konferenzband › Beitrag in Konferenzband

Abstract

In the broader effort to humanize healthcare, person-centred nursing Key Performance Indicators (KPIs) have been proposed to evaluate care quality from the patient's perspective. Patient stories offer a rich, qualitative source for understanding care experiences, highlighting what matters most to individuals and revealing both positive and negative aspects of care. However, these narratives are typically unstructured and require expert review to identify relevant KPIs and assess associated sentiments. This study investigates the use of Small Language Models (SLMs) to automate the extraction of KPIs and sentiment classification from patient stories. We demonstrate that a fine-tuned BERT model can effectively perform both tasks, achieving a Cohen's kappa agreement score of 0.56, comparable to the inter-rater agreement observed between two human expert annotators (0.58) vs a baseline agreement of 0.15 using GPT-4 and 0.34 with GPT-5. The obtained macro-average F1-Score on the KPIs was 76.65 % whereas the F1-score of GPT-4 and GPT5 were 26.54 % and 44.78 % respectively. These findings suggest that SLMs hold promise for supporting scalable, person-centred evaluation of healthcare experiences.

OriginalspracheEnglisch
Titel8th International Conference on Activity and Behavior Computing, ABC 2026
Herausgeber (Verlag)Institute of Electrical and Electronics Engineers Inc.
ISBN (elektronisch)9798331578732
DOIs
PublikationsstatusVeröffentlicht - 2026
Veranstaltung8th International Conference on Activity and Behavior Computing, ABC 2026 - Hybrid, Hokkaido, Japan
Dauer: 09 März 2026 → 12 März 2026

Publikationsreihe

Name8th International Conference on Activity and Behavior Computing, ABC 2026

Konferenz

Konferenz8th International Conference on Activity and Behavior Computing, ABC 2026
Land/GebietJapan
OrtHybrid, Hokkaido
Zeitraum09.03.2026 → 12.03.2026

ASJC Scopus Sachgebiete

  • Artificial intelligence
  • Maschinelles Sehen und Mustererkennung

Fingerprint

Untersuchen Sie die Forschungsthemen von „Using Small Language Models to Measure Nursing Key Performance Indicators From Patient Stories“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren