Beyond HCAHPS: Analysis of patients’ comments provides an expanded review of their hospital experiences

An important concern for health care professionals is that standardized patient surveys may not fully capture all the topics that are important to patients. As a result, health care professionals may not have a complete picture of what their patients experience. The purpose of this research is to utilize a state-of-the-art Natural Language Processing technique to make sense of patients’ solicited, unstructured comments to gain a deeper and broader understanding of their experiences in the hospital. We analyzed a large dataset of inpatient survey responses (48,592 patients generating 65,998 comments) by a patient experience survey vendor for an eleven-hospital health care system in a large Midwest US city. Comments were first analyzed by Top2Vec algorithm in Python and more than 650 groupings of comments were then reduced into 20 sub-domains within 4 topic domains to better understand patient feedback on their hospital experience. We find distinct domains in the textual data that are not completely captured by survey domains. Furthermore, these domains match components of a hierarchical model of health service quality: interpersonal, technical, environmental, and administrative quality. Our findings broaden and deepen understanding of domains on standardized surveys. That is, completely new issues that are not measured in structured surveys are found in patient comments, and even when patient comments can be assigned to specific domains (e.g., nurse communication, discharge, etc.) found in standardized surveys, novel sub-topics provide a more nuanced understanding of patients’ hospital experiences. Novel sub-topics found in patient comments include clinicians’ diagnostic skill, compassionate care, team coordination, transfer processes, roommates, and others. Health care organizations should utilize state-of-the-art methods to mine insights from patient comments, and ensure they have processes, resources, and capabilities needed to translate insights into action.