BACKGROUND: Timely documentation of care preferences is an endorsed quality indicator for seriously ill patients admitted to intensive care units. Clinicians document their conversations about these preferences as unstructured free text in clinical notes from electronic health records.
AIM: To apply deep learning algorithms for automated identification of serious illness conversations documented in physician notes during intensive care unit admissions.
DESIGN: Using a retrospective dataset of physician notes, clinicians annotated all text documenting patient care preferences (goals of care or code status limitations), communication with family, and full code status. Clinician-coded text was used to train algorithms to identify documentation and to validate algorithms. The validated algorithms were deployed to assess the percentage of intensive care unit admissions of patients aged >=75 that had care preferences documented within the first 48 h.
SETTING/PARTICIPANTS: Patients admitted to one of five intensive care units.
RESULTS: Algorithm performance was calculated by comparing machine-identified documentation to clinician-coded documentation. For detecting care preference documentation at the note level, the algorithm had F1-score of 0.92 (95% confidence interval, 0.89 to 0.95), sensitivity of 93.5% (95% confidence interval, 90.0% to 98.0%), and specificity of 91.0% (95% confidence interval, 86.4% to 95.3%). Applied to 1350 admissions of patients aged >=75, we found that 64.7% of patient intensive care unit admissions had care preferences documented within the first 48 h.
CONCLUSION: Deep learning algorithms identified patient care preference documentation with sensitivity and specificity approaching that of clinicians and computed in a tiny fraction of time. Future research should determine the generalizability of these methods in multiple healthcare systems.
BACKGROUND: Palliative surgical procedures are frequently performed to reduce symptoms in patients with advanced cancer, but quality is difficult to measure.
OBJECTIVE: To determine whether natural language processing (NLP) of the electronic health record (EHR) can be used to (1) identify a population of cancer patients receiving palliative gastrostomy and (2) assess documentation of end-of-life process measures in the EHR.
DESIGN/SETTING: Retrospective cohort study of 302 adult cancer patients who received a gastrostomy tube at a single tertiary medical center.
MEASUREMENTS: Sensitivity and specificity of NLP compared to gold standard of manual chart abstraction in identifying a palliative indication for gastrostomy tube placement and documentation of goals of care discussions, code status determination, palliative care referral, and hospice assessment.
RESULTS: Among 302 cancer patients who underwent gastrostomy, 68 (22.5%) were classified by NLP as having a palliative indication for the procedure compared to 71 patients (23.5%) classified by human coders. Human chart abstraction took >2600 times longer than NLP (28 hours vs. 38 seconds). NLP identified the correct patients with 95.8% sensitivity and 97.4% specificity. NLP also identified end-of-life process measures with high sensitivity (85.7%-92.9%,) and specificity (96.7%-98.9%). In the two months leading up to palliative gastrostomy placement, 20.5% of patients had goals of care discussions documented. During the index hospitalization, 67.7% had goals of care discussions documented.
CONCLUSIONS: NLP offers opportunities to identify patients receiving palliative surgical procedures and can rapidly assess established end-of-life process measures with an accuracy approaching that of human coders.
BACKGROUND: Alone, administrative data poorly identifies patients with palliative care needs.
OBJECTIVE: To identify patients with uncommon, yet devastating, illnesses using a combination of administrative data and natural language processing (NLP).
DESIGN/SETTING: Retrospective cohort study using the electronic medical records of a healthcare network totaling over 2500 hospital beds. We sought to identify patient populations with two unique disease processes associated with a poor prognosis: pneumoperitoneum and leptomeningeal metastases from breast cancer.
MEASUREMENTS: Patients with pneumoperitoneum or leptomeningeal metastasis from breast cancer were identified through administrative codes and NLP.
RESULTS: Administrative codes alone resulted in identification of 6438 patients with possible pneumoperitoneum and 557 patients with possible leptomeningeal metastasis. Adding NLP to this analysis reduced the number of patients to 869 with pneumoperitoneum and 187 with leptomeningeal metastasis secondary to breast cancer. Administrative codes alone yielded a 13% positive predictive value (PPV) for pneumoperitoneum and 25% PPV for leptomeningeal metastasis. The combination of administrative codes and NLP achieved a PPV of 100%. The entire process was completed within hours.
CONCLUSIONS: Adding NLP to the use of administrative codes allows for rapid identification of seriously ill patients with otherwise difficult to detect disease processes and eliminates costly, tedious, and time-intensive manual chart review. This method enables studies to evaluate the effectiveness of treatment, including palliative interventions, for unique populations of seriously ill patients who cannot be identified by administrative codes alone.