Customized registry tool for tracking adherence to clinical guidelines for head and neck cancers: protocol for a pilot study
Pilot and Feasibility Studies volume 6, Article number: 16 (2020)
Despite recommendations for monitoring patients with chronic and high-risk conditions, gaps still remain. These gaps are exacerbated in outpatient care, where patients and clinicians face challenges related to care coordination, multiple electronic health records, and extensive follow-up. In addition, low-income and racial/ethnic minority populations that are disproportionately cared for in safety net settings are particularly at risk to lapses in monitoring.
We aim to implement and evaluate a health information technology platform developed using systems engineering methodologies. The implementation is situated in a clinic that monitors patients with head and neck cancer within a large, urban, publicly funded hospital. Our study will evaluate the time it takes for patients to progress through key treatment milestones prior to and after implementation of the tool. We will use models controlling for secular trend to estimate the effect of the tool on improving timely and successful completion of guideline-based care processes.
This protocol details the evaluation of the effectiveness of a human-centered health information technology intervention on improving timely delivery of care for high-risk populations. Other settings, including those that face challenges related to limited resources to devote to safety programs and fragmented health information technology, may benefit from this approach.
ClinicalTrials.gov, NCT03546322. “Customized Registry Tool for Tracking Adherence to Clinical Guidelines for Head and Neck Cancers.” Registered 1 June 2018.
Since the publication of “To Err is Human” and other subsequent reports by the Institute of Medicine, there has been increasing focus on harms caused by medical care [1,2,3]. Errors of omission, or failing to deliver needed treatment, are especially common in the ambulatory setting, where patients receive only 55% of recommended preventive care. Those with chronic conditions fare no better, receiving only 56% of recommended care .
Delivery of appropriate care in the ambulatory setting frequently requires monitoring and treatment over multiple visits and often extended periods of time, particularly for those with chronic diseases or complex conditions. Missed monitoring may put patients at high-risk of adverse effects of treatment or failed detection of progression of disease. In addition to periodic monitoring, patients with complex conditions may require coordination between multiple specialties and complex treatment regimens that require timely completion of needed steps. For example, patients with cancer often require extensive diagnostic workup, evaluation by an interdisciplinary group of providers, and coordination of multiple treatment modalities. In head and neck cancer, the National Comprehensive Cancer Network (NCCN) has published guidelines for recommended pre-treatment workup, treatment, and post-treatment monitoring . Despite the presence of these guidelines, fewer than half of patients undergoing surgery for head and neck squamous cell carcinoma receive post-surgery radiation within the timeframe recommended by NCCN guidelines . Unsurprisingly, studies show an association between delays in treatment initiation and increased mortality . Treatment interruptions have also been associated with both persistent disease  and mortality . Perhaps more concerning, the proportion of patients receiving guideline-based care has decreased over time, and patients with lower socioeconomic status who receive care regardless of insurance status or their ability to pay in safety net settings appear to be most vulnerable [6, 10]. Though reasons for failure to adhere to guidelines are complex, lapses are frequently the result of the inability of clinics and health systems to proactively identify patients who meet criteria for guideline-based diagnostic or therapeutic interventions .
Traditionally, face-to-face visits have been the primary means of monitoring patients and ensuring completion of treatment steps. However, given the frequency of errors of omission—missed cancer diagnoses constitute the leading cause of paid medical malpractice claims among outpatients—solutions to systematically identify upcoming and overdue monitoring across a population of patients are needed [12, 13]. Similar to cancer screening, which is now often based on electronic registries and outreach to patients rather than purely visit-based activity, there is an opportunity to use technology and team-based workflows to enhance outpatient monitoring for high-risk conditions.
As part of the process for developing the information technology (IT) tool used in this study, we conducted a series of qualitative interviews with practitioners in a number of clinics involved in delivering care to patients with chronic or complex medical conditions at high-risk of experiencing monitoring related medical errors. We applied a technique known as journey mapping to map the process of patient monitoring in each clinic and identify difficult or high-risk steps in the monitoring process . Focusing on “pain points” identified through the journey mapping process for the Otolaryngology Clinic at the Zuckerberg San Francisco General Hospital, we applied a framework adapted from Systems Engineering Initiative for Patient Safety (SEIPS)  toward the development of a health IT tool that allows for context-specific customization of monitoring and treatment protocols for patients with head and neck cancer. This tool enables the clinic to develop custom diagnostic and treatment plans for patients with head and neck cancer, and facilitates subsequent population level tracking of completion of needed diagnostic or treatment steps.
We aim to implement the tool developed through this journey mapping process in the head and neck cancer clinic within a large safety net hospital and evaluate impact of this tool on timeliness of diagnostic evaluation, treatment initiation, and monitoring, as well as adherence to established protocols. Our objective here is to describe the study protocol for this pilot evaluation.
This study will evaluate implementation of a health IT tool designed to track patient progress toward diagnostic evaluation, treatment, and post-treatment monitoring for head and neck cancer. The study will evaluate the amount of time it takes patients to progress through key treatment milestones prior to and after implementation of the tool. The study design is a cohort study consisting of two cohorts of patients—a pre-treatment cohort including patients who have received a diagnosis of head and neck cancer but have not started treatment, and a post-treatment cohort of patients who have initiated treatment. Models controlling for secular trend will be used for data analysis to estimate the effect of tool implementation on improving timely and successful completion of guideline-based care processes. This analytic approach will help separate out the effect of the intervention from other temporal trends in care process completion. This study was approved by the University of California, San Francisco Institutional Review Board (12-09658) and is registered on ClinicalTrials.gov (Protocol ID: P30HS023558-1).
This study will take place within the Otolaryngology-Head and Neck Surgery Clinic at the Zuckerberg San Francisco General Hospital, a large county hospital affiliated with a tertiary care academic center. Patients served by this clinic are publicly insured. While the clinic has used an electronic health record for 22 years, it faces challenges, shared by many safety net systems, associated with coordinating care across multiple electronic platforms and record-keeping systems. Until adoption of this tool, the clinic did not have access to an integrated electronic registry system to monitor progress of care plans for patients undergoing diagnostic workup or treatment for head and neck cancers.
Health IT tool
The intervention evaluated in this study is an innovative workflow that incorporates an electronic registry tool. The tool integrates data from several, fragmented electronic medical records and allows clinic staff to monitor patient progress through care plans across the clinic’s entire panel (Fig. 1). Prior to development of this tool, the clinic relied on tracking methods requiring intensive manual data entry to populate a database that was not integrated with the medical record. These methods included a system of paper note cards and later an electronic spreadsheet to track patient progress. Using this tool requires changes in the clinic team’s workflow and communication.
The tool evaluated here was developed to address key process challenges identified through a “journey mapping” method with clinic staff in five clinics that monitor high-risk cancer situations: otolaryngology, pulmonary, breast, urology, and gastroenterology . The journey mapping process identified two vulnerabilities present across all five sub-specialty clinics studied, namely (1) the need to track patient progress toward diagnostic or therapeutic goals and (2) the difficulty in creating comprehensive patient lists for monitoring. Practitioners in the Otolaryngology-Head and Neck Surgery Clinic also identified challenges related to coordinating multi-specialty treatment plans. Our team mapped processes identified during the journey mapping exercise onto the SEIPS model for work system, a model that describes the interrelated components of people, tasks, technology/tools, organization, and environment that make up clinical work systems . We chose to focus on developing a solution to address challenges in work systems related to tasks and technology, as these were both the most clear areas of challenge and the most immediately amenable to change in the Otolaryngology-Head and Neck Surgery Clinic. In collaboration with CipherHealth, a health IT company, we developed a registry tool that integrates clinical data from multiple electronic health records. This tool allows for creation of clinic-specific treatment plans (Fig. 2) and tailored queries to identify patients who are overdue for completing key steps of their assigned treatment plan (Fig. 3).
Chart review will be conducted for all patients in the Otolaryngology-Head and Neck Surgery Clinic undergoing diagnostic workup, treatment, or post-treatment monitoring for head and neck cancer for the 2-year period leading up to implementation of the tool and the 2-year period after the tool has been implemented (with the 6 months prior to tool implementation excluded to avoid misclassification of exposure to the intervention, as discussed below). All patients with confirmed diagnosis of head and neck cancer who were seen in the clinic at least once will be included in the chart review. Based on prior clinic chart review, there are about 75 patients with head and neck cancer seen in the clinic each year who meet enrollment criteria. Thus, the chosen 4-year time period was selected to allow for inclusion of approximately 300 patients in our sample. Data will be extracted from patient charts and will include patient demographic factors, details of the diagnosis, and key treatment dates (Table 1). Unless noted, all clinical data will be collected from the medical record for both the intervention and control time periods, to minimize differential data collection methods that may influence comparisons.
Patients included in the study will be divided into two separate cohorts. The pre-treatment cohort will include patients with a confirmed diagnosis of head and neck cancer but who have not yet initiated treatment. The post-treatment cohort will include patients who have initiated treatment and are undergoing additional treatment modalities and monitoring. Patients who received a diagnosis of head and neck cancer and initiated any treatment will thus be present in both cohorts. Outcomes for each cohort consist of completion of key steps in the evaluation and treatment process, and are summarized in Table 2. Usual clinic protocols will be employed to address patients who are lost to follow-up (three outreach attempts, then moved to lost to follow-up list). Patients who are lost to follow-up will be included in the final analysis and will be considered to have not met the process outcome for their given cohort (i.e., it will be assumed that patients lost to follow-up from the pre-treatment cohort never initiated treatment).
Classification of exposure to the intervention
Patients will be considered unexposed to the intervention if they entered one of the cohorts at least six months prior to implementation of the health IT tool intervention. Patients who entered a cohort less than 6 months prior to implementation of the intervention will be excluded from that cohort. All patients entering a cohort after implementation of the tool will be considered exposed to the intervention for that cohort. For example, if a patient was diagnosed with head and neck cancer 3 months prior to implementation of the intervention and initiated treatment 1 week after implementation of the intervention, they would be excluded from the pre-treatment cohort and would be included in the post-treatment cohort and considered exposed to the intervention.
The key implementation outcome is feasibility. We will also measure several components of tool utilization to better understand the actor, dose, temporality, and action target of the intervention . Data collection on these parameters will occur through a quarterly survey of clinic staff using the tool and through five randomly selected clinic days when the investigators will observe clinic staff and any use of the tool that occurs. The survey will ask staff to report their role in the clinic and recall for the prior week the amount of time that the tool was used, timing of use, and number of patients outreached through the use of the tool. The feasibility objective for this pilot study is to achieve consistent use throughout all clinic sessions reaching all eligible patients. The feasibility outcome which will determine success and trigger proceeding to the main trial is achievement of significant use: use during at least 80% of clinic sessions and for 80% of eligible patients.
Incorporation of a term for calendar time into our models for primary and secondary outcomes will allow us to control for secular trend. This strengthens the analysis by eliminating temporal improvements in clinical care processes that are unrelated to the intervention. Time to event analyses will use Cox proportional hazards models. For binary outcomes including loss to follow-up and completion of treatment steps such as post-treatment radiation within 6 weeks, logistic regression will be used. Intervention effect estimates will be adjusted for potential confounders including patient demographics, substance use, housing status, and cancer stage.
Minimum detectable effects
In Cox models for time to event, the sample of 300 will provide 80% power within two-sided tests with alpha of 0.05 to detect a hazard ratio of 2.25 for the effect of the intervention, after adjusting for a linear temporal trend as well as confounders. Estimation of preliminary effects should be considered secondary objectives. The sample size we propose is based on feasibility outcomes, and therefore the simulations below demonstrate that the pilot sample size would only detect a substantial effect size. Depending on the feasibility outcomes, we would proceed to a larger trials with the ability to detect more modest effects. With the present number of patients, in Cox models for time to event, the sample of 300 will provide 80% power within two-sided tests with alpha of 0.05 to detect a hazard ratio of 2.25 for the effect of the intervention, after adjusting for a linear temporal trend as well as confounders. For binary outcomes including loss to follow-up and completion of stages, it will provide 80% power to detect intervention odds ratios of 3.4 to 6.7, depending on the number of patients included in the analysis (200–300) and the prevalence of the outcome (20 to 50%), again after adjusting for a linear temporal trend and confounders. These estimates were obtained using simulations implemented in R Version 3.4.3.
Health IT and management tools can improve the quality and safety of care delivered . The tool designed in this study has the potential to improve patient safety in diagnostic, treatment, and monitoring steps for management of patients with head and neck cancer. By mapping out care processes in conjunction with frontline clinicians and incorporating this information with the systems engineering perspective provided by the SEIPS framework, we developed a tool that clearly addresses challenging aspects of providing care that both “keep clinicians up at night” and contribute to poor clinical outcomes. This study will evaluate the effectiveness of this tool for improving timely delivery of care, and results will inform the development of a multi-center trial to evaluate effectiveness of this tool in a broader array of healthcare settings.
Although our study is limited somewhat by the retrospective nature of our pre-intervention comparison, we plan to address this limitation by adjusting for confounding by other temporal trends during the period of comparison. The study is also unable to evaluate patient progress towards diagnosis once referred to the Otolaryngology-Head and Neck Surgery Clinic. Though the tool will be used to monitor this aspect of care, there is no reliable way to identify the pre-intervention comparison cohort.
Ultimately, we hope that this tool will be adapted to other clinical contexts to expand the ability of clinics and care providers to track and intervene on patients not meeting treatment and monitoring goals. We plan to publish study results in the scientific literature, as well as compile and share results with the Otolaryngology-Head and Neck Surgery Clinic included in this study.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
National Comprehensive Cancer Network
Systems Engineering Initiative for Patient Safety
Institute of Medicine. To err is human: building a safer health system. Washington, DC: National Academy Press, Institute of Medicine Committee on Quality of Health Care in America; 2000.
Institute of Medicine Committee on Quality Health Care in America. Improving the 21st-Century Health Care System. In: Crossing the Quality Chasm: A New Health System for the 21st Century [Internet]. Washington, D.C.: National Academy Press; 2001. p. 39–60. Available from: http://www.nap.edu/openbook/0309072808/html/39.html
Committee on Diagnostic Error in Health Care. Improving Diagnosis in Health Care [Internet]. Washington, DC: The National Academies Press; 2015. Available from: http://www.nap.edu/catalog/21794/improving-diagnosis-in-health-care
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003 Jun 26;348(26):2635–45.
Pfister D, Spencer S, Adelstein D, Adkins D, Brizel D, Burtness B, et al. NCCN clinical practice guidelines in oncology. Head and Neck Cancers [Internet]. National Comprehensive Cancer Network; 2019 Sep [cited 2019 Nov 5]. Report No.: Version 3.2019. Available from: https://www.nccn.org/professionals/physician_gls/pdf/head-and-neck.pdf
Graboyes EM, Garrett-Mayer E, Sharma AK, Lentsch EJ, Day TA. Adherence to National Comprehensive Cancer Network guidelines for time to initiation of postoperative radiation therapy for patients with head and neck cancer. Cancer. 2017 Jul 15;123(14):2651–60.
Murphy CT, Galloway TJ, Handorf EA, Egleston BL, Wang LS, Mehra R, et al. Survival impact of increasing time to treatment initiation for patients with head and neck cancer in the United States. J Clin Oncol. 2016 Jan 10;34(2):169–78.
Patel UA, Patadia MO, Holloway N, Rosen F. Poor radiotherapy compliance predicts persistent regional disease in advanced head/neck cancer. Laryngoscope. 2009 Mar;119(3):528–33.
Fesinmeyer MD, Mehta V, Blough D, Tock L, Ramsey SD. Effect of radiotherapy interruptions on survival in medicare enrollees with local and regional head-and-neck cancer. Int J Radiat Oncol Biol Phys. 2010 Nov 1;78(3):675–81.
Munck K, Ali MJ, Murr AH, Goldberg AN. Impact of socioeconomic status on the diagnosis to treatment interval in Waldeyer’s ring carcinoma. Laryngoscope. 2005 Jul;115(7):1283–7.
Hall SF, Irish JC, Gregg RW, Groome PA, Rohland S. Adherence to and uptake of clinical practice guidelines: lessons learned from a clinical practice guideline on chemotherapy concomitant with radiotherapy in head-and-neck cancer. Curr Oncol. 2015 Apr;22(2):e61–8.
Gandhi TK, Kachalia A, Thomas EJ, Puopolo AL, Yoon C, Brennan TA, et al. Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med. 2006 Oct 3;145(7):488–96.
Annual benchmarking report: malpractice risks in the diagnostic process. Cambridge: CRICO Strategies; 2014 p. 1–20.
McDonald KM, Su G, Lisker S, Patterson ES, Sarkar U. Implementation science for ambulatory care safety: a novel method to develop context-sensitive interventions to reduce quality gaps in monitoring high-risk patients. Implement Sci. 2017 24;12(1):79.
Carayon P, Hundt AS, Karsh B, Gurses AP, Alvarado CJ, Smith M, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care. 2006 Dec;15(Suppl 1):i50–8.
Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013 Dec 1;8:139.
Hysong SJ, Sawhney MK, Wilson L, Sittig DF, Espadas D, Davis T, et al. Provider management strategies of abnormal test result alerts: a cognitive task analysis. J Am Med Inform Assoc. 2010;17(1):71–7.
This research is supported through a grant from the Agency for Healthcare Research and Quality (P30HS023558) and the National Cancer Institute (K24CA212294). The funders had no role in the design or presentation of results.
Ethics approval and consent to participate
We received ethics approval from the Institutional Review Board of the University of California, San Francisco (Study Number 12-09658).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Hickey, M.D., Lisker, S., Brodie, S. et al. Customized registry tool for tracking adherence to clinical guidelines for head and neck cancers: protocol for a pilot study. Pilot Feasibility Stud 6, 16 (2020). https://doi.org/10.1186/s40814-020-0552-0
- Patient monitoring
- Patient safety
- Ambulatory care
- Organizational interventions
- Systems engineering