Designing for implementation: user-centered development and pilot testing of a behavioral economic-inspired electronic health record clinical decision support module
Pilot and Feasibility Studies volume 5, Article number: 28 (2019)
Current guidelines recommend less aggressive target hemoglobin A1c (HbA1c) levels based on older age and lower life expectancy for older adults with diabetes. The effectiveness of electronic health record (EHR) clinical decision support (CDS) in promoting guideline adherence is undermined by alert fatigue and poor workflow integration. Integrating behavioral economics (BE) and CDS tools is a novel approach to improving adherence to guidelines while minimizing clinician burden.
We will apply a systematic, user-centered design approach to incorporate BE “nudges” into a CDS module and will perform user testing in two “vanguard” sites. To accomplish this, we will conduct (1) semi-structured interviews with key informants (n = 8), (2) a 2-h, design-thinking workshop to derive and refine initial module ideas, and (3) semi-structured group interviews at each site with clinic leaders and clinicians to elicit feedback on three proposed nudge module components (navigator section, inbasket refill protocol, medication preference list). Detailed field notes will be summarized by module idea and usability theme for rapid iteration. Frequency of firing and user action taken will be assessed in the first month of implementation via EHR reporting to confirm that module components and related reporting are working as expected as well as assess utilization. To assess the utilization and feasibility of the new tools and generate estimates of clinician compliance with the Choosing Wisely guideline for diabetes management in older adults, a 6-month, single-arm pilot study of the BE-EHR module will be conducted in six outpatient primary care clinics.
We hypothesize that a low burden, user-centered approach to design will yield a BE-driven, CDS module with relatively high utilization by clinicians. The resulting module will establish a platform for exploring the ability of BE concepts embedded within the EHR to affect guideline adherence for other use cases.
Intensive glycemic control is of unclear benefit and carries increased risk for older adults (defined here as 76 years and older) with diabetes . A number of randomized controlled trials, including the Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial , the Action in Diabetes and Vascular Disease: Preterax and Diamicron Modified Release Controlled Evaluation (ADVANCE) trial , and the Veterans Affairs Diabetes Trial (VADT) , found that intensive glycemic control was not protective for macrovascular complications of diabetes including myocardial infarction or stroke. These trials demonstrate the potential for harm with tight glycemic control, notably increased risk of hypoglycemia , and a suggestion of increased all-cause mortality . Older adults are particularly susceptible to harms related to hypoglycemia in diabetes, including emergent hospitalization and neurologic complications [5,6,7,8,9]. Intensive glycemic treatment may also lead to increased risk of polypharmacy and adverse medicine interactions for older adults with multiple chronic conditions .
Choosing Wisely guideline for older adults with diabetes
Due to evidence that tight glycemic control in older adults with diabetes may be medically harmful, in 2013 (revised in 2015), the American Geriatrics Society (AGS) developed ten Choosing Wisely (CW) guidelines, of which the third guideline states the following: “Avoid using medications other than metformin to achieve hemoglobin HbA1c<7.5% in most older adults; moderate control is generally better.”  Specifically, this guideline incorporates the balance between the number of comorbidities and life expectancy of older adults to provide target ranges for glycemic goals. Reasonable targets include an HbA1c of 7.0–7.5% in healthy, older adults with long life expectancy; 7.5–8.0% for patients with moderate comorbidity and a life expectancy of less than 10 years; and 8.0–9.0% for patients with multiple comorbid conditions and a shorter life expectancy [4, 12].
These recommendations build on previous work categorizing older adults with diabetes into three clinical groups based on health status  and have been endorsed by numerous expert panels and guidelines [8, 14, 15]. The target ranges for glycemic control are similar to HbA1c values that have been associated with the best outcomes for older adults with comorbid conditions in observational or modeling studies [16,17,18]. The American Diabetes Association (ADA) and others have recommended similar health status categories related to HbA1c targets in older adults [8, 14, 15]. Although the ADA guidelines do not identify less aggressive targets for HbA1c , other societies, including AGS, recommend similar less aggressive thresholds for glycemic control [15, 19,20,21].
Despite the CW recommendations, a substantial number of older adults have intensive glycemic control that may not be necessary [22,23,24]. Additionally, older patients with intensive glycemic control generally do not undergo de-intensification of therapy, suggesting opportunity for improving appropriate care . To combat this problem, the proposed study utilizes a unique approach: using behavioral economics via electronic health records to influence provider behavior with respect to diabetes care for older adults.
Behavioral economics and the EHR
The field of behavioral economics (BE) combines principles from economics and psychology to recognize the limitations of the classical economic framework that views human decision-makers as purely rational actors . In reality, humans are predictably irrational , making common decision errors that are explicable through a set of psychological principles, and are therefore predictable. Traits contributing to decision errors include loss aversion, anchoring, overweighting of small probabilities, present bias, regret aversion, sensitivity to defaults, and the power of social comparisons . Once recognized, each of these decision errors can be harnessed and overcome, often in the form of gentle “nudges” that make a desired behavior more likely [29,30,31,32].
Meanwhile, electronic health records (EHRs) now dominate the landscape, influencing nearly every clinical decision, workflow, and order placed by health care providers. Clinical decision support (CDS) is the primary EHR tool for influencing clinical decision-making and promoting adherence to clinical guidelines. CDS is an effective tool for improving provider performance and patient outcomes [33, 34]. Moreover, best practices for maximizing CDS effectiveness have been identified [35, 36]. Successful CDS must deliver accurate information in the right clinical context at the point of care and must be integrated into the relevant provider’s workflow . Large, systematic reviews of CDS have demonstrated a moderate ability to reduce morbidity, utilization, and costs [34, 38]. These modest improvements, however, are undermined by the well-documented problems of alert fatigue and poor workflow integration, which together blunt the potential of the EHR and CDS to improve healthcare outcomes .
Integrating behavioral economics strategies and electronic health records using various CDS tools is a novel approach to improving guideline adherence that also seeks to minimize negative impacts on clinical workflow and cognitive load. For example, Meeker et al. integrated three BE concepts (suggested alternatives, accountable justification, and peer comparisons) into the EHR at 50 primary care practices to significantly (~ 5–7%) reduce inappropriate antibiotic prescribing for upper respiratory infections . New approaches like these are needed to complement the traditional alerts, reminders, and other CDS tools that disrupt clinical workflow, increase cognitive load, and stress the limited capacity of clinicians to rationally process and evaluate the diverse and competing demands on their attention. By leveraging the opportunity presented by the EHR to combine modalities and extend the power of BE, this project seeks to develop a scalable intervention to reduce overtreatment in older adults with diabetes with minimal negative impact on clinician workflow or cognitive workload.
The objective of this study is to develop and pilot test a scalable, EHR customization toolkit that applies behavioral economic insights to promote appropriate diabetes care in older adults based on the AGS’s CW guideline to reduce overtreatment for the benefit of older adults with diabetes.
This study employs a pragmatic, user-centered approach as illustrated in Fig. 1 to achieve two aims: (1) develop new BE CDS modules to improve provider adherence to the CW guideline targeting overtreatment among older adults with diabetes and (2) test the effectiveness of these BE CDS modules in a naturalized, clinical setting.
Aim 1: User-centered development of a behavioral economic-inspired CDS tool
Implementing a user-centered design process, we will incorporate behavioral economic approaches into existing CDS tools and clinical workflows to design, develop, iteratively refine, and user-test the module in two “vanguard” sites.
User-centered design of initial module (aim 1, phase 1)
The development and refinement process for the new BE CDS modules involves a series of linked steps that have been successfully used in previous CDS development studies [41,42,43,44,45]. The process includes user research, prototyping, and usability testing (see Fig. 1) [46, 47].
These processes will be used to identify or “discover” candidate BE concepts and potential CDS tools, which will then be analyzed according to their potential for workflow integration, their likely impact, and the complexity of their EHR development.
Identification of behavioral economic approaches and module opportunities and utilization measures
Initially, the BE and CDS literature will be further reviewed and candidate approaches selected for evaluation by the research team. Based on prior work and the current literature, Table 1 outlines example BE concepts and relevant CDS tools to support the implementation of each BE approach, along with utilization measures available to assess process outcomes.
Key informant interviews
The team will conduct key informant interviews with BE, CDS, and clinical experts using a semi-structured interview guide to explore (1) the potential impact of BE principles and strategies on improving guideline adherence for diabetes management in older adults, (2) the functionality of established CDS tools amenable to implementing the BE approach, (3) the clinical workflow footprint of the BE CDS nudges, and (4) how well candidate BE CDS nudges combine into a modular package of BE CDS tools.
The team will also conduct a workflow analysis adapted from the Agency for Healthcare Research and Quality (AHRQ) recommendations on workflow assessments . We will conduct short observations in two selected vanguard clinic locations. The literature, interviews, and workflow assessments will be used in generation of the initial prototype BE-EHR module.
Workflows will be validated in a design-thinking workshop (described below), in follow-up interviews with key informants, and within-group feedback meetings with clinicians during user testing. During these follow-up meetings, any additional workflow variations that the module may need to support will be identified. Candidate workflows will then be discussed with the project team and other health system stakeholders to finalize the optimal workflow integration approach.
After initial key informant interviews and workflow analysis, a design-thinking workshop will be held to bring together findings from the interviews and workflow analysis. The design workshop will feature a structured, multidisciplinary workgroup composed of BE experts, CDS developers, relevant clinicians, and the research team. The sessions will be guided by a user-centered design facilitation protocol that sequentially leads the group through presentation and idea generation exercises around relevant behavioral economic concepts, CDS opportunities, and workflow obstacles and opportunities. Participants will be divided into small groups and provided materials and tasks designed to prompt new thinking on potential nudges. For example, the “crazy eights” exercise is an exercise in which individual team members are directed to draw eight ideas in 8 min. Each participant will be provided opportunities to “share back” their small group’s ideas as well as provide feedback on other groups’ ideas.
The primary goal of the workshop is to prompt divergent thinking or “discovery” (see Fig. 1) where participants focus on generating as many ideas as possible, as exemplified by the crazy eights exercise. Toward the end, the facilitator directs the group to “define” or converge on the best ideas for more explicit definition and development; to this end, the workshop will complete with the whole group voting on nudge ideas believed to be most promising. The session will be video- and audio-recorded, as well as summarized and converted into recommendations and revised workflow diagrams.
User testing (aim 1, phase 2)
Prior studies also established a clear link between the user-centered design process and successful implementation of clinical decision support tools within the EHR [49,50,51]. Upon completion of the design-thinking workshop and subsequent development of nudge prototypes, nudges will be further refined through a tailored, multi-phase feedback gathering and user testing process . This pre-clinical testing serves as a clinical laboratory for building successful workflow-integrated tools with a high likelihood of adoption and adherence [41, 44, 52]. In addition to key informant interviews and group interviews with users as described above, a variety of methods for collecting both qualitative as well as quantitative user feedback will be selected. See Table 2 for user research methodologies to be employed on an as needed basis according to fit based on type of module, content, frequency of clinician “exposure,” and response to the module.
Development of algorithms to determine activation of CDS
In order to build a user-centered CDS tool that triggers appropriately for the target patient population, algorithms will be built-in to drive timing and content of module firings that incorporate both patient life expectancy (high, medium, low) and target glycemic index. A life expectancy algorithm (based on a scoring approach used by Quan et al. and prior research by DuGoff et al.) will be developed by the research team [53, 54]; it will be used to assign patients to low, medium, and high life expectancy categories based on patient characteristics including age, gender, and both the number and type of comorbidities. Depending on a patient’s life expectancy categorization, the individual will be assigned a target HbA1c range according to the specifications stated by the CW guideline targeting overtreatment of older patients with diabetes.
Developing an algorithm that accurately predicts life expectancy using electronic health record data poses challenges. One limitation is that the number and type of comorbidities recorded in the electronic medical record may be associated with the length of patient registration and frequency of visits. This would lead to under-reporting of comorbidities for some patients, which could lead to a higher predicted life expectancy categorization and consequential bias toward over-reporting compliance with the Choosing Wisely guidelines. While we are not able to validate the life expectancy algorithm by obtaining death certificate data, the peer-reviewed literature that we will utilize for building this algorithm is well-known and was created using data sources that verified accuracy using death certification data or sources directly applicable to analyzing EHR data. For example, Quan et al. developed a weighted scoring approach for the comorbidities using hospital and death certification data, and DuGoff et al. formulated their life expectancy tables assuming chronic conditions using Medicare beneficiary data from a sample of over 1.3 million patients [53, 54]. Therefore, we are confident that our adaptation of these algorithms will perform well in this setting.
Clinical decisions in accordance with the CW guideline will be drafted and approved by clinical experts on the research team and will guide actions prompted by text and visuals in the CDS modules. Example clinical actions include switching a patient’s medication to metformin, reducing the dose, or removing the prescription of an alternative medication for patients with an HbA1c laboratory value too tightly controlled given their calculated life expectancy. Once the new module has stabilized within the vanguard sites (workflow issues are resolved and clinician usage is stable) (aim 1), the tool will be activated at the pilot sites (aim 2).
Aim 2: Pilot testing the BE-EHR modules (phase 3)
To assess the utilization and feasibility of the new tools and generate preliminary estimates of clinician compliance with the CW guideline, a 6-month, single-arm pilot study of the BE-EHR module will be conducted in six outpatient primary care clinics.
The BE CDS module will trigger for appropriate patients according to the logic built into the module based on the patient’s age, life expectancy (as calculated by the algorithm described above), and current/target HbA1c. Based on these characteristics, the CDS may, for example, leverage the behavioral economic principle of defaults and suggest metformin if appropriate. Otherwise, the CDS will suggest not adding a new prescription if the HbA1c is above the lower bound of the target HbA1c threshold among the three life expectancy categories (7.0% for healthy patients, 7.5% for those with moderate comorbidity, and 8.0% for those with a shorter life expectancy [4, 12]). The CDS will suggest stopping or reducing the medication dose if the HbA1c is below the lower bound of the target HbA1c target threshold per life expectancy category, as that would indicate the patient’s HbA1c is being too tightly controlled. The BE CDS module (see Table 1) will be activated at the system level for all relevant diabetes EHR components. For example, a diabetes order set tailored to older adults might have default medications updated to be metformin and to reflect the CW-recommended glycemic targets for calculated life expectancy.
Setting and population
The New York University Langone Medical Center (NYULMC) primary care practices serve patients with a diverse range of socio-demographic characteristics; they range from faculty-based practices with predominately privately insured and/or Medicare patients to Federally Qualified Healthcare Centers with a large Medicaid population (see Table 3). For the pilot study, four practices will be recruited, purposely chosen to reflect key practice setting characteristics (number of full-time providers, depth of support staff, transition to medical home model, insurance mix, patient socio-demographics).
Eligible patients within the practice sites will be those aged 76 or older with a diagnosis of diabetes. Patients will be categorized into one of the three glycemic target ranges inspired by the CW recommendation: (1) healthy older adults with an HbA1c target range of 7–7.5% and long life expectancy (defined here as 10+ years); (2) those with moderate comorbidity and a life expectancy of 3+ to 10 years, with a target range of 7.5–8%; and (3) those with multiple comorbidities and life expectancy of less than or equal to 3 years, with a target range of 8–9% . These categories are determined using the life expectancy algorithm described earlier. We expect the majority of patients in the selected population to fall into the “high life expectancy” category (10+ years), as the average life expectancy of a 75-year-old is between 11 and 12 years , and about two thirds of patients in the Medicare population have multiple chronic conditions [54,55,56,57]. Twenty percent of the population is estimated to be moderately healthy older adults with a medium life expectancy, and 10% are estimated to have an end-stage illness, leading to low life expectancy.
Pilot study outcomes will measure implementation as well as provide initial estimates of the BE-EHR modules’ effectiveness.
Process outcomes include utilization, measured via the EHR, such as the percentage of alerts in which the clinician changes the default option. Other outcomes are noted in Table 1. We will also collect qualitative user feedback using user research methods deemed most appropriate (see Table 2) to understand clinicians’ experience with the BE-EHR module and evaluate the specific nudges, as well as explore acceptability, implementation, and adoption issues surrounding them, including assessing their impact on guideline-focused care for older adults with diabetes.
The primary clinical outcome is the percent of eligible patients who are compliant with the AGS CW guideline targeting overtreatment of older adults with diabetes. For each patient, the HbA1c level, diabetes medication prescription status, and life expectancy status will be determined. All patients whose HbA1c are within the target range for their life expectancy status will be considered compliant. Those whose HbA1c are below the target range and who are being treated with a non-metformin agent (i.e., are over-controlled) will be deemed non-compliant. If an eligible patient is on metformin only but the HbA1c is below the upper board of the target range, they will also be deemed non-compliant. These situations are delineated in Table 4. Patient-level compliance outcomes will be aggregated to produce a provider-level compliance rate as a proportion of each provider’s eligible patients.
Statistical methods and analysis
Utilization rates will be estimated for each provider for every BE module component to which they apply (see Table 1); these rates will be summarized across physicians with 95% confidence intervals. To assess clinical value, each patient will be classified as CW compliant or not using the definition above; this indicator will be averaged across patients within a provider to calculate a provider-level compliance rate. These will also be summarized with 95% confidence intervals. The pilot study will provide critical estimates of utilization rates and clinical outcomes, allowing for refinement of the clinical trial design (Additional file 1). Specifically, following completion of the pilot study, the pruning decisions described below can be made to eliminate less effective approaches and guide future module development decisions.
Pilot phase qualitative user feedback
As outlined in Table 2, user feedback collected in the pilot phase will be analyzed with both pragmatic and academic goals. In order to maintain the timeline necessary for iterative product development, all user feedback will be summarized from researcher notes, providing near real-time feedback for module development and iteration. Simultaneously, systematically collected user feedback will be analyzed for more detailed usability findings deemed valuable by the research team for module implementation and academic dissemination using Dedoose, a software package that can integrate transcripts, pictures, memos, and other materials. Procedures recommended by Patton and others that focus on developing coding protocols to highlight issues, problems, and potential recommendations will be used . The goal of these analyses is to identify key barriers and facilitators, as well as any emerging themes related to the use of the BE-EHR module to influence provider adherence to the CW guidelines.
We will apply mixed methods “pruning criteria” to eliminate less effective approaches. To address the evaluation of individual components, utilization measures will be assessed to determine the value of the tool for incorporation into the eventual BE-EHR module. If the CDS tool is utilized in more than 30% of opportunities (e.g., if a suggested default is accepted in more than 30% of cases), then this tool will be retained in the module. While there is no gold standard threshold for what a “useful” CDS tool utilization rate should be, studies have noted common CDS tools with utilization rates between 5 and 40% depending on the type of CDS and the risk of ignoring it [59,60,61,61]. Furthermore, while low utilization rates may prove clinically worthwhile in high-risk patient safety situations, lower risk scenarios like the CW diabetes guidelines for older adults dictate the need for a stronger mean adoption rate . Certain tools will not be evaluable with this criterion, for example, the effect of adjusting the language on HbA1c lab reports. For these tools, qualitative user feedback during usability testing and brief interviews with key informants during piloting will be used to complement the utilization pruning criterion.
Scalability and dissemination
The finalized BE-EHR module will guide other EHR users through a menu of customizable options that can switch on (or off) various BE-derived CDS tools to replicate the current intervention for improving clinician adherence to diabetes management guidelines in older adults. Additionally, to facilitate scalability and widespread dissemination, the tools will follow standards-based development approaches, enabling widespread adoption across healthcare systems and diverse EHR platforms. For information regarding other key components addressed in this study, please consult the Additional file 1: SPIRIT Checklist.
This study involves the development and pilot testing of an innovative CDS tool, implementing behavioral economic principles within the electronic health record to promote clinician adherence to the CW guideline for diabetes management in older adults. Interventions for diabetes care in older adults have typically focused on addressing under-treatment of hyperglycemia ; however, the intervention described presently focuses on a system-wide approach to reduce risks resulting from overtreatment of older adults with diabetes. Evidence and lessons learned from this study can potentially inform the design, testing, and implementation of similar interventions for other CW target conditions and beyond.
Novelty of combining behavioral economics with clinical decision support tools
The incorporation of behavioral economic principles into EHR clinical decision support tools shows promise as a strategy to improve guideline adherence by addressing stubborn barriers, such as alert fatigue, that prevent the CDS from having a desired impact on clinician behavior. The proposed BE-EHR module will serve as a highly scalable platform for embedding a BE-based CDS into any EHR system. More importantly, this module can be easily applied to many other conditions in older adults and other populations where combining BE with EHR-based clinical decision support will be useful for improving guideline adherence such as AGS CW recommendations related to preventative screening procedures (e.g., colonoscopies), or increasing compliance to tobacco cessation.
Unlike most new CDS systems, we anticipate that the proposed BE-EHR module will have limited negative impact on clinical workflow and cognitive load. BE tools inherently bypass the central processing route that requires clinicians to actively think about decision-making . Instead, these tools leverage the peripheral route, which uses contextual cues and other influencing tools to nudge clinicians to choose actions consistent with stated guidelines.
While the present study aims to test the effectiveness of behavioral economic principles in the EHR as they relate to the overtreatment of elderly patients with diabetes, central to the CDS is the logic and clinical decision-making guided by the life expectancy algorithm and corresponding HbA1c targets. Such a clinical decision support tool currently does not exist within our vendor EHR (Epic™), suggesting a possible future area for development within the EHR. This life expectancy algorithm could easily be applied to other alternatives to clinical practice or additional CW guidelines as a tool to nudge clinician behavior across a variety of medical concerns affecting elderly adults.
Important role of behavioral economics in achieving user-centered tools
Prior research, including that of the research team, show the value of taking a user-centered approach to development of decision support tools [41, 64, 65]. In doing so, the appropriate behavioral economic principles can be identified and strategies developed and deployed in a way that is most likely to leverage the motivations and workflows of end users (clinicians in the case of most CDS). Our multi-phased, mixed methods approach to design and user testing as well as multidisciplinary approach as supported by our diverse team and key informants (with expertise in behavioral economics, informatics, geriatrics, diabetes, social science, user research, and rigorous evaluation of clinical interventions) enables the design of a user-centered intervention more likely to achieve the adoption necessary to impact clinical outcomes.
As with any study design, there exist some limitations. First, the sample size that would be required to adequately power the present study is beyond the scope of this work. Hence, results from this pilot study will influence the decision of whether to push forward with a fully developed randomized controlled trial using only those nudges that show promise. Furthermore, with the incorporation of nudges (see Table 1) simultaneously across the EHR, it will be challenging to determine exactly which BE strategy or intervention has the strongest impact. Care will be taken to perform statistical analyses using adjustments and gathering summary statistics for different groups of nudges that are less likely to co-occur. Furthermore, we will collect information longitudinally to assess the order and timing with which clinicians were exposed to various nudges. As a pragmatic study, however, we are testing effectiveness in a real-world setting, for which clinicians in practice will be exposed to multiple nudges in combination or simultaneously, making the present design useful for assessing the overall impact in the EHR.
In summary, the proposed research is a highly innovative, multi-phase study to determine the feasibility and impact of EHR-embedded BE approaches on provider adherence to the CW guideline for older adults with diabetes. The resulting BE-EHR module will establish a platform for exploring the ability of BE concepts embedded within the EHR to affect guideline adherence for other CW target areas. Moreover, it represents an exciting new channel for influencing provider behavior through less cognitively burdensome methods.
Action to Control Cardiovascular Risk in Diabetes
American Diabetes Society
Action in Diabetes and Vascular Disease: Preterax and Diamicron Modified Release Controlled Evaluation
American Geriatrics Society
Agency for Healthcare Research and Quality
Clinical decision support
Electronic health record
New York University Langone Medical Center
Veterans Affairs Diabetes Trial
Gerstein HC, Miller ME, Byington RP, Goff DC Jr, Bigger JT, Buse JB, et al. Effects of intensive glucose lowering in type 2 diabetes. N Engl J Med. 2008;358(24):2545–59.
Patel A, MacMahon S, Chalmers J, Neal B, Billot L, Woodward M, et al. Intensive blood glucose control and vascular outcomes in patients with type 2 diabetes. N Engl J Med. 2008;358(24):2560–72.
Duckworth W, Abraira C, Moritz T, Reda D, Emanuele N, Reaven PD, et al. Glucose control and vascular complications in veterans with type 2 diabetes. N Engl J Med. 2009;360(2):129–39.
Workgroup AGSCW. American Geriatrics Society identifies five things that healthcare providers and patients should question. J Am Geriatr Soc. 2013;61(4):622–31.
Shorr RI, Ray WA, Daugherty JR, Griffin MR. Incidence and risk factors for serious hypoglycemia in older persons using insulin or sulfonylureas. Arch Intern Med. 1997;157(15):1681–6.
Bremer JP, Jauch-Chara K, Hallschmid M, Schmid S, Schultes B. Hypoglycemia unawareness in older compared with middle-aged patients with type 2 diabetes. Diabetes Care. 2009;32(8):1513–7.
Huang ES, Laiteerapong N, Liu JY, John PM, Moffet HH, Karter AJ. Rates of complications and mortality in older patients with diabetes mellitus: the diabetes and aging study. JAMA Intern Med. 2014;174(2):251–8.
American Diabetes A. 11. Older adults. Diabetes Care. 2017;40(Suppl 1):S99–S104.
Geller AI, Shehab N, Lovegrove MC, Kegler SR, Weidenbach KN, Ryan GJ, et al. National estimates of insulin-related hypoglycemia and errors leading to emergency department visits and hospitalizations. JAMA Intern Med. 2014;174(5):678–86.
Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA. 2005;294(6):716–24.
AGS. Choosing WIsely: an initiative of the ABIM Foundation: American Geriatric Society; 2017. Available from: http://www.choosingwisely.org/clinician-lists/american-geriatrics-society-medication-to-control-type-2-diabetes/.
Workgroup AGSCW. American Geriatrics Society identifies another five things that healthcare providers and patients should question. J Am Geriatr Soc. 2014;62(5):950–60.
Blaum C, Cigolle CT, Boyd C, Wolff JL, Tian Z, Langa KM, et al. Clinical complexity in middle-aged and older adults with diabetes: the Health and Retirement Study. Med Care. 2010;48(4):327–34.
Kirkman MS, Briscoe VJ, Clark N, Florez H, Haas LB, Halter JB, et al. Diabetes in older adults: a consensus report. J Am Geriatr Soc. 2012;60(12):2342–56.
American Geriatrics Society Expert Panel on Care of Older Adults with Diabetes M, Moreno G, Mangione CM, Kimbro L, Vaisberg E. Guidelines abstracted from the American Geriatrics Society guidelines for improving the care of older adults with diabetes mellitus: 2013 update. J Am Geriatr Soc. 2013;61(11):2020–6.
Huang ES, Zhang Q, Gandra N, Chin MH, Meltzer DO. The effect of comorbid illness and functional status on the expected benefits of intensive glucose control in older patients with type 2 diabetes: a decision analysis. Ann Intern Med. 2008;149(1):11–9.
Blecker S, Park H, Katz SD. Association of HbA1c with hospitalization and mortality among patients with heart failure and diabetes. BMC Cardiovasc Disord. 2016;16:99.
Aguilar D, Bozkurt B, Ramasubbu K, Deswal A. Relationship of hemoglobin A1C and mortality in heart failure patients with diabetes. J Am Coll Cardiol. 2009;54(5):422–8.
Huang ES. Potential overtreatment of older, complex adults with diabetes. JAMA. 2015;314(12):1280–1.
International Diabetes Federation Guideline Development G. Global guideline for type 2 diabetes. Diabetes Res Clin Pract. 2014;104(1):1–52.
Allcott H, Rogers T. The short-run and long-run effects of behavioral interventions: experimental evidence from energy conservation. Am Econ Rev. 2014;104(10):3003–37.
Lipska KJ, Ross JS, Miao Y, Shah ND, Lee SJ, Steinman MA. Potential overtreatment of diabetes mellitus in older adults with tight glycemic control. JAMA Intern Med. 2015;175(3):356–62.
McCoy RG, Lipska KJ, Yao X, Ross JS, Montori VM, Shah ND. Intensive treatment and severe hypoglycemia among adults with type 2 diabetes. JAMA Intern Med. 2016;176(7):969–78.
McAlister FA, Youngson E, Eurich DT. Treated glycosylated hemoglobin levels in individuals with diabetes mellitus vary little by health status: a retrospective cohort study. Medicine. 2016;95(24):e3894.
Sussman JB, Kerr EA, Saini SD, Holleman RG, Klamerus ML, Min LC, et al. Rates of deintensification of blood pressure and glycemic medication treatment based on levels of control and life expectancy in older patients with diabetes mellitus. JAMA Intern Med. 2015;175(12):1942–9.
Mullainathan S, Thaler R. Behavioral economics. International encyclopedia of the social & behavioral sciences: Elsevier; 2001. p. 1094–1100. ISBN: 978-0-08-043076-8.
Ariely D. Predictably irrational. New York: HarperCollins; 2008.
Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica. 1979;47(2):263–91.
Thaler R, Sunstein C. Libertarian paternalism. Am Econ Rev. 2003;93:175–9.
Thaler R, Sunstein C. Nudge: improving decisions about health, wealth, and happiness. 2nd ed. New York: Penguin Books; 2009.
Camerer C, Issacharoff S, Loewenstein G, O’Donoghue T, Rabin M. Regulation for conservatives: behavioral economics and the case for “asymmetric paternalism”. Univ Pa Law Rev. 2003;101:1211–54.
Loewenstein G, Brennan T, Volpp KG. Asymmetric paternalism to improve health behaviors. JAMA. 2007;298(20):2415–7.
Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc : JAMIA. 2011;18(3):327–34.
Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29–43.
Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.
Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc : JAMIA. 2003;10(6):523–30.
Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012;45(6):1202–16.
Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, et al. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health. 2014;104(12):e12–22.
Khalifa M, Zabani I. Improving utilization of clinical decision support systems by reducing alert fatigue: strategies and recommendations. Stud Health Tech Inform. 2016;226:51–4.
Meeker D, Linder JA, Fox CR, Friedberg MW, Persell SD, Goldstein NJ, et al. Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA. 2016;315(6):562–70.
Mann DM, Kannry JL, Edonyabo D, Li AC, Arciniega J, Stulman J, et al. Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care. Implement Sci : IS. 2011;6(1):109.
Mann DM, Palmisano J, Lin JJ. A pilot randomized trial of technology-assisted goal setting to improve physical activity among primary care patients with prediabetes. Prev Med Rep. 2016;4:107–12.
Mishuris RG, Yoder J, Wilson D, Mann D. Integrating data from an online diabetes prevention program into an electronic health record and clinical workflow, a design phase usability study. BMC Med Inform Decis Mak. 2016;16(1):88.
Li AC, Kannry JL, Kushniruk A, Chrimes D, McGinn TG, Edonyabo D, et al. Integrating usability testing and think-aloud protocol analysis with “near-live” clinical simulations in evaluating clinical decision support. Int J Med Inform. 2012;81(11):761–72.
Mann DM, Quintiliani LM, Reddy S, Kitos NR, Weng M. Dietary approaches to stop hypertension: lessons learned from a case study on the development of an mHealth behavior change system. JMIR Mhealth Uhealth. 2014;2(4):e41.
Kushniruk A, Nohr C, Borycki E. Human factors for more usable and safer health information technology: where are we now and where do we go from here? Yearb Med Inform. 2016;1:120–5.
Kushniruk A, Nohr C. Participatory design, user involvement and health IT evaluation. Stud Health Technol Inform. 2016;222:139–51.
AHRQ. Workflow assessment for health IT toolkit. Agency for healthcare research and quality 2016. Available from: https://healthit.ahrq.gov/health-it-tools-and-resources/workflow-assessment-health-it-toolkit/workflow.
McGinn TG, McCullagh L, Kannry J, Knaus M, Sofianou A, Wisnivesky JP, et al. Efficacy of an evidence-based clinical decision support in primary care practices: a randomized clinical trial. JAMA Intern Med. 2013;173(17):1584–91.
Kushniruk A, Borycki E, Kuwata S, Kannry J. Predicting changes in workflow resulting from healthcare information systems: ensuring the safety of healthcare. Healthc Q. 2006;(9 Spec):114–118.
Goodman E, Kuniavsky M, Moed A. Observing the user experience: a practitioner’s guide to user research. San Francisco: Elselvier; 2003.
Mann DM. Making clinical decision support more supportive. Med Care. 2011;49(2):115–6.
DuGoff EH, Canudas-Romo V, Buttorff C, Leff B, Anderson GF. Multiple chronic conditions and life expectancy: a life table analysis. Med Care. 2014;52(8):688–94.
Quan H, Li B, Couris CM, Fushimi K, Graham P, Hider P, et al. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173:676–82.
Wolff JL, Starfield B, Anderson G. Prevalence, expenditures, and complications of multiple chronic conditions in the elderly. Arch Intern Med. 2002;162(20):2269–76.
Lochner KA, Cox CS. Prevalence of multiple chronic conditions among Medicare beneficiaries, United States, 2010. Prev Chronic Dis. 2013;10:E61.
Laiteerapong N, Iveniuk J, John PM, Laumann EO, Huang ES. Classification of older adults who have diabetes by comorbid conditions, United States, 2005-2006. Prev Chronic Dis. 2012;9:E100.
Patton M. Qualitative research and evaluation methods. Newberry Park: SAGE Publications, Inc.; 2002.
Nanji KC, Slight SP, Seger DL, Cho I, Fiskio JM, Redden LM, et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc : JAMIA. 2014;21(3):487–91.
Isaac T, Weissman JS, Davis RB, Massagli M, Cyrulik A, Sands DZ, et al. Overrides of medication alerts in ambulatory care. Arch Intern Med. 2009;169(3):305–11.
Shah NR, Seger AC, Seger DL, Fiskio JM, Kuperman GJ, Blumenfeld B, et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc : JAMIA. 2006;13(1):5–11.
Munshi MN, Segal AR, Suhl E, Ryan C, Sternthal A, Giusti J, et al. Assessment of barriers to improve diabetes management in older adults: a randomized controlled study. Diabetes Care. 2013;36(3):543–9.
Petty R, Cacioppo J. The elaboration likelihood model of persuasion. Adv Exp Soc Psych. 1986;19:124–92.
McCullagh LJ, Sofianou A, Kannry J, Mann DM, McGinn TG. User centered clinical decision support tools: adoption across clinician training level. Appl Clin Inform. 2015;5(4):1015–25.
Feldstein DA, Hess R, McGinn T, Mishuris RG, McCullagh L, Smith PD, et al. Design and implementation of electronic health record integrated clinical prediction rules (iCPR): a randomized trial in diverse primary care settings. Implement Sci : IS. 2017;12(1):37.
Research reported in this publication was supported by the National Institute of Aging under Award Number R21AG057382. HMB was supported in part by NIH 5TL1TR001447-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
This study was approved by the institutional review board of the New York University School of Medicine’s Institutional Review Board (Reference # i17-01308) in New York.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Chokshi, S.K., Belli, H.M., Troxel, A.B. et al. Designing for implementation: user-centered development and pilot testing of a behavioral economic-inspired electronic health record clinical decision support module. Pilot Feasibility Stud 5, 28 (2019). https://doi.org/10.1186/s40814-019-0403-z
- Electronic health records
- Behavioral economics
- Clinical decision support