Skip to main content

Testing the feasibility of the QuitAid smoking cessation intervention in a randomized factorial design in an independent, rural community pharmacy

Abstract

Background

Adult smoking rates in the USA are highest in economically depressed rural Appalachia. Pharmacist-delivered tobacco cessation support that incorporates medication therapy management (such as the QuitAid intervention) is a promising approach to address this need.

Methods

Twenty-four adult smokers recruited between September and November 2021 through an independent pharmacy in rural Appalachia were randomized in a non-blinded 2 × 2 × 2 factorial design to (1) pharmacist delivered QuitAid intervention (yes vs. no); (2) combination nicotine replacement therapy (NRT) gum + NRT patch (vs. NRT patch); and/or (3) 8 weeks of NRT (vs. standard 4 weeks). Participants received 4 weeks of NRT patch in addition to the components to which they were assigned. Participants completed baseline and 3-month follow-up assessments. Primary outcomes were feasibility of recruitment and randomization, retention, treatment adherence, and fidelity.

Results

Participants were recruited in 7 weeks primarily through a referral process, commonly referred to as ask-advise-connect (61%). Participants were on average 52.4 years old, 29.2% were male and the majority were white (91.6%) and Non-Hispanic (91.7%). There was a high level of adherence to the interventions, with 85% of QuitAid sessions completed, 83.3% of the patch used, and 54.5% of gum used. Participants reported a high level of satisfaction with the program, and there was a high level of retention (92%).

Conclusions

This demonstration pilot randomized controlled study indicates that an ask-advise-connect model for connecting rural smokers to smoking cessation support and providing QuitAid for smoking cessation is feasible and acceptable among rural Appalachian smokers and independent pharmacists. Further investigation into the efficacy of a pharmacist-delivered approach for smoking cessation is needed.

Trial registration

The trial was retrospectively registered at ClinicalTrials.gov. Trial #: NCT05649241.

Peer Review reports

Key messages regarding feasibility

  • What uncertainties existed regarding the feasibility?

    • ◦ It was unclear whether a pharmacist-delivered approach to smoking cessation is feasible in community pharmacies throughout rural Appalachia, where smokers have higher rates of tobacco use and are less likely to seek cessation support.

  • What are the key feasibility findings?

    • ◦ This feasibility study indicates that the ask-advise-connect approach used by community pharmacists was successful in recruiting hard-to-reach smokers, and the QuitAid intervention was well received by both pharmacists and participants.

  • What are the implications of the feasibility findings for the design of the main study?

    • ◦ This demonstration pilot randomized controlled study indicates that an ask-advise-connect model for connecting rural smokers to smoking cessation support and providing medication therapy management for smoking cessation is a promising study design to be tested in a full-scale trial among rural Appalachian smokers and independent pharmacists.

Adult smoking rates in the USA are highest in economically depressed rural Appalachia, specifically the Central and South Central subregions of Appalachia that include parts of Virginia, West Virginia, Tennessee, Kentucky, and North Carolina (henceforth referred to as “rural Appalachia”) [1, 2]. For example, 25–35% of residents report current smoking, far above the US national average of 14% [3, 4]. This region is also characterized by a high level of social vulnerability, which measures socioeconomic status, household composition and disability, minority status and language, and housing type and transportation. In a study comparing social vulnerability levels in Virginia, social vulnerability was on average 0.49 in Virginia compared to 0.63 in rural areas and 0.58 in rural Appalachian counties (range =0–1, with higher values indicating more vulnerability), and social vulnerability was directly correlated with smoking rates [5]. Smokers living in rural Appalachia are more likely to start smoking at a younger age, smoke daily and in excess of 15 cigarettes per day, and have poorer cessation-related outcomes [6, 7]. Additionally, smokers in rural Appalachia face financial barriers for pharmacotherapy, exacerbated by the fact that they often have limited awareness of available smoking cessation programs and resources [8]. Thus, there is a need to increase the reach of available smoking cessation programs to assist rural smokers to make a successful quit attempt [9].

One key smoking cessation strategy is nicotine replacement therapy (NRT)—an effective, safe, and easily accessible resource [10]. In our survey of 49 independently owned community pharmacies in rural Appalachia, 100% reported selling NRT [11]. Despite the fact that NRT increases the rate of quitting by 50–60% [10], it is often not adhered to in terms of dose and duration [10, 12, 13]. In a systematic review of barriers and facilitators to NRT adherence, the most important factors associated with adherence were related to reflective motivation, such as perceptions about NRT and quitting, physical capability, namely level of nicotine dependence and withdrawal symptoms, and automatic motivation, such as stress, depression and alcohol use, consistent with the capability, opportunity, motivation, and behavior (COM-B) model [14]. Similarly, population-based studies have found that over a third of smokers believe pharmacotherapy would be of little help and may even reduce their chances of successfully quitting [15]. However, the evidence suggests that combination NRT (fast-acting form [gum] + patch) further increases cessation (RR 1.3; 95% CI 1.2–1.4; 14 trials) although the optimal duration is unclear [16]. The lack of evidence on the optimal duration is problematic in terms of informing public policy related to NRT; for example, there is not a standard amount of free NRT provided by state quitlines; 39% provide 2 weeks, 16% provide 4 weeks, 29% provide 8 weeks, and 16% provide another amount [13].

Given that higher levels of NRT adherence are associated with cessation, it is critical that interventions for rural smokers promote adherence to NRT [17]. A meta-analysis found that medication adherence interventions led to a small improvement in adherence and long-term smoking cessation rates (RR 1.16, 95% CI 0.96 to 1.40) [18], but the available evidence was of low quality. Therefore, interventions are needed to improve suboptimal NRT adherence rates [18]. One possible avenue for disseminating NRT adherence interventions to rural communities is through independent pharmacies, given their clinical expertise, repeated patient interactions, and centralized placement in the community.

In the USA, 91% of Americans live within 5 miles of an independent pharmacy [19,20,21]. However, to date, independent pharmacies have been under-utilized in the provision of tobacco cessation [22], with only 14% of independent pharmacists in the USA reporting providing smoking cessation services [23]. Given the broad availability of smoking cessation medications in pharmacies and the fact that 17 states currently allow pharmacists to prescribe smoking cessation medications [24], an opportunity exists to train pharmacists to become a recognized local resource for smoking cessation [20]. Additionally, we surveyed independent pharmacists in rural Appalachia and found that 90.3% were interested in providing smoking cessation resources and treatment [11]. A recent meta-analysis of pharmacy-assisted smoking cessation interventions found that interventions were effective, but the evidence was low quality [25]. Thus, the optimal pharmacist-led intervention for smoking cessation remains unclear [25].

Furthermore, barriers to pharmacist delivered smoking cessation support need to be addressed [26]. For instance, only 11.4% of independent pharmacists who responded to our survey felt that they had a “great deal” of experience with providing smoking cessation resources [11]. Additionally, pharmacists are not currently compensated for providing smoking cessation services. As such, pharmacists tend to curb these activities in lieu of maximizing their medication dispensation for which there is clear reimbursement.

Medication therapy management (MTM) may be a strategy to overcome financial barriers for pharmacist-delivered smoking cessation support. MTM services, which have been used to manage chronic diseases such as diabetes, generally consist of medication review, an individual medication record and medication-related action plan, intervention and/or referral, documentation, and follow-up [27]. The Medicare Prescription Drug, Improvement, and Modernization Act paved the way in 2013 for pharmacists to receive compensation for providing medication expertise [28, 29]. Recently, MTM services have been widely adopted due to standard documentation and billing systems [28]. Research has found that independent pharmacies that deliver MTM lead to positive clinical outcomes for patients [28, 30,31,32] and are well received [33]. While MTM has not been used for the provision of smoking cessation, these findings suggest that a MTM-based approach to smoking cessation that utilizes a standard documentation and billing platform could provide a highly disseminable avenue for pharmacist-delivered smoking cessation support for hard-to-reach rural smokers.

To determine the feasibility of a pharmacist-delivered MTM-based approach to smoking cessation in rural Appalachia, we conducted a factorial randomized controlled trial. A factorial design was chosen to determine the feasibility of conducting a factorial experiment in independent pharmacies with rural Appalachian smokers. We also sought to determine feasibility of recruitment and randomization, retention, treatment adherence, and treatment fidelity of rural smokers through a single independently owned pharmacy in the region.

Methods

The current study used a 2 × 2 × 2 factorial design to implement three tobacco treatments with 24 rural smokers in Appalachia: (1) pharmacist delivered novel MTM intervention, QuitAid (yes vs. no); (2) combination NRT gum + NRT patch (vs. NRT patch alone); and/or (3) 8 weeks of NRT (vs. standard 4 weeks). All participants received at least 4 weeks of NRT patch in addition to any of the other components they were assigned to receive. These components were chosen for their ability to be easily disseminated in a rural setting.

Participants and procedures

Between September and November 2021, we recruited 24 adult smokers through their local independent pharmacy in rural Appalachia. Participants were primarily recruited through an ask-advise-connect model in which pharmacists asked potential participants if they were smoking, advised them to quit, and connected them to our study if they were interested [34]. Other recruitment methods included flyers on prescription bags and store signage. Adult smokers interested in participating in the study were then screened telephonically by research staff to determine eligibility. To be eligible, smokers must have smoked at least 5 cigarettes per day for the past 6 months and be willing to set a quit date in the next 30 days, own a cell phone, be over 18 years of age, not be pregnant or planning to become pregnant in the next 6 months, and not have any medical contraindications to using NRT. If eligible, participants were consented and provided an opportunity to ask questions. Participants signed their consent form through a mailed paper consent form that they signed and returned to the study staff in a pre-stamped envelope, electronically through DocuSign, or were given the option to sign a paper copy at their local pharmacy. Individuals who were not eligible or did not wish to participate were provided with alternative smoking cessation resources (e.g., referral to the state quitline). To account for the low literacy levels of the target population, participants were given the option to have the consent form and all assessments read to them either over the phone by a member of the study team, or for the assessments, they could also go into the pharmacy and have someone blinded to their treatment condition read them the survey. Consented participants then completed the baseline assessment and were randomized to one of the 8 treatment combinations (see Table 1). Equal allocation across treatment combinations was used and balance between treatment combinations was maintained by use of randomly permutated blocks with unequal block sizes of 8 and 16 generated by the study biostatistician; study staff were blinded to assignment until it was revealed by the study database. This study was approved by the Health Sciences Research Institutional Review Board at the University of Virginia.

Table 1 Randomized conditions

Treatment components

Nicotine replacement therapy

NRT was distributed by the pharmacy through a standing physician order which allowed the pharmacists to bill the participant’s insurance for the cost of NRT. The participant’s insurance was billed first, and the study paid any remaining balance from co-insurance or co-pays. For participants who did not have insurance, or their insurance did not cover the NRT, the study covered the entire cost of the medication. Participants had the option to pick up their NRT at the pharmacy or have the pharmacy mail the amount of NRT they were randomized to receive (4- or 8-week supply, plus patches only or patches plus gum). NRT was distributed in 4-week increments to align with insurance coverage (many insurance plans only cover for 4-weeks of NRT at one time). The first 4-week supply of NRT was filled within 7 days of enrollment. For participants randomized to receive an 8-week supply of NRT, the second 4-week supply was distributed during week 3 to ensure that the participant did not run out. NRT shipments included standard NRT use guidelines with details on proper use and potential side effects.

QuitAid intervention

The QuitAid MTM intervention addressed perceptions (e.g., motivation, self-efficacy, beliefs) and practicalities of using NRT (e.g., monitoring NRT use, providing reminders) that can be mapped onto the capability, opportunity, motivation, and behavior (COM-B) model [35]. The intervention targets include (1) goals and planning (e.g., strategies to remember to use NRT), (2) shaping knowledge (e.g., instructions on how to properly use NRT), (3) natural consequences (e.g., importance of continuing to use NRT throughout the entirety of the quit attempt), (4) associations (e.g., strategies to remember to use NRT), (5) comparison of outcomes (e.g., decisional balance about using NRT), (6) reward and threat (e.g., identifying self-rewards for achieving goals related to using NRT), (7) regulation (e.g., encouraging adherence to NRT), (8) antecedents (e.g., avoiding exposure to cues to smoke), and (9) self-belief (e.g., focusing on past successful behavior change attempts). These targets are in line with a systematic review of barriers and facilitators to adherence to NRT [14]. These strategies have previously been used to effectively promote cessation in clinical trials [36,37,38,39]. The QuitAid treatment included 1 in-person coaching session with the participant’s local pharmacist, and 5 follow-up telephonic coaching sessions. The first session included a discussion to address any negative beliefs about the use of NRT while strengthening the participant’s motivation and commitment to using NRT [18]. Participants were also instructed on proper use of the NRT they were randomized to receive (patches only or gum plus patches), useful tips to avoid common problems with NRT (e.g., skin irritation, cravings, use patch plus gum in combination), and strategies that have previously been used to increase adherence to NRT [18, 40]. Participants were instructed to begin using the NRT medication immediately, even if they were still smoking. Within 2 days of the first session, participants received a call from their pharmacist or technician to address any questions or concerns they may have had about their NRT use. The remaining weekly proactive check-in calls occurred 7, 14, 21, and 28 days from their initial call. During these weekly calls, the pharmacists addressed any negative beliefs about NRT, monitored the use of NRT, provided feedback regarding NRT use, and provided additional support to overcome any barriers to adherence [18]. All sessions occurred within 4 weeks from the time of enrollment.

Pharmacist training

Prior to the start of the study, pharmacists and technicians participated in two virtual trainings led by research staff to introduce key concepts and skills required by the program, build self-efficacy and comfort with the approach, and generate enthusiasm and commitment to the program. The trainings presented an overview of the theoretical underpinnings and evidence-base for the intervention, as well as detailed instruction about the ask-advise-connect method for identifying and recruiting smokers into the study and delivering the QuitAid intervention. They also completed a 1-h BMJ learning course on motivational interviewing in brief consultations [41]. Lastly, given their access to participant data, all pharmacists and technicians were required to complete online human subjects training [42].

Program implementation

The pharmacy had a dedicated on-site implementation leader to oversee the daily operations of the study and provide on-going support to other pharmacists or technicians related to study activities. The implementation leader worked directly with the study team to ensure open communication and fidelity to the treatment protocol and received a monthly stipend during the phases of active recruitment and implementation. Additionally, the pharmacy was reimbursed for providing the QuitAid MTM sessions, similar to what they would be reimbursed by an insurance company for providing MTM interventions for other chronic diseases, such as chronic heart failure, hypertension, and diabetes. We reimbursed pharmacies $40 for the initial in person visit and $20 for each additional visit (total of $140 per patient if they completed all 6 sessions).

Measures

Participant measures

The baseline and 3-month follow-up assessments were self-administered via either a secured web-based platform or mailed paper copies. The baseline survey assessed demographics (e.g., gender, age, ethnicity, marital status, race, education), pharmacist provider trust (11 items; α = 0.89; 1 = strongly disagree to 5 = strongly agree) [43], and previous behavioral smoking cessation aids used in the past 12 months (i.e., quitline, one-on-one in person counseling, class or support group, web-based program, smartphone application, and text-messaging program). At baseline and 3-month follow-up, tobacco use, quit attempts, and nicotine dependence using the Fagerström Test for Nicotine Dependence (FTND) were assessed [44]. The FTND includes six items that measure cigarette consumption, compulsion to use, and dependence. Items are summed to create a total score of 0–10 with higher scores indicating more intense physical dependence on nicotine.

Primary outcome measures (measures of feasibility)

Process measures were assessed through screening logs and participant surveys. Pace of recruitment was measured by (a) the proportion of smokers recruited from each source (e.g., ask-advise-connect, posters, prescription bag advertisements) and (b) the number recruited per month. We defined recruitment success if we were able to recruit eight smokers per month over the course of 3 months from a single pharmacy, which we anticipate would be sufficient to conduct a larger study within the typical budget and time period. Feasibility of randomization was determined by the number of smokers that were approached and screened in order to randomize 24 smokers (e.g., ineligible smokers, smokers that do not consent) [45, 46]. We hypothesized a 25% screen failure rate and 5% dropout/withdrawal rate. Retention was assessed by the proportion of smokers who completed the 3-month follow-up [47]. We defined retention success as more than 80% of the sample was retained at the 3-month follow-up. Receipt of the QuitAid intervention was defined as the number of QuitAid sessions the participant received [47]. Receipt of NRT was defined as the percent of NRT used and was collected from participants at the 3-month follow-up [18, 47]. We defined dose receipt of the interventions success as more than 75% of the intervention components were delivered. At the 3-month follow-up, satisfaction with treatment components were assessed (e.g., “The program kept my interest and attention”; see Table 3) [47]. Items were assessed on a 6-point Likert scale with 1 = strongly disagree to 6 = strongly agree [48, 49]. We defined satisfaction with treatment components success as mean scores greater than or equal to 4.

Secondary outcomes (tobacco use)

Tobacco use outcomes to examine overall changes in tobacco use were collected. Biochemically verified self-reported point prevalence tobacco abstinence at the 3-month follow-up was collected [50]. If a participant reported no tobacco use in the previous 7 days at the 3-month follow-up, they were asked to visit their local pharmacy to provide a saliva sample within 48 hours to biochemically verify their abstinence. Previous research has determined that optimal serum cotinine concentrations for discriminating tobacco abstinence were 3.08 ng/mL with a high degree of sensitivity and specificity (> 96%) [51]. Thus, in the current study, a concentration of≥ 3 ng/mL cotinine was set as the cutpoint for those who were smoking.

Pharmacist measures

Pharmacists and technicians completed a baseline survey related to their demographic (i.e., age, gender, and education) and professional characteristics (i.e., years of experience as a pharmacist, and experience providing tobacco cessation resources) via a secured web-based platform. Following completion of the factorial experiment, pharmacists and technicians completed a follow-up survey to assess confidence in the approach (e.g., “How confident are you that you answered participants’ questions about NRT use adequately?” 0 = not at all to 10 = extremely confident), perceived efficacy in the program (e.g., “How successful do you think smoking cessation MTM will be in reducing tobacco use among your customers?” 1 = not at all successful to 10 = extremely successful), and fidelity to the program (e.g., “Overall, how much did you adhere to the lesson plans (deliver them as written)?” 0 = not at all to 10 = extremely close).

Sample size

We chose a sample size of 3 per group based on the recommendations outlined in Julious [52], which recommends 12 participants per treatment for use in pilot feasibility trials based on the rationale about feasibility, the precision regarding the mean and variance, and regulatory considerations [52, 53]. Although the current study was a factorial design with 8 conditions, every treatment was delivered to half the sample, or 12 participants per treatment. Thus, while only 3 participants were randomized to each of the 8 conditions, 12 participants received each of the treatment components, in accordance with the recommendations by Julious (see Fig. 1) [52].

Fig. 1
figure 1

CONSORT diagram. QuitAid MTM (medication therapy management)

Data analysis

Descriptive statistics were used to summarize demographics, feasibility of recruitment, feasibility of randomization, retention, fidelity of implementation, program satisfaction, and overall changes in tobacco use behaviors. Statistical analyses were conducted in SPSS version 9.0 (SPSS Inc, Chicago, IL).

Results

Feasibility of recruitment, randomization and retention

We successfully recruited all 24 smokers in 7 weeks (approximately 3 smokers per week) from a single pharmacy. The majority were recruited using ask-advise-connect (61%), while the rest were self-referred from in store posters (18%), prescription bag advertisements (6%), and word of mouth (15%). Of the 56 smokers referred to the study, 39 (69.6%) were interested in being screened for eligibility (see Fig. 1). Following screening, 4 (10.3%) were ineligible and 7 (17.9%) were no longer interested in participating. Among the 28 eligible individuals, 85.7% were enrolled. There was a high level of retention, with 22 (92%) of the participants completing the 3-month follow-up assessment.

As shown in Table 2, participants were on average 52.4 years old, 29.2% (n = 7) were male and the majority were white (n = 22, 91.6%) and Non-Hispanic (n = 22, 91.7%). A third of the sample was married (n = 8), and only 20.8% (n = 5) were currently employed. Less than half (n = 9, 39.1%) had some college education, 62.5% (n = 15) reported less than $30,000 in annual household income, and the majority had some form of health insurance coverage (n = 21, 87.5%). All participants reported smoking at least 100 cigarettes in their lifetime. The average age at initiation was 16, the average FTND score was 5.2 (standard deviation [SD] = 2.3), and the majority of participants reported smoking everyday (n = 22, 91.7%). Roughly 45% (n = 13) of participants reported trying to quit in the previous 12 months, but none reported using a behavioral-based smoking cessation aid (not shown in the table). Finally, participants reported a high level of trust in local independent pharmacists (mean = 3.9, SD = 0.6).

Table 2 Baseline characteristics of participants (N = 24)

Dose of the interventions

There was a high level of adherence to the interventions, with 85% (n = 61) of MTM sessions completed, and 83.3% (n = 15) reporting more than 75% of the patch used and 54.5% (n = 6) reporting more than 75% of gum used.

Program satisfaction

Overall, there was a high level of satisfaction with the program (see Table 3), with a mean score for the overall program of ≥ 4.2. The most strongly endorsed item was the belief that the program helped them quit or cut down their cigarette use (mean = 4.7, SD = 1.0). Participants also felt that the NRT patch was helpful and easy to use ≥ 4.6. Finally, participants reported satisfaction with the QuitAid component.

Table 3 Summary of intervention satisfaction by participants (N = 22)

Secondary outcomes

At the 3-month follow-up, 31.8% (n = 7) of participants reported abstinence and 88.9% (n = 16) reported attempting to quit during the study. Additionally, there was a decrease in the FTND between baseline and follow-up (mean = −1.56, SD = 1.7) (Table 4).

Table 4 Secondary tobacco use outcomes (N = 22)

Pharmacist measures

Participating pharmacists were on average 39.7 years old (SD = 1.53), with 15.7 years of experience working in a pharmacy (SD = 5.51). Two (66.7%) were female, all (n = 3) had received a PharmD. They reported “a little” previous experience with providing tobacco cessation resources to customers (mean = 2.67, SD = 0.58). Overall, pharmacists felt confident in their delivery of the program (mean = 7.5, SD = 0.71), had a high level of enthusiasm towards the program (mean = 9, SD = 1.41), and perceived the program to be effective (mean = 8, SD = 1.41; see Table 5). They also reported a high degree of comfort with the approach (mean = 7.5, SD = 0.71), adherence to the program guides (mean = 8, SD = 0), and ease of delivery (mean = 8, SD = 0).

Table 5 Pharmacist and technician perceptions of the program (N = 3)

Discussion

We demonstrated the feasibility of delivering an MTM smoking cessation intervention, QuitAid, and conducting a randomized controlled trial with a 2 × 2 × 2 factorial design in an independent pharmacy in rural Appalachia. We successfully recruited participants and retained 92% of them at the 3-month follow-up. We accomplished our randomization goal with a complex randomization scheme and provided the appropriate intervention components without cross-contamination. Overall, the intervention was well received by participants and pharmacists, with high satisfaction ratings across the treatment components and high level of adherence to the intervention components. In addition, participants found their independent pharmacists to be knowledgeable and approachable. The program helped participants cut down and quit, with most participants setting a quit date. Finally, 31.8% of participants reported abstinence at the follow-up, which is a typical quit rate in a combined behavioral and pharmacological interventions in both pharmacy [54] and non-pharmacy settings [10, 55,56,57]. In the pharmacy setting, 7-day point prevalence abstinence rates at a 3-month follow-up ranged from 20.7 to 27.5% [54], which is similar to the findings in the current study.

There was a high level of adherence to NRT in the current study, particularly the NRT patch. While there was a lower utilization rate of the NRT gum, given that this was an adjunct treatment to the NRT patch, and participants who received the gum were told to use it when they had a breakthrough craving, one would expect this rate to be lower [15]. Previous research has found individual differences in the preference for various forms of NRT products [58]. Comparing the NRT gum, patch, nasal spray, and inhaler, the NRT gum was the least popular option among smokers attempting to quit, while the NRT patch was the most popular, and these findings were consistent across both men and women, although the difference was more marked for women [58]. Interestingly, prior experience using the NRT product had no effect on preference for the patch, but did affect preference for the gum, with individuals who had prior experience using the gum less likely to prefer that form of NRT [58]. This could be because the name “gum” misleads users to think that they chew the NRT gum like traditional gum, when in fact it is mean to be activated by biting down on it until one feels a tingling sensation, and then “parked” between the inside of one’s cheek and gums [59]. This process is repeated until the tingling stops, usually after about 30 min of use. Thus, individuals may not be using it correctly, or may find the gum to be strange due to their expectations about the product. Given the importance of proper utilization of NRT, both in terms of dose and duration, understanding individual preferences for NRT and the effect on utilization should be considered when designing smoking cessation trials.

Participants were very positive about their independent pharmacists and reported a high level of trust. We hypothesize that it was this trust that allowed us to reach our recruitment goal in 3 months primarily utilizing an ask-advise-connect method and could also explain the high level of program satisfaction. An ask-advise-connect model has been adopted nationally to promote utilization of publicly available smoking cessation programs. In 2019, there were over 200,000 referrals from physicians and pharmacists connecting smokers to state quitlines [13]. In our previous research with independent pharmacists in rural Appalachia, an ask-advise-connect model was well received; 83.7% were comfortable asking a customer about their tobacco use, 90.7% were confident they could advise a customer on the use of NRT, and 93.3% felt that it was feasible to connect tobacco users to a state quitline [60]. In the current study, pharmacists reported a high level of confidence in the approach and perceived participant engagement. In general, the pharmacists felt that they implemented the program as written and believed it was effective in reducing tobacco use among their participants. Thus, the current study suggests that this method is also effective at recruiting potentially hard to reach rural Appalachian smokers. Interestingly, all participants who signed up for the current study reported not having sought any behavioral smoking cessation support in the 12 months before enrolling. Thus, an ask-advise-connect model to recruit rural Appalachian smokers out of independent pharmacies shows great promise for engaging this population.

While it appears that pharmacists had the necessary time to recruit participants and deliver the QuitAid intervention, several adaptions were made at the onset of the study to ensure project success. Given that the pharmacy had several roles in this study (i.e., recruitment, NRT distribution, QuitAid implementation, biochemical verification of abstinence) in addition to routine pharmacy activities (e.g., medication dispensing, vaccinations, COVID testing, and MTM delivery for other chronic conditions), we recognized the need for an on-site implementation leader. This individual needed to be located within the pharmacy (pharmacist or technician) to oversee the daily operations of the study and provide on-going support to other pharmacists or technicians to achieve a high level of fidelity to study-related activities. Given that this was a research study with additional documentation and monitoring requirements, it is unclear if this individual would be needed in a real-world implementation of the program.

We also adapted our initial plans for training pharmacists and technicians. We had originally planned to hold a 1-day in-person training prior to launching the study. However, based on feedback from our independent pharmacists and the ongoing COVID-19 pandemic, we revised our training plans to provide self-paced web-based training and two virtual coaching sessions. In the future, it will be important to incentivize pharmacists to receive this training. One option is to provide CME credits for completing tobacco treatment specialist training and training on motivational interviewing. There are also freely available self-paced comprehensive tobacco cessation trainings for pharmacists through the Rx for Change: Clinician-Assisted Tobacco Cessation [61]. These modules cover ask-advise-connect, prescribing tobacco cessation medication, behavioral counseling, and pharmacotherapy, and could be successfully utilized in implementation trials to train large numbers of pharmacists and technicians.

There were several weaknesses of the current study that should be considered when interpreting the results. First, the study was intentionally small, recruiting 24 smokers from 1 independent pharmacy to determine feasibility of the approach. Therefore, future fully powered studies are necessary to determine efficacy of both the approach (e.g., connecting rural smokers to tobacco cessation services) as well as the QuitAid intervention. Additionally, because we only had 3 participating pharmacists and 1 pharmacy technician involved in the study, the results related to pharmacists’ and technician’s perceptions of the program are very imprecise and so interpretation is limited. As a result, it is unclear whether the current approach and intervention would be feasible in other independent pharmacies. For example, we had a high level of QuitAid session delivery (82% of sessions were delivered). It is possible in a larger trial with pharmacists with more diverse workflows and demands on their time that such a high level of implementation might not occur. However, the current study was conducted during COVID-19, when the pharmacy was overtaxed with conducting COVID-19 tests and administering COVID-19 vaccines. Given that this pharmacy was still able to achieve a high level of implementation during a very challenging time is promising. To ensure similarly high rates of implementation in the future, all pharmacists and technicians within each pharmacy should be trained in the ask-advise-connect method and QuitAid intervention delivery to provide coverage whenever the pharmacy is open. Additionally, it is possible given the association between cost of smoking cessation medication and adherence that participants without insurance may have had higher adherence rates. However, because of the small sample size, it was beyond the scope of this study to explore this relationship further. Future studies should consider examining whether there is differential adherence based on health insurance coverage, and thus differential quit rates.

Most of our sample (70.8%) were female, despite the fact that the limited epidemiologic data on smoking prevalence among rural Appalachian residents suggests that men are more likely to smoke compared to women [62, 63]. However, previous research has found that women are more likely to utilize recommended cessation resources, such as calling a quitline or using the nicotine patch [64]. Thus, it is unclear whether the current study is appealing to male smokers, and whether the current methods are feasible for recruiting balanced samples in terms of sex. Future studies should determine whether other modalities increase recruitment of male smokers into clinical trials.

Finally, given the nature of the current study to determine the feasibility of the approach, we were not powered to detect differences between treatment components. Therefore, future studies should explore whether these treatments increase cessation among this population.

This pilot randomized controlled study indicates that an ask-advice-connect model for connecting rural smokers to smoking cessation support and providing MTM for smoking cessation is feasible and acceptable among rural Appalachian smokers and independent pharmacists. The ask-advice-connect model was successful in recruiting hard-to-reach smokers, and the QuitAid intervention was well received by both pharmacists and participants. Therefore, an MTM-based approach for smoking cessation delivered by independent pharmacists could provide a ready and highly disseminable avenue for smoking cessation support for hard-to-reach rural smokers.

Availability of data and materials

The data from the current study is not publicly available due to institutional review board requirements but can be made available from the corresponding author on reasonable request.

References

  1. Marshall J, Alcalde G. Creating a culture of health in Appalachia. In: The first report in a series of exploring health disparities in Appalachia. 2018.

  2. American Lung Association. Trends in cigarette smoking rates. 2021. Accessed 10 Sept 2022.

  3. Creamer MR, Wang TW, Babb S, et al. Tobacco product use and cessation indicators among adults — United States, 2018. MMWR. 2019;68:1013–9. https://doi.org/10.15585/mmwr.mm6845a2.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Virginia Department of Health. Health behavior by health district. Richmond: Virginia Department of Health; 2021.

  5. Pilehvari A, You W, Krukowski RA, Little MA. Examining smoking prevalence disparities in Virginia counties by rurality, Appalachian status, and social vulnerability, 2011–2019. Am J Public Health. 2023;113(7):811–4. https://doi.org/10.2105/ajph.2023.307298.

    Article  PubMed  Google Scholar 

  6. Northridge ME, Vallone D, Xiao H, et al. The importance of location for tobacco cessation: rural-urban disparities in quit success in underserved West Virginia Counties. J Rural Health. 2008;24(2):106–15. https://doi.org/10.1111/j.1748-0361.2008.00146.x.

    Article  PubMed  Google Scholar 

  7. Hodge FS, Fredericks L, Kipnis P. Patient and smoking patterns in northern California American Indian clinics: urban and rural contrasts. Cancer. 1996;78(7 Suppl):1623–8.

    Article  CAS  PubMed  Google Scholar 

  8. Hutcheson TD, Greiner KA, Ellerbeck EF, Jeffries SK, Mussulman LM, Casey GN. Understanding smoking cessation in rural communities. J Rural Health. 2008;24(2):116–24. https://doi.org/10.1111/j.1748-0361.2008.00147.x.

    Article  PubMed  Google Scholar 

  9. Kruger TM, Howell BM, Haney A, Davis RE, Fields N, Schoenberg NE. Perceptions of smoking cessation programs in rural Appalachia. Am J Health Behav. 2012;36(3):373–84. https://doi.org/10.5993/AJHB.36.3.8.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Stead LF, Perera R, Bullen C, et al. Nicotine replacement therapy for smoking cessation. Cochrane Database Syst Rev. 2012;11:CD000146. https://doi.org/10.1002/14651858.CD000146.pub4.

    Article  PubMed  Google Scholar 

  11. Little MA, Krukowski RA, Wiseman KP, et al. Determining the feasibility of a pharmacistdelivered smoking cessation intervention for rural smokers in partnership with independent community pharmacies. Tob Prev Cessation. 2022;8(Supplement):A31. https://doi.org/10.18332/tpc/150967.

  12. Tobacco Use and Dependence Guideline Panel. Treating tobacco use and dependence: 2008 update. US Department of Health and Human Services. http://www.ncbi.nlm.nih.gov/books/NBK63952/. Accessed 22 Nov 2013.

  13. North American Quitline Consortium. Results from the 2019 NAQC Annual Survey of Quitlines. 2019.

  14. Mersha AG, Gould GS, Bovill M, Eftekhari P. Barriers and facilitators of adherence to nicotine replacement therapy: a systematic review and analysis using the capability, opportunity, motivation, and behaviour (COM-B) model. Int J Environ Res Public Health. 2020;17(23):8895. https://doi.org/10.3390/ijerph17238895.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Hammond D, McDonald PW, Fong GT, Borland R. Do smokers know how to quit? Knowledge and perceived effectiveness of cessation assistance as predictors of cessation behaviour. Addiction. 2004;99(8):1042–8. https://doi.org/10.1111/j.1360-0443.2004.00754.x.

    Article  CAS  PubMed  Google Scholar 

  16. Lindson N, Chepkin SC, Ye W, Fanshawe TR, Bullen C, Hartmann‐Boyce J. Different doses, durations and modes of delivery of nicotine replacement therapy for smoking cessation. Cochrane Database Syst Rev. 2019;(4). https://doi.org/10.1002/14651858.CD013308.

  17. Raupach T, Brown J, Herbec A, Brose L, West R. A systematic review of studies assessing the association between adherence to smoking cessation medication and treatment success. Addiction. 2014;109(1):35–43. https://doi.org/10.1111/add.12319.

    Article  PubMed  Google Scholar 

  18. Hollands GJ, Naughton F, Farley A, Lindson N, Aveyard P. Interventions to increase adherence to medications for tobacco dependence. Cochrane Database Syst Rev. 2019;(8). https://doi.org/10.1002/14651858.CD009164.pub3.

  19. National Association of Chain Drug Stores. NACDS: face-to-face with community pharmacies. www.nacds.org/pdfs/about/rximpact-leavebehind.pdf. Accessed 8 Nov 2021.

  20. Prokhorov AV, Hudmon KS, Marani S, et al. Engaging physicians and pharmacists in providing smoking cessation counseling. Arch Intern Med. 2010;170(18):1640–6. https://doi.org/10.1001/archinternmed.2010.344.

    Article  PubMed  PubMed Central  Google Scholar 

  21. American Public Health Association. The role of the pharmacist in public health. Policy Statements and Advocacy. www.apha.org/policies-and-advocacy/public-health-policy-statements/policy-database/2014/07/07/13/05/the-role-of-the-pharmacist-in-public-health. 2006.

  22. Casserlie LM, DiPietroMager NA. Pharmacists’ perceptions of advancing public health priorities through medication therapy management. Pharm Pract (Granada). 2016;14(3):792. https://doi.org/10.18549/PharmPract.2016.03.792.

    Article  PubMed  Google Scholar 

  23. Doucette WR, Kreling DH, Schommer JC, Gaither CA, Mott DA, Pedersen CA. Evaluation of community pharmacy service mix: evidence from the 2004 National Pharmacist Workforce Study. J Am Pharm Assoc. 2006;46(3):348–55. https://doi.org/10.1331/154434506777069471.

    Article  Google Scholar 

  24. Ismail N. Prescribing authority for pharmacists: rules and regulations by state. GoodRx. www.goodrx.com/hcp/pharmacists/prescriber-authority-for-pharmacists. Accessed 5 Mar 2021.

  25. Carson-Chahhoud KV, Livingstone-Banks J, Sharrad KJ, et al. Community pharmacy personnel interventions for smoking cessation. Cochrane Database Syst Rev. 2019;2019(10). https://doi.org/10.1002/14651858.CD003698.pub3.

  26. Patwardhan PD, Chewning BA. Ask, advise and refer: hypothesis generation to promote a brief tobacco-cessation intervention in community pharmacies. Int J Pharm Pract. 2009;17(4):221–9. https://doi.org/10.1211/ijpp/17.04.0005.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Burns A. Medication therapy management in pharmacy practice: core elements of an MTM service model (version 2.0). J Am Pharm Assoc. 2008;48(3):341–53. https://doi.org/10.1331/JAPhA.2008.08514.

    Article  Google Scholar 

  28. Viswanathan M, Kahwati LC, Golin CE, et al. Medication therapy management interventions in outpatient settings: a systematic review and meta-analysis. JAMA Intern Med. 2015;175(1):76. https://doi.org/10.1001/jamainternmed.2014.5841.

    Article  PubMed  Google Scholar 

  29. Pellegrino AN, Martin MT, Tilton JJ, Touchette DR. Medication therapy management services: definitions and outcomes. Drugs. 2009;69(4):393–406. https://doi.org/10.2165/00003495-200969040-00001.

    Article  PubMed  Google Scholar 

  30. Bunting BA, Cranor CW. The Asheville project: long-term clinical, humanistic, and economic outcomes of a community-based medication therapy management program for asthma. J Am Pharm Assoc. 2006;46(2):133–47. https://doi.org/10.1331/154434506776180658.

    Article  Google Scholar 

  31. Bunting BA, Smith BH, Sutherland SE. The Asheville Project: clinical and economic outcomes of a community-based long-term medication therapy management program for hypertension and dyslipidemia. J Am Pharm Assoc. 2008;48(1):23–31. https://doi.org/10.1331/JAPhA.2008.07140.

    Article  Google Scholar 

  32. Lounsbery JL, Green CG, Bennett MS, Pedersen CA. Evaluation of pharmacists’ barriers to the implementation of medication therapy management services. J Am Pharm Assoc. 2009;49(1):51–8. https://doi.org/10.1331/JAPhA.2009.07158.

    Article  Google Scholar 

  33. Moczygemba LR, Barner JC, Brown CM, et al. Patient satisfaction with a pharmacist-provided telephone medication therapy management program. Res Soc Adm Pharm. 2010;6(2):143–54. https://doi.org/10.1016/j.sapharm.2010.03.005.

    Article  Google Scholar 

  34. Fiore MC. Treating tobacco use and dependence: 2008 update: clinical practice guideline. Rockville: DIANE Publishing; 2008.

  35. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. https://doi.org/10.1186/1748-5908-6-42.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Fucito LM, Toll BA, Salovey P, O’Malley SS. Beliefs and attitudes about bupropion: implications for medication adherence and smoking cessation treatment. Psychol Addict Behav. 2009;23(2):373.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Mooney ME, Sayre SL, Hokanson PS, Stotts AL, Schmitz JM. Adding MEMS feedback to behavioral smoking cessation therapy increases compliance with bupropion: a replication and extension study. Addict Behav. 2007;32(4):875–80.

    Article  PubMed  Google Scholar 

  38. Schmitz JM, Sayre SL, Stotts AL, Rothfleisch J, Mooney ME. Medication compliance during a smoking cessation clinical trial: a brief intervention using MEMS feedback. J Behav Med. 2005;28(2):139–47.

    Article  PubMed  Google Scholar 

  39. Schlam TR, Fiore MC, Smith SS, et al. Comparative effectiveness of intervention components for producing long-term abstinence from smoking: a factorial screening experiment. Addiction. 2016;111(1):142–55. https://doi.org/10.1111/add.13153.

    Article  PubMed  Google Scholar 

  40. Horne R, Chapman SC, Parham R, Freemantle N, Forbes A, Cooper V. Understanding patients’ adherence-related beliefs about medicines prescribed for long-term conditions: a meta-analytic review of the Necessity-Concerns Framework. PLoS One. 2013;8(12):e80633. https://doi.org/10.1371/journal.pone.0080633.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  41. Motivational interviewing in brief consultations. https://new-learning.bmj.com/course/10051582. Accessed 4 Nov 2021.

  42. CITI Program. IRB-HSR researcher basic course. citiprogram.org. Accessed 14 Dec 2021.

  43. Hall MA, Camacho F, Dugan E, Balkrishnan R. Trust in the medical profession: conceptual and measurement issues. Health Serv Res. 2002;37(5):1419–39. https://doi.org/10.1111/1475-6773.01070.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Heatherton TF, Kozlowski LT, Frecker RC, Fagerstrom KO. The Fagerstrom test for nicotine dependence: a revision of the Fagerstrom Tolerance Questionnaire. Br J Addict. 1991;86(9):1119–27.

    Article  CAS  PubMed  Google Scholar 

  45. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45(5):626–9. https://doi.org/10.1016/j.jpsychires.2010.10.008.

    Article  PubMed  Google Scholar 

  46. Cramer H. Learning to fly: conceptions and misconceptions, uses and misuses of pilot studies in clinical research. J Altern Complement Med. 2021;27(7):531–4. https://doi.org/10.1089/acm.2021.0197.

    Article  PubMed  Google Scholar 

  47. Teresi JA, Yu X, Stewart AL, Hays RD. Guidelines for designing and evaluating feasibility pilot studies. Med Care. 2022;60(1):95–103. https://doi.org/10.1097/mlr.0000000000001664.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Salgado García FI, Derefinko KJ, Bursac Z, et al. Fit & quit: an efficacy trial of two behavioral post-cessation weight gain interventions. Contemp Clin Trials. 2019;76:31–40. https://doi.org/10.1016/j.cct.2018.11.009.

    Article  PubMed  Google Scholar 

  49. Little MA, Ebbert JO, Bursac Z, et al. Enhancing the efficacy of a smoking quit line in the military: study rationale, design and methods of the Freedom quit line. Contemp Clin Trials. 2017;59:51–6. https://doi.org/10.1016/j.cct.2017.04.011.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Hughes JR, Keely JP, Niaura RS, Ossip-Klein DJ, Richmond RL, Swan GE. Measures of abstinence in clinical trials: issues and recommendations. Nicotine Tob Res. 2003;5(1):13–25. https://doi.org/10.1093/ntr/5.1.13.

    Article  PubMed  Google Scholar 

  51. Benowitz NL, Bernert JT, Caraballo RS, Holiday DB, Wang J. Optimal serum cotinine levels for distinguishing cigarette smokers and nonsmokers within different racial/ethnic groups in the United States between 1999 and 2004. Am J Epidemiol. 2009;169(2):236–48. https://doi.org/10.1093/aje/kwn301.

    Article  PubMed  Google Scholar 

  52. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat. 2005;4(4):287–91. https://doi.org/10.1002/pst.185.

    Article  Google Scholar 

  53. Billingham SA, Whitehead AL, Julious SA. An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database. BMC Med Res Methodol. 2013;13:104. https://doi.org/10.1186/1471-2288-13-104.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Dent LA, Harris KJ, Noonan CW. Tobacco interventions delivered by pharmacists: a summary and systematic review. Pharmacotherapy. 2007;27(7):1040–51.

    Article  PubMed  Google Scholar 

  55. Schlam TR, Cook JW, Baker TB, et al. Can we increase smokers’ adherence to nicotine replacement therapy and does this help them quit? Psychopharmacology. 2018;235(7):2065–75. https://doi.org/10.1007/s00213-018-4903-y.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Smith SS, Keller PA, Kobinsky KH, et al. Enhancing tobacco quitline effectiveness: identifying a superior pharmacotherapy adjuvant. Nicotine Tob Res. 2013;15(3):718–28. https://doi.org/10.1093/ntr/nts186.

    Article  CAS  PubMed  Google Scholar 

  57. Stead LF, Hartmann-Boyce J, Perera R, Lancaster T. Telephone counselling for smoking cessation. Cochrane Database Syst Rev. 2013;8:Cd002850. https://doi.org/10.1002/14651858.CD002850.pub3.

    Article  Google Scholar 

  58. West R, Hajek P, Nilsson F, Foulds J, May S, Meadows A. Individual differences in preferences for and responses to four nicotine replacement products. Psychopharmacology. 2001;153(2):225–30. https://doi.org/10.1007/s002130000577.

    Article  CAS  PubMed  Google Scholar 

  59. Centers for Disease Control and Prevention. How to use nicotine gum. Centers for Disease Control and Prevention. Accessed 19 Nov 2023.

  60. Little MA, Porter KJ, Pebley K, et al. Evaluating the feasibility of pharmacist-facilitated tobacco cessation interventions in independent community pharmacies in rural Appalachia. J Am Pharm Assoc. 2022. https://doi.org/10.1016/j.japh.2022.06.015.

  61. Pharmacy CE Courses Office of Continuing Education. CE tobacco cessation. Purdue University. https://ce.pharmacy.purdue.edu/courses/course/index.php?categoryid=20. Accessed 14 Oct 2021.

  62. Schoenberg NE, Huang B, Seshadri S, Tucker TC. Trends in cigarette smoking and obesity in Appalachian Kentucky. South Med J. 2015;108(3):170–7.

    Article  PubMed  Google Scholar 

  63. Cardarelli K, Westneat S, Dunfee M, May B, Schoenberg N, Browning S. Persistent disparities in smoking among rural Appalachians: evidence from the Mountain Air Project. BMC Public Health. 2021;21(1):270. https://doi.org/10.1186/s12889-021-10334-6.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Jayakumar N, Chaiton M, Zhang B, Selby P, Schwartz R. Sex differences in use of smoking cessation services and resources: a real-world study. Tob Use Insights. 2020;13:1179173x20901500. https://doi.org/10.1177/1179173x20901500.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the independent community pharmacists and technicians who took time out of their busy schedules to participate in this project.

Funding

This research was supported from resources within the Cancer Control and Population Health (CPH) program at the University of Virginia Cancer Center, provided through the NIH Cancer Center Support Grant (P30CA044579), and a grant from that National Cancer Institute (R01CA267963).

Author information

Authors and Affiliations

Authors

Contributions

ML conceived the intervention and the study and wrote the first draft of the manuscript. RK provided guidance on the study and design and assisted with the draft of the manuscript. TR was responsible for study oversight and assisted with the draft of the manuscript. MM conducted statistical analyses, supervised by WY. All authors contributed to the design and implementation of the study, provided feedback on manuscript drafts, and approved the final version.

Corresponding author

Correspondence to Melissa A. Little.

Ethics declarations

Ethics approval and consent to participate

At the time of submission, ethics approval for the study was granted by the University of Virginia Institutional Review Board for Health Sciences Research on August 31, 2021. The research trial was registered on clinicaltrials.gov on December 4, 2022 (NCT05649241). Informed consent was obtained from all participants in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Little, M.A., Reid, T., Moncrief, M. et al. Testing the feasibility of the QuitAid smoking cessation intervention in a randomized factorial design in an independent, rural community pharmacy. Pilot Feasibility Stud 10, 41 (2024). https://doi.org/10.1186/s40814-024-01465-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40814-024-01465-9

Keywords