Skip to main content

Protocol for a feasibility study incorporating a randomised pilot trial with an embedded process evaluation and feasibility economic analysis of ThinkCancer!: a primary care intervention to expedite cancer diagnosis in Wales



Compared to the rest of Europe, the UK has relatively poor cancer outcomes, with late diagnosis and a slow referral process being major contributors. General practitioners (GPs) are often faced with patients presenting with a multitude of non-specific symptoms that could be cancer. Safety netting can be used to manage diagnostic uncertainty by ensuring patients with vague symptoms are appropriately monitored, which is now even more crucial due to the ongoing COVID-19 pandemic and its major impact on cancer referrals. The ThinkCancer! workshop is an educational behaviour change intervention aimed at the whole general practice team, designed to improve primary care approaches to ensure timely diagnosis of cancer. The workshop will consist of teaching and awareness sessions, the appointment of a Safety Netting Champion and the development of a bespoke Safety Netting Plan and has been adapted so it can be delivered remotely. This study aims to assess the feasibility of the ThinkCancer! intervention for a future definitive randomised controlled trial.


The ThinkCancer! study is a randomised, multisite feasibility trial, with an embedded process evaluation and feasibility economic analysis. Twenty-three to 30 general practices will be recruited across Wales, randomised in a ratio of 2:1 of intervention versus control who will follow usual care. The workshop will be delivered by a GP educator and will be adapted iteratively throughout the trial period. Baseline practice characteristics will be collected via questionnaire. We will also collect primary care intervals (PCI), 2-week wait (2WW) referral rates, conversion rates and detection rates at baseline and 6 months post-randomisation. Participant feedback, researcher reflections and economic costings will be collected following each workshop. A process evaluation will assess implementation using an adapted Normalisation Measure Development (NoMAD) questionnaire and qualitative interviews. An economic feasibility analysis will inform a future economic evaluation.


This study will allow us to test and further develop a novel evidenced-based complex intervention aimed at general practice teams to expedite the diagnosis of cancer in primary care. The results from this study will inform the future design of a full-scale definitive phase III trial.

Trial registration NCT04823559.

Peer Review reports



Cancer survival in the UK lags behind other western countries [1]. Referral rates and adherence to guidelines are lower [2, 3], primary care providers (PCPs) are less likely to take action on potential cancer symptoms [4] and cancer tends to be diagnosed at a later stage, often only after patients have presented to acute or emergency secondary care services [3]. Compared with the rest of Europe, the UK has relatively low 1-year survival, which could be due to later diagnosis [5].

Timely diagnosis is key to improving cancer outcomes [6] and cancer survival [7]. Earlier diagnosis could also reduce the proportion of cancer patients diagnosed through emergency care [7]. Early diagnosis is a rising priority in cancer policy [6, 8], because it is cost-effective [1] and the incidence of cancer is increasing [9]. Policies in Wales emphasise the importance of early diagnosis and recognise that increasing demand and a slow referral process are significant barriers to a quick cancer diagnosis [3]. Following the implementation of these policies, progress has been slow, with late stage diagnosis continuing to be an issue [10].

The timely diagnosis of cancer has become even more relevant as we enter a period in which primary care and cancer management have changed dramatically due to the ongoing COVID-19 pandemic [11,12,13]. Early figures have shown a 76% decrease in urgent cancer referrals across the UK and predict a 20% increase in excess deaths for patients with newly diagnosed cancers [14]. The increased use of remote consultation as a result of the pandemic will have implications for the early diagnosis of cancer, as important consultation techniques such as the use of visual cues and physical examination [15] may be impacted.

Primary care providers (PCPs) play a vital role in the early diagnosis of cancer [16]. A key diagnostic stage is the Primary care interval (PCI), which is the time from first presentation to a GP with a symptom that could be cancer, to the subsequent referral to a specialist in secondary care [17]. However, with an ever expanding role, PCPs are presented with a plethora of non-specific symptoms, of which only a small proportion are caused by cancer, and many overlap with other diseases [6]. Furthermore, with certain cancers, patients may not present with any alarm symptoms [6, 16] which often results in a delayed cancer diagnosis [2]. In addition, guidelines to expedite early cancer diagnosis are often unclear, with great variation in strategies between different GPs [18].

Rationale and previous work

This study comprises work package 4 of the Wales Interventions for Cancer Knowledge and Early Diagnosis (WICKED) programme. Work package 1 consisted of a review of reviews and a realist review, Work Package 2 comprised qualitative interviews with 20 GPs, four focus groups with primary care practice staff, and a combined quantitative survey and Discrete Choice Experiment (DCE), sent to GPs in Wales. The data garnered from work packages 1 and 2 fed into the development of the intervention (work package 3) via the Behaviour Change Wheel (BCW) [19]. A target behaviour was identified—GPs thinking of and acting on clinical presentations that could be cancer—and through application of the BCW, education and training, enablement and restructuring of the environment were selected as intervention functions. The findings of the earlier work packages and the development of the intervention are described in more detail elsewhere [20].

Clinical behaviour change interventions targeting PCPs have the potential to address barriers to suspected cancer identification and referral and could expedite the diagnosis of cancer and improve cancer outcomes overall [21]. Behaviour change at PCP level can be achieved through various mechanisms, including feedback, electronic prompts and training [22,23,24]. Financial incentives have also been found to be effective, but once withdrawn may lead to a drop in the quality of care [25]. There is some evidence that educational interventions targeted at PCPs could reduce the PCI, providing they encompass a multidimensional, interactive and tailored approach [21, 26, 27]. A recent systematic review of primary care interventions suggested that a whole-practice approach providing opportunities for peer review and feedback could have a positive effect on referral practices, in addition to existing guidelines being revisited through training and reinforcement [28]. This was also confirmed by the findings of the qualitative work in the previous WICKED work packages, where GPs highlighted positive practice culture and a whole-practice approach as important themes [20].

While the timely diagnosis of cancer is crucial, urgent referral can lead to over-diagnosis and over-investigation which can be harmful to the patient [7, 29]. This risk is especially high if the patient presents with vague symptoms [30]. Safety netting, a tool used to manage diagnostic uncertainty [31], can address these issues by ensuring that patients with non-specific symptoms are not ignored [29, 32]. Instead of immediate referral, patients are monitored according to a set step-wise investigational plan, while ensuring they are referred in a timely manner as and when required [29, 32]. Although safety netting is currently recommended by national guidelines [33], there are no clear recommendations on how to do it [18, 29, 31, 32].

In summary, it is relevant and befitting to develop and test interventions aimed at improving the quality and consistency of primary care approaches to ensure timely diagnosis of cancer in the UK. This will require multicomponent and complex behavioural change interventions, which utilise a multidimensional, interactive, tailored, whole-practice approach.

The ThinkCancer! intervention is a complex behaviour change intervention aimed at general medical practice teams. It consists of a multi-component workshop that includes educational early diagnosis and awareness sessions, evaluation of current practice-based safety netting systems and the appointment of a safety netting champion. The workshop will be led by an educational facilitator who will guide the development of a bespoke safety netting plan for each practice.

The aim of this study is to assess the feasibility of delivering the ThinkCancer! intervention and conducting a future, definitive randomised UK-wide controlled trial (RCT) to assess effectiveness and cost-effectiveness, in order to establish whether the intervention can be rolled out in practice.

Study objectives

The objectives of this study are as follows:

  1. 1)

    To assess the feasibility of a future definitive RCT by monitoring recruitment and retention, outcome measure completion and reasons for decline.

  2. 2)

    To assess the acceptability, feasibility, and utility of the ThinkCancer! intervention as a whole and of each of its individual components, and refining the intervention as necessary.

  3. 3)

    To determine the most feasible and appropriate primary outcome measure for a definitive RCT and producing means and confidence intervals for calculating effect sizes for the design of a definitive trial.

  4. 4)

    To describe current contextual differences, and similarities, between general medical practices and their usual safety netting practices.

  5. 5)

    To identify and test the methods and outcome measures for a process evaluation of a future definitive RCT.

  6. 6)

    To undertake a feasibility analysis of the most appropriate approach for an economic evaluation alongside a future definitive trial.


Study design

This feasibility study incorporates a pragmatic, multisite, two-armed, superiority, pilot RCT. There is an embedded process evaluation and feasibility economic analysis. The unit of randomisation is the general medical practice, and the primary clinical outcome is collected at the practice level.

The term ‘feasibility’ is used in accordance with the conceptual framework developed by Eldridge and colleagues [34], where it is described as an umbrella term within which pilot trials are a component. Furthermore, the study has been designed in accordance with the MRC Framework for evaluating complex interventions [35]. The trial will be conducted according to NIHR guidance [36] and recommendations for good practice in pilot studies [37].

The process evaluation, which will be based on a mixed-methods approach, will follow the MRC guidance for process evaluations of complex interventions [38]. During the initial piloting or feasibility testing stage of an intervention, process evaluation has a vital role in understanding and planning the future potential implementation of the intervention and optimising its design and evaluation [38].

The feasibility economic analysis will explore the appropriate future perspective of analysis, most appropriate methods of gathering costs, and range and value of outcome measures and undertake a feasibility budget impact analysis of the ThinkCancer! intervention developed through a range of blended methods that it is delivered online (either in a live format or pre-recorded) or face-to-face in general practices across north Wales.

This study protocol was developed in line with the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines [39]; the SPIRIT checklist (Appendix 1) and the schedule of procedures can be seen in Table 1. The SPIRIT checklist has been adapted in accordance with the CONSORT extension to pilot and feasibility trials [40].

Table 1 SPIRIT protocol schedule of procedures for the ThinkCancer! study

Study setting

The setting for this study is primary care. The intervention will be delivered in-practice or online to individual general medical practices and incorporates a whole-practice approach. The trial will be conducted across Wales and practices will be recruited from all seven Welsh health boards.


The proposed intervention, the ThinkCancer! workshop, has four chief components. The first are two educational sessions, one for all clinical staff (the ‘early diagnosis’ session) and one for non-clinical but patient-facing staff (the ‘cancer aware’ session). The early diagnosis session is delivered as a teaching seminar with learning outcomes focussed on NICE NG12 Suspected Cancer: recognition and referral guidelines [27], hot topics exploring the harder to recognise cancer presentations and consultation-level safety netting. As a proposed aid to support and formalise safety netting, a new tool—the Shared Safety Net Action Plan (SSNAP) [41] will be introduced. The SSNAP tool is a co-produced patient-facing information leaflet that can be used by a clinician to give specific safety netting advice to a patient about when and how they should re-present for further medical assessment [41]. This tool can be used in support of patients who present with ‘low risk, but not no risk’ symptoms or in situations where it would be too soon to trigger an urgent referral or diagnostic investigation. Proposed benefits of the tool include increasing patient empowerment to navigate primary care systems in gaining follow-up appointments, and by keeping copies of the tool within the patient record, better record advice and guidance given. This session will also see the introduction of the ThinkCancer! handbook, which will contain all the resources used in the workshop as well as external resources regarding early diagnosis and safety netting, such as NICE guidance and online learning resources. The cancer aware session is less formal with more convenor-led discussion around cancer red flag symptoms that non-clinical staff may encounter. The secondary aim of this session is to gauge and explore issues and norms around raising concerns within the practice team. The third session (the ‘safety netting session’) involves the two final components of the intervention, the co-production of a bespoke Cancer Safety Netting Plan (CSNP) and appointment of a Cancer Safety Netting Champion (CSNC). This session is attended by a combination of clinical and administrative staff who will be involved in the design and implementation of a new plan. The CSNP will evolve from discussions built on three components, learning from the earlier educational parts of the workshop, evaluation of the current practice safety netting systems reported in the practice questionnaire and the attendee’s personal reflections of cancer diagnosis and safety netting. Following this discussion, a summary document highlighting potential new action points will be sent back to the practice for them to take forward and develop. Success in developing and implementing a new practice plan may be increased by the appointment of a champion to drive change and therefore the appointment of a CSNC is explored during this part of the workshop.

Members of the research team will deliver the intervention; the GP Educator (AS) will oversee the workshop, supported by up to two researchers. The workshop was originally designed to be delivered face-to-face in participating practices during practices’ allocated protected time for educational and professional development. However, due to the ongoing COVID-19 pandemic, the workshop has been adapted into a digital format and can be delivered in one of three ways: (i) fully remote and live via online conferencing platforms; (ii) blended delivery, where practices are offered a combination of pre-recorded versions and live remote delivery of the different sessions; and (iii) face-to-face in the practice, as originally intended, if the situation allows. Practices that opt for the blended delivery of the workshop can choose to receive pre-recorded videos of sessions 1 and 2, presented by the GP Educator, allowing participants to engage with the materials in their own time. The final session, which focuses on the Cancer Safety Netting Plan, will still need to be delivered as a live session due to its interactive components. There will be flexibility to work with the practice to allow the bespoke design of the workshop format to improve the reach of the intervention. Practices receiving the intervention in any of the remote forms will be sent all of the workshop materials, including the handbook and SSNAP tool, via post. If there is the possibility to deliver the workshop face-to-face, materials will be distributed in the practice at the beginning of the workshop. Practices randomised to the control group may also receive the pre-recorded videos at the end of the study period, along with the intervention materials.

A logic model (Fig. 1) has been developed to describe the intervention components and how they link to the intended outcomes and will be adapted throughout the study period.

Fig. 1
figure 1

ThinkCancer! logic model

The previous work packages of the WICKED programme have involved extensive work with primary care staff throughout, which has directly fed into the development of the intervention. GPs were the focus and source of data collection in the early work packages and have been involved in the refinement of the intervention via a small pilot of the workshop in a general practice local to the research team. In addition, we have had a member of the public actively involved from the very beginning of the project, who cooperated with the team in developing the intervention and the trial protocol.

Outcome measures

The outcomes that will be reported in this feasibility study are as follows:

Recruitment will be assessed quantitatively by capturing the numbers of practices

  • ▪ Approached

  • ▪ Interested in participating

  • ▪ Consented

  • ▪ Randomised

A description of the excluded practices will be included to help identify potential future eligibility criteria and reasons for non-participation will be sought.

Retention will be assessed quantitatively by the numbers of practices providing both baseline and follow-up data.

Data on individual practice characteristics will be collected to:

  • ▪ Describe the studied sample

  • ▪ Identify potential effect modifiers

  • ▪ Allow identification of ‘usual practice’

The overarching goal of the ThinkCancer! intervention is to change GPs’ and primary care practices’ behaviours and systems, encouraging them to be more cancer-aware and act sooner on clinical presentations that could be cancer, or more effectively safety net cases where concern exists but criteria for immediate referral are not met. The proposed primary clinical outcomes for the definitive RCT relate to the early referral of suspected cancer. They include the 2-week wait (2WW) referral rate and the PCI. The 2WW referral rate is defined as the crude rate of 2WW referrals multiplied by 100,000 and divided by practice list size [42]. The PCI is defined as the time between the date of first presentation and the date of referral [17].

Secondary clinical outcome measures include the conversion rate and the detection rate. The conversion rate is defined as the ‘proportion of 2WW referrals that are subsequently diagnosed with cancer’ [42] and the detection rate consists of the ‘proportion of new cancer cases treated who were referred through the 2WW route’, [42] also known as the sensitivity [43]. These measures will allow us to further explore potential clinical outcomes.

Although it is unlikely that we will detect a difference in the clinical measures in the feasibility study due to the small sample and short duration of follow-up, we do expect these outcome measures to be affected by the ongoing COVID-19 pandemic. However, the feasibility of using these outcomes will be assessed.

The feasibility of using potential clinical primary outcomes, 2WW referral rate, PCI, conversion rates, and detection rates will be assessed quantitatively by determining:

  • ▪ Ability to collect/capture data from the practices, based on completion rates of data collection forms

  • ▪ Ability to extract relevant data from routinely collected data at Health Board level and from individual practices, based on whether we are able to obtain the data via Health Board contacts and whether these data are comparable with those collected by individual practices

  • ▪ Suitability and variability of the data to perform as primary outcome measures for clinical effectiveness

Acceptance, adherence to and fidelity of the intervention will be assessed by:

  • ▪ Reviewing participant views, reflections and perceptions expressed via post-workshop feedback forms and interviews

  • ▪ Post-workshop reflections from the intervention delivery staff

  • ▪ The ability to organise/schedule and deliver workshops

  • ▪ The number of practices that actively nominate a safety netting champion

  • ▪ The number of practices that demonstrate the use of the safety netting plan

To inform the process evaluation for the future definitive trial, we will do the following:

  • ▪ Evaluate how to scale up for any future process evaluation,

  • ▪ Determine the acceptability and appropriateness of the interview process

  • ▪ Identify barriers and facilitators to successful implementation,

  • ▪ Test measures for assessing reach, dose and fidelity of the intervention

For the health economics component, we will do the following:

  • ▪ Determine the feasibility of collecting data relating to the costing of the intervention via costings forms

  • ▪ Identify variables necessary for the design of a future economic evaluation alongside a definitive trial

  • ▪ Review relevant literature on the cost-effectiveness of online and mixed-methods Continuing Professional Development (CPD) programmes for health professionals in a community setting

  • ▪ Consult the DIRUM database [44] to identify the most appropriate way of capturing the costs of the ThinkCancer! intervention (online delivery and mixed-methods delivery)

Adaptations to the intervention and logic model will also be recorded throughout the trial period.

Progression criteria

The outcome measures relating to recruitment, retention and adherence/fidelity will be assessed using RAG criteria, as defined below:

  1. 1.

    Confirmation of adequate recruitment for a definitive trial at practice level. Go: 20 or more general medical practices recruited; Review: 15–19 recruited; Stop: < 15 practices recruited.

  2. 2.

    Confirmation of adequate retention for the definitive trial at practice level. Go: 80% or more practices retained; Review: 65–79% practices retained; Stop: < 65% practices retained

  3. 3.

    Confirmation of adequate fidelity of the intervention. Go: 80% or more of all intervention sessions delivered, Review: 50–79% of all interventions delivered; Stop: < 50% of interventions delivered.

  4. 4.

    Confirmation of adequate fidelity at individual practice staff level: From each general practice:

    1. a.

      At least 50% of the clinical staff should attend the workshops;

    2. b.

      At least 50% of the administrative staff should attend the workshops, comprising at least 50% of the reception and secretarial staff as well as the practice manager.

    3. c.

      Staff who do not attend the training should have the information cascaded to them by a member of the team who did attend the training. At least 75% of the staff should receive the training either directly or indirectly.

Progression criteria relating to obtaining data regarding completion of outcome measures will be assessed using the following progression criteria:

  1. 1.

    Routine data. Go: data from 70% or more practices obtained; Review: data from < 70% of practices obtained

  2. 2.

    Individual data. Go: data from 70% or more of individuals of each practice obtained; Review: data from < 70% of individuals from each practice obtained

These criteria would reflect the potential outcomes for exclusion/inclusion at a full definitive trial stage rather than prevention of the study progressing.

Sample size

We aim to recruit 23–30 general medical practices. As the intention is to have a minimum of 15 practices randomised to the intervention, it was determined that a minimum of 23 practices would be needed to provide sufficient data for the trial and to allow for development of the intervention. The maximum that was considered possible to recruit is 30 practices. Practices will be randomised using a randomisation allocation ratio of 2:1 of intervention versus control. Randomising in preference to the intervention will allow us to iteratively develop the intervention more effectively.

As this is a feasibility study, there is no requirement for a formal power calculation. This study is not intended to be powered to identify a clinically meaningful difference between the intervention groups for the primary outcome measures, rather this study aims to provide robust estimates for the likely recruitment and retention rates and give an indication of the potential variability in the proposed outcome measures, which will in turn be used to inform the power calculation for a future definitive RCT. This is discussed further under statistical analyses.

Recruitment and consent


General practices will be identified through contact details publicly available via practice websites and through contact lists provided by the health boards across Wales. Practices will be invited to participate using a standardised invitation via email addressed to the practice manager, along with information about the study. Practice managers will be asked to consult with their team and indicate their interest in participating in the study by responding to the email. They will also need to advise of their practice’s availability for potential workshop dates.

If no response to the initial email is received, a reminder email will be sent followed by a telephone call. A participant flow diagram can be seen in Fig. 2.

Fig. 2
figure 2

Participant flow diagram

Practices that take part in the study will be financially reimbursed for their time; the reimbursement amount was decided upon based on input from local GPs. In addition, participants will receive a copy of the intervention handbook that includes the NG12: Suspected cancer: recognition and referral summary tables, and certificates of attendance. Staff members involved in the design and implementation of a new cancer safety netting plan can consider using this as the basis of a quality improvement project for revalidation and appraisal purposes. Educational accreditation of the workshop will be sought for a future main trial.

We are aware that some challenges in the recruitment lay ahead, but we plan to work with the various research infrastructures in Wales to overcome this. We plan to work closely with the Primary Care Specialty Lead and Primary Care Research Managers within Health and Care Research Wales, and we will establish contacts within regional primary care clusters, with health board staff and with R&D departments in order to maximise potential recruitment opportunities. We will also work closely with the practices that agree to take part in order to support their participation in the study. The recruitment methods will be continually assessed and iteratively developed to determine the most appropriate recruitment strategy for a future definitive RCT.

Eligibility to participate

As feasibility is the main objective of this study, all types of general medical practice will be eligible for inclusion. This will aid intervention refinement and allow for a better understanding of what is feasible across a range of practices and also why some practices may not be able to take part.


Any general practice in Wales is eligible for inclusion. The target audience for the intervention, based on a ‘whole team approach’, includes all practice staff members. These may include, for example, GPs, nurse practitioners, nurses, health care support workers (HCSWs), practice managers, administrators, receptionists or any other practice staff, clinical and non-clinical. We would aim to include any new forms of primary care organisations such as managed GP practice networks or other general medical practice amalgamations, as well as traditional GP partnerships. However, some of these may be unstable practices with no regular GP staff, and as such, they may be difficult to recruit or find it difficult to participate. Practices participating in other research are also eligible for inclusion; they will be asked to notify us of any cancer-related studies they may be participating in via the baseline questionnaire.


There are no exclusion criteria.


Practice managers will be sent more detailed study information in the form of a ‘Research Information Sheet for Practices’ (RISP) and a link to an online baseline practice questionnaire after they have expressed an interest to take part in the study. Participating practice managers will need to indicate that they have read the study information and have agreed with consent statements on the first page of the electronic questionnaire before they can proceed. Prior to the commencement of the ThinkCancer! workshop, Participant Information Sheets (PIS) will be provided by members of the research team and written consent will be obtained from all participating members of staff. This will include consent to use anonymised data recorded on paper or audio-file during workshops and workshop feedback forms. At this time point, participants will also be given the option to provide their contact details should they be happy to be contacted for a telephone interview. Those who indicate that they would like to take part will be contacted at least 2 months after the intervention. Participants from practices randomised to the control arm, who do not take part in a workshop, will also have the opportunity to take part in a telephone interview and will be sent an invitation letter via their practice manager. Response to interview invitations and supplying of contact details will be taken as consent to be contacted and informed consent will be obtained verbally at the time of interview.

Although we will initially contact practice managers, and they will most likely be the person who completes the questionnaire, this task may be delegated to another member of the practice team with a particular interest in the study.

Pre-trial pilot

A local practice, known to the research team, has agreed to participate in a ThinkCancer! pilot workshop prior to full rollout of the feasibility study. The practice is an urban, large 12,000 patient training practice in a moderately deprived region of North East Wales. Data will not be collected or recorded for trial purposes and feedback from those participating will only be used to refine the intervention prior to its delivery across recruited practices.

Randomisation and blinding

The general medical practice will be the unit of randomisation. Randomisation will be achieved online, through the remote randomisation centre at the North Wales Organisation for Randomisation Trials in Health (NWORTH) at Bangor University. The randomisation system will use a dynamic adaptive allocation algorithm [45] to achieve randomisation, stratified for health board. ThinkCancer! is an open trial where blinding of participants, researchers and the statistician is not possible due to the nature of the intervention and 2:1 ratio for randomisation.

Withdrawal criteria

Practices (and individuals within a practice) will be free to withdraw from the trial at any time, and their right to refuse participation will be respected throughout. We will seek to understand their reasons where possible. In terms of the primary outcome measures, as long as it is possible to collect the data, intention-to-treat analysis will be utilised, whether or not the intervention was received or adhered to.

Data collection

The feasibility study will be used to rehearse data collection approaches and assess their ease of use. Data will be collected at time-points specific to each item and depending on the type of data. All data collected in this study will be anonymised.

Proposed clinical effectiveness outcomes

Data relating to the proposed primary outcomes for the future definitive RCT will be collected at baseline and 6 months after randomisation. Two-week wait referral data and PCI data will be collected directly from participating practices via Case Report Forms containing full instructions on how to extract the data from practice IT systems. We will work with the Practice Manager, CSNC or other delegated individuals to achieve this. It is recognised that this is likely to be too short a follow-up period for meaningful differences to be observed, but the main purpose in this case will be to test the feasibility of collecting the data in this way. Additionally, we will explore the availability of 2WW data at health board level.

Practice questionnaires

The baseline and endline practice questionnaires will be available online to both intervention and control practices and are to be filled out by the practice manager or other designated person, ideally in collaboration with the practice team; SurveyMonkey™ will be the most likely platform. The questionnaires will consist of closed questions and some open, free-text questions, and will be used to collect data for each individual practice on the practice characteristics and current systems, and existing practice systems relating to cancer diagnosis and safety netting. The baseline data may be used to inform some workshop planning—i.e. workshop content and delivery may be tailored to some extent to suit individual practice needs and circumstances. Further process evaluation data will also be collected from the practice questionnaires as they will incorporate questions exploring contextual factors known to influence the success of quality improvement approaches used to improve health care [46]. The baseline questionnaire will be completed by all practices prior to randomisation. The endline questionnaire will assess any differences in practice, knowledge or systems in comparison with those measured at baseline and will be completed at 6 months post-randomisation.

Baseline measures will include the following:

  • ▪ Demographic information and practice characteristics (practice size, research-accredited status, number of clinical and non-clinical staff members, whether a teaching practice, etc.)

  • ▪ Practice culture (e.g. team structure, diversity of team member roles, team decision-making processes)

  • ▪ Practice knowledge with regard to safety netting and cancer awareness

  • ▪ Current safety netting systems in place, if any, including:

    • What systems are in place

    • How widely they are used within the practice

    • How safety netting issues are communicated:

      • ▪ Between clinicians

      • ▪ To the wider practice team

      • ▪ To patients

    • How safety netting is recorded

Feasibility and piloting data

Recruitment, retention and questionnaire completion numbers will be recorded throughout the trial. Spreadsheet systems will be put in place to record practice responses and to track their progress in the trial (e.g. number of practices approached, whether they have responded to the initial invitation, whether they have agreed to be randomised, etc.). Separate spreadsheets will also record feasibility data relating to the workshop itself, such as participant numbers.

Post-workshop reflections and participant feedback

Data specific to the intervention will be collected via participant feedback and observation and reflections of the research staff. Participant feedback forms will be distributed to practice staff; these can be completed in paper format or online. Responses will be requested using a combination of Yes/No choices, Likert scales and free-text comments. The questions will cover a number of areas including acceptability, usefulness, learning outcomes and the potential to change practice [47].

Relevant ad hoc communications with practices throughout the study will also be collected on a spreadsheet, which may contribute to understanding the intervention in terms of what works, why and how.

The same research team members will deliver the intervention in all practices; their observations will be collected and will inform any refinements of the intervention. Observations and reflections recorded by the research team may provide valuable data on the potential effects of contextual factors, site-by-site and component-by-component measures, and the appropriateness of individual questions included in the practice questionnaires. They will also describe the cancer safety netting plan proposed by the practice and whether the SSNAP tool is used.

The researchers will keep a diary, which will include a record of any modifications made to the intervention and data collection methods.

Health economics

Health economics data collection sheets will be completed by the researchers following each workshop, and costings specific to the practice will be recorded. We will also use the feedback forms to determine staff roles within the practice for costing purposes.

NoMAD instrument

At least 2 months after the intervention, participants who consented to be contacted will be sent a link to complete an adapted Normalisation Measure Development (NoMAD) instrument [48]. This will assess the implementation of the cancer safety netting plan using Normalization Process Theory (NPT) principles, which may or may not include the SSNAP tool depending on uptake.

Telephone interviews

We will conduct qualitative telephone interviews lasting up to 30 min with a purposive sample of up to 45 clinical and non-clinical practice staff. Practice staff in both arms of the trial will be eligible to participate in the interviews as they will be invited to give feedback on all aspects of the trial process including the intervention where appropriate. The qualitative interviews are designed to achieve an in-depth understanding of the views and perceptions of practice staff involved in the trial. The interviews will allow participants to explain how they were able to utilise aspects of the trial and how they worked in practice. Informed consent will be obtained and interviews recorded and transcribed verbatim. The interviews will be semi-structured and will follow a pre-defined topic guide, although not every participant will be engaged with every section of the topic guide (i.e. only the specific areas of the topic guide that are relevant to an individual’s role and experience will be explored). Topics may include acceptability, safety netting, data collection, uptake of the intervention and SSNAP tool and implications. These interviews will occur at least 2 months after the intervention has taken place; control practices will be invited two months post-randomisation.

Adverse events

A risk assessment has found this trial to be low risk. Participating practices are encouraged to report any adverse events. The Chief Investigator (CI) will determine whether they are Adverse Events (AEs) or Serious Adverse Events (SAEs). They will be reported in line with current ICH-GCP Standard Operating Procedures [49]. SAEs are defined as follows:

…an untoward occurrence that (a) results in death; (b) is life-threatening; (c) requires hospitalisation; (d) results in persistent or significant disability or incapacity; (e) consists of a congenital anomaly or birth defect; or (f) is otherwise considered medically significant by the investigator [49]

We do not expect any related SAEs for this study.

Data analysis

Quantitative analysis

A fully documented statistical analysis plan will be prepared by NWORTH, a registered clinical trials unit, and agreed by the co-investigators and approved by the trial governance structure, which will be known as the Trial Steering Committee (TSC). The statistical analysis plan will be approved prior to data collection being completed.

Baseline characteristics will be summarised for all practices, the intervention group and control groups separately. Feasibility and process evaluation data such as practice recruitment rate, implementation and uptake of and adherence to the intervention, and follow-up rates will be summarised and presented as percentages.

Determining differences in clinical outcomes between the control and intervention is not the primary purpose of this study; therefore, the focus of the results will be on the estimates of the treatment effects rather than statistical significance and as such, no hypothesis testing will be undertaken. As recommended in guidelines for good practice for the analysis of pilot studies [37], summary estimates of effects will be developed along with their 95% confidence intervals. Differences between the two comparison groups will be presented in the form of an unadjusted mean difference for continuous outcomes, and an odds ratio for binary outcomes. Exploratory analysis using ANCOVA for continuous outcomes and logistic regression for binary outcomes will consider adjustment for the stratification variables in assessment of the treatment effects.

Factors associated with the ability to implement the intervention will be tentatively explored using logistic regression with the focus on identifying deterministic barriers to implementation rather than probabilistic factors. The nature of the intervention may vary, directed by real-time feedback during the course of the trial, and this will need to be taken into consideration during analysis.

As this is a feasibility study, there will be no imputation of missing data over and above any scoring rules established for the outcomes. This information will be used to feed into the suitability and applicability of the chosen outcome measures.

Economic analysis

Alongside the statistical analysis plan, a Health Economics Analysis Plan (HEAP) will be produced setting out the objectives and methods for data collection and analysis of the health economics findings [50]. The HEAP will be signed-off by the lead health economist and CI prior to any data being shared with the health economics researchers. From a NHS perspective, a micro-costing approach will be used to gather sufficient economic data to cost the ThinkCancer! intervention through the use of health economics data collection sheets. In micro-costing, each component of resource use is estimated and then a unit cost is derived using 2019 national reference costs (in Great British Pounds Sterling) where available [51,52,53]. In addition and separately, the costing of the development of the ThinkCancer! intervention will include researcher time, piloting, materials development, printing, publication, development of online materials, etc. Delivery costs of the ThinkCancer! intervention will be determined based on the following:

  • ▪ Online delivery format—live seminars/webinars, staff time and materials, exploration of whether health professional time should be collected in a full trial to reflect the co-production nature of CPD in own time or reflecting the opportunity cost of CPD in terms of time not spent on direct patient care activities.

  • ▪ Mixed-format delivery—potential costs of a face-to-face/online delivery format across Wales in future after COVID-19.

Qualitative analysis

The transcribed telephone interviews, the free text responses from the feedback forms and the observational data, in text form, will be analysed for the process evaluation using Framework Analysis [54]. Framework is a five-stage matrix-based system for analysing qualitative data which is highly appropriate for a feasibility study which is iterative in its development. Initially, all transcripts and textual data will be read thoroughly by the same researcher who conducted the interviews to achieve data familiarisation and immersion. An index of emergent themes will then be created and data coded according to the index. Charts will be created according to the themes and coded data will be synthesised into the appropriate thematic charts. The completed charts will then be used for final stage which is in-depth interpretation [54].

Trial management

The Sponsor is Bangor University. The study will be supported by the North Wales Organisation for Randomised Trials in Health (NWORTH), which is a fully registered Clinical Trials Unit.

There will be no on-site monitoring as there are no local research teams at sites. Therefore, the monitoring of data will have a more internal focus in the form of self-audits to ensure compliance with regulations.

Trial governance

Operational group

The operational working group will be responsible for the overall conduct, supervision and progress of the study. They consist of the immediate research team, supported by a wider group of experts.

Trial management group

The trial management group (TMG) will meet once a month, consisting of the operational group and a wider team of experts, including a PPI member. The group will be responsible for the overall management of the trial and ensuring the study adheres to the protocol.

Trial governance structure

A Trial Steering Committee (TSC) committee will provide independent oversight for the study, ensuring it is conducted according to the standards set out by the HRA Research Governance Framework [55]. As the study includes an element of ongoing intervention refinement and is deemed low risk with very minimal likelihood of stopping early due to patient safety, a Data Monitoring and Ethics Committee (DMEC) will not be required [56].

Meetings are expected to be biannual and the Sponsor and Funder will be updated following each meeting. The TSC will have an independent chairperson (Aneez Esmail) and at least three independent members including Patient and Public Involvement (PPI) representation, trial co-applicants, a statistician (Rhian Gabe), a health economist (Laura Ternent) and other independent members.

Data management

A detailed data management plan will be written by NWORTH staff and approved as soon as possible following commencement of data collection, if not before. This plan will include the definition of the data quality checks that will be performed on the data throughout the life course of the trial. These will include source data validation, random data checks and timelines for data entry.

Quality control

Quality control will be maintained through adherence to the study protocol, Betsi Cadwaladr University Health Board/Bangor University Standard Operating Procedures (SOPs), principles of Good Clinical Practice, research governance and clinical trial regulations.

Data protection and participant confidentiality

All investigators, trial site and research staff will comply with the requirements and regulations of the EU General Data Protection Regulation 2018 (GDPR) regarding the collection, storage, processing and disclosure of personal information and will uphold the Regulation’s core principles. All research staff involved will have up to date GCP training. Research data will be retained as per the Sponsor’s research data management policy. Bangor University is the data custodian.

Data archiving

As per the Sponsor’s research data management policy, research data and records will be archived along with the data management policy of the Sponsor.

In line with legal requirements, trial documents will be archived centrally at a secure facility with appropriate environmental controls and adequate protection from fire, flood and unauthorized access. Archived material will be stored in tamper-proof archive boxes that are clearly labelled. Electronic archiving will be provided by the Sponsor for post-project deposit and retention of data.

Destruction of essential documents will require authorisation from the Sponsor.

Dissemination policy

On completion of the study a final report will be prepared for Cancer Research Wales.

Findings will be disseminated through various media, including open-access peer-reviewed publications, national and international conferences, the programme web pages, social media, and through an end-of-programme symposium for key stakeholders. Findings will also be disseminated to participating practice teams via a newsletter.

Publications arising directly from the WICKED programme and authorship on the final trial report will adhere to the BMJ guidelines on authorship and contribution, based on the International Committee of Medical Journal Editors Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals (ICMJE) 2013 [57].

Patient and public involvement

The study team recognises that the involvement of those with lived experiences will be vital in this research. Furthermore, a lay perspective is essential in the development and undertaking of research for the promotion of equality, diversity and transparency. Two patient representatives were initially recruited to the WICKED programme, one of whom has maintained active involvement in the study design, the development of the protocol and conduct throughout. Additionally, the trial PPI has been active in providing feedback on participant-facing documents. Two more PPI representatives have been recruited to the TSC through the North Wales Cancer Forum and have directly relevant experience. Their perspective as both a patient and a member of the public will inform the overall supervision of the trial.


This study aims to test the feasibility of the ThinkCancer! intervention. The ThinkCancer! study comprises Work Package 4 (WP4) of a programme of research called the Wales Interventions and Cancer Knowledge about Early Diagnosis (WICKED). The intervention will consist of a workshop aimed at the entire general practice team, as previous work packages have demonstrated the value of a whole-practice approach. If the intervention is shown to be feasible, we will proceed with designing a full-scale definitive trial.

One of the key strengths of this intervention is that it can be iteratively developed throughout the study period, which will ensure the future definitive trial will adopt an optimal approach. The design process of the study is also a strength in that a strong multidisciplinary team and advisory groups have been involved throughout. Furthermore, the mixed methods approach will allow us to capture a variety of data from this complex intervention. Although recruitment will be challenging, we hope to maximise it by working closely with various research infrastructures across Wales. Participating practices will be supported throughout the trial through regular communication with the research team, and we will encourage all staff from participating practices to get in touch should they have any queries or concerns about the study. We acknowledge that interventions aiming to increase cancer referrals bring with them a risk of over-diagnosis and over-investigation of patients. However, safety netting can minimise the risk of harm by ensuring referrals are appropriate [7, 29, 32].

Although safety-netting has garnered more attention in recent years, there currently are no recommendations on how best to do it [31]. To our knowledge, there are no interventions targeting primary care with a focus on safety netting. In addition, involving the entire practice is a relatively novel approach, with great potential benefit. This study encompasses a multicomponent and complex behavioural change intervention comprising a multidimensional, interactive, tailored and whole-practice approach, which is timely and needed to optimise primary care approaches to the timely diagnosis of cancer.

Trial status

The trial is currently open for recruitment.

Availability of data and materials

On completion of the trial, the final datasets generated and/or analysed will be available from the corresponding author on reasonable request. Access to the final datasets will be in accordance with governance policies, Good Clinical Practice (GCP) guidelines and funder arrangements.



Two-week wait


Adverse event


Cancer Safety Netting Champion


Cancer Safety Netting Plan


Data Monitoring and Ethics Committee


Good Clinical Practice


General Data Protection Regulations


General Practitioner


Health Care Support Worker


Health Economics Analysis Plan


Health Research Authority


Medical Research Council


National Institute for Health Research


North Wales Organisation for Randomisation Trials in Health


Primary care interval


Primary care provider


Participant Information Sheet


Patient and public involvement


Research and development


Randomised controlled trial


Research Information Sheet for Practices


Research Ethics Committee


Serious adverse event


Standard Operating Procedures


Trial Steering Committee


Wales Interventions and Cancer Knowledge about Early Diagnosis


Work package


  1. CRUK. Cancer Research UK position paper: the diagnostic workforce in Wales. 2018;, 2019.

    Google Scholar 

  2. Nicholson BD, Mant D, Neal RD, Hart N, Hamilton W, Shinkins B, et al. International variation in adherence to referral guidelines for suspected cancer: a secondary analysis of survey data. Br J Gen Pract. 2016;66(643):106–13.

    Article  Google Scholar 

  3. Wales Cancer Network. Cancer delivery plan for Wales 2016-2020. 2016;, 2019.

    Google Scholar 

  4. Rose PW, Rubin G, Perera-Salazar R, Almberg SS, Barisic A, Dawes M, et al. Explaining variation in cancer survival between 11 jurisdictions in the International Cancer Benchmarking Partnership: a primary care vignette survey. BMJ Open. 2015;5(5):e007212.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Butler J, Foot C, Bomb M, Hiom S, Coleman M, Bryant H, et al. The International Cancer Benchmarking Partnership: an international collaboration to inform cancer policy in Australia, Canada, Denmark, Norway, Sweden and the United Kingdom. Health Policy. 2013;112(1-2):148–55.

    Article  PubMed  Google Scholar 

  6. Rubin GP, Berendsen A, Crawford M, Dommett R, Earle C, Emery J, et al. The expanding role of primary care in cancer control. Lancet Oncol. 2015;16(12):1231–72.

    Article  PubMed  Google Scholar 

  7. Hamilton W, Walter FM, Rubin G, Neal RD. Improving early diagnosis of symptomatic cancer. Nat Rev. 2016;13(12):740–9.

    Article  Google Scholar 

  8. Koo MM, Hamilton W, Walter FM, Rubin GP, Lyratzopoulos G. Symptom signatures and diagnostic timeliness in cancer patients: a review of current evidence. Neoplasia. 2018;20(2):165–74.

    Article  PubMed  Google Scholar 

  9. WHO. Key statistics. 2020; Accessed 12 Oct 2020.

    Google Scholar 

  10. Weller D, Menon U, Falborg AZ, Jensen H, Barisic A, Knudsen AK, et al. Diagnostic routes and time intervals for patients with colorectal cancer in 10 international jurisdictions; findings from a cross-sectional study from the International Cancer Benchmarking Partnership (ICBP). BMJ Open. 2018;8(11):e023870.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Jones D, Neal RD, Duffy SRG, Scott SE, Whitaker KL, Brain K. Impact of the COVID-19 pandemic on the symptomatic diagnosis of cancer: the view from primary care. Lancet. 2020;21(603):748–9.

    Article  CAS  Google Scholar 

  12. Looi M, Coombes R. Risky Business: lessons from COVID-19. BMJ. 2020;369:m2221.

    Article  PubMed  Google Scholar 

  13. Spicer J, Chamberlain C, Papa S. Provision of cancer care during the COVID-19 pandemic. Nat Rev Clin Oncol. 2020;17(6):329–31.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Lai A, Pasea L, Banerjee A, Denaxas S, Katsoulis M, Chang WH, et al. [In press] Estimating excess mortality in people with cancer and multimorbidity in the COVID-19 emergency. 2020. Accessed 15 Oct 2020.

  15. Smith CF, Drew S, Ziebland S, Nicholson BD. Understanding the role of GPs’ gut feelings in diagnosing cancer in primary care: a systematic review and meta-analysis of existing evidence. Br J Gen Pract. 2020;70(698):e612–21.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Round T. Primary care and cancer: facing the challenge of early diagnosis and survivorship. Eur J Cancer Care. 2017;26(3):e12703.

    Article  Google Scholar 

  17. Weller D, Vedsted P, Rubin G, Walter FM, Emery J, Scott S, et al. The Aarhus Statement: improving design and reporting of studies on early cancer diagnosis. Br J Cancer. 2012;106(7):1262–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Evans J, Ziebland S, MacArtney JI, Bankhead CR, Rose PW, Nicholson BD. GPs’ understanding and practice of safety netting for potential cancer. Br J Gen Pract. 2018;68(672):505–11.

    Article  Google Scholar 

  19. Michie S, Atkins L, West R. The Behaviour Change Wheel. A guide to designing interventions. Great Britain: Silverback Publishing; 2014.

    Google Scholar 

  20. (Pre-print) Surgey A, Disbeschl S, Lewis R, Hiscock J, Nafees S, Law RJ, et al. ThinkCancer! The multi-method development of a complex behaviour change intervention to improve the early diagnosis of cancer in primary care. medRxiv. 2020:2020.11.20.20235614.

  21. Mansell G, Shapley M, Jordan JL, Jordan K. Interventions to reduce primary care delay in cancer referral: a systematic review. Br J Gen Pract. 2011;61(593):821–35.

    Article  Google Scholar 

  22. Johnson MJ, May CR. Promoting professional behaviour change in healthcare: what interventions work, and why? A theory-led overview of systematic reviews. BMJ Open. 2015;5(9):e008592.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Chauhan BF, Jeyaraman M, Mann AS, Lys J, Skidmore B, Sibley KM, et al. Behavior change interventions and policies influencing primary healthcare professionals’ practice—an overview of reviews. Implement Sci. 2017;12(1):3.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Dikomitis L, Green T, Macleod U. Embedding electronic decision-support tools for suspected cancer in primary care: a qualitative study of GPs’ experiences. Prim Health Care Res Dev. 2015;16(6):548–55.

    Article  PubMed  Google Scholar 

  25. Minchin M, Roland M, Richardson J, Rowark S, Guthrie B. Quality of Care in the United Kingdom after Removal of Financial Incentives. N Engl J Med. 2018;379(10):948–57.

    Article  PubMed  Google Scholar 

  26. Baskerville NB, Liddy C, Hogg W. Systematic review and meta-analysis of practice facilitation within primary care settings. Annal Fam Med. 2012;10(1):63–74.

    Article  Google Scholar 

  27. Schichtel M, Rose PW, Sellers C. Educational interventions for primary healthcare professsionals to promote the early diagnosis of cancer: a systematic review. Educ Prim Care. 2013;24(4):274–90.

    Article  PubMed  Google Scholar 

  28. Blank L, Baxter S, Woods HB, Goyder E, Lee A, Payne N, et al. Referral interventions from primary to specialist care: a systematic review of international evidence. Br J Gen Pract. 2014;64(629):765–74.

    Article  Google Scholar 

  29. Nicholson BD, Mant D, Bankhead C. Can safety-netting improve cancer detection in patients with vague symptoms? BMJ Qual Saf. 2016;355:i5515.

    Google Scholar 

  30. Rubin GP, Saunders CL, Abel GA, McPhail S, Lyratzopoulos G, Neal RD. Impact of investigations in general practice on timeliness of referral for patients subsequently diagnosed with cancer: analysis of national primary care audit data. BJC. 2015;112(4):676–87.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Jones D, Dunn L, Watt I, Macleod U. Safety netting for primary care: evidence from a literature review. BJGP. 2019;69(678):e70–9.

    Article  PubMed  Google Scholar 

  32. Tompson A, Nicholson BD, Ziebland S, Evans J, Bankhead C. Quality improvements of safety-netting guidelines for cancer in UK primary care: insights from a qualitative interview study of GPs. Br J Gen Pract. 2019;69(689):e819–26.

    Article  PubMed  PubMed Central  Google Scholar 

  33. NICE. Suspected cancer: recognition and referral (NG12). 2015;, 2019.

    Google Scholar 

  34. Eldridge S, Lancaster G, Campbell M, Thabane L, Hopewell S, Coleman C, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. Plos One. 2016;11(3):e0150205.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Craig P, Dieppe P, Macintryre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.

    Article  PubMed  PubMed Central  Google Scholar 

  36. NIHR. NIHR Research for Patient Benefit (RfPB) Programme Guidance on Applying for Feasibility Studies. 2017;, 2019.

    Google Scholar 

  37. Lancaster G, Dodd S, Williamson P. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.

    Article  PubMed  Google Scholar 

  38. Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions. UK Medical Research Council (MRC) guidance. 2015. Accessed 12 Dec 2019.

  39. Chan A, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 Statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Heyhoe J, Reynolds C, Lawton R. Developing a safety-netting intervention for the earlier diagnosis of cancer in primary care: the Shared Safety Net Action Plan (SSNAP). Br J Gen Pract. 2019;69(suppl 1):bjgp19X703049.

    Article  Google Scholar 

  42. NCIN. General Practice Profiles for cancer: meta-data for profile indicators. 2010;, 2019.

    Google Scholar 

  43. Meechan D, Gildea C, Hollingworth L, Richards MA, Riley D, Rubin G. Variation in use of the 2-week referral pathway for suspected cancer: a cross-sectional analysis. Br J Gen Pract. 2012;62(602):e590–7.

    Article  PubMed  PubMed Central  Google Scholar 

  44. DIRUM. Database of instruments for resource use measurement. 2019;, 2019.

    Google Scholar 

  45. Russel D, Hoare ZSJ, Whitaker R, Whitaker CJ, Russel IT. Generalized method for adaptive randomization in clinical trials. Statist Med. 2011;30(9):922–34.

    Article  Google Scholar 

  46. Kaplan H, Provost L, Froehle C, Margolis P. The Model for Understanding Success in Quality (MUSIQ): building a theory of context in healthcare quality improvement. BMJ Qual Saf. 2012;21(1):13–20.

    Article  PubMed  Google Scholar 

  47. Kirkpatrick Partners. The Kirkpatrick Model. 2019;, 2019.

    Google Scholar 

  48. May C, Rapley T, Mair FS, Treweek S, Murray E, Ballini L, et al. The normalization process theory on-line user's manual, toolkit and NoMAD instrument. 2015;

    Google Scholar 

  49. ICH-GCP. CH E6 Good Clinical Practice (GCP) Guideline. 2016;, 2019.

    Google Scholar 

  50. Thorn JC, Brookes ST, Ridyard C, Riley R, Hughes DA, Wordsworth S, et al. Core items for a standardized resource use measure: expert Delphi consensus survey. Value Health. 2018;21(6):640–9.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Charles JM, Edwards RT, Bywater T, Hutchings J. Micro-costing in public health economics: steps towards a standardized framework, using the incredible years toddler parenting program as a worked example. Prev Sci. 2013;14(4):377–89.

    Article  CAS  PubMed  Google Scholar 

  52. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 4th edition. Oxford: Oxford University Press; 2015.

  53. Edwards RT, McIntosh E. Applied health economics for public health practice and research. Oxford: Oxford University Press; 2019.

  54. Ritchie J, Spencer L, O'Connor W. Carrying out qualitative analysis. In: Ritchie J, Lewis J, editors. Qualitative Research Practice. A guide for social science students and researchers. London: SAGE Publications Ltd; 2003.

    Google Scholar 

  55. HRA. UK Framework for Health and Social Care Research. 2017;, 2019.

    Google Scholar 

  56. EMEA. Guideline on data monitoring committees. 2005;, 2019.

    Google Scholar 

  57. International Committee of Medical Journal Editors, (ICMJE). Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. Updated December 2019.

Download references


The authors would like to thank Maggie Hendry for her significant contributions to the trial protocol on which this paper is based, as well as her contributions to the development of the intervention, and Stella Wright, for her contributions to the trial protocol and the participant-facing materials. We would also like to acknowledge Nic Nikolic for her work on the study costings. We would like to thank Jan Rose, Brian Lewin and Rosemary Birch, our patient representatives, for their valuable contributions to the development of the protocol and the participant-facing materials. We would also like to acknowledge the contributions from Dr Jane Heyhoe from the Yorkshire and Humber Patient Safety Translational Research Centre and Dr Caroline Green from RedWhale GPUpdate.

Trial sponsor

Bangor University. The Sponsor had no role in the design of this trial and will not have any role during its execution, analyses, data interpretation or decision to submit results.

Contact: Dr Huw Roberts. School of Healthcare Sciences, The Normal Site, Holyhead Road, Gwynedd, LL57 2AS, UK; Tel: 01248 383 136; Email:


Bangor University currently has in place appropriate Clinical Trials Indemnity and Professional Indemnity insurance covering harm to participants arising from the management and/or the design of the research and member of the research team to conduct the research as per protocol. On sites that are not covered by the NHS Indemnity Scheme (e.g. GP surgeries in primary care), investigators/collaborators will need to ensure that their activity on the study is covered under their own professional indemnity. Health and Care Research Wales assisting with recruitment will have NHS contracts and will be responsible for ensuring they are appropriately insured.


This article presents independent research funded by Cancer Research Wales (CRW). The views expressed are those of the authors and not necessarily those of CRW. This funding source had no role in the design of this trial and will not have any role during its execution, analyses, data interpretation or decision to submit results. This research is also linked to the CanTest Collaborative, which is funded by Cancer Research UK (C8640/A23385), of which RDN is an Associate Director.

Author information

Authors and Affiliations



RDN and CW are the Chief Investigators for the study. CW and RDN contributed to all aspects of this work including: conceiving and design of the study, obtaining initial and additional funding, intervention development, management of the study, obtaining approvals and writing the report. SD is the Research Project Support Officer for the study. SD drafted the manuscript integrating comments from JR, AS, AH, NG, CW, RN, NW, RTE, BFA, RL, RJL and ACS. SD was also involved in the development of the intervention and its adaptation in response to COVID-19. AS was involved in the development of the intervention, its adaptation in response to COVID-19 and contributed to the protocol via the intervention description and by developing the logic model. The workshop materials were designed by AS, with assistance and advice from the wider ThinkCancer! team. AS is also the GP Educator and will be delivering the intervention. JR is the trial manager for the study. She oversaw the writing of the protocol, facilitated gaining approvals from the seven health boards and contributed to adaptations of the methodology in response to COVID-19. AH is the qualitative researcher for the study and contributed to the protocol via the embedded qualitative component and design of qualitative data collection materials. AH also contributed to gaining R&D approvals from the seven health boards. RL was involved in the development of the intervention, the study design and the previous work packages leading up to the ThinkCancer trial. RL contributed to the methodological components of the protocol. NG is the Trial Statistician and contributed to the protocol via the statistical component. ZH is the Principal Statistician and will oversee all statistical analysis. ZH was involved in the design of the study and contributed to the protocol via the statistical component and the outcome measures. RTE is Lead Health Economist. BFA is Health Economics Research Support Project Officer and drafted the health economics analysis plan and is responsible for data analysis. RJL was involved in the intervention development and the previous work packages leading up to the ThinkCancer trial. JH was involved in the intervention development and the previous work packages leading up to the ThinkCancer trial. ACS contributed to the design of the study and the development of the intervention. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Stefanie Disbeschl.

Ethics declarations

Ethics approval and consent to participate

This study has been approved by the Health Research Authority/Health and Care Research Wales (IRAS 256824) and Bangor University (School of Healthcare Sciences, 2019-16498). We did not seek REC Approval; this was not required as the study does not involve patients.

Participants are consented at two different time points in the study. Practice managers will consent to participate in the study on behalf of the general medical practice. If the practice has been randomised to the intervention, individual staff participants will be asked for their informed consent at the start of the workshop.

Consent for publication

No participant identifiable information will be published.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:.

WHO Trial Registration Data Set.

Additional file 2:.

ThinkCancer! Logic Model.

Additional file 3.

participant flow diagram.

Additional file 4:.

SPIRIT Checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Disbeschl, S., Surgey, A., Roberts, J.L. et al. Protocol for a feasibility study incorporating a randomised pilot trial with an embedded process evaluation and feasibility economic analysis of ThinkCancer!: a primary care intervention to expedite cancer diagnosis in Wales. Pilot Feasibility Stud 7, 100 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: