Evaluating ‘enhancing pragmatic language skills for young children with social communication impairments’ (E-PLAYS): protocol for a feasibility randomised controlled trial study
Pilot and Feasibility Studies volume 5, Article number: 75 (2019)
A number of children experience difficulties with social communication and this has long-term deleterious effects on their mental health, social development and education. The proposal presented in this article describes a feasibility study for a trial to test an intervention (‘E-PLAYS’) aimed at supporting children with social communication impairments. E-PLAYS harnesses technology in the form of a novel computer game in order to develop collaborative and communication skills. Preliminary studies by the authors show that when E-PLAYS was administered by the research team, children with social communication impairments showed improvements on communication test scores and on observed collaborative behaviours. The study described here is a pragmatic trial to test the application of E-PLAYS delivered by NHS speech and language therapists together with schools.
This protocol outlines a two-arm feasibility cluster-randomised controlled trial of the E-PLAYS intervention with treatment as usual control arm, with randomisation at the level of the speech and language therapist. The aim of this study is to ascertain whether it will be feasible to progress to running a full-scale definitive trial to test the effectiveness of E-PLAYS in an NHS setting. Data relating to recruitment and retention, the appropriateness of outcomes and the acceptability of E-PLAYS to participants will be collected.
Speech and language therapists will select suitable children (ages 4–7 years old) from their caseloads and deliver either the E-PLAYS intervention (experimental group) or treatment as usual (control group). Assessments will include blinded language measures and observations, non-blinded teacher-reported measures of peer relations and classroom behaviour and parent-reported use of resources and quality of life. There will also be a qualitative process evaluation.
The findings of this study will inform the decision as to whether to progress to a full-scale definitive randomised controlled trial to test the effectiveness of E-PLAYS when delivered by speech and language therapists and teaching assistants within schools. The use of technology in game form is a novel approach in an area where there are currently few available interventions.
ISRCTN 14818949 (retrospectively registered).
Children with social communication impairments experience difficulties with using linguistic context to understand speakers’ meanings and, more broadly, with applying conventional norms and expectations from the wider society to relate to others . To illustrate, children with social communication impairments are likely to find it difficult to take turns during conversation, maintain a topic of conversation appropriately, recount coherent narratives, respond contingently, make inferences, understand non-literal language such as jokes, irony or sarcasm, repair communication breakdowns and generally follow conversational conventions. Pragmatic language skill may be defined as selection of the appropriate message or interpretation in relation to the communicative context . The terms ‘social communication’ and ‘pragmatic language’ are commonly used to refer to similar skills.
Recent systematic reviews of interventions for ‘social communication impairments’ or ‘pragmatic language impairments’ [3, 4] have adopted broad definitions for these terms encompassing both verbal and nonverbal aspects of communication to include abilities such as facial expression recognition and production . Whilst the terms ‘social communication’ and ‘pragmatic skill’ have often been used interchangeably in the research literature , for the current proposal, we follow the suggestion by Matthew et al.  and define ‘pragmatics’ more specifically as the linguistic component (excluding facial communication and other non-verbal communication) of social communication.
Children with social communication impairments are a heterogeneous group and encompass both clinical and non-clinical groups. Clinical groups include the categories of ‘Language Impairment’ (also commonly known as ‘Specific Language Impairment’ and, more recently, ‘Developmental Language Disorder’), ‘Social (Pragmatic) Communication Disorder’ and ‘Autism Spectrum Disorders’ as defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM-V; ). ICD 10  as well as listing a number of pervasive developmental disorders (autism, Asperger’s and other pervasive developmental disorders) also describes a number of developmental disorders of language. Children experiencing difficulties with social communication are not limited to those covered by these DSM-V and ICD 10 diagnoses, however. Social communication and pragmatic language difficulties have also been shown to be associated with other clinical populations, notably those with ADHD  or conduct disorder [10, 11] and non-clinical groups such as shy children [12, 13]. There are also indications that social communication disorders are severely under-diagnosed, particularly in children of low socio-economic status [10, 14].
Research concerning children identified as having pragmatic language difficulties demonstrates that effects are far-reaching. Recent studies have shown associations between pragmatic language difficulties and behavioural problems , especially of the disruptive, externalising kind. Healthcare costs (excluding education costs) have been shown to be 36% higher for 4-year-old children with language disorders than for typically developing children . A number of studies have shown long-term negative effects: children with social communication impairments experience problematic peer relationships  and are commonly rejected and victimised by peers at school [18, 19]. Individuals diagnosed with pragmatic language impairments as children were found to experience continuing problems with establishing social relationships (such as friendships) later in adulthood .
A particularly important skill for successful adult functioning, to which social communication is central, is the ability to collaborate with others. Collaborative and team-building skills are recognised as vital to future adult employment and participation in society . A number of collaborative activities between children (such as collaborative learning, cooperative learning or peer tutoring) now commonly take place in classrooms and have generally been judged to be beneficial in terms of improved learning (e.g., [22, 23, 24]). Collaborative working has also been shown to improve peer relations and facilitate children’s feelings of belonging [25, 26]. However, group and dyadic work is not invariably beneficial, only collaborations where children are actively engaged and that are characterised by high-quality questioning, explanation, clarification of ideas, discussion and generally positive affect have been found to be productive .
Given their difficulties, it is to be expected that children with social communication impairments will struggle with collaborative group working and recent studies do indeed bear this out. By comparison to typically developing children in collaborative contexts, they can be aggressive or withdrawn , show more irrelevant behaviours and share less , ignore others’ questions and requests and give poorer directions .
Interventions to support pragmatic language and collaborative working are limited. The use of technology and gaming has been highlighted as a positive tool for facilitating communication and collaboration with peers in children with communication difficulties (e.g. [30, 31]). The application of computer technology for collaborative games to support children’s communication has a number of unique advantages over non-technological approaches. Children with social communication impairments are frequently overwhelmed by the complexity and unpredictability of social interaction. Computer games designed for two or more children to play together can be formatted so that they provide a ‘scaffold’ that guides and structures the interaction between children (for a review, see ). Properly designed, use of a computer game can support communication by structuring collaboration between children so that they solve problems within a fun environment together in partnership [33,34,35]. Typically, in collaborative activities, children with social communication disorders either participate very little, lacking the confidence to become involved or, alternatively, may attempt to dominate and run the activity solely according to their own agenda . Managing these behaviours presents a serious challenge for speech and language therapists, teachers and teaching assistants . Technology can be used to regulate turn-taking, ensuring equal participation between all players and preventing exclusion of any children. Furthermore, rule enforcement and reward-giving (such as virtual ‘prizes’) by a computer, rather than an adult, is viewed by children as more consistent, fairer and less arbitrary . The flexibility afforded can also be harnessed to sustain children’s interest and concentration by adding surprises, colourful animations and unusual sounds and keeping the game flowing at a suitable pace [31, 38].
We have developed and carried out initial testing for a computer game intervention which aims to improve pragmatic language skill and the ability to collaborate in children with social communication impairments. Preliminary efficacy testing for ‘E-PLAYS’ (formerly known as the ‘Maze Game’) is described in Murphy et al. [29, 39] on 32 children randomised either to the intervention or to usual school practice. Children who received the computer game intervention showed significant increases on communication test scores, in the use of high-quality questioning and listening skills and in their positive ratings about collaborative work with peers . It is important to note that for these studies, the intervention was delivered to the children by a trained, post-graduate psychology research assistant who was supervised by the research team. This represents an ‘ideal’ scenario for intervention and it is likely that results would differ from implementation in ‘real-life’ by professionals who would not have the benefit of intensive supervision, time and training. Successful implementation by the National Health Service (NHS) speech and language teams throughout the UK would require the E-PLAYS intervention to translate to a setting in which speech and language therapists and teaching assistants could learn to use it together without research team input or supervision.
Following on from our preliminary work with E-PLAYS [29, 39] therefore, the aim of the present study is to establish the feasibility of running a full-scale clinical trial to evaluate its clinical- and cost-effectiveness when implemented within the National Health Service (NHS) in the UK. Speech and language therapists are employed by and work within the NHS, but their day to day work frequently takes place in the school setting, rather than in clinics, particularly for young children.
This feasibility study will explore potential challenges and aim to develop a protocol for a follow-on large-scale RCT. Specific objectives include testing the recruitment procedure and recording the number of participants recruited over time. An important aspect will focus on determining treatment fidelity; E-PLAYS will be delivered by teaching assistants who have been trained by NHS speech and language therapists. We will also aim to establish the acceptability of E-PLAYS with speech and language therapists and with teachers, teaching assistants, parents and children and also explore the acceptability of randomisation to a control group. We will also seek to determine the feasibility of collecting resource-use data and quality of life measures for the purpose of calculating relative cost-effectiveness.
The design is a two-arm feasibility cluster-randomised controlled trial (cRCT) with randomisation at the level of the speech and language therapist (SLT). Each participating child receiving the E-PLAYS intervention will be a child with social communication impairments. For each participating child, there will be an associated teacher and parent who will complete questionnaires, a speech and language therapist and teaching assistant who will deliver the intervention and a typically developing child from the same class who will partner the child to play the E-PLAYS game. For ease of reference, these will be called ‘participant groups’; each participant group comprises one child with communication impairment, plus one parent, one teacher, one SLT, one teaching assistant and one typically developing partner child. The randomisation will occur at the level of the SLT because once an SLT has become familiar with E-PLAYS, it is possible that their approach to children with social communication difficulties may be influenced. Therefore, SLTs will receive immediate training if randomised to the intervention group and if allocated to the control group, will receive training after all study measures have been completed.
Participating children in the intervention arm will receive the E-PLAYS intervention while participating children in the control arm will receive treatment as usual. The trial will be pragmatic in nature with the interventions being delivered in the way that they would be delivered within the NHS. The effectiveness of E-PLAYS will not be addressed in this feasibility trial but left to a subsequent full-scale RCT, with the key outcomes of the present study relating to the feasibility of conducting such an RCT. Assessment of these feasibility outcomes will inform whether a full-scale trial is viable and inform its design and implementation.
We will recruit SLTs and suitable children on their caseloads from within North East London NHS Foundation Trust (NELFT) Speech and Language Services.
Participants will comprise the following in a ‘participant group’ comprising one child with social communication impairment (SCI) with associated relevant others:
SLTs from NELFT
Participating children (ages 4–7 years old) with SCI as selected by SLTs ‘focal children’
Teaching assistants trained to use E-PLAYS by participating SLTs
Teachers who will complete pre-, post- and follow-up test measures relating to participating children
‘Partner’ children; classmates for the focal children for the E-PLAYS intervention who will play the dyadic collaborative computer games that comprise E-PLAYS with them.
Parents of the focal children who will complete post-test measures for health economics.
Selection, recruitment and consenting of participants
The procedure for distribution of information sheets and consent forms for the different kinds of participants is described below. In all cases, potential participants will be given at least 48 h in which to make a decision.
SLTs from NELFT
The research team will approach suitably employed paediatric SLTs, that is SLTs working with children aged 4- to 7-years-old attending mainstream (not special) schools. The SLTs will be invited to a presentation by the team, will have the opportunity to ask questions about the study and will be given information sheets and consent forms to take away to read. Consent forms can be returned by post, electronically or by hand to the research team. SLTs not responding within a week will be reminded and invited again by email once more only.
Participating children with social communication impairments (SCI) ‘focal children’.
SLTs will review their caseloads and identify suitable children using the Social Communication Behaviour Checklist devised by Adams et al. . The checklist requires the SLT to decide on the applicability of each of the following five statements to a given child:
The child has trouble understanding and interpreting the social context and friendship, e.g. social roles, emotions
The child has trouble understanding and/or using non-verbal aspects of communication, e.g. facial expression, intonation
The child has trouble with aspects of conversation, e.g. beginning and ending, taking turns, giving relevant and sufficient information
The child makes bizarre, tangential or inappropriate comments
The child has difficulty using and understanding non-literal language
Children who are judged as having at least two of the five statements applying to them will be considered eligible for inclusion in the study provided they also meet the following additional criteria:
Four- to seven-years-old at the time of the SLT’s assessment of eligibility
Have at least minimum levels of English (including any children with English as an additional language)
Not suffering with a hearing, visual or physical impairment severely affecting speech production
Once eligibility is confirmed, the research team will provide schools with participant information sheets to send to parents, one written for parents and one written in simpler terms for the child to share with their parent, as well as a consent form for parents to sign.
Teaching assistants trained to use E-PLAYS by participating SLTs
Once the children with SCI have been selected by SLTs and consent has been received from parents, the teaching assistants who are supporting these children will be invited to take part in the study and will receive written information sheets and consent forms at school from the research team.
Teachers who will complete pre-, post- and follow-up test measures relating to participating children
Teachers of recruited children will also be invited to take part in the study and will receive written information sheets and consent forms at school from the research team.
‘Partner children’ for focal children for the E-PLAYS intervention
‘Partner children’ will be suggested by teachers and teaching assistants in consultation with the relevant SLT. These children will be typically developing children without language disorders in the same class as the focal children. E-PLAYS comprises a series of dyadic computer games, some of which are played with an adult and some with the partner child. The importance of the role that peers can play in interventions has been the subject of a recent systematic review by ). These authors concluded that the inclusion of peers was one of the most promising strategies for intervention and resulted in positive impacts on social communication. An important aspect of peer-mediated interventions concerns the peers chosen to interact with focal children. These peers can provide appropriate language models and also an opportunity for children with SCI to practice newly acquired skills. In general, it has been reported that teachers children who are popular, prosocial, and self-confident as peer partners for interventions . Locke et al.’s longitudinal study  did not report any adverse outcomes for typically developing children as a result of participating in a peer-mediated intervention. An important feasibility question will concern recruitment of these children and willingness of parents to allow participation. Schools will send participant information sheets to parents of the peer partner children, one written for parents and one written in simpler terms for the child to share with their parent (as above for children with social communication impairment), as well as a consent form for parents to sign.
Parents of the children with SCI who will complete post-test measures for health economics.
Parents will be asked to consent to completion of these forms at the same time as they will be asked to provide consent on behalf of their children (b) above.
Whilst research assistants will pass on information sheets to participants via schools and request the return of consent forms, all participants will be given the option to contact the Chief Investigator in the event of additional questions. All participants will be free to withdraw at any time from the protocol treatment without giving reasons and without affecting their usual standard of care. The Chief Investigator will preserve the confidentiality of participants taking part in the study and is registered under the Data Protection Act.
Randomisation and allocation process
Participating children with SCI will be cluster-randomised at the level of the SLTs. SLTs will be randomised 1:1 to receive either immediate briefing from the research team together with the manual on the E-PLAYS intervention or at to receive this at the end of the trial. Randomisation will take place after SLTs have consented to participate in the study, and have identified and recruited children on their caseloads to participate in the trial but before they receive briefing. Allocation will be via minimisation to ensure balance across the two groups based on the borough of the SLT (there are five boroughs within the district covered by NELFT, each served by a different SLT team) and number of children recruited (dichotomised around the median number of children recruited to form a two level factor). All SLTs will be randomised together as opposed to having a rolling recruitment/randomisation period. The minimisation will be implemented by the trial statistician at the York Trials Unit using MinimPy version 0.3. Once the allocations have been generated, they will be communicated to members of the research team responsible for training the SLTs, while ensuring that the outcome assessors (research assistants) remain blind to these allocations.
Parent- and teacher-completed measures cannot be blinded; however, measures collected by research assistants (RAs) will be blinded. Teachers and teaching assistants at the schools will be reminded not to reveal allocations to the research assistants at every visit. For the qualitative data collection (Fun Tool Kit, open-ended training questionnaires, observations and focus groups) in which blinding is not possible, research assistants will not collect data from the same schools in which they are collecting quantitative data in order to preserve blinding.
The SLTs in the intervention group will receive training from the research team and will then, in turn, train teaching assistants in schools to deliver the E-PLAYS intervention to the children. Training for SLTs will be kept to the minimum as the aim of this feasibility study is to explore the feasibility of implementing E-PLAYS within the NHS nationally without research team support. Therefore, SLTs will be introduced to the E-PLAYS game and to the manual only. The SLTs will use a training manual with which to train teaching assistants which covers the delivery of E-PLAYS in depth. They will train teaching assistants for around 1 h.
Quantitative outcome measures
In addition to basic demographic data on age, gender and ethnicity, selected quantitative measures have been chosen for sound psychometric properties (validity, reliability, internal consistency, inter-rater reliability). The specific feasibility outcomes to ascertain recruitment, measure treatment fidelity, test acceptability of both the intervention and of the measures used, and test the feasibility of data collection for language and communication measures, social and mental health measures, generalisation and cost effectiveness for an economic evaluation are given below, with details of the instruments, scales, assessments and interviews to be used:
(a) Recruitment measures
A recruitment log will be kept to determine participation, drop-out and completion rates to inform a possible future full-scale trial.
(b) Language and communication outcome measures
(i) The Children’s Communication Checklist-2, (CCC-2, )
The CCC-2 is the most widely used, standardised questionnaire of communication impairment in research and clinical contexts and will be completed by teachers.
(ii) Test of Pragmatic Skills, (TPS, ).
This is an observational elicitation measure. The tester (RA) engages the child in structured play in which test questions are embedded. The TPS was a sensitive indicator and successful at detecting improvements in communication in our pilot study; it has been standardised on 650+ children by the author. It will be administered by blinded RAs.
(iii) Clinical Evaluation of Language Fundamentals-5 (CELF-5, ).
We will use one of the subscales; recalling sentences. This sentence repetition assessment is a frequently used measure and is generally regarded as a measure of overall language ability drawing upon a wide range of language processing skills . This will also be administered by blinded RAs.
(c) Social behaviour, friendship and mental health measure
The Strengths and Difficulties Questionnaire
The Strengths and Difficulties Questionnaire (SDQ) is regarded as the ‘gold standard’ measure and is widely used as a mental health indicator with subscales assessing behavioural, emotional and peer problems. This will be completed by teachers .
(d) Measure of generalisation to social contexts
Dyadic collaborative construction task (Magformers®)
A frequently reported issue with interventions targeting children with SCI is that the skills learned do not generalise to contexts beyond those of the intervention. In order to explore whether communication skills learned through E-PLAYS do in fact generalise to other contexts, children will be video-recorded undertaking a typical classroom collaborative task with a peer. The task we will ask the children to complete involves constructing a Magformers® model following a set of video-recorded instructions; this will take around 10 min. Magformers® are plastic, brightly coloured, magnetised blocks (similar in size to Lego® but different in shape and operation) with which it is possible to construct small models (see magformers.co.uk). All children in intervention and control groups will undertake this task and be recorded; children’s use of different kinds of communication will be observed and coded from the video-recording. The Magformers® construction task will be facilitated and video-recorded by blinded RAs who will also undertake coding of the transcripts from the video-recordings.
(e) Health-economic measures
(i) Bespoke Questionnaire
A bespoke questionnaire designed by the team’s health economist based on previous work in studies with children will measure the child’s health services-use (parent-completed, non-blinded).
For the feasibility study, we will not undertake a full economic evaluation; we will assess intervention and trial costs only. We will estimate research and implementation costs of undertaking a full trial.
Data collection points for outcome measures
Data collection for the TPS, CCC-2, CELF-5 Recalling Sentences subscale, SDQ and Magformers® construction task will take place at baseline, 15–20 weeks and 35–40 weeks post-randomisation. These time points are equivalent to baseline, immediate post-intervention and 3-month follow-up for the intervention group. All health economic measures will be collected at 35–40 weeks post-randomisation (see Fig. 1. SPIRIT figure for E-PLAYS trial for data collection timings). Data will be collected at three times points from teachers, parents and the children themselves. As we are working with young children who may tire easily, measures will be collected in two to three short (< 30 min) sessions when necessary.
As recommended by the Medical Research Council (MRC) guidance for RCTs , we will supplement quantitative feasibility testing of E-PLAYS with a qualitative process evaluation which will examine the processes involved in intervention delivery. The processes we will explore are: whether instructions for delivery of E-PLAYS are adequate and clear; whether staff in a real-life NHS and school settings can use E-PLAYS faithfully; how acceptable staff find E-PLAYS; and whether children engage with and enjoy E-PLAYS. By conducting this process evaluation, we hope we may identify any possible influencing mechanisms or unintended negative effects not revealed by quantitative measures. In the event that, for example, we fail to recruit to target, or that fidelity appears compromised, the process evaluation may be illuminative. The process evaluation that forms part of this feasibility trial aims to investigate the following processes linked to implementation:
(a) Speech and language therapists’ and teaching assistants’ views
Focus groups with the speech and language therapists and teaching assistants who delivered E-PLAYS will be conducted once delivery is complete, and their views sought on acceptability and ease of use.
(b) Children’s views on E-PLAYS
The children’s view will be solicited using the Fun Toolkit  (which has been designed to measure children’s views of technology) by an RA immediately after each of two sessions.
(c) Fidelity evaluation
On-line recording of the number of intervention sessions completed for each child and their content and duration in minutes will be automatically logged by the E-PLAYS software. For the E-PLAYS computer game, each child will have their own unique login, which will record the number of sessions undertaken, the time that they lasted and which parts of the programme were completed.
E-PLAYS sessions will be observed live by RAs to assess how faithfully teaching assistants actually use it in practice.
We will also explore fidelity issues in the focus groups described above.
(d) Training and instructions
Shortly after receiving the instruction pack and manual, SLTs will be asked to complete a short, open-ended questionnaire giving their impressions of its utility and clarity. Teaching assistants will receive a similar questionnaire after they have received the manual and instruction from SLTs (Fig. 2).
Recruitment data analysis
The number of children recruited and the rate of attrition will be reported by allocation and in total. The number of E-PLAYS sessions completed and their duration will be summarised descriptively for children in the intervention arm.
Analysis of outcome measures
A detailed analysis plan will be produced prior to the final analysis and will be reviewed by the independent trial steering committee. Analyses will follow intention to treat principles, with all data analysed as randomised. Analyses will be entirely descriptive with scores on the TPS, CCC-2, CELF-5 Recalling Sentences Subscale and Magformers® construction task presented unadjusted at each time point.
For the Magformers® construction task, video-recordings will be professionally transcribed and video-coding will comprise observation and categorisation of communicative (e.g. questions, directives, clarifications) and affective (e.g. positive and negative behaviours) items. The coding system is based on micro-analytic theory [52, 53] and was developed by the team specifically to analyse children’s collaborative interaction and has been shown in our studies to successfully detect changes in talk [29, 39, 54, 55]. The coding system incorporates elements from research into collaborative learning, conversation analysis and language impairment [29, 39, 53,54,55,56,57,58,59,60]. Video interaction will be coded by blinded RAs trained to fidelity by the Chief Investigator who has used this coding system extensively. RAs will train by watching and practice-coding training videos until they reach inter-rater reliability with these previously coded video-recordings. Areas of disagreement with the training videos will be discussed and explained by the Chief Investigator. Inter-rater reliability will further be tested between all RAs taking part in the study using a sample of 15% of all video-recordings.
The analysis of all measures will not be concerned with the effectiveness of the study intervention, but instead will seek to inform the feasibility objectives. In particular, this data will be used to characterise the groups at baseline, assess the proportion of analysable data at follow up and provide estimates of key population parameters, all of which will inform the design of a future full scale trial. The analysis of the health economic data will be principally concerned with the completeness and quality of this data. This will be reported together with a calculated estimate of the costs of running E-PLAYS within the NHS and schools and costs of a possible future full-scale RCT.
Process evaluation analysis
(a) Speech and language therapists’ and teaching assistants’ views
The focus groups (a) will last around 1 h and will be audio-recorded and subsequently professionally transcribed. Nvivo 9 software will be used to aid a thematic analysis which will follow the guidelines of Braun and Clarke :
(1) Becoming familiar with the data, (2) generating initial codes, (3) searching for themes, (4) reviewing themes and (5) defining and naming themes.
(b) Children’s views on E-PLAYS
Immediately after participating in sessions, children will be asked to rate their enjoyment and use of E-PLAYS using the Fun Toolkit . The tool is easy to complete and requires no writing. It comprises a ‘Smileyometer’ scale; visual analogue scale with ‘smiley’ faces, ‘Fun Sorter’ (children sort which attributes of the game they like best) and ‘Again Again table’ (children rate which aspects of the game they would like to play again). The tool measures, besides user satisfaction, usability (in software terms) and ease of use. The Fun Toolkit has shown good test-retest reliability in previous research . The Smileyometer, Fun Sorter and Again Again table produce numerical data for scoring; however, they are also designed to stimulate and encourage children to talk about the different aspects of the game and their enjoyment.
(c) Fidelity evaluation
(i) Number of intervention sessions
Descriptive statistics will be prepared of the number of intervention sessions completed for each child and their content and duration. The possible impact of treatment fidelity and adherence on outcomes will be explored.
Teaching assistants will be observed by RAs whilst delivering two sessions each to see how they actually deliver E-PLAYS in practice. A crucial aspect of the intervention is the extent to which the instructions in the manual are adhered to. Observers will also record instances of direct verbal feedback and children’s requests for help and comments.
The data from the open-ended questionnaire given to SLTs and teaching assistants will be analysed by coding responses into categories known as a ‘coding frame’ . Given that this is a feasibility study and therefore exploratory in nature, the coding frame will be developed after the data is collected to cover a range of potential responses.
(a) Quantitative measures
It is not intended that this feasibility study will be powered to detect clinical differences between intervention and treatment as usual groups, this is left for the main trial. We are proposing to collect data on outcomes that will be used to inform a future large-scale trial. Sample sizes of between 24 and 70 have been recommended for feasibility trials to allow for the reliable estimation of a standard deviation for use in future sample size calculations [63, 64]. We therefore aim to recruit 70 children (35 in each group). We anticipate that attrition will be low; this is usually the case for school-based studies with young children, e.g. in our pilot study , we lost only one child over a follow-up period of 6 months. However, even with 20% loss to follow-up, we will retain 56 children, which is more than double the recommended minimum sample size. This number will also be sufficient to obtain a reasonably precise estimate of the intra-cluster correlation coefficient for the children grouped by SLT.
(b) Qualitative evaluation
All SLTs and teaching assistants in the intervention group (around seven speech and language therapists and 19 teaching assistants) will receive the open-ended questionnaire. Ten teaching assistants will be observed live delivering E-PLAYS and ten children will give their views using the Fun Toolkit. There will be four focus groups with four to five speech and language therapists or teaching assistants in each group.
The protocol outlined in this article describes a feasibility study for a trial to test an intervention (‘E-PLAYS’) aimed at supporting children with social communication impairments.
E-PLAYS uses novel computer technology to support and stimulate children to improve communication whilst collaborating with their peers. The protocol is based on our earlier pilot studies [29, 39], indicating that the intervention was efficacious when delivered by the research team. Whilst efficacy for E-PLAYS looks promising, it is important to remember that implementation in a ‘real-world’ context brings with it a number of challenges not immediately apparent in the closely controlled research setting of a pilot study. Thus, the study described here aims to investigate a number of important feasibility questions before proceeding to a full-scale trial. The context for the implementation of E-PLAYS is the NHS in the UK. NHS treatment for children with social communication disorders is the responsibility of SLTs. SLTs are specialist health professionals and frequently adopt a ‘consultation model’ whereby they provide training for teaching assistants in schools to deliver interventions rather than providing intervention directly themselves . Therefore, E-PLAYS will fit well within typical NHS service delivery if it proves effective when provided via the consultation model and exploring the means of evaluating this is the aim of the present study.
The first aim of the feasibility study is to establish whether sufficient participants can be recruited for a full trial. Recruitment is complex, involving as it does a number of the adults concerned with each child. We will aim to determine whether sufficient numbers of SLTs, children and their parents, teachers and teaching assistants can be recruited and over what period of time for a definitive large-scale RCT.
The second major potential challenge to implementation in the NHS concerns fidelity. E-PLAYS is fully manualised, but is the manual sufficiently clear and comprehensive to ensure faithful delivery? Unlike in our previous pilot studies, should E-PLAYS be distributed nationally, it would not be viable for the research team to train numerous speech and language teams throughout the country. A key research task therefore involves investigating aspects of the indirect delivery whereby training will be provided to teaching assistants by SLTs. This differs fundamentally from delivery by a trained and supervised RA as in our pilot studies and feedback from SLTs and teaching assistants will be crucial.
Other challenges relate more broadly to difficulties with research concerning children with social communication impairments. SCI is notoriously difficult to measure  as it generally manifests itself only during dynamic social interaction, thus rendering testing with standardised questionnaires largely unachievable. A number of different approaches have been used to address this issue . One approach is to ask teachers or another adult who knows the child to rate their pragmatic language. In this regard, the CCC-2  is the most commonly used research measure. However, it is possible that global reports from adults may not be sensitive enough to detect changes resulting from interventions . Furthermore, adult (particularly parent) report for conditions relating to autism and social communication may be especially susceptible to placebo effects [66, 67]. Another approach is to use an elicitation test, that is, to use a series of structured but naturalistic contexts which are designed to prompt particular communication behaviours for observation. The TPS is a well-validated elicitation measure for the age group targeted by E-PLAYS. Finally, observation and analysis of children’s spontaneous interactions using coding derived from conversation analysis  should provide an ecologically valid indication of any improvement. We will be video-recording children’s communications whilst playing with Magformers. The study will therefore incorporate all three of the approaches (adult report—CCC-2, TPS—elicitation test and video-observation) normally used to measure social communication to minimise the well-documented difficulties with measuring this construct. This is a particular strength of the study. In addition, the observational data from the Magformers task will provide a measure of generalisation, that is, an indication of whether the skills and strategies learned during the E-PLAYS intervention transfer to other, analogous collaborative contexts. Further, more distal measures of generalisation are included in the form of the SDQ which will measure the potential impact of communication changes on peer relations and classroom behaviour.
Ultimately, there is high potential for E-PLAYS to make a positive impact on children’s language learning. The intervention, being a computer game, can easily be scaled up and relies on simple technology available in all schools and NHS trusts. In the long-term, should the study progress to a full-scale RCT and demonstrate effectiveness, benefits to children with social communication impairments could be substantial and could include enhanced communication, more productive peer collaboration and improved classroom relations.
The current study status is that ethical approval was obtained on 4th December 2017 (Cambridge Central Research Ethics Committee, REC ref: 17/EE/0320, IRAS Project ID: 227864). The study opened to recruitment on 1st January 2018 and completed recruitment on 30th March 2018. Intervention delivery is currently proceeding.
Children’s Communication Checklist-2
Clinical Evaluation of Language Fundamentals-5
Diagnostic and Statistical Manual of Mental Disorders
Enhancing Pragmatic Language skills for Young children with Social communication impairments
EuroQol 5-Dimension scale
Medical Research Council
North East London NHS Foundation Trust
National Health Service
Paediatric Quality of Life inventory
Social communication impairment
Strengths and Difficulties Questionnaire
Speech and language therapist
Test of Pragmatic Skills
Norbury CF. Practitioner review: social (pragmatic) communication disorder conceptualization, evidence and clinical implications. J Child Psychol Psychiatry. 2014;55(3):204–16.
Bishop DVM. Uncommon understanding: development and disorders of language comprehension in children. Hove: Psychology Press; 1997.
Parsons L, Cordier R, Munro N, Joosten A, Speyer R. A systematic review of pragmatic language interventions for children with autism spectrum disorder. PLoS One. 2017;12(4):e0172242.
Wieckowski A, White S. Application of technology to social communication impairment in childhood and adolescence. Neurosci Biobehav Rev. 2017;74:98–114.
Insel T, Cuthbert B, Garvey M, Heinseen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatr. 2010;167(7):748–51.
Matthews D, Biney H, Abbot-Smith K. Individual differences in children’s pragmatic ability: a review of associations with formal language, social cognition, and executive functions. Language Learning Dev. 2018;14(3):186–223.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Washington: American Psychiatric Association; 2013.
World Health Organisation. Multiaxial classification of child and adolescent psychiatric disorders. Cambridge: Cambridge University Press; 1996.
Bishop DV, Baird G. Parent and teacher report of pragmatic aspects of communication: use of the children’s communication checklist in a clinical setting. Dev Med Child Neurol. 2001;43:809–18.
Donno R, Parker G, Gilmour J, Skuse D. Social communication deficits in disruptive primary-school children. Br J Psychiatry. 2010;196:282–9.
Gilmour J, Hill B, Place M, Skuse D. Social communication deficits in conduct disorder: a clinical and community survey. J Child Psychol Psychiatry. 2004;45(5):967–78.
Coplan RJ, Weeks M. Shy and soft-spoken: shyness, pragmatic language, and socioemotional adjustment in early childhood. Infant Child Dev. 2009;18:238–54.
Mewhort-Buist TA, Nilsen ES. What are you really saying? Associations between shyness and verbal irony comprehension. Infant Child Dev. 2013;22:180–97.
Bishop DVM, McDonald D. Identifying language impairment in children: combining language test scores with parental report. Int J Lang Commun Disord. 2009;44(5):600–15.
Ketelaars MP, Cuperus J, van Dall J, Jansonius K, Verhoeven L. Pragmatic language impairment and associated behavioural problems. Int J Lang Commun Disord. 2010;45:204–14.
Sciberras E, Westrupp EM, et al. Healthcare costs associated with language difficulties up to 9 years of age. Int J Speech Lang Pathol. 2015;17(1):41–52.
Weismer SE. Specific language impairment. In: Cummings L, editor. Communication disorders. New York: Cambridge University Press; 2013. p. 73–87.
Laws G, Bates G, Feuerstein M, Mason-Apps E, White C. Peer acceptance of children with language and communication impairments in a mainstream primary school: associations with type of language difficulty, problem behaviours and a change in placement organization. Child Lang Teach Ther. 2012;28(1):73–86.
Mok PLH, Pickles A, Durkin K, Conti-Ramsden G. Longitudinal trajectories of peer relations in children with specific language impairment. J Child Psychol Psychiatry. 2014;55(5):516–27.
Whitehouse AJO, Watt HJ, Line EA, Bishop DVM. Adult psychosocial outcomes of children with specific language impairment, pragmatic language impairment and autism. Int J Lang Commun Disord. 2009;44(4):511–28.
Howe C. Peer groups and children’s development. Chichester: Wiley-Blackwell; 2010.
Azmitia M. Peer interactive minds: developmental, theoretical and methodological issues. In: Baltes PB, Staudinger UM, editors. Interactive minds: life-span perspectives on the social foundation of cognition. New York: Cambridge University Press; 1998. p. 133–62.
Christie D, Tolmie A, Thurston A, Howe C, Topping K. Supporting group work in Scottish primary classrooms: improving the quality of collaborative dialogue. Camb J Educ. 2009;39(1):141–56.
Johnson DW, Johnson RT. An educational psychology success story: social interdependence theory and cooperative learning. Educ Res. 2009;38:365–79.
Johnson-Pynn JS, Nisbet VS. Preschoolers effectively tutor novice classmates in a block construction task. Child Study J. 2002;32:241–55.
Tolmie A, Topping K, Christie D, Dolnaldson C, Howe C, Jessiman E, et al. Social effects of collaborative learning in primary schools. Learn Instr. 2010;20(3):177–91.
Brinton B, Fujiki M, Montague EC, Hanton JL. Children with language impairment in cooperative work groups: a pilot study. Lang Speech Hear Serv Sch. 2000;31:252–64.
Kimhi Y, BaumingerZviely N. Collaborative problem solving in young typical development and HFASD. J Autism Dev Disord. 2012;42:1984–97.
Murphy SM, Faulkner DM, Farley LR. The behaviour of young children with social communication disorders during dyadic interaction with peers. J Abnorm Child Psychol. 2014;42(2):277–89.
Holt S, Yuill N. Facilitating other-awareness in low-functioning children with autism and typically-developing pre-schoolers using dual-control technology. J Autism Dev Disord. 2014;44:236–48.
Ploog BO, Scharf A, Nelson D, Brooks PJ. Use of computer-assisted technologies (CAT) to enhance social, communicative, and language development in children with autism spectrum disorders. J Autism Dev Disord. 2013;43:301–22.
Fischer F, Kollar I, Mandl H, Haake JM. Scripting computer-supported collaborative learning. New York: Springer; 2007.
Cress U, Wodzicki K, Bientzle M, Lingnau A. Computer–supported collaborative learning for intellectually disabled pupils: stimulating interaction by using a floor control mechanism. Comput Support Collab Learn. 2011;6:307–21.
Gal E, Lamash L, Bauminger-Zviely N, Zancanaro M, Weiss PL. Using multitouch collaboration technology to enhance social interaction of children with high-functioning autism. Phys Occup Ther Pediatr. 2016;36(1):46–58.
Piper AM, O'Brien E, Morris MR, Winograd TSIDES. A cooperative tabletop computer game for social skills development. The 20th anniversary conference on computer supported cooperative work, 4–8 November Banff. Alberta: ACM; 2006. p. 1–10.
Baines E, Blatchford P, Kutnick P. Promoting effective group work in the primary classroom. London: Routledge; 2009.
Ben-Sasson A, Lamash L, Gal E. To enforce or not to enforce? The use of collaborative interfaces to promote social skills in children with high functioning autism spectrum disorder. Autism. 2013;17(5):608–22.
Grynszpan O, Weiss P, Perez-Diaz P, Gal E. Innovative technology-based interventions for autism spectrum disorders: a meta-analysis. Autism. 2014;18(4):346–61.
Murphy SM, Faulkner DM, Reynolds LR. A randomised controlled trial of a computerised intervention for children with social communication difficulties to support peer collaboration. Res Dev Disabil. 2014;35(11):2821–39.
Adams C, Lockton E, Freed J, Gaile J, Earl G, McBean K, et al. The social communication intervention project: a randomized controlled trial of the effectiveness of speech and language therapy for school-age children who have pragmatic and social communication problems with or without autism spectrum disorder. Int J Lang Commun Disord. 2012;47(3):233–44.
Chang Y-C, Locke J. A systematic review of peer-mediated interventions for children with autism spectrum disorder. Res Autism Spectr Disord. 2016;27:1–10.
Locke J, Rotheram-Fuller E, Kasari C. Exploring the social impact of being a typical peer model for included children with autism spectrum disorder. J Autism Dev Disord. 2012;42:1895–905.
Bishop DVM. The children’s communication checklist. 2nd ed. London: Harcourt Assessment; 2003.
Shulman BB. Test of Pragmatic Skills–Revised. Tucson: Communication Skill Builders; 1986.
Wiig EH, Semel E, Secord WA. Clinical Evaluation of Language Fundamentals®-Fifth Edition (CELF®-5). Bloomington: Pearson; 2013.
Klem M, Melby-Lerva M, Hagtvet B, Halaas Lyster S, Gustafsson J, Hulme C. Sentence repetition is a measure of children’s language skills rather than working memory limitations. Dev Sci. 2015;18(1):146–54.
Goodman R. Psychometric properties of strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40:1337–45.
Wille N, Badia X, Bonsel G, Burström K, Cavrini G, Devlin N, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Qual Life Res. 2010;19(6):875–86.
Varni JW, Seid M, Rode CA. The PedsQL™: measurement model for the pediatric quality of life inventory. Med Care. 1999;37(2):126–39.
Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J. 2015;350:h1258.
Read JC. Validating the fun toolkit: an instrument for measuring children’s opinions of technology. Cogn Tech Work. 2008;10(2):119–28.
Bull P. Communication under the microscope: the theory and practice of microanalysis. Hove: Routledge; 2002.
Markell R, Asher S. Children's interactions in dyads: interpersonal influence and sociometric status. Child Dev. 1984;55:1412–24.
Murphy S, Faulkner D. Gender differences in verbal communication between popular and unpopular children during an interactive task. Soc Dev. 2006;15(1):83–109.
Murphy S, Faulkner D. The relationship between bullying roles and children's everyday dyadic interactions. Soc Dev. 2011;20(2):272–93.
Anderson AH, Clark A, Mullin J. Interactive communication between children: learning how to make language work in dialogue. J Child Lang. 1994;21:439–63.
Gottman JM, Parker JG, editors. Conversations of friends. Cambridge: Cambridge University Press; 1986.
Kruger A. Peer collaboration: conflict, co-operation or both. Soc Dev. 1993;2:165–82.
Lloyd P, Boada H, Forns M. New directions in referential communication research. Br J Dev Psychol. 1992;10:385–403.
Radziszewska B, Rogoff B. Influence of adult and peer collaborators on children’s planning skills. Dev Psychol. 1988;24:840–8.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.
Bryman A. Social research methods. Oxford: Oxford University Press; 2016.
Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat. 2005;4:287–91.
Teare DM, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials. 2014;15:264.
Ebbels SH, McCartney E, Slonims V, Dockrell JE, Norbury CF. Evidence-based pathways to intervention for children with language disorders. Int J Lang Commun Disord. 2019;54(1):3–19. https://doi.org/10.1111/1460-6984.12387.
Lord C, Wagner A, Rogers S, Szatmari P, Aman M, Charman T, et al. Challenges in evaluating psychosocial interventions for autistic spectrum disorders. J Autism Dev Disord. 2005;35(6):695–708.
Smith T, Scahill L, Dawson G, Guthrie D, Lord C, Odom S, et al. Designing research studies on psychological interventions in autism. J Autism Dev Disord. 2007;37(2):354–65.
Sidnell J. Conversation analysis. Chichester: Wiley-Blackwell; 2010.
We are very grateful for the support and guidance of our Trial Steering Committee, including Professor Chloe Marshall of University College, London who is the independent chair and Dr. Susan Smith of Essex University, Dr. Allan Clark of University of East Anglia and Katherine Farrow, Highly Specialist Speech and Language Therapist, our independent members, also Corinne Fox, our parent representative and Mary Hartshorne, Director of Outcomes and Information at I CAN, the UK’s communication charity. We are also grateful to the on-going support of NELFT in particular Fiona Horton, Research Business Operations Manager, Sandeep Toot, deputy director of R&D and Nita Madhani, Assistant Director for Children and Young People.
This article presents independent research funded by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-0416-20035). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.
Availability of data and materials
The datasets generated during and/or analysed during the current study will be stored in non-publically available repositories within the Universities of York and Bedfordshire by the end of 2019. Anonymised data may be accessed and analysed by members of the project team and with researchers collaborating with members of the project team on the analysis of these data. Consent from participants was not sought for sharing raw data publicly. Therefore, external researchers seeking to access the data for use in future projects can do so via any reasonable request to the Chief Investigator Suzanne Murphy (or her delegate), and projects using the data must have been approved in accordance with contemporary UK ethical and regulatory processes pertaining to the release of anonymised data under these circumstances.
Ethics approval and consent to participate
Ethical approval was obtained from Cambridge Central Research Ethics Committee, REC ref.: 17/EE/0320, IRAS Project ID: 227864. All participants will provide informed written consent and in the case of children their parents will provide consent on their behalf. The trial will be conducted in accordance with the principles of the Declaration of Helsinki (1996) and the principles of Good Clinical Practice (GCP). The chief investigator is required to submit regular reports and a final report at conclusion of the trial to NIHR, the funder.
Consent for publication
Not applicable—this manuscript does not contain data from any individual person.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Murphy, S., Joffe, V., Messer, D. et al. Evaluating ‘enhancing pragmatic language skills for young children with social communication impairments’ (E-PLAYS): protocol for a feasibility randomised controlled trial study. Pilot Feasibility Stud 5, 75 (2019). https://doi.org/10.1186/s40814-019-0456-z