Skip to main content

Evaluating ‘Enhancing Pragmatic Language skills for Young children with Social communication impairments’ (E-PLAYS): a feasibility cluster-randomised controlled trial

Abstract

Background

This article reports the results from a feasibility study of an intervention (‘E-PLAYS’) aimed at supporting children who experience difficulties with social communication. E-PLAYS is based around a dyadic computer game, which aims to develop collaborative and communication skills. A pilot study found that when E-PLAYS was delivered by researchers, improvements on communication test scores and on collaborative behaviours were observed. The aim of this study was to ascertain the feasibility of running a full-scale trial to test the effectiveness of E-PLAYS in a National Health Service (NHS) setting with delivery by speech and language therapists and teaching assistants.

Methods

The study was a two-arm feasibility cluster-randomised controlled trial of the E-PLAYS intervention with a treatment as usual control arm. Data relating to recruitment and retention, treatment fidelity, acceptability to participants, suitability of outcomes and feasibility of collecting health economic measures and of determining cost-effectiveness were collected.

Speech and language therapists selected suitable children (ages 4–7 years old) from their caseload. E-PLAYS intervention (experimental group) was then delivered by teaching assistants overseen by speech and language therapists. The control group received usual care. Assessments included blinded language measures and observations, non-blinded teacher-reported measures of peer relations and classroom behaviour and non-blinded parent-reported use of health and education resources and quality of life.

Results

Planned recruitment was for 70 children, in the event, 50 children were recruited which was sufficient for feasibility purposes. E-PLAYS was very highly rated by children, teaching assistants and speech and language therapists and treatment fidelity did not pose any issues. We were able to collect health economic data which suggests that E-PLAYS would be a low-cost intervention.

Conclusion

Based on recruitment, retention and adherence rates and our outcome measures, a full-scale randomised controlled trial estimated appears feasible and warranted to assess the effectiveness of E-PLAYS for use by the NHS and schools.

Trial registration

ISRCTN 14818949 (retrospectively registered).

Peer Review reports

Key messages

  • What uncertainties existed regarding the feasibility? The study aimed to establish the feasibility of delivering the E-PLAYS intervention in schools. An earlier pilot study by the authors had demonstrated that, when delivered by researchers, an earlier version of E-PLAYS produced significant gains in pragmatic language test scores and communication skills compared to a control group. The areas of uncertainty concerned delivery of E-PLAYS by speech and language therapists and teaching assistants and the impact that this may have on intervention fidelity and acceptability, also the ease of recruitment for a large-scale study.

  • What are the key feasibility findings? Acceptability was very high and intervention fidelity was good. However, there were some challenges in relation to recruitment and retention. Recruitment was via speech and language therapists who examined their caseloads for children fitting the inclusion and exclusion criteria and then approached schools. One unintended consequence of this recruitment strategy was that the demographic profiles of the parents of the children recruited were largely White British, graduates and in fulltime employment. This was not reflective of the North East London area in which we were recruiting which was ethnically highly-diverse and where several boroughs experience high levels of poverty. However, this was consistent with previous research which has found that well-educated parents with higher socio-economic status are better able to access services such as speech and language therapy.

  • What are the key implications of the feasibility findings for the main study? Based on the outcomes of our study, a full-scale trial appears feasible and warranted to assess the effectiveness of E-PLAYS for use by the NHS and schools. The low-cost, computer-based nature of E-PLAYS makes it highly suitable for national distribution. E-PLAYS was designed ultimately to be widely shared and we have used existing, easily updated technology which is available within the NHS and in primary schools. A vast number of commercially-available games are targeted at parents of children with autism spectrum disorders and other children with social communication impairments. Reviewers of the computer game literature have urgently called for large-scale studies to ensure these are based on a sound evidence-base. A possible recruitment alternative would be to approach schools directly without using speech and language services as an intermediary. It is likely that this would reach a wider demographic in terms of socio-economic status and ethnicity.

Background

Children with social communication impairments struggle to communicate effectively in social contexts; our E-PLAYS intervention aims to provide them with support. Difficulties with social communication can manifest themselves as an inability to maintain a topic of conversation or take turns appropriately, misunderstanding of non-literal language such as jokes, irony or sarcasm, failure to make inferences and repair communication breakdowns. Responding appropriately, interpreting others’ communications whilst at the same time understanding society’s norms and expectations constitutes pragmatic language skill [1, 2].

The terms ‘social communication’ and ‘pragmatic language’ are often used interchangeably. Recent systematic reviews of interventions for ‘social communication impairments’ or ‘pragmatic language impairments’ [3, 4] have adopted broad definitions for these terms encompassing both verbal and nonverbal aspects of communication to include abilities such as facial expression use as well as spoken language [5]. For the purposes of our study, however, we have adopted Matthews et al.’s (2018) [6] suggestion and defined ‘pragmatics’ more specifically as the linguistic component (excluding facial communication and other non-verbal communication) of social communication.

Children with social communication impairments are highly heterogeneous. The Diagnostic and Statistical Manual of Mental Disorders (DSM-V [7];) and the International Statistical Classification of Diseases and Related Health Problems (ICD 10 [8]) both list a number of developmental disorders (such as autism) and language disorders concerned with social communication. Furthermore, ADHD [9] and conduct disorders [10, 11] have also been shown to impact on social communication.

It is important to remember, however, that many children will experience difficulties with social communication without ever receiving diagnoses and there are indications that this is particularly the case for children of low socio-economic status [10, 12, 13].

Pragmatic language difficulties can have profound and long-lasting effects on emotional functioning and peer relations [14, 15]. Children are frequently rejected and victimised by peers [16, 17], and in adulthood, social relationships continue to be a challenge [18].

Schools aim to facilitate children’s learning and train team-building skills for adulthood through collaborative activities [19,20,21,22]. Collaborative working has also been shown to improve peer relations and facilitate children’s feelings of belonging [23, 24]. A systematic review by Chang and Locke (2016) [25] concluded that the inclusion of peers was one of the most promising strategies for social skills intervention. A recent review [26] concluded that collaborative learning activities are particularly beneficial for low-ability children at younger ages (4–7 years old).

Studies which have observed peer collaborations involving children with social communication impairments [27,28,29] have consistently shown that such children fail to contribute appropriately, sharing less, making more irrelevant statements, ignoring others and showing aggressive or withdrawn behaviours and consequently, they are often excluded by peers. Enabling social interaction between children with social communication impairments and their peers demands considerable skill from adult facilitators. The difficulties and resource implications of providing support for such children have tended to result in their isolation within the classroom; Blatchford et al. (2009) [30] have reported that typically, the most common activity for children with special educational needs is one-to-one working with teaching assistants and that consequently, rates of interaction with teachers and peers are reduced by almost half [31].

There are few interventions available to SLTs and schools for pragmatic language difficulties [3, 4, 32]; the Social Communication Intervention Programme (SCIP) is an example of an intervention with emerging evidence [33] but it demands a substantial time input from SLTs. The use of technology and gaming have been promoted as particularly useful means of facilitating communication and collaboration for children with communication difficulties as they are especially appealing to this group of children e.g. [34,35,36]. A systematic review and meta-analysis of computer-assisted collaborative learning [37] concluded that the intrinsic socio-cognitive scaffolding can enhance collaborative skills substantially; thus, computer games can facilitate communication and collaboration between children [38,39,40]. Alt et al. (2012) [41] have pointed out that currently, peer collaborative activities for children with communication and language impairments rely heavily on adult specialist skill for facilitation; little time or thought has been invested in the creation and development of these activities. Technology can fill this gap and be used to regulate turn-taking, participation and enforce the rules [42] and can also add surprises, colourful animations and unusual sounds and keep the game flowing at a suitable pace [35, 43].

Preliminary testing (n = 32) for ‘E-PLAYS’ (formerly known as the ‘Maze Game’) is described in Murphy et al. (2014a, 2014b) [29, 44]. Children in the intervention group showed significant increases on communication test scores by comparison to a control group [44]. In this preliminary study, the intervention was delivered to the children by the research team The aim of the present study was to establish the feasibility of running a full-scale clinical trial to evaluate E-PLAYS’ clinical- and cost-effectiveness when implemented within the NHS.

The study had the following feasibility objectives; to assess:

  1. 1.

    Participant recruitment and retention (via NHS trusts and schools)

  2. 2.

    The acceptability of E-PLAYS to children, SLTs, teachers, parents and teaching assistants

  3. 3.

    Treatment fidelity.

  4. 4.

    The suitability of outcome measures

  5. 5.

    The feasibility of collecting health economic measures and of determining cost-effectiveness for a full trial

Method

Feasibility trial design

The study was a two-arm, cluster-randomised trial (cRCT) to investigate the feasibility of conducting a sufficiently powered trial to evaluate the effectiveness and cost-effectiveness of E-PLAYS (compared with treatment as usual) for young children with social communication impairment.

Setting

We recruited speech and language therapists (SLTs) employed by North East London NHS Foundation Trust (NELFT) Speech and Language Services.

Participants

Speech and language therapists

The research team approached all suitably employed paediatric SLTs. They were invited to presentations by the team where they had the opportunity to ask questions and receive an information sheet and consent form to take away. Consent forms were returned by post, electronically or by hand to the research team. SLTs not responding within a week were reminded and invited again by email once more only.

Focal children

Inclusion criteria

SLTs screened all children on their caseloads and identified children aged 4 to 7 years who attended mainstream schools using the Social Communication Behaviour Checklist devised by Adams et al. (2012) [45]. Children were required to meet at least two of the five criteria as based on the SLT’s clinical judgement below:

  • The child has trouble understanding and interpreting the social context and friendship, e.g. social roles, emotions

  • The child has trouble understanding and/or using non-verbal aspects of communication, e.g. facial expression, intonation

  • The child has trouble with aspects of conversation, e.g. beginning and ending, taking turns, giving relevant and sufficient information

  • The child makes bizarre, tangential or inappropriate comments

  • The child has difficulty using and understanding non-literal language

Children were also required to have at least minimum levels of English (children with English as an additional language were included). We wished to be as inclusive as possible; teachers were consulted about children’s level of English and all those regarded as having sufficient levels of English were included. The parents of every child meeting the above criteria were invited to participate in the study.

Exclusion criteria

  • Hearing, visual or physical impairment severely affecting speech production

Parents

The research team provided schools with participant information sheets to send to parents of eligible children, one for parents and one written in simpler language for the child, with consent forms for parents to sign to authorise their child’s participation and to indicate their willingness to complete health economic questionnaires.

Teachers and teaching assistants

The teachers and teaching assistants of focal children were invited to take part in the study and received information sheets and consent form. Some teachers and teaching assistants were associated with more than one focal child.

Partners for focal children for the E-PLAYS intervention

Suitable partner children were suggested by teachers and teaching assistants in consultation with the SLT. These partner children were typically-developing children without language disorders in the same class as the focal children. Schools were provided with participant information sheets for parents of the partner children, again one for parents and one for children, with a consent form.

All participants were given the option of contacting the Chief Investigator in the event of additional questions and were informed that they were free to withdraw at any time from the study.

Randomisation and allocation process

Focal children were cluster-randomised at the level of their treating SLT. The SLTs were randomised 1:1 to the intervention or comparator group after they had consented to participate in the study and had identified and recruited children on their caseloads but before they received a briefing on E-PLAYS delivery from the research team. Allocation was via minimisation to ensure balance across the two groups on the borough of the SLT (five boroughs covered by NELFT, each served by a different SLT team) and the number of children recruited (dichotomised around the median). Minimisation was implemented by a trial statistician at York Trials Unit using MinimPy version 0.3.

Blinding

Due to the nature of the intervention, SLTs, teachers, teaching assistants, children and parents could not be blinded to allocation. However, the research assistants collecting outcome data were blinded (see the ‘Measures’ section). Several different research assistants were associated with this project and they did not conduct pre- and post-test measures on the same children. We also ensured that same research assistants did not conduct qualitative assessments (where they were unblinded) with the same children to whom they administered the TPS, CCT and Recalling Sentences tests. At each visit to schools, staff were reminded not to unblind research assistants.

Procedure and intervention

Intervention group: E-PLAYS

E-PLAYS has been designed to be administered by non-specialists such as teaching assistants (as opposed to SLTs or teachers) in 12 weekly sessions. SLTs randomised to the intervention group received a 30-min briefing from the research team together with a manual. The SLTs then trained the teaching assistants in small groups for around 2 h in their schools and gave them the manual with which to deliver E-PLAYS. SLTs gave teaching assistants further support in the same way as they would usually when introducing a new intervention if needed. Generally, most teaching assistants were able to use the game readily, the support that was given was very brief and amounted to a maximum of 30 min per teaching assistant.

The E-PLAYS intervention is built around a dyadic collaborative computer game. Children play the computer game with a teaching assistant for eight, 30-min sessions and with a classmate for four, 15-min sessions. In their sessions, teaching assistants use the game to guide the child through (a) requesting optimally useful information, (b) giving helpful directions and answering questions and (c) asking for clarification. Sessions with the classmate give the focal child an opportunity to practice in live interaction the content of the sessions led by the teaching assistants. Sessions one, five, six and twelve were those with a peer; the other sessions were with the TA only. Session one was designed as an introductory session for the children to find their way around E-PLAYS, sessions five and six were in the middle, for children to practice new skills and the final session, twelve, aimed to consolidate the skills learned.

Comparator group: usual practice

Usual practice for a child with social communication impairment on an SLT’s caseload typically comprises a programme of activities, devised by the SLT who supports schools’ teaching assistants to deliver them (a ‘consultancy model’ [46]). Activities typically [32, 47] include exercises on turn-taking, topic management and conversational skills, sometimes with role-playing or modelling. These are generally taught directly by an adult (e.g. a teaching assistant) to the child one-to-one or in a small group.

For both the intervention and the comparator group, SLT visits to support the teaching assistants took place around 3–6 times per year.

Measures

(a) Measures administered and rated by blinded research assistants

(i) Test of Pragmatic Skills, (TPS, [48])

A major difficulty inherent in researching pragmatic language skills is that social communication difficulties generally manifest only during social interaction itself and are therefore usually missed by standard language tests [1]. Social communication impairments are therefore best assessed by direct observation. The TPS is an observational elicitation measure. The tester engages the child in structured but naturalistic play to elicit target behaviours and responses, which are audio-recorded and later scored. Scoring time for the TPS is approximately 25 min. The TPS was a sensitive indicator and successful at detecting improvements in communication in our pilot study [44]; it has been standardised on 650+ children by the author. The instrument shows good reliability (test-retest r = 0.96, interrater r = 0.92). A score between 0 and 42.5 is given, where a higher score indicates greater pragmatic language skill.

(ii) Clinical Evaluation of Language Fundamentals-5 (CELF-5, [49]).

CELF-5 is one of the most widely-used standard language tests by SLTs. We used one of the subscales only, Recalling Sentences. This sentence repetition assessment is generally regarded as a measure of overall language ability drawing upon a wide range of language processing skills [50]. The type of language structures that are modelled in the intervention (e.g. requesting clarification, providing directions) could potentially lead to greater syntactic competence and an ability to structure more coherent sentences. This subscale therefore gives us an indication of the impact of E-PLAYS on children’s language as a whole by comparison to the more specific social communication skills targeted by the TPS.

Raw scores (0–78) are used to calculate age-adjusted scores between 1 and 19, where higher scores indicate greater ability to recall and reproduce spoken sentences accurately.

(iii) Dyadic collaborative construction task (CCT)

A frequently reported issue with interventions targeting children with social communication impairments is that the skills learned do not generalise beyond those of the intervention context; furthermore, this skill transfer is rarely measured [4]. A 10-min, collaborative construction task using Magformers® (a construction toy made with plastic, brightly coloured, magnetised blocks) was devised by the team as a transfer measure to observe children’s collaborative and communicative skills pre- and post-intervention. Children undertook the task with a peer (this was the same classmate pre- and post-test and also for the intervention) and were video-recorded by research assistants. The children followed instructions and were asked to work together without adult input. What we were observing, in this instance, was the ability, not only of the peer to scaffold the focal child, but also of the focal child to build up a collaborative working relationship with the peer over time.

(b) Teacher-completed measures

(i) The Children’s Communication Checklist-2 (CCC-2, [51])

The CCC-2 is the most widely used, standardised questionnaire of communication impairment in research and clinical contexts [1]. The CCC-2 has eight subscales which can be combined to yield the General Communication Composite, an indicator of overall communication difficulty, on a scale from 3 to 133 (age-dependent; higher scores indicate fewer difficulties). It also includes four subscales (Initiation, Stereotyped Language, Use of Context and Non-verbal Communication) that may be considered to be more specifically concerned with pragmatic language and can be combined to give a Pragmatic Language Score [6] between 3 and 69 (age-dependent, higher scores indicate fewer difficulties).

(ii) The Strengths and Difficulties Questionnaire (SDQ, [52]).

The SDQ is a widely used mental health indicator with subscales assessing behavioural, emotional and peer problems. A total score for the SDQ was calculated from 0 to 40 (higher scores indicate greater difficulties) and for the Prosocial subscale from 0 to 10 (higher scores indicate greater prosocial behaviour).

(c) Parent-completed measures

(i) Quality of Life: EQ-5D-Y (proxy 1) [53] and Paediatric Quality of Life (PedsQL, [54])

The EQ-5D-Y (proxy 1) was developed as a child-friendly version of the EQ-5D, the preferred instrument recommend by NICE for clinical decision-making. The proxy version asks caregiver to rate the child’s health-related quality of life. Given the population of the study, we were uncertain as to whether the EQ-5D-Y (proxy-1) could offer sufficient sensitivity, thus we also included the PedsQL as a comparative instrument.

(ii) Resource use

The ability to assess resource implications was piloted using a bespoke resource use questionnaire for parents. The costing approach was undertaken from an NHS perspective and also considered the perspectives of both Social Services and education providers.

(d) Fidelity measures

Delivery of the computer game within E-PLAYS (duration and number of sessions) was automatically recorded by the E-PLAYS software and transmitted to the research team; each child could be identified individually by their unique PIN login.

Process evaluation – qualitative investigation

A process evaluation was included in our study (as recommended by the Medical Research Council (MRC) guidance for RCTs, [55]) to elucidate processes which may impact on intervention delivery and measurement. Processes examined were whether:

  1. (a)

    Instructions for delivery of E-PLAYS were considered to be adequate

  2. (b)

    Staff could use E-PLAYS with sufficient fidelity

  3. (c)

    How acceptable staff and children found E-PLAYS

SLTs and teaching assistants in the intervention arm received open-ended questionnaires within 2 weeks of receiving the manual (SLTs) and of training (teaching assistants); these assessed the clarity of instructions for delivery of E-PLAYS. Ten teaching assistants in the intervention group were invited to be observed whilst delivering E-PLAYS and together with the corresponding children were viewed live by research team members for two E-PLAYS sessions each, observation of these sessions concerned teaching assistants’ adherence to manual instructions and teaching assistants were made aware of this. These same children were also interviewed about their experiences of E-PLAYS. Focus groups with teaching assistants were conducted at the end of the summer term in which E-PLAYS was delivered. The child interviews and focus groups were concerned with acceptability.

Data collection points for measures and qualitative assessments

Data collection for the TPS, CCC-2, CELF-5 and SDQ took place at baseline, 15–20 weeks and 35–40 weeks post-randomisation. These time points are approximately equivalent to baseline, immediate post-intervention and 3-month post-intervention follow-up for the intervention group.

Analysis of the CCT is a resource-intensive research activity requiring substantial input of research assistant time. Therefore, as this was a feasibility study, the decision was taken to collect at baseline and 15–20 weeks only to save resources as the study was primarily concerned with the viability and acceptability of collecting this data rather than producing definitive analyses. All health economic measures and demographic data were collected at 35–40 weeks post-randomisation. These data were collected at one time point only as we have observed in previous trials that whilst baseline data collection tends to be good, completion rates decline over time hence administering the questionnaire at the end time point gives us an indication of the likely minimum level of data attainable whilst also reducing participant burden.

Sample size

We aimed to recruit 70 focal children to provide reliable estimates of sample size parameters [56].

Analysis

Analyses were conducted in Stata v15. Following CONSORT recommendations [57] for feasibility studies, quantitative outcome measures were summarised descriptively by group and time point, with no formal between-group comparisons undertaken. Parameters required to calculate the sample size for a future full-scale RCT were estimated. The variance of the TPS at each time point was calculated with a 95% confidence interval (CI). To estimate the intra-cluster correlation coefficient for TPS scores at week 20 and 40, we used a three-level mixed effects model (measurements nested in children nested in SLTs), adjusting for the main effects of time and allocation and their interaction, baseline TPS score and the factors used in the minimisation algorithm as fixed effects with random intercepts for SLT and child nested in SLT. The correlation and 95% CI between the TPS score at baseline and the two post-randomisation time points were calculated.

The focus groups were audio-recorded and professionally transcribed. Observations of the teaching assistants were live and followed a checklist. Thematic analysis for both followed the guidelines of Braun and Clarke (2006) [58]: (1) becoming familiar with the data, (2) generating initial codes, (3) searching for themes, (4) reviewing themes and (5) defining and naming themes.

Data from the open-ended questionnaires were analysed by coding responses into categories known as a ‘coding frame’ [59]. Children’s interviews were conducted using the Fun Toolkit [60], a schedule designed to probe children’s views of technology. The Fun Tool Kit asks children to rank the aspects of the game that they liked best and to give an overall indication of how much they liked the game.

For the CCT, video-recordings were transcribed by a professional company and then coded by blinded research assistants. Coding consisted of observation and categorisation of communicative (e.g. Information-Seeking Questions, Directives, Clarification Requests) and affective (e.g. positive and negative behaviours) items which have been found to differentiate children with social communication impairments from typically-developing children [44]. The coding system is based on micro-analytic theory [61, 62] and was developed by the team specifically to analyse children’s collaborative interaction. It was based on research in collaborative learning, conversation analysis and language impairment [29, 44, 63,64,65,66,67,68,69]. We have used this system extensively and it has previously successfully detected changes in talk in our studies [29, 44, 64]. Research assistants were trained to fidelity by the Chief Investigator by watching and coding training videos until they achieved a threshold of concordance (that is, agreement with 80% of the codes on the training videos). Inter-rater reliability was also calculated between research assistants taking part in this study using a random sample of 15% of all video-recordings.

The costs of providing E-PLAYS were estimated including accounting for development and ongoing maintenance costs where possible. An estimate of the costs of running E-PLAYS within the NHS and schools was calculated.

Results

Participant recruitment and flow

We approached 72 SLTs for participation, of which 45 (63%) did not respond, 14 (19%) were associated with a school which declined to participate and one (1%) could not identify any potentially eligible children on their caseload (Fig. 1). The remaining 12 SLTs (17%) were randomised into the trial (six to each group). Recruitment of focal children commenced on 1st January 2018 and closed on 20th April 2018. Recruitment of children (via parents) did not take place until schools had agreed to participate. Recruitment was originally planned for September 2017–December 2017 but had to be delayed. We recruited 50 focal children out of a target sample size of 70 (24 in the intervention group and 26 in the control). These children were recruited from 14 schools (six intervention and eight control). One school (four children) withdrew immediately following the baseline visits due to flooding at the school. Another school (six children) was unresponsive at the final data collection point meaning no assessments could be undertaken. SLTs recruited a mean of 4.2 children (SD 1.7, range 2 to 7). Children were aged, on average, 6.2 years (0.8) and 74% were male (Table 1). Characteristics relating to age, gender and baseline assessments conducted appear comparable between the two groups.

Fig. 1
figure1

Study flow diagram

Table 1 Characteristics of participants for intervention (E-PLAYS) and comparator groups

Fifty typically-developing children were recruited to partner the focal children; 100% of parents (n = 50) gave consent for these children to take part in the study. All participating SLTs (n = 12) and 35 out of 39 participating teaching assistants returned the open-ended questionnaires. There were 19 teaching assistants in the intervention group, 14 of these participated in the focus groups which was as many as were available on the days in question, all had been invited. Our aim was to observe ten teaching assistants; ten were randomly selected and approached. Of these, one declined, and therefore, nine teaching assistants were observed delivering two E-PLAYS sessions and nine children were observed in the same E-PLAYS sessions and were then interviewed by research assistants using the Fun Tool Kit [60].

The response rates for research assistant-administered measures were 86% at 20 weeks and 78% at 40 weeks. For teacher-completed measures response rates were 82% at 20 weeks and 62% at 40 weeks. Forty parent questionnaires were distributed at 40 weeks, of which 32 were returned.

Acceptability

Speech and language therapists

Responses to the questionnaire from SLTs indicated that the training manual was favourably regarded by SLTs and they commented on its clarity:

I found the manual very clear (SLT 2)

…the manual is so self-explanatory(SLT 5)

…the session by session breakdown [is] very helpful (SLT 12)

Teaching assistants

Similarly, teaching assistants commented positively on the manual and especially on the illustrations:

Simply written. Easy with diagrams (TA4)

When it was unclear, I looked at the pictures and it helped me (TA7)

Step by step explanations also very easy to follow (TA10)

There were few problems with running the game itself, which had been designed to be easy to load and use. During focus groups, teaching assistants were overwhelmingly enthusiastic about the game.

He (name of child receiving intervention) loved it… he really did love it. (TA4)

The game is brilliant (TA8)

It does make it fun that it is a game and it does help them and make them want to

do it (TA9)

Teaching assistants were especially positive about the involvement of typically-developing partner children and stressed that both the partners and the focal children enjoyed the game.

I think they are more relaxed with peers than an adult (TA2)

…he really listened well; he listened better to (partner name) than he does in class. (TA8)

Children

Children appeared to like all aspects of the game, particularly finding the ‘hidden treasures’. Six of the nine children interviewed rated the game as ‘good’, ‘really good’ or ‘brilliant’. Teaching assistants explained that one of the children who rated the game as ‘not very good’ was upset at having to finish (because of school timetabling) the game sooner than they wanted and the two others were ‘having a bad day’ but did in general at other times enjoy E-PLAYS.

School use

A further indication of the acceptability and popularity of E-PLAYS is that after completion of data collection, control group schools were offered the use of E-PLAYS. All control group schools took up this offer; furthermore, many of the intervention group schools asked to continue using E-PLAYS, with the result that we extended the licence to all participating schools until December 2019. In total, nine of the 14 participating schools (2 of 6 in the intervention and 7 of 8 in the control group) were still using E-PLAYS 12 months after study completion.

Intervention delivery and fidelity

Intervention fidelity: qualitative observations

During observations of the teaching assistants, we found that all were able to use the software and set up the game and all were able to keep the children on task. Less positively, only three of the observed teaching assistants followed manual recommendations to pause to give the children time to respond to questions and remind children of the collaborative nature of the task. Almost all teaching assistants tended to intervene with the children more than recommended instead of allowing the children to explore the game.

The unanticipated delays in recruitment had a detrimental, knock-on effect on intervention delivery. E-PLAYS is a 12-session intervention with one (recommended) session per week. Due to the recruitment delays, many schools were left with only the end of the summer term in which to deliver 12 sessions of E-PLAYS and this was often less than 12 weeks. Schools were therefore asked to deliver as many sessions as was reasonably practicable. The mean number of sessions received by the children in the intervention group was 6.2 (SD 4.3). Five children randomised to the intervention group did not receive any E-PLAYS sessions. This was due to one school being unable to commence E-PLAYS for one child and another school (four children) withdrawing immediately following the baseline visit due to flooding at the school. Excluding the children from the school which withdrew, then the mean number of sessions received was 7.4 (SD 3.6). However, once schools were able to begin delivering E-PLAYS, the average frequency with which they delivered the E-PLAYS sessions was within the recommended range. The mean frequency of session completion was 1.1 sessions per week (range 0.3–2.6). Recommended duration for the sessions where the focal child played with the teaching assistant was 30 min (minimum 20 min, no maximum); actual mean duration was 25.50 (SD 11.6) min. Recommended duration for child and classmate peer partner sessions was 15 min (minimum 10 min, no maximum); actual mean duration was 28.72 (SD 12.47) min.

Children in the intervention group generally received the early sessions with declining numbers receiving the later sessions; all 19 children received session 1 but only six children received session 12. Peer sessions required as much adult input (in terms of supervision) as did the teaching assistant only sessions so this was not a factor impacting delivery. To illustrate, the number of children receiving session 4 (teaching assistant only) = 17, session 5 (peer session) = 16, session 6 (peer session) = 12 and session 7 (teaching assistant only) = 11 and session 12 (peer session) = 6.

Suitability of outcome measures and qualitative processes

From the point of view of participant burden and ease of administration, these outcome measures were generally found to be suitable. Importantly, the TPS, CELF-5 and CCT were blinded independent measures and we were able to maintain blinding. On two occasions school staff did inadvertently unblind research assistants despite being reminded. On these occasions, a different (blinded) research assistant was substituted to administer the outcome measure.

Table 2 gives scores for research assistant-administered and teacher-report measures. Improvements in TPS score over time were observed in both groups, as might be expected for this population. Scores for other outcome measures were more mixed. There is generally limited evidence for large differences between groups; few conclusions can be drawn, due to the small number of participants and the possibility of informatively missing (missing not at random) outcome data.

Table 2 Scores for measures for the intervention (E-PLAYS) group and for the comparator group at baseline, 20 weeks post-randomisation and 40 weeks post-randomisation

Scores in general are typical of children with language disorders. Of the 49 children with TPS scores at baseline, 45 (91.8%) had scores below the 25th percentile of the standardisation sample used in the development of the TPS and 26 (53.1%) had scores below the 10th percentile. Similarly, 48 (96%) of the children in the sample had baseline scores on the CELF-5 Recalling Sentences subscale either at or below the mean score obtained by the standardisation sample, with 34 (68%) having scores more than one standard deviation below the standardisation sample mean score. This was also reflected in the teacher-completed instruments. Of the 41 children with a valid baseline score for the CCC-2 General Communication Composite, all of them had scores below the 20th percentile of the standardisation sample, with 37 (90.2%) of these having scores below the 10th percentile. Of the 42 children with a valid SDQ Prosocial scale score at baseline, 35 (83.3%) had scores below the mean score for British children aged 5–10 years (mean = 7.3, SD = 2.4) and 23 (54.8%) had scores more than one standard deviation below this mean.

We successfully achieved inter-rater reliability between the research assistants (weighted kappa = 0.69). According to guidelines proposed by Landis and Koch (1977) [70], kappa values ranging from 0.41 to 0.60 are rated ‘moderate’, 0.61–0.80 as ‘substantial’ and 0.81–1 as ‘almost perfect agreement’. Table 3 shows scores of E-PLAYS and comparator groups for communication codes. Directives, Clarifications and Information Questions all showed increases over time. For Positive and Negative Feelings, results varied with no consistent pattern. Overall, no firm conclusions could be drawn especially as for this small sample, the impact of the time taken for the task cannot be controlled for.

Table 3 Collaborative Construction Task; number of codes appearing in each task for each pair of children for the intervention (E-PLAYS) group and for the comparator group at baseline, 20 weeks post-randomisation and 40 weeks post-randomisation

Cost and resource use data collection

(i) Quality of Life

Table 4 gives responses for EQ-5D (proxy-1) and PedsQL questionnaires. Due to the absence of published population norms for EQ-5D for children, we cannot compare how the reported levels of problems relate to those observed in the general population. However, only 35% of caregivers reported children being in a perfect health state with caregivers reporting various levels of issues across the 5 dimensions. The established national norm for the PedsQL of a healthy population is 82.3 (± 15.6) [54]. Based on the combined results of all responding caregivers, the children in the study score at the lower end of this normal range suggesting that the instrument is sensitive to the problems faced by these children.

Table 4 Proportions of reported problems by the trial arm in EQ-5D-Y dimensions and PedsQL scores

By comparison to the EQ-5D-Y (proxy-1), the PedsQL provides more in-depth information concerning emotional, social and school functioning, all dimensions that are likely to be affected by social communication impairments.

(ii) Resource use

Little NHS, Social Services and educational services use was reported across both trial arms either through the NHS or privately. Similarly, low levels of use were reported for home-based assistance, voluntary services or education and childcare services. The most widely used resource was special educational needs coordinator (SENCO) though the mean number of contacts was still very low: 2 (SD 3.5) in the comparator group and less than 1 (SD 0.5) in the E-PLAYS intervention group.

(iii) Intervention delivery: costing exercise

From an NHS perspective, low input is required from health care professionals to deliver E-PLAYS. We anticipate, therefore, the cost to the NHS to be small meaning if the intervention was effective it would likely represent good value for money. Schools are not required to purchase or provide any additional resources to facilitate the delivery of E-PLAYS. There was some time required to set up the intervention initially in schools and attend the training. Teaching assistants spent around 10 min preparing for each session. Schools participating in the feasibility study were not required to pay for the E-PLAYS software; however, in a full-scale trial, it is possible that schools would incur an annual licencing fee of approximately £50 per school to cover software maintenance costs. As with the NHS, costs for schools would be low.

Estimating the sample size of a future trial to evaluate the effectiveness of E-PLAYS

The variance of the available TPS scores at baseline, week 20 and week 40 was 51.3 (95% CIFootnote 1 29.3 to 93.9), 49.1 (95% CI1 30.0 to 80.6) and 47.8 (95% CI1 23.5 to 93.3), respectively. These suggest a reasonable estimate of the standard deviation of 7.0 (although the available data are compatible with values between approximately 4.8 and 9.7). Conditional on the fixed effects, the intra-cluster correlation for SLT was essentially 0. In order to be conservative, we assume an intra-cluster correlation of 0.05 in the following calculation which, together with a (conservative) estimate of the mean cluster size of 5, gives an estimated design effect due to clustering of 1.20. The observed correlations between the baseline TPS score and the TPS scores at week 20 and week 40 were strong at 0.84 (95% CIFootnote 2 0.71 to 0.91) and 0.79 (95% CI2 0.63 to 0.89), respectively. We can account for a more conservative correlation of 0.6 in our sample size calculation by multiplying the sample size required for an unadjusted analysis by(1 − 0.62). Finally, we assume a conservative rate of attrition of 25%. These parameters mean that 356 children would be required for 90% power in a two-sided test of size 5%. Assuming an average of 5 children per SLT, approximately 71 NHS SLTs would need to be recruited and randomised.

Adverse events and safety

No adverse events were recorded.

Discussion

The aim of the present study was to assess the feasibility of conducting a full-scale randomised controlled trial to evaluate the effectiveness and cost-effectiveness of the E-PLAYS intervention when delivered by SLTs and teaching assistants by comparison to usual care for children with social communication impairments.

To assess feasibility, we examined recruitment of children through the Speech and Language Services of the NHS, the acceptability of E-PLAYS to the children and teaching assistants using it, methods of determining intervention fidelity and delivery and the suitability of outcome measures and cost-effectiveness measures. Important lessons were learned that will be applied to any follow-on trial.

Recruitment and retention

Our original target was to recruit 70 children over a period of 3 months [56, 71]. We actually recruited 50 children in 3.5 months although the recruitment period had to be delayed. For a full trial, increasing the number of children to be recruited could be addressed straightforwardly by simply recruiting more NHS trusts to the study. Although 70 children had been our original target, there are no set guidelines for participant numbers for feasibility trials; sample sizes of between 24 and 70 have been recommended to allow for the reliable estimation of the standard deviation for use in the sample size calculation of a future fully powered trial [56, 72] and 50 was in fact sufficient for this purpose. Importantly, our E-PLAYS intervention has already indicated a signal of efficacy in pilot studies [29, 44].

The time period within which to recruit children and the rate of recruitment are more complex issues as they are constrained by the necessity of operating within the school year. We delayed the recruitment period as far as possible to allow SLTs time to examine their caseloads and approach schools. Even with this delay, however, we still did not manage to recruit our target of 70 children. We were obliged to bring recruitment to a close in order to begin intervention delivery within the school year. A major factor causing recruitment delays was that SLTs often did not know which children were to be on the caseload until well into the school year. However, for a full trial, recruitment could be more easily managed in waves over a 2- or 3-year period; once a school and SLT are part of the study, children from subsequent school years can be recruited relatively easily. E-PLAYS was highly appealing, and thus, schools and SLTs were generally keen to participate. We recruited 50 children from a single NHS trust; our sample size calculation suggests that we would need 356 children and 71 NHS SLTs for a full trial. This would necessitate the recruitment of around 6–7 NHS trusts, which would appear feasible.

An important feasibility question concerned recruitment of the typically developing peer children to partner the focal children and the willingness of parents to allow their participation. We found that 100% of parents gave consent for these children to take part. However, concerns have been expressed elsewhere that this kind of participation could have negative consequences for typically-developing children [73, 74] as it may take up their own educational time for the benefit of other children. Locke et al.’s (2012) longitudinal study [75] did not report any adverse outcomes for typically developing children as a result of participating in a peer-mediated intervention, and the burden of our E-PLAYS intervention is relatively light (4 × 15-min sessions). E-PLAYS aims to provide a positive educational experience for typically-developing children as well as those with communication impairments and teaching assistants reported that they enjoyed it just as much.

Retention was satisfactory; 70% retention is considered the minimum for inclusion in Cochrane Reviews [76]. Participant burden on the children did not appear to be onerous which seems to be borne out by children’s willingness when approached by research assistants. Teacher and parent responses were slightly lower than 70% at 40 weeks. Although the questionnaire burden on teachers was relatively light (15 min per child), we found that this was a stressed occupational group with high staff turnover. As we were relying on schools to pass questionnaires on, this also impacted the response rate from parents. For a full trial, we will explore possibilities for incentives for schools. This has worked successfully in other large-scale school-based NIHR trials.

Acceptability

E-PLAYS was highly acceptable, with SLTs, teaching assistants and children all rating it very positively. It is likely that this reflects the high level of intervention development, testing and stakeholder involvement in the pilot study that preceded the feasibility study [29, 44].

The questionnaires to SLTs and teaching assistants concerning training and the manual elicited favourable responses, suggesting that we had been able to develop accessible instruction material. All reported that the computer game was easy to run. There were few issues around acceptability. Feedback overwhelmingly consisted of requests for a higher standard of graphics and additional games features which will be considered for future versions.

A further reason for the popularity of E-PLAYS was that it had been designed to fit within normal school and SLT working patterns. Thus, additional staff and resources were not required from schools and NHS SLTs taking part.

Tolmie et al. (2010) [24] found that collaborative work led to improved student relations. This is a particularly important consideration for children with special educational needs who are generally less liked and accepted by their peers; Pinto et al. (2019) [31] found that the level of meaningful interaction with peers was the strongest predictor of acceptability for these children. Collaborative computer games are generally enjoyable interactions but also meaningful in this instance as the children in E-PLAYS are directed and obliged to collaborate to succeed in the game. It is possible, therefore, that E-PLAYS may have an impact on children’s peer relations and this is one of the measures we propose to include in a future trial. Teaching assistants reported that sessions with peer classmates were particularly popular, with most suggesting that these should be increased in number and duration.

Treatment fidelity and delivery

Should E-PLAYS ultimately become nationally available and distributed, it is important that SLTs and teaching assistants can use it autonomously. For maximum fidelity, E-PLAYS was web-based and manualised with the content for each session specified step-by-step. The manual was generally well-liked and sessions were delivered without major problems thereby showing promise for good fidelity in a national roll-out. Delays tended to occur around the commencement of the intervention, chiefly due to staff shortages. However, once schools began with E-PLAYS, they were keen to continue.

We were able to gain precise assessment of treatment delivery through automatic recording by the E-PLAYS software. This recording was supplemented with live observations of the teaching assistants delivering E-PLAYS. These two methods confirmed that delivery was acceptable in terms of number and length of sessions.

Suitability of outcome measures

Pragmatic language skills are notoriously difficult to measure; the problem being that they manifest themselves only during dynamic social interaction, thus rendering testing with standardised questionnaires largely unachievable [1]. Furthermore, adult (particularly parent) report for conditions relating to autism and social communication are thought to be particularly susceptible to placebo effects [77, 78]. In spite of this, the majority of social communication literature is based on non-blinded parent-, teacher- or clinician-report [79]. It was one of our chief objectives to address this limitation and use effective measures administered by independent, blinded outcome assessors. We therefore used the TPS and CELF-5 subscales administered by research assistants. For consistency with other studies and to provide more global measures we also included reports from teachers and parents.

A key outcome of language and communication skill-learning is measuring subsequent use of the skill in contexts other than the one in which it was learned; skills can fail to generalise to novel contexts. Wieckowski and White’s (2017) [4] systematic review of technological interventions for social communication impairments reports that this is rarely assessed. To measure generalisation we included the CCT, previously devised by the authors.

The suitability of our measures was indicated by the good response and retention rates. Also, completion of the measures was excellent with very few missing items; almost all were analysable (see Fig. 1). Comparison of scores with norms suggests that these measures will detect changes as a result of intervention in a future full trial. We were able to preserve blinding for the research assistant-administered measures and for the CCT to successfully train research assistants to high levels of inter-rater reliability.

Health economic measures and cost-effectiveness

At present, neither of the instruments tested (ED-5D-Y proxy-1 and PedsQL) can be used to calculate QALYs that are required for a utility analysis, nor is there an alternative that can be used for this age range. However, EuroQol are currently looking to establish a value set for children and this may be available in the near future. On this basis, we would plan to include both the EQ-5D-Y proxy-1 and PedsQL in a full trial to ensure the best representation of health-related quality of life with the view of calculating QALYs if possible. Completion rates for EQ-5D proxy-1 and PedsQl were very similar to each other but PedsQL provides more information on social and emotional functioning.

As the feasibility study was conducted with a small number of participants in a geographically small area, the low uptake of resources may reflect low levels of availability within this area. This is at odds with previous research; health economic evaluations have been called for in this field as it has been shown that healthcare costs are 36% higher for children with language disorders aged 4–5 years [80]. In a full-scale trial, we may observe higher uptake owing to greater availability resources at a national level. This would also allow us to examine the effect of the intervention from a societal perspective.

Strengths and limitations

The low-cost, computer-based nature of E-PLAYS makes it highly suitable for national distribution. E-PLAYS was designed ultimately to be widely shared and we have used existing, easily updated technology which is available within the NHS and in primary schools. A vast number of commercially-available games are targeted at parents of children with autism spectrum disorders and other children with social communication impairments [81]. Reviewers [36] of the computer game literature have urgently called for large-scale studies to ensure that exploitation of the opportunities available with these new technologies is founded on a sound evidence-base to benefit the children to whom they are marketed.

Particular strengths of this study are that we have included independent, blinded measures and also a measure of generalisation of the skills taught. Through the use of technology, we were also able measure delivery of the intervention sessions precisely; other studies have been obliged to depend on less reliable methods such as teacher reports and diary methods.

Fidelity was further assessed via direct live observation of teaching assistants delivering some of the E-PLAYS sessions. Teaching assistants knew that they were being observed and this may have impacted on their behaviour when delivering these particular sessions. Possible alternatives could involve video-recording every session and then selecting random sessions for fidelity assessment. This would have the advantage that teaching assistants would not know which sessions were to be observed and could not alter their behaviour accordingly. However, this would have considerable resource implications for a large-scale study requiring either that a research assistant attend every session to record it or that teaching assistants were asked to make all the recordings themselves. A further drawback would be that teaching assistants and children would only use E-PLAYS with constant observation, thereby possibly not reflecting natural usage of the intervention which could impact on outcomes. A limitation of our study concerns the demographic makeup of our sample. Demographics were taken at the 40-week data collection point thereby probably resulting in a lower response rate than had we collected this information at baseline. We acknowledge that this is a shortcoming of our study and that the value of this data is correspondingly reduced. Responses came primarily from white British participants with an above-average level of education; this did not reflect the highly diverse areas of London in which the study took place. However, our finding is consistent with those of Safer-Lichtenstein et al.’s systematic review (2019) [82] who report that study samples for children with autism generally lack diversity, with an overrepresentation of participants who are male, White, and from upper-middle class backgrounds. This is particularly concerning in view of the fact that children from lower socio-economic backgrounds are less likely to receive diagnoses and support from health and education services for autism or language disorders [10, 12]. Sample composition is an important consideration, particularly if we fail to include groups that are often under-represented. We will explore means of recruiting a more diverse sample for a future RCT.

Conclusion

Based on the outcomes of our study, a full-scale trial appears feasible and warranted to assess the effectiveness of E-PLAYS for use by the NHS and schools. Participants in this study (focal children, teaching assistants, SLTs) enjoyed using E-PLAYS; overall acceptability was high. Few concerns were reported regarding participation in a future trial. However, strategies to improve recruitment rate and retention and to recruit a more diverse sample would need to be adopted. Despite some limitations, this feasibility study prepares the ground for one of the first interventions for children with social communication impairments to harness the potential of computer technology. A full-scale trial would result in making a valuable resource available as well as advancing our knowledge of how to design technological interventions tailored for this group of children.

Trial status

Ethical approval was obtained on 4th December 2017 (Cambridge Central Research Ethics Committee, REC ref: 17/EE/0320, IRAS Project ID: 227864). The study opened to recruitment on 1st January 2018 and completed recruitment on 30th March 2018.

Availability of data and materials

The datasets generated during and/or analysed during the current study have been stored in non-publically available repositories within the Universities of York and Bedfordshire. Anonymised data may be accessed and analysed by members of the project team and with researchers collaborating with members of the project team on the analysis of these data. Consent from participants was not sought for sharing raw data publicly. Therefore, external researchers seeking to access the data for use in future projects can do so via any reasonable request to the Chief Investigator Suzanne Murphy (or her delegate), and projects using the data must have been approved in accordance with contemporary UK ethical and regulatory processes pertaining to the release of anonymised data under these circumstances.

Notes

  1. 1.

    95% bias corrected bootstrap confidence intervals.

  2. 2.

    95% confidence intervals for correlations based on Fisher’s z-transformation.

Abbreviations

CCC-2:

Children’s Communication Checklist-2

CCT:

Collaborative construction task

CELF-5:

Clinical Evaluation of Language Fundamentals-5

DSM-V:

Diagnostic and Statistical Manual of Mental Disorders

E-PLAYS:

Enhancing Pragmatic Language skills for Young children with Social communication impairments

EQ-5D:

EuroQol 5-Dimension scale

MRC:

Medical Research Council

NELFT:

NorthEast London NHS Foundation Trust

NHS:

National Health Service

PedsQL:

Paediatric Quality of Life inventory

SDQ:

Strengths and Difficulties Questionnaire

SLT:

Speech and language therapist

TPS:

Test of Pragmatic Skills

References

  1. 1.

    Norbury CF. Practitioner review: Social (pragmatic) communication disorder conceptualization, evidence and clinical implications. J Child Psychol Psychiatry. 2014;55(3):204–16.

    PubMed  Article  Google Scholar 

  2. 2.

    Bishop DVM. Uncommon Understanding: Development and Disorders of Language Comprehension in Children. Hove: Psychology Press; 1997.

    Google Scholar 

  3. 3.

    Parsons L, Cordier R, Munro N, Joosten A, Speyer R. A systematic review of3. pragmatic language interventions for children with autism spectrum disorder. PLoS ONE. 2017;12(4):e0172242.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  4. 4.

    Wieckowski A, White S. Application of technology to social communication impairment in childhood and adolescence. Neurosci Biobehav Rev. 2017;74:98–114.

    PubMed  Article  Google Scholar 

  5. 5.

    Insel T, Cuthbert B, Garvey M, Heinseen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. 2010;167(7):748–51.

    PubMed  Article  Google Scholar 

  6. 6.

    Matthews D, Biney H, Abbot-Smith K. Individual differences in children’s pragmatic ability: A review of associations with formal language, social cognition, and executive functions. Language Learning and Development. 2018;14(3):186–223.

    Article  Google Scholar 

  7. 7.

    American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington: American Psychiatric Association; 2013.

    Google Scholar 

  8. 8.

    International Statistical Classification of Diseases and Related Health Problems (10th Revision, ICD-10), World Health Organisation, 2016.

  9. 9.

    Bishop DV, Baird G. Parent and teacher report of pragmatic aspects of communication: Use of the children’s communication checklist in a clinical setting. Dev Med Child Neurol. 2001;43:809–18.

    CAS  PubMed  Article  Google Scholar 

  10. 10.

    Donno R, Parker G, Gilmour J, Skuse D. Social communication deficits in disruptive primary-school children. Brit J Psychiatry. 2010;196:282–9.

    CAS  Article  Google Scholar 

  11. 11.

    Gilmour J, Hill B, Place M, Skuse D. Social communication deficits in conduct disorder: a clinical and community survey. J Child Psychol Psychiatry. 2004;45(5):967–78.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  12. 12.

    Bishop DVM, McDonald D. Identifying language impairment in children: Combining language test scores with parental report. Int J Lang Commun Disord. 2009;44(5):600–15.

    PubMed  PubMed Central  Article  Google Scholar 

  13. 13.

    Coplan RJ, Weeks M. Shy and soft-spoken: Shyness, pragmatic language, and socioemotional adjustment in early childhood. Infant Child Dev. 2009;18:238–54.

    Article  Google Scholar 

  14. 14.

    Helland WA, Lundervold AJ, Heimann M, Posserud M. Stable associations between behavioral problems and language impairments across childhood – the importance of pragmatic language problems. Res Dev Disabil. 2014;35:943–51.

    PubMed  Article  PubMed Central  Google Scholar 

  15. 15.

    Weismer SE. Specific language impairment. In: Cummings L, editor. Communication disorders. New York: Cambridge University Press; 2013. p. 73–87.

    Google Scholar 

  16. 16.

    Laws G, Bates G, Feuerstein M, Mason-Apps E, White C. Peer acceptance of children with language and communication impairments in a mainstream primary school: Associations with type of language difficulty, problem behaviours and a change in placement organization. Child Language Teaching and Therapy. 2012;28(1):73–86.

    Article  Google Scholar 

  17. 17.

    Mok PLH, Pickles A, Durkin K, Conti-Ramsden G. Longitudinal trajectories of peer relations in children with specific language impairment. J Child Psychol Psychiatry. 2014;55(5):516–27.

    PubMed  PubMed Central  Article  Google Scholar 

  18. 18.

    Whitehouse AJO, Watt HJ, Line EA, Bishop DVM. Adult psychosocial outcomes of children with specific language impairment, pragmatic language impairment and autism. Int J Lang Commun Disord. 2009;44(4):511–28.

    PubMed  PubMed Central  Article  Google Scholar 

  19. 19.

    Howe C. Peer groups and children’s development. Wiley-Blackwell: Chichester; 2010.

    Google Scholar 

  20. 20.

    Baines E, Blatchford P, Chowne A. Improving the effectiveness of collaborative group work in primary schools: effects on Science attainment. Brit Educ Re J. 2007;33(5):663–80.

    Article  Google Scholar 

  21. 21.

    Baines E, Blatchford P, Kutnick P. Promoting Effective Group Work in the Primary Classroom. London: Routledge; 2009.

    Google Scholar 

  22. 22.

    Kutnick P, Ota C, Berdondini L. Improving the effects of group working in classrooms with young school-aged children: Facilitating attainment, interaction and classroom activity. Learning and Instruction. 2008;18(1):83–95.

    Article  Google Scholar 

  23. 23.

    Johnson DW, Johnson RT. An educational psychology success story: Social interdependence theory and cooperative learning. Educ Res. 2009;38:365–79.

    Article  Google Scholar 

  24. 24.

    Tolmie A, Topping K, Christie D, Dolnaldson C, Howe C, Jessiman E, et al. Social effects of collaborative learning in primary schools. Learning and Instruction. 2010;20(3):177–91.

    Article  Google Scholar 

  25. 25.

    Chang YC, Locke J. A systematic review of peer-mediated interventions for children with autism spectrum disorder. Res Autism Spectrum Disorders. 2016;27:1–10.

    Article  Google Scholar 

  26. 26.

    Sills J, Rowse G, Emerson LM. The role of collaboration in the cognitive26. development of young children: a systematic review. Child: care, health and development. 2016;42(3):313–24.

    CAS  Google Scholar 

  27. 27.

    Brinton B, Fujiki M, Montague EC, Hanton JL. Children with language impairment in cooperative work groups: A pilot study. Language, Speech, and Hearing Services in Schools. 2000;31:252–64.

    PubMed  Article  PubMed Central  Google Scholar 

  28. 28.

    Kimhi Y, Bauminger-Zviely N. Collaborative problem solving in young typical development and HFASD. J Autism Dev Disord. 2012;42:1984–97. https://doi.org/10.1007/s10803-012-1447-6.

    Article  PubMed  PubMed Central  Google Scholar 

  29. 29.

    Murphy SM, Faulkner DM, Farley LR. The behaviour of young children with social communication disorders during dyadic interaction with peers. Journal Abnormal Child Psychol. 2014;42(2):277–89.

    Article  Google Scholar 

  30. 30.

    Blatchford P, Bassett P, Brown P, Webster R. The effect of support staff on pupil engagement and individual attention. Brit Educ Res J. 2009;35(5):661–86.

    Article  Google Scholar 

  31. 31.

    Pinto C, Baines EM, Bakopoulou I. The peer relations of pupils with special educational needs in mainstream primary schools: the importance of meaningful contact and interaction with peers. Br J Educ Psychol. 2019. https://doi.org/10.1111/bjep.12262.

  32. 32.

    Lindsay G, Dockrell J, Law J. Roulstone. Better communication research programme: 2nd interim report. Research Report DFE-RR172. 2011:1–130.

  33. 33.

    Adams C, Gaile J. The SCIP Manual: Social Communication Intervention Programme. Cheshire: Napier Hill Press; 2015.

  34. 34.

    Holt S, Yuill N. Facilitating other-awareness in low-functioning children with autism and typically-developing pre-schoolers using dual-control technology. J Autism Dev Disord. 2014;44:236–48.

    PubMed  Article  PubMed Central  Google Scholar 

  35. 35.

    Ploog BO, Scharf A, Nelson D, Brooks PJ. Use of computer-assisted technologies (CAT) to enhance social, communicative, and language development in children with autism spectrum disorders. J Autism Dev Disord. 2013;43:301–22.

    PubMed  Article  PubMed Central  Google Scholar 

  36. 36.

    McKinney A, Hotson KL, Rybicki A, Weisblatt EJL, Días C, Foster J, Villar SS, Murphy S, Belmonte MK. Point OutWords: protocol for a feasibility randomised controlled trial of a motor skills intervention to promote communicative development in non-verbal children with autism. Trials. 2020;21(109):1–12.

    Google Scholar 

  37. 37.

    Vogel F, Wecker C, Kollar I, Fischer F. Socio-cognitive scaffolding with computer-supported collaboration scripts: a meta-analysis. Educ Psychol Rev. 2017;29:477–511.

    Article  Google Scholar 

  38. 38.

    Cress U, Wodzicki K, Bientzle M, Lingnau A. Computer –supported collaborative learning for intellectually disabled pupils: Stimulating interaction by using a floor control mechanism. Computer-Supported Collaborative Learning. 2011;6:307–21.

    Article  Google Scholar 

  39. 39.

    Gal E, Lamash L, Bauminger-Zviely N, Zancanaro M, Weiss PL. Using multitouch collaboration technology to enhance social interaction of children with high-functioning autism. Phys Occup Ther Pediatr. 2016;36(1):46–58.

    PubMed  Article  PubMed Central  Google Scholar 

  40. 40.

    Piper AM, O'Brien E, Morris MR, Winograd T. SIDES: a cooperative tabletop computer game for social skills development. The 20th anniversary conference on Computer supported cooperative work, 4-8 November Banff, Alberta, Canada: ACM; 2006; 1-10.

  41. 41.

    Alt M, Meyers M, Ancharski A. Using principles of learning to inform language therapy design for children with specific language impairment. Int J Lang Commun Disord. 2012;47(5):487–98.

    PubMed  Article  PubMed Central  Google Scholar 

  42. 42.

    Ben-Sasson A, Lamash L, Gal E. To enforce or not to enforce? The use of collaborative interfaces to promote social skills in children with high functioning autism spectrum disorder. Autism. 2013;17(5):608–22.

    PubMed  Article  Google Scholar 

  43. 43.

    Grynszpan O, Weiss P, Perez-Diaz P, Gal E. Innovative technology-based interventions for autism spectrum disorders: A meta-analysis. Autism. 2014;18(4):346–61.

    PubMed  Article  Google Scholar 

  44. 44.

    Murphy SM, Faulkner DM, Reynolds LR. A randomised controlled trial of a computerised intervention for children with social communication difficulties to support peer collaboration. Res Dev Disabilities. 2014;35(11):2821–39.

    Article  Google Scholar 

  45. 45.

    Adams C, Lockton E, Freed J, Gaile J, Earl G, McBean K, et al. The social communication intervention project: a randomized controlled trial of the effectiveness of speech and language therapy for school-age children who have pragmatic and social communication problems with or without autism spectrum disorder. International Journal of Language and Communication Disorders. 2012;47(3):233–44.

    PubMed  Article  Google Scholar 

  46. 46.

    Pring T, Flood E, Dodd B, Joffe V. The working practices and clinical experiences of paediatric speech and language therapists: a national UK survey. Int J Lang Commun Disord. 2012;47(6):696–708.

    PubMed  Article  Google Scholar 

  47. 47.

    Adams C, Baxendale J, Lloyd J, Aldred C. Pragmatic language impairment: case studies of social and pragmatic language therapy. Child Language Teaching and Therapy. 2005;21(3):227–50.

    Article  Google Scholar 

  48. 48.

    Shulman BB. Test of Pragmatic Skills–Revised. Communication Skill Builders: Tucson; 1986.

    Google Scholar 

  49. 49.

    Wiig EH, Semel E, Secord WA. Clinical Evaluation of Language Fundamentals®-Fifth Edition (CELF®-5). Pearson: Bloomington, MN; 2013.

    Google Scholar 

  50. 50.

    Klem M, Melby-Lerva M, Hagtvet B, Halaas Lyster S, Gustafsson J, Hulme C. Sentence repetition is a measure of children’s language skills rather than working memory limitations. Developmental Science. 2015;18(1):146–54.

    PubMed  Article  PubMed Central  Google Scholar 

  51. 51.

    Bishop DVM. The Children’s Communication Checklist. 2nd ed. London: Harcourt Assessment; 2003.

    Google Scholar 

  52. 52.

    Goodman R. Psychometric properties of Strengths and Difficulties Questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40:1337–45.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  53. 53.

    Wille N, Badia X, Bonsel G, Burström K, Cavrini G, Devlin N, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Quality of Life Research. 2010;19(6):875–86.

    PubMed  PubMed Central  Article  Google Scholar 

  54. 54.

    Varni JW, Burwinkle TM, Seid M, Skarr D. The PedsQL 4.0 as a pediatric population health measure: Feasibility, reliability, and validity. Ambulatory Pediatrics. Ambul Pediatr. 2003;3(6):329–41.

    PubMed  Article  PubMed Central  Google Scholar 

  55. 55.

    Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, Baird J. Process evaluation of complex interventions UK Medical Research Council (MRC) guidance. London: Medical Research Council; 2014. www.populationhealth-sciences.org/Process-Evaluation-Guidance.html.

    Google Scholar 

  56. 56.

    Teare DM, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials. 2014;15:264.

    PubMed  PubMed Central  Article  Google Scholar 

  57. 57.

    Schulz KF, Altman DG, Moher D; CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010 Mar23;340:c332.

  58. 58.

    Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

    Article  Google Scholar 

  59. 59.

    Bryman A. Social research methods. Oxford: Oxford University Press; 2016.

    Google Scholar 

  60. 60.

    Read JC. Validating the Fun Toolkit: an instrument for measuring children’s opinions of technology. Cognition, Technology & Work. 2008;10(2):119–28.

    Article  Google Scholar 

  61. 61.

    Bull P. Communication Under the Microscope: The Theory and Practice of Microanalysis. Hove: Routledge; 2002.

    Google Scholar 

  62. 62.

    Markell R, Asher S. Children's interactions in dyads: interpersonal influence and sociometric status. Child Development. 1984;55:1412–24.

    Article  Google Scholar 

  63. 63.

    Murphy S, Faulkner D. Gender differences in verbal communication between popular and unpopular children during an interactive task. Social Development. 2006;15(1):83–109.

    Article  Google Scholar 

  64. 64.

    Murphy S, Faulkner D. The relationship between bullying roles and children's everyday dyadic interactions. Social Development. 2011;20(2):272–93.

    Article  Google Scholar 

  65. 65.

    Gottman JM, Parker JG, editors. Conversations of friends. Cambridge: Cambridge University Press; 1986.

    Google Scholar 

  66. 66.

    Kruger A. Peer collaboration: conflict, co-operation or both. Social Development. 1993;2:165–82.

    Article  Google Scholar 

  67. 67.

    Lloyd P, Boada H, Forns M. New directions in referential communication research. Brit J Dev Psychol. 1992;10:385–403.

    Article  Google Scholar 

  68. 68.

    Radziszewska B, Rogoff B. Influence of adult and peer collaborators on children’s planning skills. Developmental Psychology. 1988;24:840–8.

    Article  Google Scholar 

  69. 69.

    Anderson AH, Clark A, Mullin J. Interactive communication between children: Learning how to make language work in dialogue. J Child Language. 1994;21:439–63.

    CAS  Article  Google Scholar 

  70. 70.

    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  71. 71.

    Murphy S, Joffe V, Messer D, Crafter S, Radley J, Sunthararajah S, Bell K, Corbacho B, Fairhurst C, Rodgers S, Torgerson D, Welch C. Evaluating ‘enhancing pragmatic language skills for young children with social communication impairments’ (E-PLAYS): protocol for a feasibility randomised controlled trial study. Pilot and Feasibility Studies. 2019;5(75):1–13.

    CAS  Google Scholar 

  72. 72.

    Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharmaceutical Statistics. 2005;4:287–91.

    Article  Google Scholar 

  73. 73.

    Ferraioli SJ, Harris SL. Effective educational inclusion of students on the autism spectrum. J Contemporary Psychotherapy. 2011;41:19–28.

    Article  Google Scholar 

  74. 74.

    Reiter S, Vitani T. Inclusion of pupils with autism: the effect of an intervention program on the regular pupils’ burnout, attitudes, and quality of mediation. Autism. 2007;11:321–33.

    PubMed  Article  PubMed Central  Google Scholar 

  75. 75.

    Locke J, Rotheram-Fuller E, Kasari C. Exploring the social impact of being a typical peer model for included children with autism spectrum disorder. J Autism Dev Disord. 2012;42:1895–905.

    PubMed  PubMed Central  Article  Google Scholar 

  76. 76.

    Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from http://handbook.cochrane.org.

  77. 77.

    Lord C, Wagner A, Rogers S, Szatmari P, Aman M, Charman T, Dawson G, Durand VM, Grossman L, Guthrie D, Harris S, Kasari C, Marcus L, Murphy S, Odom S, Pickles A, Scahill L, Shaw E, Siegel B, Sigman M, Stone W, Smith T, Yoder P. Challenges in evaluating psychosocial interventions for autistic spectrum disorders. J Autism Dev Disord. 2005;35(6):695–708.

    PubMed  Article  PubMed Central  Google Scholar 

  78. 78.

    Smith T, Scahill L, Dawson G, Guthrie D, Lord C, Odom S, Rogers S, Wagner A. Designing research studies on psychological interventions in autism. J Autism Dev Disord. 2007;37(2):354–65.

    PubMed  Article  Google Scholar 

  79. 79.

    McConachie H, Parr JR, Glod M, Hanratty J, Livingstone N, Oono IP, Robalino S, Baird G, Beresford B, Charman T, Garland D, Green J, Gringras P, Jones G, Law J, Le Couteur AS, Macdonald G, McColl EM, Morris C, Rodgers J, Simonoff E, Terwee CB, Williams K. Systematic review of tools to measure outcomes for young children with autism spectrum disorder. Health Technology Assessment. 2015;19(41):1–538.

    PubMed  Article  Google Scholar 

  80. 80.

    Sciberras E, Westrupp EM, Wake M, Nicholson JM, Lucas N, Mensah F. Healthcare costs associated with language difficulties up to 9 years of age: Australian population-based study. Int J Speech Language Pathology. 2015;17(1):41–52.

    Article  Google Scholar 

  81. 81.

    Brady LJ, Apps for Autism, Texas: Future Horizons. 2011.

  82. 82.

    Safer-Lichtenstein J, Hamilton JC, McIntyre LL. Examining Demographics in Randomized Controlled Trials of Group-Based Social Skills Interventions for Individuals with Autism Spectrum Disorder. J Autism Dev Disord. 2019;49:3453–61. https://doi.org/10.1007/s10803-019-04063-4.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We are very grateful for the support and guidance of our Trial Steering Committee, including Professor Chloe Marshall of University College, London, who acted as the independent chair and Dr Susan Smith of Essex University, Dr Allan Clark of University of East Anglia and Katherine Farrow, Highly Specialist Speech and Language Therapist, our independent members, also Corinne Fox, our parent representative and Mary Hartshorne, Director of Outcomes and Information at I CAN, the UK’s communication charity. We are also grateful for the support of NELFT in particular Fiona Horton, Research Business Operations Manager, Sandeep Toot, deputy director of R&D and Nita Madhani, Assistant Director for Children and Young People.

Funding

This article presents independent research funded by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-0416-20035). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Author information

Affiliations

Authors

Contributions

Suzanne Murphy led on the design of the study and writing the article. All other authors contributed to the design of the study and drafting of the paper. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Suzanne Murphy.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was obtained from Cambridge Central Research Ethics Committee, REC ref: 17/EE/0320, IRAS Project ID: 227864. All participants gave informed written consent and in the case of children parents provided consent on their behalf. The trial was conducted in accordance with the principles of the Declaration of Helsinki (1996) and the principles of Good Clinical Practice (GCP). The chief investigator submitted regular interim reports and a final report at conclusion of the trial to NIHR, the funder.

Consent for publication

Written consent was obtained for the individual quotations above, all have been anonymised.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Murphy, S., Joffe, V., Donald, L. et al. Evaluating ‘Enhancing Pragmatic Language skills for Young children with Social communication impairments’ (E-PLAYS): a feasibility cluster-randomised controlled trial. Pilot Feasibility Stud 7, 5 (2021). https://doi.org/10.1186/s40814-020-00724-9

Download citation

Keywords

  • Social communication
  • Pragmatic language
  • Randomised controlled trial
  • Feasibility study
  • Young children
  • Peer collaboration
  • Communication impairment
  • Computer game