We understand that in the context of tightening budgets and scare resources it is becoming increasingly important to spend money on ‘what works’.
The RTK offer a range of robust evaluation services including impact, process and implementation evaluation. We are committed to understanding and meeting the needs of our clients by tailoring our services to ensure that they have a clear understanding of whether their services are having the desired impact. Commonly this involves projects that incorporate different types of evaluation into one high quality piece of work.
We have a wealth of experience of delivering evaluations for emergent, complex and heterogeneous interventions such as complex community-based programmes. We specialise in theory-based and realistic approaches with particular expertise in theory of change as is discussed further in our blog page. We work with closely with clients to unpick what they’re hoping to achieve through their intervention so that they can effectively measure whether this intended change has taken place.
Client: Leeds City Council
Title: Family Valued Evaluation
Brief: Leeds City Council received the largest grant of the DfE Social Care Innovation Fund for its programme ‘Family Valued’. This was an ambitious programme of whole system change which involved implementing restorative practice across children’s services and the social work service. LCC required a large-scale evaluation of the family valued programme.
Our Approach: We designed and delivered a comprehensive evaluation of this programme. Firstly, we worked closely with the client to develop an Outcomes Based Accountability (OBA) indicator framework and incorporated it within a theory of change. There were multiple strands within this evaluation including qualitative interviews, observations of practice, surveys of practitioners and families, analysis of administrative data and an impact analysis. This was important to give a holistic picture and to include a range of perspectives, we analysed this information and synthesised it into a clear report.
Impact: The robust, holistic and detailed evaluation gave LCC a clear view of the impact of the Family Valued programme in terms of what was working well and what could be improved. Vitally, we ensured that we worked closely with LCC throughout so that when the project ended, we had ensured there was capacity within the council to complete ongoing evaluation work. We provided them with tools and advised them on how to collect data to populate the tools in order to track ongoing performance management.
Client: The Youth Sport Trust (YST)
Brief: YST commissioned the RTK to an outcome and process evaluation to establish the impact of their Lead Your Generation: An Inclusive Futureprogramme, funded by the Big Lottery.
Our Approach: We used a Theory of Change approach and built an Outcomes Based Accountability framework from this. We gathered quantitative data and gathered further qualitative data through interviews and observations to add depth to our understanding.
Impact: The results of the project enabled the Youth Sport Trust:
- To identify how far the programme has delivered its intended outcomes;
- To understand how elements of programme delivery have either helped or hindered effective delivery; and
- To understand the relative financial costs of benefits of the programme.
Client: London Youth
Title: Evaluation of Athan 31- My team, My Club, My Community Programme project.
Brief: London Youth commissioned the RTK to deliver an impact evaluation of their project Athan 31–My team, My club, My community Programme project. Additionally, they also wanted an economic evaluation using a cost-benefit analysis.
The programme was designed to build on the organization’s aim of supporting young people coming together to lead and learn from projects they have created themselves with the support of a youth worker. Having successfully attracted additional funding, London Youth rolled it out across 40 youth clubs in the London area, involving up to 636 young people.
Our Approach: In-keeping with the ethos of the project, we incorporated partnership working into the project so that by the end of the evaluation we had upskilled staff at London Youth to continue their own performance management data collection. We assessed the impact of the project using a number of quantitative and qualitative measures. The mixed methods approach gave a comprehensive evaluation.
Impact: London Youth had a good understanding of how the project was working and the impact that it was having. They also had a good understanding of the economic case for the intervention, this understanding will assist them in building a sound economic case for future projects and applying for funding.
Client: Lambeth Council
Brief: Lambeth Council is at the forefront of a new approach to the way local authorities deliver public services; it has become one of the country’s first Co-operative Councils. This new approach redefines the relationship between the council and the citizens it services. At its heart is an assumption that public services are more responsive when power is shared between citizens and government. To support the effective roll out of this approach, Lambeth have commissioned theRTK Ltd to review ten existing evaluation reports in a way that provides a strong, structured narrative enabling future development to be based on a rigorous, evidence-based understanding of how best to create the conditions that lead to the successful delivery of coproduced outcomes.
As part of its commitment to rolling out the processes and procedures that will enable it to become a fully Cooperative Council, Lambeth Council is rolling out a major new prototype of Community-Based Commissioning (CBC), cooperative commissioning at a local level. The objective is to use the roll out of the prototype as an opportunity to improve the Council’s understanding of how CBC might enhance the delivery of importance services and outcomes, at the same time exploring whether and what internal organisational changes might be required were the decision scale up CBC across Lambeth. To that end, Lambeth has commissioned an evaluation of the prototype from theRTK Ltd and Queen Mary, University of London.
Our Approach: The consortium delivered an evaluation to support Lambeth in understanding the processes that underpin the effective delivery of CBC and to assess the extent to which CBC activity leads to the development of stronger communities and more cost effective delivery of service outcomes. We used a Theory of Change approach to deliver this project and worked in close partnership with the client to develop this.
Impact: This work provided the Council with a sound evidence base to support decisions concerning the future of CBC as a borough-wide service delivery strategy.
Client: Royal College of Speech and Language Therapists (RCSLT)
Brief: The Royal College of Speech and Language Therapists (RCSLT) required a process evaluation of their programme known as The Box. The Box is a training package and screening tool aimed at improving the awareness amongst criminal justice professionals of the contribution speech and language services can make to the delivery of an efficient, effective and equitable criminal justice system.
Our Approach: We delivered a process evaluation to monitor the delivery of the work at five different criminal justice settings across the UK. Our final report concluded that all planned activities were being delivered, we did identify some variations and made clear suggestions for how each of these could be resolved by adapting delivery.
Impact: The report allowed the RCSLT to understand whether the planned activities were being delivered and to resolve situations where there were variations to the planned delivery.
Client:General Medical Council
Title: Identifying unmet needs from the Gateways to the Professions guidance
Brief: The General Medical Council (GMC) wanted to update their guidance for medical education providers about how to support disabled learners.
Our Approach: We used a mixed methods approach to draw out the barriers and enablers to using the guidance to support disabled learners.
We developed and disseminated an online survey for staff involved at undergraduate and postgraduate stages of medical education.
For the qualitative elements of the research, we took a case study approach and designed a sampling framework to select ten medical schools across the UK. We conducted depth interviews with staff at both undergraduate and postgraduate levels in a range of roles including heads of medical schools, disability support officers and postgraduate deans. We also interviewed disabled medical students who were at different stages in their medical education.
We exceeded our participation targets and in total we interviewed over 40 staff and stakeholders and 25 students despite the research taking place in a short timeframe and during the students’ summer break. Our creative approach ensured that participation was not burdensome and our sampling methods (convenience and snowball sampling) along with our personable nature meant we got a high level of participation.
We conducted rigorous analysis of the data, for the qualitative data this involved developing an analysis framework and conducting a thematic analysis and triangulating this data with the quantitative data.
Impact: We supported the GMC with dissemination activities including presenting at a meeting for key stakeholders and submitting an article for an open journal. The findings from the project will directly influence the content of the new guidance, the format of it and how it will be introduced and disseminated.