CYP(3)- CP 12

Children and Young People Committee

Inquiry into Child Poverty

Reducing Risk and Increasing Resilience

How Did 'On Track’ Work?
Research Report DCSF-RR035
Deborah Ghate, Kirsten Asmussen, Yang Tian and Hanan Hauari
Policy Research Bureau, May 2008

Summary

The following consists of directly quoted extract from the Introduction and the Executive Summary of "How Did On Track Work”: text in boxes is directly quoted from the study itself.

The extracts primarily concern On Track’s work with Primary School age children and their families (unless otherwise stated).

PRB was unable to draw positive conclusions regarding adolescent behaviour / offending (although they were able to comment on the precursors to anti-social behaviour / offending) because:

• On Track was moved from the Home Office to the Department of Education in 2002 (Assembly in Wales) and its "mission” widened to a child well-being / social inclusion brief.
• On Track’s brief was to work with the 4 - 12 years age group
• The period of study did not allow a significant throughput from the On Track age group to adolescence.)

1. Introduction

This report synthesises results from separate stand-alone publications from eight studies that together comprised the national evaluation. The studies included:

• A tracking and monitoring study of approximately 1,100 On Track services and 17,000 service users
• Two longitudinal surveys (separate samples) of pupils in primary and secondary schools in On Track areas, involving approximately 30,000 pupils in

2001 and 20,000 pupils in 2004

• A longitudinal community profiling study of change over time in community indicators in On Track areas and matched comparison areas
• A longitudinal cohort study (panel sample) of over 500 On Track area residents and service users, and a matched comparison sample of families in non-On Track areas, measuring change for parents and children over the course of 12 months in the mid-point of On Track’s development (2004-2005)
• A qualitative follow-up of 36 families who took part in the cohort study
• A qualitative study of service providers and stakeholders in six 'exemplar’ On

Track project areas

• Qualitative research with head teachers and their colleagues from 21 primary

schools in On Track areas

2. Getting started; the development of multi-agency partnerships

• On Track was at heart a multi-agency, cross-sectoral initiative with partners drawn from education, social services, youth offending services, health, local councils and local and national voluntary organisations.
• Some agencies proved harder to engage than others… engaging schools as active partners took time and persistence in some areas. However, once schools had become convinced of On Track’s benefits and 'added value’ they became enthusiastic advocates and supporters of the programme.
• Factors that facilitated local projects’ survival over the national programme’s life course included strong leadership from On Track project managers; senior staff who acted as champions in participating agencies; project staff who thrived on the experimental nature of the work; a strong strategic lead from the local authority; designated project staff with specified duties aimed at promoting multi-agency partnership; projects who cultivated a high profile and high visibility locally; and the development of shared protocols and administrative procedures.
• Overall, the research suggested that projects that were clearly identifiable both in the community and to other agencies had better sustainability than projects that had a low or unclear profile.

3. The On Track workforce

The principal of working across disciplinary boundaries is now well established in the UK, underscored by key policy and practice restructuring into multi-agency Children’s Trusts… On Track projects to some extent nurtured the 'new breed of professional’ that is now required to operate in the new environment of increasingly integrated services.

• Project managers were identified as playing a crucial role. Many described their work in On Track as extremely challenging. There appeared to be no standard background that guaranteed success; rather, project managers needed to be highly flexible, and skilled in a range of areas from direct work with service users to business management and human resources. Those who were described as most successful were people with excellent local networks, able to pull in support from a wide range of external agencies, and also a good knowledge of the local community. They were often described by colleagues and stakeholders as having strong personal charisma and evident commitment to the On Track mission. Frequent changes of project leadership, on the other hand, were not good for projects’ longer-term survival.
• Project staff were generally rated highly by service users for competence and approachability.

4. The impact of On Track

Impact at the level of individual children

• A validated instrument for measuring emotional and behavioural problems used in the cohort study showed a decline in overall levels of difficulty for children in On Track areas and for children using On Track services. Over time, although children in the On Track samples still had higher rates than those in comparison areas (as they also had at the outset), the differences ceased to be statistically significant. The change over time was especially notable for one area of difficulty: peer problems.
• Reducing youth crime per se was not the aim in the short to medium term. Accordingly, there was only weak evidence of impact in reducing youth crime and antisocial behaviour at the level of individual children. However, in respect of the attitudinal and behavioural precursors to youth crime, the results were more promising, especially for younger (primary school aged) children.

General emotional and behavioural problems

• Individual reports gathered from the qualitative strands do suggest…  that On Track services resulted in improvements in some children’s well-being, with service users and providers both providing strong examples of how services were thought to have improved some children’s self-confidence, self-control and self-esteem.
• Over time, primary school aged children in On Track areas reported increased rates of ‘happiness’ with family life.
• Attitudes to bad behaviour grew markedly less antisocial over time amongst primary school children.
• In the primary schools survey disruptive behaviour at school had declined at Wave 2, with mean scores on a composite scale at Wave 2 significantly lower that those at Wave 1 A composite measure we labelled ‘challenging behaviour’, which measured children’s immature behaviour, also declined significantly.

Impact at the level of the family

• Some of the strongest evidence of impact was found at the level of the family.
• Protective factors at the family level – including parental coping, home-school interaction, relationships and involvement with children, and use of both formal and informal sources of social support – showed strong and consistent evidence of positive change that appeared to be associated with the presence of On Track.
• This was especially true for families with primary school children. Impacts were detected in relation to parents’ attitudes and practices, and were especially notable in relation to interactions between home and school. There was also evidence from several sources that parent-child relationships had improved over time.

Parenting attitudes, skills and behaviours

Monitoring, supervision and consistency of discipline

• Children reported significant increases in levels of parental supervision and monitoring... The increase was noticeably driven by boys: girls’ levels of supervision started and remained high, but boys reported increased supervision over time. In the primary schools survey, children (both boys and girls) reported significantly higher levels of monitoring and consistent discipline.

Parental warmth and involvement

• The findings were consistent with a picture of increasingly good relationships in families in the On Track areas, especially among
  those using On Track services with On Track samples reporting significantly increased rates of praise, whilst the rates decreased for the comparison groups. Change over time was significant.
• Children reported significantly more parental warmth and involvement… Looking at change between Wave 1 and 2, children at Wave 2 reported significantly more parental warmth and involvement at Wave 2

Feeling involved in school life

• There was also a substantial increase in the proportions of parents who reported feeling ‘very involved’ in their child's school life between the two waves of the cohort study… by Wave 2, On Track residents and, especially, On Track users were substantially more likely to feel this than comparison parents

Coping with parenting

• On Track services reported a number of encouraging improvements in relation to their ability to cope. Since parents’ perceptions of how well they are coping have been associated in other studies with a host of risk factors for poor child outcomes… this is an important and encouraging result.

Impact at the level of the peer group

T

he evidence for improvement in children’s peer relationships was encouraging. On Track did seem to have helped children in the highest need groups in particular to make new friends and to have more positive relationships with other children.

Peer antisocial behaviour

• Secondary school age children in the schools survey reported decreases over time in the level of antisocial behaviour by siblings and peers, as did primary school aged children in the cohort study, though in the latter the changes were not large enough to reach statistical significance…

Peer networks and relationships

• More positively, there was some evidence that the extensiveness and quality of children’s own peer networks and relationships had improved where they were resident in On Track areas or were users of On Track services. Peer problems, as measured by a validated instrument, diminished over time for children living in On Track areas or using On Track services.
• Children in the ‘booster’ cohort of high intensity services users (i.e., the highest need children) reported the greatest decreases of any group, and also gained the greatest number of new friends over time (having started with the smallest networks). Qualitative data supported these findings.

Impact at the level of the school

• Overall, there were positive changes at the school level in a number of dimensions, especially in relation to protective factors, although data on risk factors were subject to substantial inconsistencies.

Peer antisocial behaviour

Secondary school age children in the schools survey reported decreases over time in the level of antisocial behaviour by siblings and peers, as did primary school aged children in the cohort study, though in the latter the changes were not large enough to reach statistical significance…

Peer networks and relationships

• More positively, there was some evidence that the extensiveness and quality of children’s own peer networks and relationships had improved where they were resident in On Track areas or were users of On Track services. Peer problems, as measured by a validated instrument, diminished over time for children living in On Track areas or using On Track services.
• Children in the ‘booster’ cohort of high intensity services users (i.e., the highest need children) reported the greatest decreases of any group, and also gained the greatest number of new friends over time (having started with the smallest networks). Qualitative data supported these findings.

Impact at the level of the school

Overall, there were positive changes at the school level in a number of dimensions, especially in relation to protective factors, although data on risk factors were subject to substantial inconsistencies.

Impact at the level of the community or neighbourhood

• The 'logic model’ for the On Track programme suggested at the outset of the research that we would be unlikely to find community level impacts so early in the life of the programme… but some of the protective factors that were measured did in fact show positive movement over time.
• Truancy and exclusions, and attainment and performance
• Findings for these indicators… were mixed…. Overall, the evidence was not strong for an impact on truancy, with rates rising over time in some strands of the evaluation and falling in others… though there were stronger indications from the cohort study that for primary school children and for children and young people in the booster sample of high-need families, temporary exclusions had dropped substantially over time.
• Self-assessed attainment and performance at school showed positive changes for primary school…

Bad behaviour at school

• Primary school children taking part in different waves of the schools survey self-reported a substantial decrease in bad behaviour at school over time, and self-reports in the cohort study of bullying other children dropped in both On Track area and On Track user samples…
• Qualitative data included observations by professionals that general behaviour and levels of concentration in class had improved in some primary schools following On Track services being established.

School ethos, and satisfaction and involvement with school

• School level protective factors showed (an) encouraging picture…older children’s attachment to and enjoyment of school showed substantial and significant positive changes over time, as did secondary school pupils’ reports of involvement and participation at school.
• Primary school aged children also reported substantial increases on a measure of satisfaction at school, and very interestingly, these rates were also statistically significantly higher in schools with a ‘high’ level of On Track activity… Qualitative data from a range of sources supported these findings and overall, we concluded that protective factors connected with school life showed promising evidence of improvement as the On Track programme matured.

Impact at the level of the community or neighbourhood

• The ‘logic model’ for the On Track programme suggested at the outset of the research that we would be unlikely to find community level impacts so early in the life of the programme… but some of the protective factors that were measured did in fact show positive movement over time

Youth views of the neighbourhood, social support for youth, and out of school activities

• Youth views of the local neighbourhood got more positive over time for primary school children and especially so for children at schools with high levels of On Track activity.
• On the protective side of the equation… .in primary schools with a high level of On Track activity, children reported statistically significantly higher levels of this kind of activity compared to those in other schools.

Service uptake (services other than On Track)

• Service uptake appeared to increase in On Track areas once families were in contact with On Track, and showed strong increase amongst hard to reach section of the community. Some agencies may have increased their activity in On Track areas… with the exception of child and adolescent mental health services (CAMHS). This may indicate that On Track projects were absorbing some referrals that formerly would have gone into the CAMHS case load.
• In other words, those already in direct contact with On Track services also reported increasing use of other services over time.
• There were signs that increases in service use had been especially sharp amongst households… where children’s school performance was rated by parents as poor, or where children had abnormal levels of emotional and behavioural problems.
• These findings are good indicators that On Track projects were successfully reaching out to parents and children in higher need groups…

5. On Track as a multi-modal intervention

• The qualitative studies suggested that the ability to work in a cross-disciplinary way with families was an important factor in successfully meeting their needs.
• Perhaps the biggest 'added value’ of multiple service use came from multiple intervention use within families, where both children and their parents were accessing On Track services. The ability to produce simultaneous change in both parent and child was seen as important for sustained good outcomes. Workers did feel it was possible to see positive outcomes from services that only reached children, but that these might be slower to occur.

6. Mainstreaming On Track services

• Although the integration of specific On Track services into local core provision (for example, in schools) was the most common form of mainstreaming, there were also examples of On Track projects having changed or improved ongoing practice, refocused services for specific at-risk groups, provided training to statutory and voluntary agencies, and influenced strategic decisions...
• Processes that facilitated the mainstreaming of services included the availability of additional funding, support at the higher strategic level, and perceptions that the service was effective.
• On Track services that were recognised as effective in schools were particularly likely to be mainstreamed, especially if the school was able to obtain additional funding, or the local authority saw it as a priority.
• In addition, On Track services that were seen to support the Every Child Matters outcomes framework were also more likely to become integrated into the ongoing work of core health or educational services.
• Overall, it was clear that the On Track programme, both in terms of its staff and activities, added value to core services both for at risk families and for the community in general.

7. Conclusions

• The On Track 'journey’ has not been an entirely smooth one, including substantial problems created by changes in governance at national and local level and cuts in funding mid-way. In spite of this, many projects survived long enough to develop, test, and eventually mainstream the best elements of their service packages, and it was clear that in respect of multi-agency working, the On Track programme will leave a positive legacy in many, if not all, the areas in which it functioned.
• Schools became an increasingly major part of On Track’s identity… Family and parenting work was also an area in which On Track’s work appeared to be making a particularly valuable contribution.
• … There are many possible competing drivers of change, not least of which was the proliferation of other community-based initiatives that were co-located in (some) On Track areas. Use of other mainstream services locally may also have contributed to positive change, where it occurred.
• However… (taking) into account all the evidence gathered, we concluded that the presence of On Track was an influential factor in reducing risk factors and boosting protective factors for children and parents.
• For example, for some factors there were significant differences between primary schools with higher levels of On Track activity compared with those with lower or no On Track activity. There was also a range of significant differences between On Track service users and On Track area residents compared to carefully selected comparison areas containing families who were not exposed to On Track.

Children and Young People Committee
Child Poverty Inquiry
Note of visits to On-Track, Tylorstown
10.07.08

Mick Millman - Manager, On-Track
Peter Ford     - Teacher attributed to project
Helen Mary Jones - Chair CYP Committee
Claire Griffiths - Deputy Clerk
Helen Roberts - Legal Adviser

22.07.08

Mick Millman - Manager, On-Track
Catherine Silver - On-Track
Angela Burns - Member CYP Committee
Tom Jackson -Clerk
Sophie Williams - AMSS

These visits were arranged as part of the evidence gathering for the child poverty inquiry. Committee had received a response from the project to the consultation and following the visits additional information has also been submitted which is attached to this brief note.

Project initially successful research pilot funded by Home Office in 1999/2000. In 2001, funding moved to DELLS and consequently WAG. The original funding was for seven years enabling sufficient time to establish and revaluate the project.

80 LAs originally applied to be part of pilot; 2 successful in Wales (Tylorstown and Bridgend) out of 24 projects. Project ended in 2007 but RCT funding enabled the project to continue. All pilots different and one in Cornwall collapsed.

How pilot facilitated up to each project - Tylorstown built around Secondary school catchment area.

The Chair asked about evaluation. Mick Millman has sent copies of the National Evaluation (Executive Summary and Qualitative Summary).

One fact that shows success is the large decrease in umbers of pupil referrals from an average of around 80 to 7-12 per year.

While considering that there is no magic bullet to identifying children at risk, Mick Millman suggested there were three particularly high risk periods:

• Transition into pre-school/primary school
• Transition into secondary school
• Leaving secondary school

Mick Millman suggested that additional "one-off” project monies must be used with the ultimate aim of refiguring services, though this does not exclude trying new things. He added that the long term aim of this On-Track project was for mainstream services to undertake what the project is achieving.

Committee Service
August 2008

The National Evaluation of On Track, Phase Two.

QUALITATIVE STUDY OF SERVICE PROVIDERS’ PERSPECTIVES

Brief No: RB754, May 2006, ISBN 1 84478 742
Graham, J., Corlyon, J., Bhabra, S., Woodfield, K. & Ghate, D. with Hauari, H.

Summary

• The following consists of directly quoted extract from the published findings from qualitative research with agencies and workers who were either providing, or working in partnership with On Track services in six exemplar areas, and with key stakeholders and schools. Projects were anonymised in the published study - Tylorstown On Track was Project E. The extract summarises material relating only to Tylorstown On Track, or all six projects (including Tylorstown On Track).
• The research was conducted by the National Centre for Social Research and the Policy Research Bureau between September 2004 and September 2005.)

2. Understanding the remit of on Track Objective and Underpinning Theories

2.1 Managerial perspectives on the On Track brief

• Project E (was) much more focused on services and interventions to promote social inclusion… (but) were confident that in shaping their service provision around this focus they would naturally contribute to the prevention of crime and anti social behaviour and therefore did not feel their interpretations were out of step with the original intentions of On Track.  
• On Track was about showing how each of the services children and families may need should work together to support a child’s development.
• Overall, the flexibility and evolution of (the project’s) remit over time was seen as appropriate and a positive feature of the initiative… broadly speaking the essence of On Track as a flexible initiative evolving to meet need over time was an important element.

2.2 On Track service workers’ perspectives on the On Track brief.

• 'I think a lot of it is getting parents to realise what they actually do and supporting them in a positive role - to try and increase their self-esteem and confidence within their parenting role.’ (Project E service worker).
• 'Ultimately it’s about relationships, about expressing yourself, it’s about confidence, it’s about hope…opening different opportunities…instead of a blinkered or very narrow tube channel view of things.  It’s about trying to open a more obtainable world…and that’s about issues of hope and about not giving up.  It’s about issues of motivation and trying to help families with questions of a different array of ideas and tools.’ (Project E service worker).
• Mainstreaming of On Track services was visible in each of the six sites and was well received by most of those associated with On Track.

2.3 Stakeholders’ perspectives on the On Track brief.

• Understandably… their understanding of the role and remit of the project (was) generally less detailed than managers or workers although… participants were clearly well informed about the On Track brief… (as) the On Track project had a high profile and high levels of multi-agency interaction.

3. On Track models and referral Pathways

3.1 The Projects’ structural organisation

The On Track project operates) using a combination of universal services (open access offered… irrespective of need) and targeted services (restricted access aimed at specific children or parents, dependent upon need) both in small group and one to one work… Some universal services were delivered to all within the area such as, for example, transition work undertaken with all children in year six in a primary school.  

3.2 Partnership working

3.2.1 Co-delivery of services.

On Track services were typically delivered by the On Track Project staff.  However there were some exceptions to this general pattern.  These exceptions appeared in a range of different forms.

• Mainstreaming; involved co-delivery with other professionals with the aim of those professionals taking on service delivery.
• Linked parallel delivery; involved communication with other professionals outside On Track who were working with the same child, parent or family.  This happened in several ways…

3.2.2 Case discussions

There were several different ways in which (Project E was) involved in sharing information about children, parents or families involved in the On Track programme with other professionals.

• Within On Track case discussions
• Multi-agency case meetings convened by On Track
• Attendance at multi-agency case meetings convened by other services

Project E… held regular internal case review meetings for discussion between the On Track staff, but also regularly attended case meetings called by external agencies.

3.3 Strategic Partnerships

The On Track manager and staff… (were involved) in a range of strategic forums

3.4 Routes and pathways through On Track for service uses

A complex interplay was taking place… between a range of factors which influenced how integrated the On Track project was with wider networks of services for family and children.  These factors included:

• The level and formality of the referral mechanism
• The extent to which case reviews took place…
• The professional backgrounds and networks of individual On Track managers and front-line workers
• The nature of (the) local strategic partnership…

4. Implement on Track organisational structures and strategy

4.1 Organisational structures over time

In April 2001 responsibility for On Track passed from the Home Office… All projects experienced some difficulties and not all came through this period of transition equally well… Managing the project during a period of change which presented problems over and above the financial ones was an even more difficult task.  It is interesting to note that, at the time of the interviews, three of the projects had retained the same managers since the projects got underway but the remaining three has experienced several changes of manager.  The adverse effects of this lack of continuity on the projects had been further compounded by the fact that these projects had also gone through periods of time without an overall manager.

4.3 Contracts and retention

There had however) been workforce stability over practically the whole period of (Project E). Moreover, they expressed a considerable degree of job satisfaction and felt that they were engaged in valuable work in the community.

4.4 Secondment

(Secondment) … provided skills and expertise - valuable commodities in the early days of On Track - and an awareness of the local issues and key personnel.  An example of this is Project E where the manager, on secondment from a local statutory agency, had many years’ experience of working in the community.  He had drawn on his knowledge of the area and his perceptions of the main issues both when writing the On Track bid and when initiating and developing the services.  His existing network of contacts, and the degree of respect afforded him by workers in other agencies also meant that he had few problems in integrating On Track into the local infrastructure.

4.5 Team working

In Project E, staff cohesion was, according to stakeholders attending the local discussion group, a major factor in the project’s successful work in the community.  Participants commented on the extent to which the staff worked from the basis of a shared value system and were 'almost evangelical’ in their sense of purpose.  One member of the group commented on how 'everyone links with each other, and it’s an absolute pleasure to see them’.  This person speculated on how this occurred, given that staff came from a variety of disciplines and not all were employed directly by On Track. The consensus of the group was that it arose from a shared value system which stemmed from the charismatic nature of the manager.

'(the manager) is probably the key to everything’ (external agency stakeholder Project E).

'And I think you just can’t underestimate the input of (the manager) into On Track’ (External agency stakeholder, Project E).

4.6 Establishing and maintaining relationships

There was acknowledgement… that the On Track model of working had played a vital role in bringing agencies together, sharing workers’ expertise and thus, ultimately, benefiting families through the provision of appropriate services.

4.7 Strategic and frontline relationships

• Certainly a key factor in working relationships appeared to be personal contact with other professionals, this worked… in two ways: firstly, in setting up the partnerships and secondly in maintaining them… In the stakeholder group discussion held in (Project E) group members remarked that this was the key to their success:
• 'I think On Track works because of... this is my feelings, because of the personal contacts that they develop (External agency stakeholder, Project E).
• 'One of the key things has been the fact that the workers are from the community…it’s not judgemental, it’s all about helping them, about welcoming.  Much more easy for them to trust people who live and are from where they’re from.  And I think that does make a difference; (external agency stakeholder, Project E).
• The manager was also well known and well connected in the local community and this appeared to be crucial to the effective partnership working with other agencies.  One participant in the group discussion referred to his ability to 'pull people in along the way…. as a need is identified they’ve pulled the appropriate people in’ and, moreover, people had been willingly 'pulled in’.  On Track had, in their words, become a 'spider’s web’.

5. Delivering on Track Services

5.1 Mapping the services

• In Project E there was a similar range of activities and services to those offered by other projects operating mixed or contracted out models of delivery whereas project F… (for example) was strikingly different in its focus on mainly universal services based within school.
• Project E was offering both universal and targeted one to one work with parents and children, delivered in a range of locations including schools, local community venues and homes.  Its services were similarly varied in terms of their focus, some focused on transitional-support activities whilst others focused on crime prevention and raising awareness of issues like domestic violence and drug abuse…

5.2 The fit of On Track services with the original On Track model

• On Track activities were wide-ranging and often innovative… what was striking was… (that the project) had developed a customised suite of activities and services which each contributed in their own way to an agenda of social inclusion and resilience to crime and anti-social behaviour…  it has clearly demonstrated that as exemplar… (the project) had created workable initiatives which were operating on a number of different levels, and… in a number of different community contexts to help strengthen resilience to crime… and.. there was clear evidence that these types of impacts were starting to be observed.

5.3 Who were the users?

The key issues children and their families were encountering were described as Both separately and in combination:

For children

• Behavioural and emotional difficulties
• Failing to engage in peer group relationships and activities
• Being at risk of peer pressure in relation to anti-social behaviour and other risk related activities
• Being involved in risky activities such as truancy, drug use and criminal behaviour or exposed to family members engaged in these activities
• Having low confidence, little self-esteem and, or poor social skills
• Having difficulties either forming or maintaining stable and caring relationships with care givers
• Under achieving at school
• Having special educational needs, especially those resulting in disruptive behaviour or lack of concentration, for example ADHD
• … literacy and numeracy issues
• Having witnessed or been subject to domestic violence or other abuse

5.4 For parents

• Limited parenting skills or particular difficulties in relating to one or more child(ren)
• Poor family stability and, or, relationship breakdown
• Having had poor or fractured experiences of education as a child themselves, leading to mistrust and lack of engagement with their children’s schools
• Ongoing or past drug or alcohol misuse
• Ongoing or past criminal activity
• Poverty, low income and unemployment
• Ill health, metal health issues
• Families and children were not the only users of the services offered.  Professionals were another key user group, with their engagement with the initiative happening in a number of different ways.  Other workers who were being equipped through On Track training and mentoring…  should also be considered as users of On Track services.  In a similar way other local agencies working with families and children… were making use of On Track services for their clients either through referral or joint working.

5.5 Identifying the target group

• Broadly speaking On Track core staff (manager and On Track located workers) tended to have a higher awareness and greater clarity about the nature of their target group(s)… Workers were most likely to identify the On Track target group as those who fitted the specific aims and objectives of their individual services… Nevertheless this was often framed within an understanding of the early prevention remit of On Track.

5.6 Barriers to user engagement

• The location of On Track within the local community, the physical location of the services was seen as important.  
• Over time… (the project was able) to build up a local profile, maintain stability of services and respond to the needs of the community and the families…

5.7 Delivering On Track - encouraging multi-agency referral and co-operation

• Broadly speaking… the referrals that they received from other agencies were appropriate and suitable… Generally… difficulties had been countered by raising awareness and sharing information between agencies and were supported by continuity of staff and service provision.  

5.8 Delivery relationships

• Evidence of healthy, functioning delivery was found in the form of regular, effective cross-agency communication about cases and service provision, … Other indicators of good, cross-agency relationships included a broad awareness of the services provided by different agencies, respect for the different strengths offered by different approaches and a genuine willingness to place the family or child at the centre of referral and delivery decisions, rather than an emphasis on 'whose case’ the family was...
• On Track had eventually been largely successful in building relationships with schools and overcoming their initial reservations.  

6. ON TRACK IMPACTS ON PROFESSIONAL PRACTICE AND SERVICES FOR FAMILIES AND CHILDREN

6.1 On Track’s impact on local service provision

… there were also cases where other workers or agencies had decided to adopt the 'On Track model’ or approach.  Typically, this happened where other professionals had observed that the 'On Track model’ was working with families and children and had decided to try and transfer some key tenets or practices into their own practice…

6.2 The impact on practice for On Track staff and other service workers

• On Track service workers… reported a range of impacts on their own skills and practices as a result of working in an On Track team.  They also described how they felt On Track had impacted on the skills and practice of workers beyond the On Track team.  Some of these perceived impacts were echoed by members of the stakeholder focus groups suggesting wider impact on professional practice had occurred in line with the original objectives of the initiative.

6.3 Impact on On Track service workers

• Broadly speaking staff reported an increased skills base and a broadening of their approaches to working with families, children and other professionals.
• Professionals from particular sector backgrounds found that working in a collaborative way; providing joined up service delivery, inter-intervention referral and the wider discussion of cases had broadened their own perspectives and approaches, as well as heightened their understanding of the way other 'sectors’ and agencies operated.

6.4 Impact on professionals (and community members) beyond the On Track service worker

• Staff in our study described how they believed that (the On Track project) had, in some cases, impacted positively on the skills and practices of other professionals working outside of the On Track programme.  Stakeholders (from the local area) confirmed some of these perceptions.  These impacts had been achieved through a range of different actions or activities
• Another important impact of the On Track model of cross-agency working was that it enabled both On Track and non-On Track staff to increase their awareness of the wider range of services and agencies available within a local area.  This meant that professionals had other agencies to refer into, or communicate with, which helped ensure that the most appropriate packages of care could be designed for service users.
• 'On Track, the way it works and… because you know like what we were saying before is that actually they’re so good at sort of pulling people in, and different agencies, they’re very much a spider web as far as I’m concerned.  And not that I know all of it but (through On Track)… I know my place, what’s out there, who to talk to; (External agency stakeholder, Project E).
• Professionals attending some of the stakeholder focus groups… described how On Track seemed to be 'opening doors’ to work in schools for other professionals; the successful building of relationships and delivery of on Track services within these schools was felt to have laid the foundations for more professional activity within those schools in the future.  For example, a community police officer noted that… schools (in the OT area) were much more amenable to his request to deliver sessions on domestic violence than any other schools.

6.5 Impact on the workloads of other professionals

• The On Track project manager) and representatives of agencies beyond On Track commented on the impact of On Track on the workload of other professionals…
• In some cases it was reported that the level of referrals to other professionals had decreased over time…  
• In two areas (Project E and Project D) YOT managers attended the stakeholder focus groups and noted a decrease of referrals to YOT programmed over the period On Track had been working in the area.

6.6 Impact on the skills of some community members

• In addition to 'upskilling’ professionals, some On Track programmes specifically trained members of the local community to deliver services.  For instance parents (were trained) to facilitate parent support groups or networks … and yet another example included the training of young people as peer mentors in some capacity… The impacts of this 'upskilling’ on (the community) and individuals will be partly explored in the next of the qualitative evaluation focusing on service user perceptions and experiences.

6.7 The Impact of On Track on schools

Different kinds of impacts on school life were reported.  These included:

• Impacts on the range of services offered within a school environment;
• Shifts in the thinking and practice of school staff;
• Cultural shifts in how schools perceived and managed children who were seen to be disruptive and difficult; and
• Impacts on the level of referrals to other education related services.

It was felt that On Track had helped 'put another option in there’ (dealing with children who were seen as disruptive to the learning environment).  

Referrals to other education related services were also reported, to have changed over time. (The) educational psychologist noted that the emphasis in her referred case-load had changed.  She reported working with 'higher-end’ cases more, and having more time for these cases, since On Track had began working in primary schools within the area.  She attributed this to the work that On Track had undertaken.

Local stakeholders… pointed to the supportive role the (On Track team has) played in sustaining change among professionals and services through their ongoing relationships, support and accessibility to 'mainstreamed’ workers, but raised questions about how long changes would be mainstreamed after On Track dissolves.

7. IMPACT OF ON TRACK ON FAMILIES AND CHILDREN

7.1 Measuring outcomes and user progress

Stakeholders and project workers were able to identify a range of ways in which On Track had impacted upon professional practice and local service delivery

7.2 Mechanism used to undertake evaluation of change or progress

Evaluation of services was important not only to see the impact of services, but to assist in improving or refining services over their life course.

(Cases were monitored)… by a system of assessment and review, and was based on multiple interventions… within a package of care.  Case reviews looked at the progress made by a family on the particular issues they and the professionals involved had identified at the point of initial assessment.

(Feedback on single services)… was collected… from service users… from schools about the impacts seen, and through asking the parents of children who had received an intervention for their views on the impacts.

7.3 Monitoring information from questionnaires / research with service users directly

Questionnaires were employed in a range of different types of service.  Most, although not all, ascertained a base line score in a variety of measures and looked at changes to these at the end point of service intervention, or, in some cases at the mid-point too.

7.4 Monitoring changes seen in schools

On Track also sought information from the… schools in their areas.  In some cases specific figures were sought; such as school’s attendance records, the number of referrals to particular services or disciplinary panels in school… Less numerical feedback was also sought from schools… (the manager) made effort to listen to schools’ views on services and alter service provision as a result, but this type of feedback was also sought as a way of gauging the success of school-based interventions.  Service workers told how teachers had reported that specific pupils had improved their conduct in the classroom, or seemed more settled in school.

7.5 Monitoring change in children via parents

The views of parents were also sought by some services… Services that aimed to improve self-confidence, ones that looked at improving challenging behaviour and a transitions service all surveyed the parents of service users.  Response levels among this group were not often high, but there were reports that parents had given positive feedback to the projects.

7.6 Exploring impacts of On Track on families and children

A wide range of outcomes had been observed by On Track professionals… these included changes to:

• A child’s behaviour (at home or at school)
• Their engagement with and achievement at school
• Family relationships
• Parenting approaches
• Parents’ relationships with other agencies; and
• … changes at a community level.

The process of change was often described as being subtle and interrelated with other 'knock on impacts’; for example a change in achievement or engagement in school could help foster improved family relationships and improved relationships between parents and the school.  The interrelated nature of impacts was particularly heightened in cases where families were in receipt of simultaneous multiple interventions (whether these were all provided by On Track or not).

7.7 Changes to children’s behaviour

The changes reported in child behaviour related to the following:

• Improving self-esteem and self-confidence:
• Improving social skills and how children relate to their peers or adults;
• Changing attitudes toward anti-social behaviour; and
• Reducing disruptive behaviour in the classroom.

Services that reported seeing increases in self-esteem and self-confidence were often those, which were targeted and delivered to a small group or one to one with an individual child. These included therapeutic services and groups whose programme aimed to improve self-esteem.  

Improving a child’s self-esteem was also reported to improve other aspects of social interaction for them.

Other services also identified changes to children’s engagement resulting from children’s involvement with On Track initiatives.  These typically related to higher levels of engagement with other children and an increase in their social skills.

On Track counselling or therapeutic interventions also felt that they had had an impact on children’s thinking skills, which had helped children to minimise impulsive responses to situations.

Staff and stakeholders observed that these behavioural changes were carried through into the classroom and beyond.

7.8 Barriers and facilitators to changes in children’s behaviour

Parental engagement was thought to be a key factor in effecting change.  This arose in particular in relation to planning multiple interventions within a family unit.  

Service workers also felt that it was important that interventions were viewed as acceptable activities to a family, and particularly to the service user themselves. This entailed ensuring the content, location and timing of delivery were acceptable.

7.9 Changes to children’s engagement and achievement in schools

Positive changes were reported in relation to improved engagement with and enjoyment of school across a wide range of service provision

Improved attendance was reported as resulting from multiple interventions, intensive one to one work and/or effective transition work.  

On track was believed to have contributed towards (preventing exclusion) … by providing schools with a different approach to dealing with children at risk of exclusion, which avoided the use of disciplinary measures.

Support workers based in schools and mentors also reported seeing an increase in children’s enjoyment of lessons.  In other cases children seemed to enjoy school generally and particular lessons more when their own self-confidence had been increased through small group or one to one work.  

7.10 Barriers and facilitators to changes in children’s engagement and achievement in school

Flexibility and accessibility of staff working on interventions in schools were also described as key facilitators to positive outcomes.  If children were working one to one with a particular staff member it was thought to be helpful if a child could talk to their worker when a particular problem arose; in this way problems in school could be addressed before they escalated further.  Workers also discussed the importance of a speedy response to referral for one to one work.  

7.11 The impact of On Track on families

These impacts were perceived to fall into three main groups: changes to relationships within the family, improvements in parenting skills and strategies, and encouraging more openness to professional intervention and use of family support services.

7.12 Changing and improving family relationships

Although services that aimed directly at improving parenting skills and confidence described clear impacts on family relationships (see below) other services providing therapy and home visiting also felt that they had impacts on family relationships.

On Track interventions were described as helping to improve family relationships in a number of different ways: …by helping family members to recognise a problem or issue was present, the door for a solution to be sought was felt to be opened.  

7.13 Changes to parenting skills and strategies

This provision took a variety of forms: discussion groups for parents, structured group programmes around parenting skills, and in some cases, through one to one work. Parenting interventions had led to a number of different outcomes according to service workers:

• Improving parent’s confidence in their parenting skills;
• Encouraging parents to respond positively to their child(ren); and
• Helping parents to change their responses to their children’s behaviour

The group setting was believed to facilitate these changes by minimising a sense of loneliness in parenting difficulties and providing empathetic support from other group members around particular issues…

7.14 Bridging a gap between parents and other agencies

• This was felt to be a particular importance in relationships with schools, but service workers also felt they successfully supported parents’ in using other relevant services and sometimes liased directly with other service workers on parents’ behalf.
• Service workers also bridged the gap between parents and schools by ensuring that parents responded when schools called them and also by phoning or passing messages through to schools on parents’ behalf when there were issues at home they felt the school be aware of.
• In addition, service workers also felt they had successfully signposted, referred and supported parents in accessing other services beyond On Track.  

7.15 Community level changes

• Interestingly, initiatives around parenting skills and family relationships were also thought to have produced outcomes at the local community level…
• Parenting groups with positive dynamics were described as helping parents to find new social and support networks… fathers had taken to phoning each other during the week if they experienced a difficulty in relation to their child.  This was seen as a particular positive outcome of service intervention.  

7.16 Features of On Track supporting change

• The factors described above relate to specific types of outcomes.  However, there were also a series of key features of On Track, which helped to support and progress positive outcomes, such as…
• How effectively the service was reaching parents;
• How accessible the services / location of services were;
• How effective multi-agency working was in the project and the local area;
• How effectively the team worked to be adaptable to particular family contexts;
• How stable the team was and its ability to provide continuity in On Track staff; and
• How approachable, flexible and accessible staff were.

7.17 Providing multiple interventions within a family

• Another feature of On Track believed to be critical to creating change was the ability to provide provision of multiple interventions within a family where a higher level of risk factors were demonstrated.  
• The ability to engage parents and therefore produce simultaneous changes in both parents and the child was seen as important in providing sustainable change as it engendered change not just within one person but within the family… This was a key factor in ensuring the provision of an appropriate package of care for individual families at the higher end of the service needs…

7.18 The role of early intervention

• Also important was… (On Track’s) focus on working to reduce risk factors at an early point.  It was felt that the earlier services could intervene to work with these problems then the easier it was to effect sustainable positive change and prevent risk factors from escalating further.

8 Looking to the Future

8.1 The future of On Track - professionals’ views.

• The loss of a central project team with… a high local profile was… a worry for some professionals who felt the lessons learnt from the initiative might be undermined and have less potential (to) effect change without the support of an On Track team.  Others voiced concerns about how organisations like schools would be able to afford to continue with On Track projects once core funding was lost.

8.2 Suggestions for change and refinement of the On Track initiative

• In looking to the future On Track workers, managers and local stakeholders were asked how they would change the initiative for the better if it were to run again.  They had a number of different suggestions although few questioned the underlying remit of early preventative activities.
• Increased funding
• Amend the criteria for On Track participation
• Recognise the importance of multi-disciplinary expertise
• Greater clarity around objectives and national coherence

8.3 The legacy of On Track for families, children and the local area

Pprofessionals  were clear that the experience of On Track had taught them that early preventative interventions were possible and beneficial for families and increased the chance of moving a family forward.  Catching families at this point meant that workers are not always facing entrenched difficulties and therefore may have greater opportunities to effect more lasting change when compared to crisis interventions.

On Track, it was argued, was playing a key role in helping mainstream professionals to consider alternatives to their existing mechanisms for dealing with children who were at risk of becoming anti-social in their behaviour or relationships with others.

8.4 The legacy of the On Track experience for future policies targeted at families and children at risk.

The evidence collected during this strand of the evaluation has shown that it is possible to develop and sustain a proactive, multi-disciplinary early prevention agenda for work with families, children and communities (by)

• Providing a variety of interventions informed by different professional disciplines but within 'risk and resilience’ framework
• Delivering these services to a range of recipients including individual families and children, groups of parents or children, other professionals and the local community as a whole
• Making use a range of delivery models from one to one intensive work to groups based activities and self-help models
• Sustaining a change agenda

On Track did have coherence as a programme in its own right… whose distinctive features (are)

• A shared theoretical framework for early intervention.  
• A focus on identifying local need and being flexible in design and delivery to accommodate that need
• Multi-agency planning and delivery outside the standard statutory agency environment
• Multi-disciplinary team and approaches
• Multiple interventions wherever possible to strengthen outcomes
• Varying levels of formal case-management systems to facilitate professional communication and make delivery to families seamless
• A strong focus on schools