Question 1 – How have you adapted service delivery in response to the bushfires, floods and Coronavirus pandemic? When has it worked and when hasn’t it worked? How will this affect how you deliver services in the future? Have your service adaptations included better integration with other initiatives?
– YFS in Logan Queensland was quick to adapt to the Covid-19 crisis by setting up a Pandemic team and providing regular communiques to staff to assist in understanding concerns and expectations. Larger teams were split across two work sites to minimize the impact of any potential exposure and older and vulnerable staff were assisted to work from home where possible.
– YFS Social enterprise SubStation33 refurbished over 1000 computers and provided them at low cost to families – many to assist families who were required to home school to overcome the digital divide.
– Issues for client families included lack of access to digital resources, isolation and increased anxiety – adults and children, increased Domestic Violence,
The increase in financial assistance was mostly used very carefully by families to pay down debt, purchase important assets (better quality car that does not break down as frequently to assist with getting children to school), or support access to computers and data to support children’s schooling.
– Family Support model was adapted to – focus on crisis related needs rather than longer-term change, provide accurate and timely information about the Covid pandemic to isolated and vulnerable families to ensure they had access to the right supports for their needs, short safe home visits where possible using Covid safe practices, focus on sighting children to ensure their safety and wellness.
– Across YFS, clinical teams moved to electronic sessions via Zoom, Teams, What’sApp and other platforms as preferred by the family. While these were important to maintain connection and enable therapy to continue, it was identified that during the engagement period families responded better to face-to-face contact. Thorough assessments were difficult as it was difficult to read micro-cues using electronic platforms, and sighting children and home safety issues were less evident.
– YFS has updated forms to create digital versions, including consent forms, and provided staff with mobile devices to allow greater ability to work in a paperless and remote way.
Question 2 – Are the proposed key outcomes for the families and children programs the right ones? Are there any major outcomes missing? How can we include strengths-based outcomes that focus on family or child safety?
– Generally, the YFS team believe the outcome measures as illustrated by the concentric circles create a positive and comprehensive picture. Our FaRS team work predominantly with Young Parents including adolescent parents who can be seen as fitting in multiple domains; still children in their own right, with the responsibilities of empowered individuals and new families – it will not always easy to allocate to a domain. YFS has drafted an initial map against current Program Logic Outcomes for our FaRS and Special Family Violence programs against the DSS High Level Outcomes and Associated Outcomes.
– In most instances we were able to match our program outcomes with the DSS outcomes. These have been sent to FRSA.
– There was some discussion around the phrasing of the outcome statements – some indicate progress and others talk to an end aspiration – this should be made consistent to better support measuring outcomes.
– Focus on family and child safety – We are aware that many factors contribute to the safety of children. In our Program Logic we have added an aspirational outcome – Parents are able to protect and care for their children. We think it is important to consider the safety of children and are informed by the Department of Child Safety’s framework that assesses parents’ motivation (willing) and ability (able) when determining if a child is safe in the home.
– At a system level, we are very aware that, for many reasons, there is a significant bias in the family sector toward working with mothers. At program level we are seeking to consciously ensure we actively promote engagement with fathers and value their role within the family. It will be important to include outcome measures that relate to fathers so that the negative system response to fathers is not reinforced.
Question 3 – What tools or training would support you to effectively measure and report outcomes through the Data Exchange Partnership Approach?
– We have been participating in the DSS Partnership Approach for more than 5 years and have a lot of experience in reporting using SCORE. We currently use a process that involves the relevant family worker assessing outcomes in collaboration with a Senior Worker to ensure accountability, accuracy, and transparency. We use two tools, a worker rated tool and a client rated tool to feed into DEX to ensure data reflects a balanced assessment. In the past we have used the Parent Efficacy and Empowerment Measure (PEEM) tool with families to gain self-report information on program efficacy and outcomes. With the change of target group, we are now shifting to the use of the Personal Wellbeing Index-School Children (PWI-SC) to gather pre/post and intermittent self-report information from program participants.
In the past workers have rated outcomes directly against the SCORE tool. We are currently in the process of transitioning to a YFS developed Family Matrix tool to better assist with unpacking the domains, which can then be mapped to the SCORE domains.
Please note – any changes to the SCORE Matrix will have significant flow-on effects to our data gathering and reporting.
– It will be important that DSS provide clear and consistent definitions of Outcomes Statements to assist with consistent measurement across the many and varied organisations and services.
– Clarity of who should and should not be measured will be needed – during our work with families we often see other ‘family’ members come into the picture for varying lengths of time.
– To assist with ‘effective measurement’ it will be important for DSS to consider reporting timeframes. Currently families that enter our service in one reporting period and exit in a following reporting period do not show positive change because there is no way to link pre- and post-SCORE measures across different reporting periods. This means considerable valuable change information is not captured given that we use a flexible 12-month service period.
Question 4 – Do you already have a program logic or theory of change outlined for your program? Did you find the process useful? If you do not have one, what has stopped you from developing one? What capacity building support would assist service providers to develop program logics and theories of change?
– Yes, we currently have a Program Logic for our FaRS and Specialised Family Violence programs.
– Yes, we currently have Theory of Change for each program.
YFS employed an external facilitator to assist us with the development of both of these documents and we consulted with the team along the way to ensure we included their knowledge and experience, and they were clear about what was being developed and could align with the end document.
We found the process a useful way to gain clarity for the team around what our program was aiming to achieve, and the how, why, and expected timeframes. The process also helped identify staff knowledge gaps and training needs. It is important to note that the two (Program Logic and Theory of Change) are very different and serve unique purposes, and each is important in its own right
After an external evaluation of our FaRS service in 2019 by Griffith University, and informed by the Logan Together priority areas, we changed from a generalist family service to a focus on young parents and their children. During the transition process we re-visited and updated our Program Logic and Theory of Change, again using an external facilitator in collaboration with the whole team to ensure we developed a comprehensive and accurate document that workers felt accurately portrayed the work that they were doing. These documents are living documents and can be updated in line with new evidence.
YFS is currently in the process of developing an Organisational Theory of Change to assist in overcoming internal silos and creating shared understandings across the organisation.
Question 5 – If longer-term agreements are implemented, how can the department work with you to develop criteria to measure and demonstrate performance? How can the Data Exchange better support this?
YFS are very pleased that DSS has moved to five-year funding agreements and agree that this enables programs to build trust with families, community, services and staff and develop momentum and platforms to build on emerging data to create change. Currently YFS provides data 6 monthly in line with the DEX Partnership approach. We agree that an interim evaluation/ report is a useful way to reflect on the work that has been done and the data that has been generated. The Activity Work Plan may be a useful tool to assist with data analysis and promoting innovation based on emerging evidence.
YFS collect Pre and Post Organisational Outcome information across most of their programs in domains including Housing, Finances, Community Connections and Safety. This is collated and used to consider trends and themes across the organisation and across program clusters. For example, during COVID (March-September 2020), YFS experienced a surge in families experiencing domestic and family violence.
YFS has employed an Evaluation and Research Manager who regularly collates data at both program level and organisational level to reflect on shifts, trends, and themes. This information is fed back to Program Managers and teams to inform service delivery, and management to inform efficacy discussions and strategic direction decisions.
There was some discussion as to whether the current scaling of outcomes from:
Significantly Negative……through to ….Ok…….through to…….Better
is aspirational or strengths based enough as it seems to lean heavily toward the Negative end.
It will be important to also provide qualitative data to DSS eg. case studies to assist Contract Managers in understanding the data and to ‘paint the picture’ of the type of work that services are doing.
We reiterate in this section that 6 monthly windows for data reporting do not adequately capture change that occurs over a longer period of time, or even a change that occurs over a 6-month period that is spread over two reporting periods. Is it possible to measure cumulative change?
YFS appreciates the DSS consultative approach in progressing this agenda and recommends that an appreciative enquiry approach be continued when providing services an opportunity to provide qualitative information, e.g. “What have you noticed has gone well in the reporting period?” Asking the right questions can support innovation.
Question 6 – What does success look like for your service, and how do you assess the overall success of your service?
Until now, YFS has used a suite of evaluation tools including – SCORE, PEEM (Parent Empowerment and Efficacy Tool), Organisational Outcomes Measures, twice yearly Feedback Snapshot windows and client exit surveys. As previously stated, data from these tools is regularly reviewed and used to reflect on trends, themes and gaps including, where we are seeing the most and least change. YFS regularly employs external agencies including Griffith University and the Parenting Research Centre to evaluate programs from a sector informed and evidence-based perspective.
Target client group in our FaRS program shifted in 2020 after an external evaluation and Logan Together identified significant community need and a gap in service responses. We have shifted our client focus area to young parents and have now found that the PEEM tool does aligns well with this cohort, so will be moving to the Personal Wellbeing Index-School Children (PWI-SC) that better relate to young mums and dads.
For the target group for the Specialised Family Violence, YFS users the Movers Tool. Areas measured include: how the aggrieved feels about being able to respond and whether they are comfortable asking for help.
At service level our Program Logic for FaRS articulates a list of expected changes including:
– Parents are able to protect and care for their children
– Parents are able to access and maintain stable housing
– Parents are able to manage the household budget
– Parents are accessing pre-natal and antenatal care
– Children are achieving their developmental milestones
– Children are accessing playgroup, other early childhood service, or school
– Family communication and relationships including partner to partner and parent to child are improved
At service level our Program Logic for Specialised Family Violence articulates a list of expected changes including:
– increased knowledge and understanding of the impact of violence and trauma
– increased knowledge and understanding of options for improving safety
– increased capacity to action safety plans
– increased access to informal and formal supports
– improved information sharing and collaboration between police and service providers
Services are provided and tracked across multiple domains. Service provision is tracked fortnightly in Family Review Meetings between the worker and the Program Manager and documented in the casefile. Managers attend monthly supervision with Line Managers to reflect on Program related issues.
The YFS Research and Evaluation manager provides regular reviews of program data.
Question 7 – Do you currently service cohorts experiencing vulnerability, including those at risk of engaging with the child protection system? If not, how does service delivery need to adapt to provide support to these cohorts?
YFS clients come from vulnerable backgrounds including: low socio-economic status, high numbers of Aboriginal and Torres Strait Island people, refugees and culturally diverse people and people with disabilities.
A 2019 external evaluation of the YFS FaRS program identified that our generalist service had moved from an early intervention focus to working with greater complexity with little adaption of processes or tools or staff training. At the same time, the Logan Together ‘Road Map’ had identified a number of gaps in strategic services in the Logan region including support services for young parents. YFS saw an alignment between these two needs and started the process of re-designing their service to work effectively with young parents, including those who may have been in care as children and young people who are kinship carers. The YFS FaRS team includes an identified First Nations component – the Burrabilly worker, who works specifically with First Nations families.
The YFS FaRS Program Manager is also facilitating the Logan Together – Backing Young Parents community collaboration, auspiced by the Logan Together Leadership Table who are currently working to create community solutions for complex families with Child Safety involvement or at risk of entering the Child Protection system.
Young parents typically experience a significant combination of complexities including family of origin issues and lack of positive family support, housing instability, mental health, lack of education options, substance use, lack of finances, and social isolation. By intervening at a point of crisis with a young person still developing their own future objectives, in a multi-faceted and coordinated way, significant impacts can be made for both the young person and their child – in a two generational approach.
This transition has taken over 12 months (interrupted by COVID) and included refreshing of core program documentation including Operations Manual, Practice Manuals, replacement of an existing Assessment tool, considerable staff training and some staff movement, and new ways of working that better reflect preferences of this group including group programs, structured variation in service delivery intensity and increased use of electronic communication tools. YFS has devoted considerable resources to building community awareness and developing new referral pathways.
The YFS Specialised Family Violence Program works with vulnerable women who have experience domestic and family violence. We provide the second tier of response after the police have visited the home.
YFS agree that DSS funded services should be targeted toward the most vulnerable and those with little opportunity to pay for the supports needed for their family. YFS also recognises that it is more cost effective to provide ‘early intervention’ supports before crisis escalates to statutory level.
Question 9 – For all providers, are there other ways to improve collaboration and coordination across services and systems?
YFS is an active member of the Logan Together collaborative across several levels. Logan Together has links with the Logan Communities for Children Facilitation Partner and with the state funded Local Level Alliances, as well as multiple local network groups and government and non-government service providers. The aim is for Logan Together to provide an umbrella for multiple service providers and groups, maintain a helicopter view of local issues for families with 0-8 year olds, and provide strategic direction around gaps, emerging issues and local needs.
YFS CEO on behalf of the organisation has participated in the Stronger Places, Stronger People initiative in Logan.
YFS FaRS program participates in the Logan Complex Needs Panel facilitated by the Logan LLA which provides opportunity for collaborative responses for individual families who consent to involvement.
The YFS Specialised Family Violence Program known as Project HERA is a collaborative project with the Queensland Police Service, providing a holistic response to women who are experiencing domestic violence. The SFV worker participates in the Logan High Risk Domestic Violence Panel.
From a contract management perspective, it is important to recognise the evidence base reinforcing the importance of collaboration in providing better outcomes for families. This means that services need adequate resources to allow staff time to facilitate or attend meetings, conferences and training opportunities.
Question 10 – The capability building support offered under Families and Children Activity programs has gone through several iterations. What works well? What do you think should change?
YFS appreciates the emphasis and rigour of promoting evidence-based interventions combined with the flexibility to create innovative place-based responses that will contribute to the evidence base.
YFS self-funded and developed an innovative trial ‘coaching/resourcing program’ to explore new ways of working with families to achieve better outcomes. Some of our learnings from this two-year trail have been incorporated into the re-design of the FaRS service including:
– Locating specialist workers within family support teams can facilitate fast and targeted responses for crisis situations while avoiding the duplication of referring to a specialist service requiring another set of engagement and assessment functions. YFS have located a specialist Housing worker within the FaRS team as young parents frequently experience significant housing issues and are well positioned to learn effective housing skills early to prevent future homelessness.
– Using the point of crisis – in this case early pregnancy – as a window of opportunity to engage while young people are motivated to solve a problem and try new approaches.
Annual opportunities for providers to meet face-to-face at the FRSA sponsored conference allows for formal and informal sharing of learnings – these are important to building knowledge and consistency.
Question 11 – Aside from additional funding, how can the department best work with you to support innovation in your services while maintaining a commitment to existing service delivery?
We would be very interested in accessing amalgamated data that includes other DSS funded services in our region, and population data to reflect on trends and progress toward creating population level outcomes.