The Salvation Army (CaPS)

Question 1 – How have you adapted service delivery in response to the bushfires, floods and Coronavirus pandemic? When has it worked and when hasn’t it worked? How will this affect how you deliver services in the future? Have your service adaptations included better integration with other initiatives?

In response to the Coronavirus pandemic, CaPSS transitioned to online programming.
Face to face parenting workshops have been converted to interactive webinars; increasing CaPSS’ ability to connect with parents outside of the traditional CaPSS catchment areas. Manualised multi-session programs, groups, and individualised family support are also delivered. CaPSS has also developed accessible online delivery of parenting support via social media and email to maintain connection with parents and to provide additional assistance to parents in adjusting to working from home and parenting responsibilities during COVID. A key funding priority for CaPSS is community development. Over the Christmas/New Year period, CaPSS has undertaken research with parents regarding the parenting support they require in 2021. Initial responses indicate a strong interest in continued online delivery, alongside in-person programs. The therapeutic benefits of the programs have been maintained and the combination of face to face and online flexible responses foster engagement with hard to reach cohorts. Subsequently, CaPSS will continue with online delivery into the future, incorporating a hybrid model of online and in-person delivery. Delivery of this innovative model has already begun with Koorie Kids Homework Club and Parent Support Program having held one meeting per week as a ‘picnic in the park’ in the lead-up to Christmas. Other adaptions will include delivering multi-session online programs with one/two ‘meet ups’ and delivering workshops online and in-person concurrently. The online environment maintained our integration with other initiatives. This included attendance at our Network meetings focussed on parent education, early childhood, working with young people, people with disability and Aboriginal and Torres Strait Islanders (ATSI) and CALD local responses. In addition, CaPSS has established and maintained relationships with the following organisations: Mid West Health; Relationships Australia Victoria; Melton and Wyndham Councils; Spectrum; The Manor Kindergarten; Multiple schools across the catchment, primary, secondary, state and Catholic; Community centres (including Kirrip Aboriginal corporation and Wunggurrwil Dhurrung) across the catchments; Baptcare; Mackillop Family Services; IPC Health; Maternal Child Health Nurses; Djerriwarrh Health; Sunshine Hospital; Koorie Engagement Support Officers and DVVIC. The CaPSS Men as Fathers portfolio delivers the Western Fathers Inclusive Practice Network with Good Shepherd and Mid-West Health. This partnership enhances referral exchange and support to fathers, involves co-facilitation of programs, including shared learnings with other organisations, and assists to maintain professional relationships with other organisations and programs. In the future CaPSS hopes to see, as with program delivery, a hybrid model of online and in-person network meetings to facilitate new and continued connections with other services.

Question 2 – Are the proposed key outcomes for the families and children programs the right ones? Are there any major outcomes missing? How can we include strengths-based outcomes that focus on family or child safety?

Our suggestions focus on valuing identity as well as cultural differences across all families and children’s programs, empowering individuals to feel safe, empowering communities to influence improved family and child wellbeing, and ensuring the service system is equipped to support the above.
We offer the following comments for consideration:
• Weaving identity and culture into all outcomes
o All four key aims would be strengthened by recognising the importance of reinforcing families’ connection to culture and identity. We note that ARACY has recently developed resources for measuring children and young people’s sense of identity and culture.
o Celebrating culture must also include emphasising the value of communities that welcome diversity and differences.
• Family Relationships Flourish
o Suggested new strengths-based outcome: ‘Improved family functioning and harmony’ to add an outcome that specifically mentions ‘family’.
o There is scope to better recognise non-traditional families or ways of parenting. This includes those within kinship arrangements and cultures with large family (non-couple) connections and relationships.
• Children and Young People Thrive
o Amended outcome: ‘Optimal physical, mental and emotional wellbeing’, to recognise the benefits of healthy eating and the impact of childhood trauma on child development.
o Amended outcome: ‘Better engagement and connectedness at school’ to recognise the social benefits of attending school.
o Amended outcome: ‘Feel safe and supported’, removing the words ‘at home’ as children’s safety extends beyond the home. As the National Principles for Child Safe Organisations affirm, organisations and the broader community play a role in ensuring the safety of children, as do broader community protective factors.
o Amended outcome: ‘Linked to appropriate services and information’ to recognise the importance of access to reliable information and culturally appropriate information using plain English and offering options for people with low literacy.
• Empowered Individuals
o Suggested new strengths-based outcome: ‘Perceived safety and right to live free from violence and abuse’ to capture the importance of empowering individuals (in particular, women) to feel safe.
o Amended service linkages outcome as above.
• Cohesive Communities
o Suggested new outcome: ‘Communities have a say’. This acknowledges the importance of families and children programs listening to community views.
o Suggested new outcome: ‘Improved engagement between services’, to evaluate whether services are being provided in a way that is maximising accessibility for isolated families with complex needs. When families build trust and relationships through soft entry activities, this facilitates access to the targeted services they need. Organisations can enable this by working closely together.

Question 3 – What tools or training would support you to effectively measure and report outcomes through the Data Exchange Partnership Approach?

The Salvation Army agrees there is value in outcomes measurement to demonstrate where our work is and is not effective and to inform decisions that ensure services produce positive outcomes for individuals, families and our community. We appreciate the work done by DSS to support the sector to meet their Data Exchange requirements. We have reservations about SCORE as expressed in our response to Question 5.
Measuring Outcomes
We understand that the Broadmeadows Communities for Children (CfC) pilot had an ‘atlas’ – a compendium of evidence-based programs with guidance on the outcomes measurement tools best suited to measuring a range of baseline or benchmark socio economic indicators. These indicators ranged from data on pre-school and school attendance, early childhood and school readiness, and other community-based targets. Such guidance for the Families and Children Activity (FaC) programs would be invaluable.
Reporting Outcomes
The Salvation Army uses an in-house case management and statistical system to collect data across all our services and upload our data to the Data Exchange in bulk each week.
We currently report under the Partnership Approach for our Children and Parenting Support Services, as well as for other activities outside of the FaC Activity. To enable us to meet our Partnership Approach requirements, we have undertaken significant work to map our own outcomes measurement framework to SCORE. The SCORE translation guides have been useful, however the process is still time-and resource-intensive.
If Partnership Approach participation becomes mandatory across the FaC Activity, a transition period of 12 months at minimum would be necessary to allow for translation and organisational infrastructure changes.
Understanding How Data Is Used
Our experience in developing an outcomes measurement framework for The Salvation Army’s programs and services is that the value that program staff see in data collection has significant benefits for the quality (and therefore usefulness) of data collected. There is value in DSS ensuring service providers appreciate how their data is being used by government. This would also help us better communicate its worth to families.

Question 4 – Do you already have a program logic or theory of change outlined for your program? Did you find the process useful? If you do not have one, what has stopped you from developing one? What capacity building support would assist service providers to develop program logics and theories of change?

Our CaPSS program has completed program logics that align with The Salvation Army National Model of Care and National and State Agendas.

As identified in response to Question 3, the practitioner training delivered by Australian Institute of Family Studies (AIFS) is very useful and provides an opportunity for CaPSS managers to better explain to practice managers ‘why we are asking clients the questions that we do’.

AIFS are always responsive in offering assistance whether in relation to tools, procedures, and/or reviews of evidence-based data tools and approaches.

Question 5 – If longer-term agreements are implemented, how can the department work with you to develop criteria to measure and demonstrate performance? How can the Data Exchange better support this?

Performance Measurement through SCORE
Measuring social outcomes relies on active client engagement and participation. This is a particular challenge for soft entry programs, which are designed specifically to provide a ‘no-wrong-door’ approach for stressed and isolated families and to avoid actual and perceived stigmatisation of vulnerable high-risk families. These programs play an important role in engaging hard to reach populations and providing an important entry point to more specialised services. Requiring pre-engagement assessments jeopardises families’ willingness to engage.

Further, the most complex challenge is that of maintaining engagement for the ‘closure’ period of service delivery. It is common for parents simply ‘disappear’ after their period of participation, even if the experience has been extremely positive. Parents become reluctant to complete self-assessment and program evaluation feedback documentation. It would be very useful if some type of ‘incentive’ for parents is integrated into programs to assist case workers collect post session data for evaluation purposes and reporting against outcomes.

It is also challenging to capture outcomes for community development activities where we may only have one or two brief interactions with parents. Though it is useful to know how many of these interactions result in ongoing service engagement, this need must be balanced with the cost of obtaining this data.

Long Term Agreements
Long term agreements facilitate opportunities to gather longer term or even longitudinal based data at both the individual and community level. We applaud the new Activity Work Plan tool. It is a significant improvement to the previous tool and helps channel users’ thinking on evidence-based measurement and outcomes.

Access to funding of scheduled evaluations will be needed for long term agreements to provide the evidence to both organisations and the broader community on the levels of success and whether activities are achieving their intended outcomes. Ongoing access to webinars, such as the recent session run by DSS on ‘how to engage a university partner’, was welcomed. Opportunities to provide baseline data centred on the Australian Bureau of Statistics’ socio-economic indexes (SEIFA) to track community trends and changes in demographic data would be welcomed.

Community of Practice
We recommend continuing community of practice meetings. The previously scheduled bi-yearly meetings were very useful forums to share good practice and learnings. These sessions provide an opportunity for reflective practice and review of the evidence base, program outcomes and existing partnership arrangements.

Question 6 – What does success look like for your service, and how do you assess the overall success of your service?

CaPSS utilise a number of tools to assess the success of our services. These tools are included in program Activity Work Plans and include the following validated tools:
• Outcome rating scale
• Session rating scale
• Group session rating scale
• Parent empowerment and efficacy measure (PEEM)
• Feedback, suggestion and complaints mechanisms

We would greatly appreciate training and support from DSS to select and implement the most appropriate evaluation model.

Question 7 – Do you currently service cohorts experiencing vulnerability, including those at risk of engaging with the child protection system? If not, how does service delivery need to adapt to provide support to these cohorts?

As an early intervention and prevention program, CaPSS remains a critical and important part of the broader service system to divert vulnerable and at risk to the tertiary system.

Service adaption and future funding however, must align with population trends and changes in vulnerable communities to ensure that financial investments are aligned to need.

As noted in our response to Question 5, increased use of Australian Bureau of Statistics socio-economic indexes (SEIFA) as a long-term monitoring tool would be beneficial to all programs under the Family and Children’s Activity.

Question 8 – If you are a Children and Parenting Support or Budget Based Funded service provider, do you currently link with a Communities for Children Facilitating Partner or other regional planning mechanism to understand what other services are provided in the community and what the community identifies as their needs? How does this work in practice? Would you value the increased support of being attached to a local Facilitating Partner?

Linking CaPSS programs to a local facilitating partner would significantly enhance community development and partnership opportunities across all age groups.

This arrangement would also facilitate involvement in FaC forums and discussions, providing additional opportunities to leverage community information and good practice from existing partners.

Question 9 – For all providers, are there other ways to improve collaboration and coordination across services and systems?

Over the past year The Salvation Army CaPSS programs have been aligned to our family and domestic violence service stream to facilitate linkages and strengthen service collaboration for adult and child victim-survivors. This alignment has enabled responses to presenting individuals, parents and families across the continuum from prevention to early intervention and response. This delivers a comprehensive framework for responding to need, risk and safety to ensure a person-centred Model of Care.

As noted in Question 1, our CaPSS programs work collaboratively with a number of services to assist vulnerable families. We understand the intersectional nature of vulnerability and have established a strong partnership with Mid-West Health mental health services.

As identified in our response to Question 5, CaPSS program managers whole-heartedly support service provider forums to provide opportunities to explore further collaboration and share good practice innovative approaches.

Question 10 – The capability building support offered under Families and Children Activity programs has gone through several iterations. What works well? What do you think should change?

We appreciated the outcomes measurement training conducted by AIFS a number of years ago. We suggest that any future sessions would benefit from being better tailored to the differing levels of skill, knowledge, experience and organisational infrastructure.

We suggest at least two training sessions, a basic entry level introduction to outcome reporting and a more advanced training session. This provides opportunities for program managers to better understand data collection in the context of their own capabilities and their organisation’s infrastructure and resource constraints.

A clear demonstration of the potential benefit to organisations, as well as incentives to encourage ‘buy-in’ may also facilitate better attendance, particularly from organisations with limited resources.

Question 11 – Aside from additional funding, how can the department best work with you to support innovation in your services while maintaining a commitment to existing service delivery?

We welcome the department’s flexibility and support of innovative service delivery to fill gaps in services and deliver services in a different way for CaPSS programs. For example, the department’s flexibility allowed us to diversify and adapt responses to address emerging local needs.

Funding to evaluate programs that support innovation remains critical. We welcome the opportunities to maintain work with external organisations to help integrate innovative approaches into program and service delivery models.

Regular analysis and use of ABS SEIFA data to establish benchmarks/baseline data for communities will also strengthen opportunities for innovation to meet the needs of individuals, families and communities.

Question 12 – Is there anything else you would like to share about the ideas and proposals in the Discussion Paper?

Our CaPSS programs managers welcome any advancement in evidence-based tools and are happy to participate in any future discussions on the reporting of outcomes.

As identified in previous responses, we strongly support the building of a community of practice, access to a summary of family and children activities within communities/locations and the ongoing training provided by AIFS in supporting organisations and informing good practice.

Structured and regular analysis of ABS SEIFA data would be of great benefit to all program managers to support innovation, address service gaps and achieve outcomes.