Communities for Children FP Katherine Committee
Question 1 – How have you adapted service delivery in response to the bushfires, floods and Coronavirus pandemic? When has it worked and when hasn’t it worked? How will this affect how you deliver services in the future? Have your service adaptations included better integration with other initiatives?
_Some project adaptations:
‘Titjimbat Biginini Kaltja’, led by Jawoyn women at Barunga, demonstrated the benefit of programs being led and staffed locally. The coordinator could not visit, so local staff quickly took on increased responsibility for data collection, decision making, and delivery.
Dedication shown by front line staff during this time cannot be overstated. BabyFAST and Play2Learn projects did huge amount of work to reduce families’ experience of isolation during this time, and moved from in-person to digital format. Projects created resource kits hygienically, delivered sessions via Zoom or Facebook establishing virtual play groups. Participants valued the adapted delivery continuing to engage digitally: “I loved the way the team supported us. I feel so blessed to have been able to participate. It made isolation during COVID shut down much more bearable.” (BabyFAST participant)
Smiling Mind and the Song Room provided digital/online training, support and resources to staff in remote schools whilst communities were locked down, utilizing this opportunity to create digital resources that can be used in the future. This highlighted the digital divide and the importance of remote communities have access to adequate internet infrastructure: “I actually found the webinar very useful and the facilitator excellent. It dropped out for us a few times but that was because of technology on our end. I practice mindfulness myself but was unsure how to do this in the classroom and am very excited to try this!” (Smiling Mind participant)
_Facilitating Partner Response:
The benefits of place-based frameworks were highlighted. Successful adaptations were only possible through knowledge of the local context and collaborative relationships.
Project plans for 20-21 FY contracts were adapted to include Covid19 contingency plans. We continue to ensure we support projects to be responsive to changing community needs.
Greater ability for staff to work from home within CfC FPs and across the Smith Family as a whole. This supported staff, particularly working parents with increased flexibility to work from home.
Increased capacity to host with online platforms and hybrid formats.
_The importance of face to face delivery.
The benefits of face to face delivery must not be devalued moving forward. We strongly highlight it is not in the best interests of the region to rely on solely digital delivery, at the expense of face to face. The increased capacity to utilize digital technology complements and augments in-person delivery.
Collaborative partnerships between organizations with aligned goals is a central part of CfC FP. Opportunities for networking and human connection are already limited in a remote context. Digital means enabled relationships to be maintained through at a difficult time, when face-to-face was not possible. Feedback on online delivery: “I only wish it was in person, but thoroughly enjoyed it.” (BabyFAST participant)
Question 2 – Are the proposed key outcomes for the families and children programs the right ones? Are there any major outcomes missing? How can we include strengths-based outcomes that focus on family or child safety?
CfC FP program would improve if it were consistent over various sites and across different organizations. A National Outcomes Framework for CfC FP, or program logic would help achieve this. This would support consistent knowledge and expertise about strength-based outcomes within the FAMs, better placing them to support sites.
In the discussion paper outcomes framework diagram ‘Health’ appears as ‘not in scope’. This is inconsistent with currently CfC FP objectives. Also, all the mental health aspects within the ‘Children and Young People Thrive’ circle cannot be viewed as in scope while health is ‘not in scope’. To be successful as an outcomes framework, it needs to take a holistic perspective and incorporate public health principles.
In 2019 the Katherine CfC FP activity included the ‘Hearing Children’s Voices in the Katherine Region’ project which aimed to support the right children to have a voice on issues that relate to them. ‘Being active’ was a key theme, which is not included in the outcomes framework. Themes of feeling safe and respected were also key themes. The associated outcome ‘Feel safe and respected at home’ could be adapted to ‘feel safe, respected and supported at home and school’ to be a more accurate reflection of children’s needs, as identified by children through this project.
It would also be of benefit to provide a key aim which encourages service providers to reflect inwards. A holistic application of the strength-based approach utilised within the diagram requires a shift within the service sector to encourage service providers to work in collaboration with service participants and community members as equal contributors rather than recipients of services or charity.
Examples of outcomes related to this include:
• Sharing positive stories of community members
• Incorporating children’s perspectives into planning
• Ensuring ‘buy in’ from the local community
• Community representatives are offered genuine decision-making power
We were glad to hear at the online forum that the outcomes framework will be mapped to new appropriate DEX domains. It was reassuring to be informed that maintenance of positive results will also be valued in addition to measuring change, and that different ways of measuring are being considered. This is required for early intervention programs.
Question 3 – What tools or training would support you to effectively measure and report outcomes through the Data Exchange Partnership Approach?
If use of SCORE and Partnership Approach becomes mandatory, it is important DSS provide a tool that is fit for purpose. The current mismatch of SCORE and DEX to CfC FP projects undermines data fidelity and outcomes analysis.
The Partnership Approach would benefit from providing a better user experience and being more flexible to the needs of early intervention programs. Areas to improve:
_Users being able to define if a participant entered in DEX is statistically significant before being included in the total ‘assessable’ clients, with the ability to adjust by SCORE domain.
– Program attended by parents/carers and newborns displays low percentage of clients assessed for satisfaction. This reflects that newborns cannot take satisfaction surveys, not that program staff did not assess enough clients.
– Program guidelines dictate that participants are statistically significant for assessment after attending a minimum number of sessions. It is not currently possible with DEX to see what percentage of those eligible for assessment have been assessed.
– Playgroup style programs often have one-off guests (i.e. grandparents). Entering one-off participants artificially inflates the total.
_The ability to attach notes that provide qualitative data, and explanations for inaccuracies alongside the quantitative data. Frequently, the data does not accurately reflect service delivery and service providers are concerned they are unable to attach small notes explaining their context.
_Genuine reciprocity and commitment to ‘two-way sharing’ from the department. DEX training material suggests a commitment to two-way sharing. It is unclear how the current iteration of DEX enables this. Community Partners and Facilitating Partners input data in DEX. This is complex and time-consuming requiring specialist knowledge of the data from DEX and SCORE Partnership Approach. Often data cannot be used for project improved, or to glean a clear picture of project delivery.
_Support and training which is clear, has time for staff to ask questions and is delivered by knowledgeable/responsive DSS staff. DSS could ensure all training/helpdesk staff have time, communication skills and program knowledge to engage helpfully and respectfully with service providers.
– Recently, Katherine CfC FP hosted two Q&A sessions for community partners (CPs) with the FAM and a DEX staff member. CPs appreciated the time from the DSS staff and felt as if they were being listened to, and having the difficulties using the system acknowledged.
– When the DEX helpdesk cannot assist the FP with enquiries but can demonstrate understanding of the issue, this encourages sustained use of resources provided, through an interaction which is helpful rather than frustrating and time consuming.
(Please also see answers to Q.5, Q.10, Q.11 and Q.12.)
Question 4 – Do you already have a program logic or theory of change outlined for your program? Did you find the process useful? If you do not have one, what has stopped you from developing one? What capacity building support would assist service providers to develop program logics and theories of change?
As mentioned above, a National Program Logic for all CfC FPs mold support alignment of outcomes and deliverables.
AIFS sessions training for all CfC FP and CP staff to learn more about outcomes-based accountability and support to create translation matrixes would be useful.
Funding support for outcomes evaluation capacity development for CPs (particularly with high staff turnover in many projects) would help CPs meet required outcomes and have a greater understanding of why DSS is moving towards an outcomes-based model.
The process of developing a project logic and/or theory of change for the CfC FP program could enable the committee to clearly articulate their vision. However, as CfC FP activities are so varied, it is important any support or templates provided by the department are not too restrictive or imposing inflexible reporting requirements which would make them unfit for purpose.
It is also important to make sure program logics and theories of change further enhance innovation and program design and do not act as an impediment. Sometimes, in the process of doing the work, the underlying assumptions of your program logic are found to be incorrect and can be improved. Program logics and theories of change can be useful as a communication tool and to promote reflexive thinking. However, as a reporting tool it would need not to limit those applications.
Question 5 – If longer-term agreements are implemented, how can the department work with you to develop criteria to measure and demonstrate performance? How can the Data Exchange better support this?
Outcomes-based accountability (OBA) is important for the purpose of reflecting on service delivery, demonstrating the successes of a project and finding areas for improvement. To have continuous improvement, outcomes data should not be compliance driven as this often negatively impacts on relationships. Strong relationships are inherent to the delivery of early intervention programs, and there is a risk that compliance could become focused on justifying rather than learning.
Transparency around what the outcomes produced in DEX are used for at the Department level would increase willingness to participate and create more meaningful engagement of organizations with the data reporting process. DEX also needs to demonstrate that it operates within the principals of Indigenous Data Sovereignty.
Having the flexibility to measure what is appropriate to a project, in clear language that is relevant to the project would help with clarity in DEX. As an example:
– The description ‘GOAL SCORE improved for 60% of clients’ is unnecessarily complicated, relying on familiarity with DEX jargon. What it means is:
60% of clients improved in a project outcome, which the project staff have entered under the one of the domains (example ‘skills’ and ‘behaviours’) within the ‘GOAL’ component of SCORE.
It is possible to write ‘60% of clients improved their GOAL for [domain x] and [domain y]’ (example ‘60% of clients improved their GOAL for skills and behaviours’). This would help to clarify measurement criteria and have productive conversations about demonstrating performance that don’t get lost in jargon. SCORE outcomes data could be easily read with their corresponding translation matrices per project.
Flexibility to use outcomes data measures other than ‘change over time’ likert scales and a review of the ‘one size fits all’ approach. Not all programs are suitable for this style of data collection. This needs to be recognized.
Examples of impact:
– An external evaluation of the Titjimbat Biginini Kaltja project found that: ‘The Program coordinator and evaluator have identified several issues with the quality of the data in DEX. The system is a ‘one size fits all’ for DSS funded projects but some aspects do not suit the TBK project design’
– Program staff from the Holding Children Together project have expressed concern that the DEX data system assumes a very linear experience of trauma, which does not align with the experiences of children within the program or principles of trauma informed care.
Question 6 – What does success look like for your service, and how do you assess the overall success of your service?
Success, for a Facilitating Partner, looks like:
• An engaged and active committee, representative of our local community.
• FP is aware of and responsive to community needs. Awareness of our own work and of the broader work and relationships across the community and how they interact.
• Prioritising both others’ engagement with us and our engagement with others.
• Valuing and investing in relationship building
• Our work and that of our community partners is culturally appropriate
• Upholding the rights of the child in all we do, including their right to have a say in decisions that affect them.
We measure this through:
• CCC engagement and attendance; meeting diversity and representation targets in our CCC membership.
• Engaging with and participating in: Network meetings and community events
• Success of the CP projects funded through CfC FP. Alignment of these projects to our CSP.
• Being responsive to community needs and able to adapt or support adaptations of projects in a timely manner where necessary.
• Evidence of children’s voices having meaningful input into the work we do.
• Examples of sharing the positive stories of the region.
Question 7 – Do you currently service cohorts experiencing vulnerability, including those at risk of engaging with the child protection system? If not, how does service delivery need to adapt to provide support to these cohorts?
• From the discussion paper it was unclear how much and whether the perspectives and input of Aboriginal and Torres Strait Islander leaders and organisations were included. There seems to be an underlying assumption that Aboriginal and Torres Strait Islander children and families and culturally diverse families be viewed only as families requiring intensive support and service recipients, rather than the people with the knowledge and insight that will lead to solutions to the very complex problems CfC FP is aiming to address.
• The Smith Family’s submission to the productivity commission inquiry into the expenditure on children in the Northern Territory highlights that it is vital for commissioning processes to prioritise self-determination and culturally appropriate services. We suggest expanding evidence-based program requirement to include programs delivered by Aboriginal Organisations (even if not on the AIFS list or having gone through an AIFS evaluation). The advantages of this would be twofold: firstly, it is a practical way to recognise the overwhelming evidence that Aboriginal organizations are best positioned to deliver culturally appropriate and place-appropriate services; and secondly, few of the AIFS approved evidence based programs, have been evaluated for use in remote Aboriginal communities.
• Consider reviewing the Katherine Region with a place-based approach, rather than population/geographical area. As noted in the Smith Families submission to the expenditure on children in the NT, the Katherine region is geographically the largest CfC FP site, covering over 326,326.7km^2. To authentically apply a place-based approach requires local governance with ongoing meaningful ties to community which is simply unattainable across a footprint that large. Currently, the region is defined by SA2s, which means our footprint overlaps with some programs such as WYDAC and Yugul Mangi’s Stronger Communities for Children programs. More flexibility to define the region outside of pre-determined SA2s would support place-based and needs-based model. (Also see response to question 9 regarding Pine Creek.)
Question 9 – For all providers, are there other ways to improve collaboration and coordination across services and systems?
• As recommended in the expenditure on children in the Northern Territory report it would be beneficial to have better coordination between the NT government and federal government both in terms of funding and services.
Utilising the same delivery areas, with some flexibility for service providers to use their best judgement whilst this is established would be beneficial. Staff and organisations on the ground in the Katherine Region service sector have strong working relationships, the different boundaries established by different governments create barriers, contributing to fragmented service delivery and leave communities some marginalized.
For example, the town of Pine Creek is part of the Big Rivers Region, defined by NT Government. But it is not included in the SA2s defined by the Communities for Children FP boundaries. This results in a town an hour from Katherine and locally considered to be part of the region that is ineligible to engage with Communities for Children programs, despite similar demographic needs, a shared identity, and a desire to do so.
Better coordination and more flexibility to collaborate across different service sectors would strengthen the services system. Addressing the needs of children aged 0-12 in a small community necessitates strong collaborative relationships between service sector staff and education staff. ‘On the ground’ in Katherine there is an effort to work collaboratively, but this can be limited by reporting requirements, funding agreements, policies etc. The goals for children are similar and a culture of collaboration at a high level would improve the capacity of CfC FP projects to be collaborative.
Question 10 – The capability building support offered under Families and Children Activity programs has gone through several iterations. What works well? What do you think should change?
DEX must be supported and endorsed by AIFS to ensure that ‘outcomes’ produced are relevant to and benefit organisations and contribute to improving delivery of services.
CPs are eager to incorporate evidence informed approaches into their service delivery, and to demonstrate the long-term impact of their projects. As noted in the discussion paper, a lack of buy-in and being time poor can limit service providers’ ability to incorporate evidence informed practice. Improving the user experience of DEX, decreasing the complexity and the time required to interact with the system, would support a shift towards outcomes-based thinking in services’ project planning.
Question 11 – Aside from additional funding, how can the department best work with you to support innovation in your services while maintaining a commitment to existing service delivery?
If additional funding is unavailable, providers’ capacity to provide same-service or implement changes to improve service is limited and, due the increasing cost of remote service delivery over time, a decrease in service provision may be unavoidable. The funding levels for Communities for Children have remained roughly the same over the last decade. This will cause, and has in some cases already caused, a shift towards town-based projects and away from delivery in more remote community, as the cost of delivery in remote areas has increased, while the funding has not. This is ultimately a decrease in investment of Communities for Children activities in more remote areas, causing already vulnerable populations to be more isolated and have less access to services.
The Katherine CfC has had 4 different FAM’s over the last 12 months. Having longer term, FAMs who have the time to learn the different program iterations and to become familiar with their specifically allocated CfC FP and region would strengthen the department’s ability to support the CfC FP.
The amount of material provided by the department can be overwhelming and jargon heavy. Ensuring departmental training material, directories, protocols, guidelines and handbooks etc. are succinct, in plain English and not overwhelming in quantity would promote use of and engagement with the materials available.
Question 12 – Is there anything else you would like to share about the ideas and proposals in the Discussion Paper?
There could be more flexibility to be able to use DEX data in a way that is helpful to services providers. Decreasing admin time would assist in making use of DEX a higher priority and would increase buy in. Data Exchange promotes the ability to produce reports and internally utilise the data collected as one of its benefits. However, because the program is so restrictive this function is largely unused or not able to be used.
Demographic markers and categorizations are predetermined by DEX and inflexible and therefore don’t reflect what is of use to individual service providers. Administration time is often prohibitive.
DEX does not allow for the age ranges to be edited. The pre-determined age ranges do not align with those required for The Smith Family Annual Report. Getting numbers of participants for the desired age ranges must be calculated manually, significantly increasing the administrative load, and decreasing the usefulness of the data in DEX.
Seeing change in engagement over time is not possible beyond 12 months. Diverse and sustained engagement of our CCC members is a priority for Katherine CfC FP. It would be useful to chart attendance and demographics at our committee meetings over time. The system allows us to see each year individually, however when multiple years are charted it combines the totals for each month (i.e 15 attendees in January 2019 and 12 in January 2020 is represented as January = 27 attendees total).
In both instances we reached out to the DEX help desk to see if this way possible, were provided with workaround which requires downloading the data as an excel file. This is a time-consuming process and it’s much easier to gather the data independently.