Question 1 – How have you adapted service delivery in response to the bushfires, floods and Coronavirus pandemic? When has it worked and when hasn’t it worked? How will this affect how you deliver services in the future? Have your service adaptations included better integration with other initiatives?
We have had to be very flexible and creative over the past year with the pandemic. Our program involves outreach to multiple community groups, such as playgroups and a friendship cafe as well as running our own storytimes and community events. With nothing allowed to run in person, we’ve had one program continue online, but the rest has been trying to maintain support for families via telephone or email, care packages, at home activities, newsletters for support services, events and linking in to the community. It’s been really hard to reach some families, especially the ones under the most pressure with managing multiple children at home during remote learning as well as the stress of income pressures. It’s an extremely careful balance maintaining that level of contact that is supportive but where they don’t feel like it’s an extra burden to maintain engagement because they are stretched to the max with so many extra pressures. A constant evaluation of what you can provide that is likely to be truly helpful. Isolation is just as big of a worry, so it requires a huge amount of care and sensitivity. These changes have really made us think even more carefully and be more creative with everything that we do. Certainly I think our communications have improved because everything has needed to be really clear with not being able to see people in person to convey information.
Question 2 – Are the proposed key outcomes for the families and children programs the right ones? Are there any major outcomes missing? How can we include strengths-based outcomes that focus on family or child safety?
I thought they were quite good – particularly when expanded out into the visual diagram that provides much more clarity and expansion on the concepts. I work very closely with programs that are focused on connection to culture and connection to community, so it was great to see those aspects so well represented. For the families and communities that I work with, they are absolutely the key to improving outcomes. I think it probably needs to be clear that services can’t focus on everything though. Different parts of this diagram are going to be more significant for different programs at different times – but I’m assuming that this will all become clear in the program logic and theory of change that will be produced by all of the programs under this umbrella.
Question 3 – What tools or training would support you to effectively measure and report outcomes through the Data Exchange Partnership Approach?
We already do this type of reporting as much as we can. At times it is quite difficult – I work with clients whose particular circumstances mean that their experiences with government organisations have often been negative. Especially historically, and that experience and the ongoing intergenerational trauma means that they are understandably highly suspicious of what will be done with their information once it leaves my hands. They trust me, but they don’t necessarily trust what is done with the information, if that makes sense. They also find formulaic surveys and pre-constructed tools alienating and it really can damage the relationship if evaluation isn’t done with the utmost sensitivity. We have found that group yarning about the program, talking about past & present experience and future ideas is often a great bonding experience both between participants but also between participants and facilitators, but it is often difficult to translate such data into the Data Exchange. Reporting becomes even more important, but I would really like to see the Data Exchange developed to allow a much greater capacity for different types of data entry. I would also really like the ability to report survey results for clients who have responded to an anonymous survey. Currently, I have to ask for a name (or a code) to attach their data as a SCORE. I believe this is a potential source of bias (and a discouragement to answering). I understand that this allowing individual SCORE data that isn’t tied to a client won’t be able to be tied to a later SCORE, but I think even averages across time would still be valuable, with multiple individuals entered. At the moment I can only add a Community SCORE with anonymous results, which doesn’t allow for the richness of response as it is limited in response type and also doesn’t allow you to see the breakdown of individual responses. And yes, more training is always welcome!!
Question 4 – Do you already have a program logic or theory of change outlined for your program? Did you find the process useful? If you do not have one, what has stopped you from developing one? What capacity building support would assist service providers to develop program logics and theories of change?
Yes, we have a program logic. It was already developed when I joined the program, but I was involved in reviewing and making changes to it when we were going through the AIFS evaluation process. I found the whole process incredibly useful, especially the literature review and writing the reports which really forced me to clarify my thinking and our practice. I think it would be great to further develop a theory of change. Our theory of change is actually fairly well explained in our submission to AIFS I think, but it would be fantastic to go through the process to distill this into a succinct diagram/document that really gets to the heart of the how and the why.
Question 5 – If longer-term agreements are implemented, how can the department work with you to develop criteria to measure and demonstrate performance? How can the Data Exchange better support this?
Certainty is so incredibly important. I work with very vulnerable communities, and it takes huge amounts of time to build trust. Not having certainty from year to year means that it is difficult to plan long-term, to build relationships for larger programs with other providers, to talk about future plans with any certainty with partners and with clients. It’s also a drain on talent, expertise and community recognition of staff. Huge amounts of time and money are put into training and developing staff, they build networks and relationships and earn trust, but as they have no certainty of ongoing funding, they are potentially lost to other, more secure roles. Regarding performance criteria, I agree that reviews are necessary. I would like to see a lot of flexibility in what this looks like though. Programs vary so much depending on clients and the community context. Different research methods should be embraced, as should recognition that not all evaluation methods are appropriate in all communities.
Question 6 – What does success look like for your service, and how do you assess the overall success of your service?
Success looks different for every client. It isn’t about what my organisation sees as success, in my opinion. It’s about the hopes, dreams and goals of our clients. Of course we want to improve literacy, to reduce isolation, to connect families to services, to make sure they have their material needs met, etc. But at the end of the day, the most important thing that we can ever do is listen to their voices. We have a lot of measures for all sorts of things, but I measure myself against their expectations and if I am helping them reach their vision for themselves and their families. And this isn’t always an easy thing to measure in a survey! It’s about relationships; it’s about trust; it’s about listening, and it’s about respect. This does come out in surveys though – usually written in free text responses about the impact of the program on the lives of clients. But it comes out in other ways too. Impromptu visits to the library to say hello, a child’s drawing, a thank you card. It just shows how we need to provide many different ways to share, and we always need to be listening. All measures are important – and sometimes the unexpected ones are the most enlightening.
Question 7 – Do you currently service cohorts experiencing vulnerability, including those at risk of engaging with the child protection system? If not, how does service delivery need to adapt to provide support to these cohorts?
Yes, I work mostly with Aboriginal families and families from culturally and linguistically diverse backgrounds, including many refugees. Many of these families have been through extremely difficult situations themselves or are living with the impacts of intergenerational trauma. We have to be very aware of the impact of both their history and the colonial history of this land. Colonial violence is still with us, and the impact pervades every interaction, especially anything to do with government – data collection is affected by this, as it the ‘evaluation’ of outcomes because quite frankly, for a parent whose community has been so intensely affected by child removals, it isn’t surprising that any sense that their parenting is being scrutinised is quite frightening, no matter if the intention is just to support as we would support any family. The fear is real, and it is understandable, especially given the extremely high rate of Aboriginal children in out of home care today. We have to build trust with the families we work with, and to do that, we must start with a position of respect and work from strengths, not deficits. Otherwise, these families will disengage.
Question 9 – For all providers, are there other ways to improve collaboration and coordination across services and systems?
I absolutely LOVE our monthly CfC meeting where we all have lunch and talk about what’s happening in our programs. It is so supportive and often begins discussions of potential collaborations. I hope this will continue. I also think it would be great to have broader forums where we focus on particular issues. Zoom provides such a great opportunity to learn from services across Australia. I’d love to hear more regularly about what’s happening outside our site. I also think a service directory would be great.
Question 11 – Aside from additional funding, how can the department best work with you to support innovation in your services while maintaining a commitment to existing service delivery?
I’ve really appreciated the many webinars provided through organisations like AIFS – they always help me reflect on my own practice and try to improve. I would like to see online webinars continue after the danger of Covid has passed. I think it makes training and connecting with colleagues so much more accessible, especially being on the outskirts of the city and so much training often takes place in the major cities or even across the other side of the city. Traveling takes us away from our programs and clients for longer. In person events are nice occasionally, but I think I get a lot more done with online options, being very time poor. Also important to have a wide range of training and innovative ideas being explored in our professional networks – great talking to similar professionals in other countries as well for fresh ideas.
Question 12 – Is there anything else you would like to share about the ideas and proposals in the Discussion Paper?
Not really about what is in the paper, more what was discussed in the webinar conversation. The prospect of targets on numbers of clients being evaluated or more of their data provided was discussed. This was really the only aspect that caused me any concern. Because I work with clients who are very distrustful of government data collection and evaluation as well as clients who have significant literacy challenges, this would potentially be extremely difficult, if not impossible, depending on the targets. Strict targets with no flexiblity do not take into account the incredible complexity in working with these clients, for reasons I have already discussed in other question responses. Targets by percentages also are difficult because I am involved with some large community events like NAIDOC which are hugely important and impactful, but they are mostly group clients (there are some individual clients, but being a large event, invariably lots of others as well), and I’m a big concerned about how group clients would affect these targets.