Friday 5 May 2017

Lessons Learned: Post-Settlement (Self) Evaluation

As our group is in the process of sponsoring additional refugees to come to Canada and we will be assisting these future newcomers in their settlement process, we wanted to learn as much as possible from our first settlement so that we can improve our effectiveness as time goes on. 

A number of people have expressed interest in learning more about the evaluation process we undertook, and the specific questions we asked, so we wanted to share this information in our blog.

The evaluation process was undertaken as we moved into the 13th month, once the formal sponsorship period of 12 months had been completed. The first survey was conducted on a one-on-one basis between the individual members of the newcomer family and an Arabic speaking volunteer, without any of our sponsorship team members being present. The questions were provided ahead of time, in Arabic, and the family members were encouraged to be as critical as possible so that we could learn from their feedback. Some key takeaways from their feedback are included below the questionnaire.

The second questionnaire was completed on an anonymous basis via Survey Monkey with members of the sponsorship team. The results were then discussed and distilled down into the most important learning points for our group members to consider. A few selected examples of our feedback are included below.

The third part of the evaluation, not included here, was an open and frank discussion with each of the newcomers, individually, with a close and trusted member of the settlement team and an Arabic interpreter. The aim of that discussion was to highlight things that the settlement team felt that the particular newcomer still needed to focus on, or opportunities they should take advantage of, or concerns that we had regarding their particular situation.
 
Some of the major areas of feedback from the family were:
  • More regular English language work with different members of the settlement team would have been appreciated. Having a regular English “class” in their apartment was very important to them and we should have been doing it more often and continue to do it into the second year.
  • Similarly, more support for the job search, including during the second year when they would be more prepared to look for work, would be appreciated 
  • They would have appreciated more advice regarding where to conduct their shopping, where to find the best bargains and how to use the Toronto public transport. [Although we did cover this it appears we did not do it as comprehensively as we should have] 
  • There was also feedback that they remained uncertain about how certain things, such as making dental appointments, should be executed on, which we had thought had been clearly communicated already. 

As a result of their feedback, our group has decided that in the future, with our next sponsorships, we will conduct a formal monthly review to ensure that we don’t take anything for granted and we ensure that no issues or unmet needs are left unaddressed as the settlement proceeds.


Highlights of feedback from members of the sponsorship group


Q2: What could we do better (Pre-arrival and/or post arrival)?
  • Clearer division of responsibilities post-arrival 
  • Pair up each family member with an RRP member for outings, English lessons etc. No need to always involve the whole family. 
  • Encourage family members to take on part time jobs early on 
  • Consider location closer to subway, not in Arabic environment 
  • Focus on different ways to improve their English (outside of ESL classes) 
  • Manage expectations of family for month 13 

Q3: How effective was the communication between our group and the newcomers? What improvements in this area should we consider?
  • Understand role and responsibility of volunteer vs professional interpreters 
  • Draw on a bigger pool of interpreters 
  • Email not a good communication method but other communication means such as text messages, What’sApp, Facebook messenger worked well 
  • Regular, formal check-ins with family with different group members to discuss progress, concerns 
  • Spend more time with the family post-arrival to clarify mutual expectations, roles and responsibilities 

Q4: What specific actions or approaches could we have done better to promote self-sufficiency and empower the newcomers?
  • Encourage them to take on part-time employment 
  • Encourage them to take on volunteer positions 
  • Be more active in English-language support for the family 
  • Earlier start of employment discussion, preparation of CV 
  • Manage expectations for month 13 

Q6: In terms of the RRP group overall, considering how we might be able to work more effectively, and in terms of our long-term future, do you have any suggestions regarding changes we should make in any of the following areas? Direction, Goals, Processes; Governance; Membership; Task-definition and assignments; Meetings; Decision-making process; Advocacy; Other
  • Defining governance, including membership guidelines/ how decisions are made. Clarifying vision and mission. 
  • We should re-visit team member roles, accountabilities and expectations of ourselves and of each other 
  • We should continue to support advocacy efforts on behalf of the private sponsorship model in general and its application to the Syrian refugee backlog in particular. 
  • Develop mechanisms to ensure engagement of members, to exclude members if needed, to bring in new members 
  • Take and circulate minutes at each meeting 
  • Continue meetings, perhaps on a more regular basis 
  • Clear delineation of roles and responsibilities of members 

Q7: Personal reflections - Please provide your thoughts on any of the following questions: What did you learn personally, what was most meaningful, memorable, about this experience?
  • It has been a wonderful personal experience both as part of the RRP itself and in getting to know the newcomers. I definitely feel like I have gained a lot more than I have given. Being a member of the group has connected me with terrific individuals that I might not have known otherwise. Our discussions have been sometimes difficult but it has felt like we have really worked as a team to arrive at decisions. And in a crazy world that seems to be getting crazier, it has felt empowering for our group to make a difference in the lives of these 8 people. 
  • The most memorable was the pick up on day on, and the small day to day interactions and light moments and I really enjoyed our Ripple team meetings 
  • I found it personally satisfying to assist the family to settle in Canada after the trauma they must have experienced over the past few years both inside and outside Syria. It's only one family from among millions of displaced Syrian refugees but what a difference our team has made in the lives of every member of the family. Our medical team members have been truly outstanding. 
  • Seeing the family so grateful to be here in a free country, and being received so warmly was a true pleasure. 
  • On the whole, this has been a really positive experience for me personally as I feel I have connected in a meaningful way with people with shared values. It has been an honour to tangibly assist with settling a vulnerable newcomer family. This journey has been an impetus for becoming involved with an advocacy organization and to learn about the problems with the Canadian Immigration system and to explore possible solutions. 
  • This has been one of the most meaningful experiences of my life and a real honour to work with such a great group of people. Has given me a lot to think about in terms of empowerment, interpersonal relationships, cultural issues, volunteering
  • Very glad to be involved, regretful that did not have more time to make myself available. Regretful also that I did not manage to make a meaningful personal connection with them. 
  • The most meaningful part has been to enable two families to have a new life, and a better hope for the future. Also, it is very special to feel part of a group, of a bigger community, and taking a small step into making the world a slightly better place. Doing something concrete to help instead of just complaining. The most memorable moments were the arrival of the family and the many beautiful moments we spend with them socially. The whole year was one big learning experience as everything we did was completely new to me. I am proud that we were able to weather many of the challenges that arose, and stuck together as a group, without a lot of conflict and disagreement. One disappointment was that a number of people in the group simply disappeared or were disengaged, especially when the going got tough, or just came to group meetings without engaging with the family. 
  • The friendship with people so very different from me in customs and values has been heart warming. On the other hand, I have come away with a renewed appreciation of the importance of making a contribution to others for both my own and the refugees mental health. Overall, I think that the joy of self sufficiency, contribution to the welfare of others and social participation is a right we need to help the newcomers to access. I am learning about the range of groups in Syrian society and their associated attitudes, and wonder whether the government could better help those on the list to come here to manage their expectations and to understand what will be expected of them in turn. 
  •  It was wonderful to be part of a team that helps to provide a safe haven and a new start for people in real distress. I learned that the barriers are larger than I had expected. 


Template for our self-evaluation, and for the newcomer’s evaluation of our work (in English and Arabic).

By Andrew FitzGerald