Quality performance in the Refugee Protection Division 2020 to 2021

​​​​​​Cat. No. MQ1-14E-PDF (Electronic PDF, English)​

​​ISSN 2816-5942

​​​​​​​​​​Report of Results

Prepared by:

Hélène Mayrand and the Strategic planning, accountability, and reporting directorate ​

​​​​​​​​​​On this page

• 1.0 Context
• 2.0 Performance results
• 2.1 Timely and complete pre-proceeding readiness
• 2.2 Respectful proceedings
• 2.3 Focused proceedings
• 2.4 Reasons state conclusions on determinative issues
• 2.5 Decisions provide findings and analysis necessary to justify conclusions
• 2.6 Reasons are transparent and intelligible
• 2.7 Supplementary Questions Related to Virtual Hearings
• 3.0 Results for specific populations
• 3.1 SOGIE cases
• Annex A - Checklist
• Annex B- Sexual orientation, gender identity and expression checklist

1.0 Context

This report describes the results of the measurement of quality in decision-making in the Refugee Protection Division (RPD).

Sample methodology

The study reviewed 80 out of a possible 2680 claims and applications that were finalized between January 1, 2021 and March 31, 2021 (the assessment period) on their merits after a virtual oral hearing before a single-member panel. The hearings were randomly selected in proportion to region, outcome, and specially selected case types, including cases related to Sexual Orientation and Gender Identity and Expression (SOGIE)^{Footnote 1}, and women fearing gender-based violence (GBV). All hearings with a duration between 45 minutes and 3 hours were included in the final population. Members who had been hearing cases for less than six months from the start of the assessment period were excluded from the sample. Within the sample, hearings related to diverse SOGIE were identified and further analyzed through a particular SOGIE lens.

The following charts illustrate the sampling makeup, which is proportionately representative of the population data:

Regional office

Outcome

Selected case types^{Footnote 2}​

Assessment methodology

This qualitative assessment was performed by an independent reviewer who is since 2012 a law professor specialized in immigration and refugee law. She clerked at the Federal Court of Appeal of Canada and holds a LL.M from the University of Cambridge and a S.J.D from the University of Toronto. The reviewer examined all evidentiary and administrative materials on file, listened to the complete audio recordings, and assessed these against qualitative indicators in a checklist developed by the Strategic Planning, Accountability and Reporting (SPAR) Directorate and approved by the Deputy Chairperson of the RPD (see Appendix A). Each indicator is assessed on a 1-to-3 rating scale. The standard checklist assesses twenty-eight indicators across six performance categories. Fourteen of the indicators are mandatory for assessment, and fourteen are assessed only when applicable. An additional four questions were added to the checklist that pertain to the virtual hearing process, all of which are assessed on an as applicable basis. The performance categories are:

1. Timely and complete pre-proceeding readiness
2. Fair and respectful proceedings
3. Focused proceedings
4. Reasons state conclusions on determinative issues
5. Reasons are transparent and intelligible
6. ​Decisions provide findings and analysis necessary to justify conclusions
7. Virtual Hearing Processes

The 1-3 rating scale is as follows:

1=Does not meet expectations: The quality requirement was not met. The evidence showed one or more key instances where the proceeding or reasons would have markedly benefited had this requirement been met. There may have been an effort to apply the requirement, but the level of achievement fell short of expectations.

2=Meets expectations: This is a level of acceptable achievement. On balance, the member satisfied this quality requirement though there is margin for minor improvement.

3=Exceeds expectations: This is a level of consistent, above-average achievement. The evidence shows a grasp of the quality requirement and an understanding of its importance to a high-quality proceeding or decision, as the case may be.

Results are also expressed as a percentage of hearings that meet or exceed expectations. A hearing is considered to meet high quality standards when 80% or more of the standard indicators for that hearing score a 2 or higher.

In addition, as part of Gender Based Analysis Plus (GBA+), a second checklist was developed for hearings involving persons of diverse SOGIE and is used in quality measurement reviews in all four of IRB's divisions (see Appendix B).

Considerations/limitations

Results are accurate to within ±9% percent, 18 times out of 20. This margin increases when data is broken down by region or claim type. However, the goal of the study was not to generate statistics but to identify areas of strength, areas for improvement, and patterns in decision-making quality.

This study acknowledges the inherent limitations of qualitative research, which does not generate precise data as does quantitative research. To mitigate the inherent limitations of qualitative research, detailed performance indicators were provided to the reviewer to help focus the assessment. To ensure quality and consistency in the assessment, a reviewer was selected based on their in-depth knowledge of the Immigration and Refugee Protection Act (IRPA) and refugee matters. Moreover, a small sample size limits the inferences that may be made about the broader caseload. Where sample sizes are too small, observations or recommendations may still have been provided but these are not based on representative findings.

This report aims to provide a perspective to improve the Division's performance overall.  The evaluation unit provided the statistics found in the table accompanying each result section as well as the information in 1.0 “Context”. However, the findings in this report, including all strengths, areas for improvement and recommendations are solely those of the reviewer. Their observations do not lend themselves to firm conclusions on legal matters such as the correct application of the law, the weighing of the evidence, or the fairness of the proceedings from a natural justice perspective. Only a court reviewing the case can arrive at such conclusions.

2.0 ​Performance results

What was measured

Each performance result in sections 2.1-2.7, contains a table representing the number of hearings assessed for each indicator, and the percentage of assessed hearings that met expectations by scoring a 2.0 or higher for that indicator. The number of hearings assessed are provided for reference and context only.

The primary performance target for this assessment is for 75% of hearings to meet quality standards. The quality standard is that the hearing achieves a score of 2.0 or higher in at least 80% of all standard indicators assessed within that hearing. This target was achieved with 91.3% of hearings (73 out of 80) meeting or exceeding expectations.

Where an indicator had many hearings that did not meet the target, it is addressed in the reviewer’s observations following the table (Areas for Improvement).

2.1 Timely and complete pre-proceeding readiness

Why measure this

The groundwork for a quality hearing and decision is set when the Registry prepares a timely, organized and complete case docket and the member assimilates the facts and key issues of the case.

Considerations

Only 1 indicator was assessed for this category. Due to COVID-19 and the transition from physical files to e-files, some files were fully or partially available in the electronic record management system, while others were only available as physical files. The files were assessed in the same manner for all types of files using the support (electronic or physical) available.

General observations

• The assessment of this indicator took into account that members had to review both physical and/or e-files, as documents were available in multiple formats. The fact that a member was not familiar with a document prior to the hearing was not automatically attributed to the member’s lack of readiness if it was apparent that the document may not have been uploaded to the e-folder in the digitization process for example, and if the member took immediate steps to address the issue.
• Given the challenges to members in this transitional year, the results for the indicator for pre-proceeding readiness is excellent. In 97.5% of cases, the member was appropriately prepared for the hearing.

Strengths

• Members have shown a great capacity to adapt to the new situation and to move from physical files to e-files. They had several challenges, including that of having to analyze both physical and electronic documents, as well as sorting duplicates from amendments. Members prepared a list of consolidated documents and consulted counsels to ensure they had all the documents submitted. When necessary, they took short breaks to read a document that was not part of the record they received. Overall, members met the challenges, being well prepared for the hearing, and accordingly, making sure that the quality of the proceeding was not impacted by technical problems.

Areas for improvement

• In only 2 cases members were not ready for the proceeding and scored a “1”. In 1 of these cases, the member was disorganized and had to take two breaks to read documents already available on file, unnecessarily lengthening the hearing. In the other case, the member was of the view that the file should have been decided on the record without a hearing. He was taken aback by the situation that involved a vulnerable person and was unprepared to conduct the hearing.

Recommendations

• Complete the transition from physical files to e-files as soon as possible, including the implementation of the new tabbing protocol to all e-files:
• To ensure that members can easily find the relevant documents for their file and prepare for the proceedings, continue current practice of informing members of the advancement of the implementation of the transition from physical files to e-files and update instructions to members and training accordingly.
• In line with the recommendation of the RPD Quality Report for 2019-20, continue to provide guidance on the importance of file information completeness and consistency.

2.2 Respectful proceedings

Why measure this

Individuals appearing before the IRB expect that they will be treated with sensitivity and respect. Any shortcoming in this regard potentially undermines tribunal integrity and public confidence. Especially in a year of transition to virtual hearings, measuring respectfulness of proceedings is important in determining the dynamic of the member and participants within the virtual environment.

Considerations

Indicators 2 and 3 are applied to all hearings while 4-6 are scored on an as-applicable basis.

General observations

• Most members (n=75) treated participants with sensitivity and respect.

• Indicator 6 “problems with interpretation”: encompasses the quality of the interpretation and its impact on fairness of the proceedings. As a result, not only were problems with the language of interpretation or interpretation accuracy assessed, but also whether interpretation was available to claimants during all steps of the proceedings, including the oral delivery of the reasons for the decision. There was a significant issue with interpretation in many cases, which impacted the quality of the proceedings. The move from in-person hearings to virtual hearings has resulted in a practice where members, especially in the Central and Western regions, did not ask consent to waive interpretation or when they did, they did not leave a real choice to claimants, suggesting that the virtual format made it impossible to have consecutive translation. This particular issue should, in the assessor’s view, be addressed in priority to ensure full and fair proceedings.

  The analysis under “Strengths” in this section of the report indicates that interpretation with virtual proceedings is possible, and such practice ensures that claimants have a fair hearing. It substantially improves the quality of the proceedings. While it is true that the transcript of the decision will be sent in the mail and that claimants will then be able to have such transcript translated in their own language, the oral delivery of the decision is an integral part of the proceedings and the actual moment when the decision occurs. Allowing claimants to fully listen to the reasons at this moment helps them understand that the reasons for the decision are important and that they are an integral part of their own proceedings, contributing to adherence to the decision and trust towards the institution.

Strengths

• Sensitivity and respect: In 93.8% of cases, members treated all participants (claimants, counsels, minister’s representatives, witnesses, and interpreters) in a sensitive and respectful manner. A significant number of cases scored a “3” for this indicator (n=18), when members clearly showed an above-average level of sensitivity and respect, significantly improving the quality of the proceedings. These members were particularly helpful to claimants and explained all steps of the hearing to them, what the legal issues were, the evidence they had to provide, how such evidence was assessed and weighed, etc. They ensured that claimants were fully participating in their own hearing, and not merely witnesses providing testimony. These members showed the same sensitivity and respect for self-represented claimants, claimants represented by counsel, and for positive and negative outcomes. Members adopting such an approach viewed their decision-making role as a dialogue with claimants.
• Presentation of evidence and representations: In 98.8% of cases, parties were given adequate opportunity to present evidence and to respond to issues. Members asked claimants and/or counsel whether all the evidence was on the record, whether claimants had anything to add to their testimony and whether counsel had any questions to ask to the claimants and/or whether they had additional representations to make.
• Questioning and clarification: In 39 files, there was an issue with the evidence that required members’ action. Of those cases, members asked clarifying questions 97.4% of the time (n=38/39). 6 cases exceeded expectations for this indicator, when members clearly identified what was problematic with the evidence and explained why it was particularly relevant, linking it to the determinative issues of the hearing.

• Interpretation: Members were efficient in addressing problems with interpretation when it was related to the language of interpretation and accuracy of interpretation. In all cases assessed involving interpreters, they confirmed the language of interpretation at the beginning of the hearing and whether the claimant could understand the interpreter and vice versa. When needed, they asked the claimants to break up their sentences and to speak slowly. Members also rephrased their questions when the interpreter and/or the claimant did not understand them.

  In 13 of the hearings which required interpretation, members maintained interpretation for the oral delivery of the reasons for the decision even though the hearing was conducted on MS Teams. There was no problem encountered in these cases, the hearing being completed in the allotted time and the transcription of the decision being accurate. Such examples show that it is possible to have interpretation for the entire virtual hearing, without compromising efficiency. A best practice conducted by members was to consult the interpreter on the best way to proceed (for example one line at a time). The full interpretation of the proceedings, including the oral delivery of the reasons for the decision, significantly improved the quality of the proceedings. Moreover, it allowed the claimant to fully understand why his or her claim was accepted or dismissed, contributing to adherence to the decision and trust towards the institution. It also helped the claimant feel included in his or her own hearing and signalled that the actual reasons for the decision are important.

Areas for improvement

• Sensitivity and respect: In only 5 cases, members lacked sensitivity and respect. In 3 of those 5 cases, members interrupted claimants abruptly, preventing them from finishing their thought, and/or asked questions in a confrontational manner. In the 2 other cases, they were from the same member, whose attitude towards claimants raised particular concerns, as he went beyond interrupting and questioning the claimant in a confrontational manner. In one hearing, he stated the claimant should not "play games with the interpreter", using a language and tone which seem to imply that the claimant was dishonest, and conducted the hearing as if to corner him. That a similar attitude was adopted in two different cases points to a potential misunderstanding of the member’s role as decision-maker and/or potential biases towards certain claimants. Adopting a neutral, yet sensitive tone when questioning claimants is instrumental for the claimant to have a fair hearing, but also for them to know that their case was decided by an impartial member.
• Communication in the absence of a party: In only 1 case out of 4, a member asked the claimant and the designated representative to leave the meeting to address some issues with counsel, but when the claimant and designated representative came back, the member did not disclose the information. The claimant in that case was a vulnerable person and had a limited ability to testify. Nonetheless, he was represented by a designated representative and should have had the same rights and considerations as other claimants.

• Interpretation: The practice of not asking claimants and their counsel for consent to waive interpretation of the oral delivery of the reasons for the decision is the main issue for this assessment. In 12 cases, members did not ask claimants nor their counsel to waive interpretation for the oral delivery of the reasons for the decision. If they did, it was not part of the recording. These 12 cases scored a “1” for this indicator. The results indicate that the practice of not asking consent to waive interpretation for the oral delivery of the reasons for the decision is more common in the Central (n=9/38) and Western (n=3/14) regions compared to the Eastern region (n=0/28). Out of the 12 cases where no consent was given, 1 had a negative outcome.

  Members did not always explain why they believed obtaining consent from the claimant or counsel was not necessary, or why interpretation of the reasons and decision should be waived. Instead, their comments gave the impression that they thought that interpretation was not possible or burdensome. For example, members explicitly stated in 3 cases that interpretation was not possible in a virtual hearing. Less common reasons provided include saving time (2 cases) and/or facilitating the transcription of the oral delivery of the reasons for the decision (1 case).

  Another case also scored a “1”, when the member suggested that the claimant should not have an interpreter since he could understand English, but he clearly needed one to express himself.

  In 7 other cases, members did ask for consent from claimants or their counsel to waive their right to interpretation. While these cases received a score of “2”, there was clearly room for improvement in some of the cases. In 4 cases, the way the question was framed implied that the claimant did not really have a choice in the matter. In 3 of these cases, members asked consent, but specified that it was not possible to have concurrent interpretation due to the MS Teams format. It is difficult to see how claimants have a choice to agree or not to waive interpretation in such context. In 1 case, the member referred to the claimant’s right to interpretation, but that waiving interpretation of the oral delivery of the reasons for the decision should be favoured to facilitate transcription. In 3 additional cases, members only asked counsel if they agreed to waive interpretation. Claimants were informed that the reasons for the decision would not be translated to them, as opposed to being asked whether they agree to waive their right.

  Another area for improvement is with respect to the use of a summary for the delivery of the reasons for the decision. In 7 cases for which interpretation for the oral delivery of the reasons for the decision was waived (3 without consent; 4 with consent), members did ask the interpreter to provide a summary. However, as one member pointed out himself, such practice remains problematic. Members must rely on an interpreter not trained in law to decide which parts of the decision was relevant when they make their summary. The explanation of legal concepts and tests, the analysis of the evidence and findings of facts are all instrumental to fully understand the rationale behind a decision, to promote adherence to the decision and trust towards the institution.

Recommendations

Interpretation:

• Clarify in the guidelines and/or instructions to members that in all proceedings, including virtual hearings on MS Teams, claimants have a right to interpretation for all parts of the proceedings, including the oral delivery of the reasons for the decision. Claimants can waive their right to interpretation; however, members have to specifically ask them or their counsel, and ensure claimants have a real choice to waive interpretation or not.
• Clarify in the guidelines and/or instructions that members should not ask or rely on the interpreter to provide a summary of the reasons for the decision. If claimants agree to waive his or her right to interpretation for the oral delivery of the reasons for the decision, members can provide their own summary which can be translated to the claimant.

Sensitivity and respect: To ensure hearings are conducted in a respectful and sensitive manner that enables claimants to fully participate in their own proceedings, the RPD should continue to implement good practices in professional development programming, by leveraging internal examples of best practices in hearings from members as learning examples for training exercises.

Vulnerable Persons: Provide further training and/or guidance to members for proceedings involving vulnerable persons, especially on how to include the claimants and their designated representatives in all steps of the proceedings.

2.3 Focused proceedings

Why measure this

Proceedings that are efficient and well managed create conditions for quality outcomes to emerge and support the IRB’s efforts to make the most effective use of its resources.

Considerations

Indicators 7 to 9 are assessed against all hearings while 10 to 17 are assessed on an as-applicable basis.

General observations

• None identified.

Strengths

• Issue agenda: In 86.3% of cases, members set out a clear issue agenda at the beginning of the hearing. The most common issue was credibility, and additional issues included identity as a member of a specific group, nexus to the Convention grounds, state protection, and internal flight alternative (IFA), In 28 cases which exceeded expectations, members clearly explained to claimants how the hearing was organized (order of questioning, breaks, questions and counsel submissions, oral decision), what they had to prove according to the law (s. 96 and 97(1) Immigration and Refugee Protection Act or IRPA), and how the determinative issues were relevant to what had to be proven. They also indicated that they did not need more evidence and/or explanation on particular points. They provided examples and/or explained in simple words legal concepts, such as credibility, balance of probabilities, Convention grounds, serious possibility of persecution, and IFA. When the issue agenda was clearly explained to claimants, it appeared to put them at ease and, in the assessor’s view, helped to build trust and an open dialogue from the start of the hearing.
• Member’s questioning – focus and organization: In 87.5% of cases, members’ questioning was focused and well organized during the entire hearing. Having set a clear agenda at the beginning helped members to keep their questioning organized. In 1 case which exceeded expectations, the member restated the issues during the course of the hearing and explained why the issues were relevant before asking questions. This helped the claimant know what the questioning would be about, to provide pointed testimony.
• Narrowing the issues: Two indicators were assessed for the more general question of narrowing the issues (indicators 13 and 14), however they were only applicable in a limited number of cases. Narrowing was not necessary during most hearings, when the only determinative issue was credibility and when there was no specific issue with the evidence. However, when it was relevant (in 8 cases), members clearly indicated that they were satisfied that one or more issues identified at the beginning of the hearing were resolved, and that the remaining part of the hearing would focus on a specific issue. Such indications were particularly relevant to ensure that counsels’ questions to claimants were specific to the determinative issues and/or problems with the evidence. With respect to narrowing the issues for final representations, in 13 out of the 14 cases for which it was applicable, members provided clear instructions to counsel that they still had some concerns with respect to a specific issue and/or evidence. Narrowing the issues in such a manner contributed significantly to the hearing’s overall efficiency.
• Adherence to the applicable legislation, regulations, rules or guidelines: This indicator was met in 97.5% of cases. In almost all files (n=78), members adhered to the applicable legislation, Rules or Guidelines. The assessment of this indicator did not look at whether members referred to specific rules or guidelines, but rather whether they followed them during the course of the hearing. Many members, however, did cite specific guidelines or rules where relevant, in particular Chairperson’s Guidelines 3, 4 and 9. Such practice should be encouraged. Good examples of adherence to the applicable legislation, regulations, Rules or Guidelines included members who avoided asking specific questions and details about certain traumatizing events.

Areas for improvement

• Issue agenda: In the 41 cases which scored a “2”, meeting the threshold for this indicator but leaving some space for improvement, members referred to these issues without explaining what they meant in plain language and/or directed the information to counsel only. In 10 cases, members did not identify any issue during the course of the hearing and in 1 case, the member identified an issue only towards the end of the hearing. Not identifying a determinative issue at the beginning of the hearing also negatively impacted indicators 8, 10 and 18 in this assessment, as these indicators also required that an issue be identified by members. While some hearings may only have one issue, for instance credibility, identifying that particular issue remains important to having a focused and efficient hearing.
• Member’s questioning is relevant:​ While this was not a common practice, in 2 cases different members asked a female claimant whether she had miscarriages. This question is too intrusive and not relevant for confirming the claimant’s identity or assessing credibility.
• Adherence to the applicable legislation, regulations, rules or guidelines: Citing the guidelines does not amount to actually applying them. In 2 cases, members’ adherence to the applicable legislation, regulation, Rules or Guidelines was lacking. In 1 case, the member did not attempt to accommodate a vulnerable claimant and his designated representative to provide testimony and was rather confused and unprepared to conduct the hearing, as he was of the view that the claim should have been allowed without a hearing. In another file, a different member did not apply the presumption set out in Maldonado v. Canada (1979), 1980 2 FC 302 that “when a claimant swears to the truth of certain allegations, this creates a presumption that those allegations are true”. The member, who also raised concerns with respect to indicator 2 (sensitivity and respect) and indicator 6 (suggesting not to use interpretation), stated that the claimant was “playing games” with the interpreter and using a language and tone giving the impression that the claimant was dishonest. However, there was no integrity issue raised by the minister or on the face of the record in that case. While the member may have concerns with respect to a claimant’s credibility and the evidence, they should approach the case with the presumption that the claimant’s allegations are true and ask questions in a neutral but sensitive manner to make the appropriate findings of facts and conclusions.

Recommendation

Issue agenda: In line with the recommendation of the RPD Quality Report for 2019-20, continue to stress the importance of setting determinative issues addressed to both claimant(s) and counsel at the beginning of the hearing when training new members as well as during professional development sessions.

Narrowing the issues: In line with the recommendation of the RPD Quality Report for 2019-20, continue to underline the utility of clearly narrowing the issues prior to questioning and final submissions by counsel when training new members as well as during professional development sessions.

2.4 Reasons state conclusions on determinative issues

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility and transparency in a decision of an administrative tribunal. Through indicators 21 to 31 this study applies the Court’s requirement in the context of IRB decision-making.

Considerations

These indicators are applied to all hearings.

General observations

• Indicators 18 and 19 are intrinsically linked with other indicators and scores for this section should be interpreted with this consideration in mind. Failure to identify determinative issues at the beginning of the hearing in accordance with indicator 7 generally implies that indicator 18 cannot be met unless an issue was subsequently identified during the course of the hearing and dealt with in the reasons. The quality of the conclusions for indicator 19 improves significantly when members have a well-organized decision with headings identifying the relevant parts and issues of the decision (indicator 28), as well as when they support their conclusions with findings of fact supported by clear examples (indicator 21), especially when they refer to the claimant’s testimony and how it supports documentary evidence.

Strengths

• Determinative issues in reasons: In 86.3% of cases (n=69), members addressed the determinative issues identified during the course of the hearing in their reasons. In 4 cases which exceeded expectations, in their decision members made it explicit that they were now providing reasons for the issues they had identified. In 1 case, the member clearly stated why he would not provide reasons for all the issues identified at the beginning of the hearing, as the outcome of the claim would be decided on one determinative issue.
• Conclusions based on issues and evidence: In 88.8% of cases (n=71), members based their conclusions on all the issues and relevant evidence adduced during the proceedings.

Areas for improvement

• Determinative issues in reasons: As underlined above, a score of “1” for indicator 18 is attributed to 10 cases that failed to set a clear agenda and identify determinative issues at the beginning of the hearing.
• Conclusions based on issues and evidence: In 9 cases, members did not base their decision on the issues and evidence adduced during the proceedings. With respect to the issues, some members based their decision on issues which were not raised during the course of the hearing. For example, in 1 case a member did not ask any question with respect to the claim of one minor claimant with a different nationality from his parents, while the member rejected the claim in his decision. If a different conclusion is drawn for different claimants, members should ask questions on all claims during the hearing or indicate why one claim does not raise any issue.
• W​ith respect to the evidence adduced during the proceedings, a common problem among the 9 cases was that members failed to refer to the claimant’s testimony in their decision. They based their conclusions only on the documentary evidence (Basis of Claim form, other documentary evidence provided by the claimant, and National Documentation Package) and did not refer to the evidence provided during the hearing.

Recommendation

Conclusions based on issues and evidence: When training new members as well as during professional development sessions, members should be reminded that they must support their conclusion on credibility on the evidence provided during the hearing, taking specific examples from the claimant’s testimony.

2.5 Decisions provide findings and analysis necessary to justify conclusions

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility and transparency in a decision of an administrative tribunal. Through indicators 21 to 31 this study applies the Court’s requirement in the context of IRB decision-making.

Considerations

Indicators 20-22 are applied to all hearings, while 23-25 are applied on an as applicable basis.

General observations

• The overall score for this section supports the conclusion that for the sample assessed, members wrote decisions that were justifiable, intelligible and transparent.
• Indicator 24 was assessed taking into account that members do not necessarily have to cite the rules, regulations, Jurisprudential Guides, Chairperson’s Guidelines or persuasive decisions if they follow and apply them properly, and provide reasons when not following them.

Strengths

• Clear, unambiguous findings of fact, supported by clear examples and credible evidence: Indicators 20, 21 and 22 met or exceeded expectations in almost all cases. Good examples exceeding expectations for indicator 20 (n=8) was when members clearly explained why all relevant facts were proven, on a balance of probabilities. With respect to indicator ​21, in 8 of the 12 cases where members exceeded expectations, they provided many examples taken from the hearing to support their findings, especially linking the testimony provided by the claimant to the Basis of Claim form and the National Documentation Package.
• Why certain evidence was preferred: In all cases where there was evidence contrary to the members’ decisions, members did refer to the piece of evidence and why they did not put weight on it.
• Legislation, rules, regulations, jurisprudential guides, chairperson’s guidelines, or persuasive decisions: In all cases members referred to s. 96 and 97(1) of the IRPA (or refugee Convention and person in need of protection). Good practices also included referring to Chairperson’s Guidelines, which was common for Chairperson’s Guidelines 3, 4 and 9. In 7 cases, members exceeded expectations and scored a “3” when they also engaged with the relevant case law to explain their decision. For example, members referred to decisions from the Federal Court, the Federal Court of Appeal and/or the Supreme Court to explain the requirement of a personalized risk for the application of 97(1) IRPA and the notion of a “particular social group” under the Refugee Convention. This practice should be encouraged when applicable, such as when referring to the relevant case law contributes significantly to support or explain the member’s conclusion on a determinative or contested issue.

Areas for improvement

• Clear, unambiguous findings of fact, supported by clear examples and credible evidence: In 4 cases, members did not make clear, unambiguous findings of fact as required by indicator 20. This was the case particularly when there was more than one claimant or when the decision would be decided on 97(1) IRPA. Moreover, clarity was undermined in 1 case when a member used a double negative formulation which created confusion about a finding of fact. Using positive formulations for findings of fact should be encouraged.

  In 5 cases which scored a “1” for indicator 21, the reasons for the decision lacked specific examples to support findings. The findings in these decisions were too general and gave the impression that it was taken from a template without adapting it to the particular situation of the claimant and the evidence he or she provided. Using the generic formulation: “The claimant testified in a straightforward manner and there were no major inconsistencies or contradictions between the testimony, the Basis of Claim, or the documentary evidence before me” should be supported with concrete examples.

• Legislation, rules, regulations, jurisprudential guides, chairperson’s Guidelines or persuasive decisions: In 1 case, a member only took into consideration Chairperson’s Guideline 9 in a case where a woman suffered from sexual and physical abuse in relation to her sexual orientation, but also her gender. In that case, Chairperson’s Guideline 4 was also relevant, but not taken into consideration. In another case, the member referred to Chairperson’s Guideline 4, but it was unclear in the reasons why it was relevant and how the member applied it.
• Social and cultural contextual factors in assessing witnesses’ testimony: In all the cases assessed, members did consider social and cultural factors when making their decision as asylum claims necessarily imply that members take into consideration the particular situation as well as the country conditions of a claimant. In only 1 file a member referenced in her decision that she took into account social and cultural factors when assessing the claimant’s testimony. However, this file was attributed a score of “1”, as the member included such reference in generic terms and did not explain what those factors were and how they impacted the findings and/or conclusions.

Recommendations

Clear, unambiguous findings of fact, supported by clear examples and credible findings: Continue to implement good practices in professional development programming by sharing concrete examples and tips on how to personalize reasons. Members should be reminded that general findings, have to be supported with specific examples.

Legislation, rules, regulations, Jurisprudential Guides, Chairperson’s Guidelines or persuasive decisions: For the purposes of professional development, share good practices on when it is relevant and how to engage with the case law in a persuasive and efficient manner in decisions.

2.6 Reasons are transparent and intelligible

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility and transparency in a decision of an administrative tribunal. Through indicators 21 to 31 this study applies the Court’s requirement in the context of IRB decision-making.

Considerations

All indicators are applied to all hearings.

General observations

• In general, the reasons for the decision were transparent and intelligible. Some members excelled at explaining their oral decisions to claimants in plain language, but on occasion lacked organization due to their informal tone. Other members chose to fully explain their findings in great detail, which sometimes resulted in lengthy reasons. The best decisions attempted to strike a balance between the need to state their reasons in simple words supported by concrete examples, and the need to be concise and to organize their reasons with headings.
• For the assessment of indicator 28, the cases were assessed taking into consideration that members should organize their reasons in a logical way with headings, for both oral and written reasons. In 46 cases, members did not include such headings for all parts of their oral decisions.

Strengths

• Plain language: Decisions could easily be understood in 97.5% of the hearings. In the 13 cases which exceeded expectations, members explained the issues and legal concepts in simple words. Best practices include plain language explanation of terms such as: the legal tests to meet the definition of a refugee Convention and a person in need of protection; the standard of proof; Chairperson’s Guidelines; and IFA. It is noteworthy that plain language applies to all cases, irrespective of the fact that a claimant is represented by counsel.
• Clear and concise reasons: Striking a balance between having a detailed decision supported by many examples, but yet ensuring that the decision remains clear and concise is an art. Members achieved this in 66 cases and mastered it in 5 cases.
• Reasons easily understood and logically sequenced: Headings significantly improved the quality of the reasons.18 cases had sufficient headings, and 9 cases had an excellent use of headings. The use of headings helped members to clearly organize their decision and for the parties to easily follow them. It also helped to avoid repetition, contributing to concision. Headings also helped to immediately see when reading the reasons whether there were findings of fact and conclusions for all the issues identified.

Areas for improvement

• Plain language: It was a rare issue, but in 2 cases, members used the acronym IFA without explaining what it meant. They should be reminded that their decision is addressed to the claimants and not their counsel.
• Clear and concise reasons: In 9 cases, the reasons for the decision were unclear and/or too long. Common problems included unnecessary repetitions, moving back and forth on certain findings and issues, long sentences with many facts, and unclear findings of fact. Some members had to correct their statements while reading their oral reasons for the decision. Having clearer notes or almost fully written reasons for the decision would prevent such problems.

• Reasons easily understood and logically sequenced: In 53 cases, members did not include clear headings in their oral reasons and/or failed to organize their reasons in a logical order. In 39 of these cases, there were no headings at all. In 3 of the 53 cases, the headings did not correspond to the content of the section. For example, the member put all issues under “Credibility”, including IFA. In other cases, it was the logical order of the reasons that was lacking. For example, in 1 case, the member put “Conclusion” before State protection and IFA. In 4 cases, there were some headings in text (“With respect to State protection”), but not for all parts of the reasons. Formal headings separate from text should be favoured, as they enable members to more efficiently organize their decision and quickly identify whether an issue or conclusion is missing.

  In 5 cases, members used a formal terminology using "the claimant" and "his", as well as a personal terminology talking to the claimant using "you" and "yours". In 4 cases which scored a “1” for this indicator, members constantly moved from second person pronouns to third person pronouns, making the reasons more difficult to understand. Consistency should be favoured.

  In 2 cases involving more than one claimant, members had difficulty explaining which issues and findings were relevant for each of the claimants.

Recommendations

Reasons easily understood and logically sequenced:

• Remind members to include clear headings in their reasons for their decision, even for oral decisions.
• In professional development programming, share tips and concrete examples of how to improve writing and organizational skills to deliver clearer and logically sequenced reasons for oral decisions.

2.7 Supplementary questions related to virtual hearings

Why measure this

During a time of transition from in person to virtual hearings, it is important to ensure that members adhere to guidelines prescribed for this new approach to adjudication. This includes ensuring that claimants are afforded accommodations when required, that the technology provides adequate quality for the hearing, that witnesses are participating with the same level of adherence to protocols as an in-person hearings, and that members follow practices in line with regional protocols. In conjunction with all other indicators, the below indicators will further support findings related to the quality of the virtual hearing process.

Considerations

All indicators are assessed only if applicable

General observations

• Given the many challenges involved in virtual hearings, members were efficient and resourceful in finding solutions to address them.

Strengths

• Appropriate steps to resolve sound, video or technical issues: There were sound, video or technical issues in 57 cases. The more common problems included echo, background noise (phones or other electronic devices ringing; paper noise; typing noise), sound cutting, image freezing, parties disconnecting, and problems showing a document on the screen. In 49 cases, members met expectations by acknowledging the problem and finding simple solutions, including asking the parties to repeat themselves and/or rejoin the meeting. Technical assistance was sometimes required.

  In 5 additional cases, members exceeded expectations by also providing direct advice on how to improve sound quality, contributing to preventing problems for the rest of the hearing. For example, members suggested using headphones or earbuds to reduce echo and background noise in 2 cases. In 3 cases, members also requested to turn off or mute cell phones and to disable notification alerts on computers.

• Participants are alone in the room: In 61 cases, at the beginning of the hearing members asked parties to confirm that they were alone in the room and that no one else could see or hear the proceedings. This practice helped members to clearly identify who was in the rooms and ensure that there were no unauthorized individuals. In 2 cases, after the claimant was asked the question and indicated that they were with a witness, the member requested that the witness leave the room while the claimant was testifying. It is noteworthy that for this indicator, the practice among regions is uneven, with the Eastern region almost always asking the question at the beginning of virtual hearings (n=26/28).
• New documents presented at the hearing: In all the cases assessed, members asked that new documents which were deemed relevant for the hearing be sent to the registry.

Areas for improvement

• Appropriate steps to resolve sound, video or technical issues: In only 3 cases, members did not acknowledge and address technical issues. In 2 cases, there were significant echo and sound problems, but the member continued the hearing with no attempt to address the issue. In 1 case, there were multiple problems: phone ringing, screen freezing, and the member even left the room to answer the door. In that case, claimant’s counsel had to suggest solutions to address technical problems, since the member did not.
• Witnesses are alone in the room: In 19 cases, members did not ask parties to the hearing whether they were alone in the room. Failure to ask this question was more common in the Central (n=15/38) and Western (n=3/14) regions. In another case, it is the claimant’s counsel who underlined that a person not part of the hearing was speaking to the claimant.

Recommendations

Sound quality:

• Ensure all members are aware of and using best practices to resolve sound and technology issues proactively and efficiently. Members should ensure they have a distraction free environment while presiding, free of noise pollution from outside sources wherever possible. For example, they should ask participants to turn off or mute their cell phones and to disable notification alerts on their computers.
• Headphones with microphones:
• Make headphones with integrated microphones available to all members and encourage their use.
• On the webpage “RPD virtual hearings – Guide for parties”, reference to headphones is made in the section “Tips to avoid technical problems”. However, such recommendation is not in the “Notice to Appear for a Virtual Hearing for your Refugee Protection Claims”, RPD 45.04 (December 3, 2020). Clearly put in all documents concerned with virtual hearings that headphones with integrated microphones are strongly recommended, while not mandatory.

Confirmation that all parties are alone: Harmonize practices among the Eastern, Central and Western regions and require members to add in their questions at the beginning of virtual hearings whether all parties are alone in the room.

3.0 Results for specific populations

3.1 Consideration for sexual orientation and gender identity and expression

Why measure this

For the purposes of Gender Based Analysis Plus (GBA+), which is a priority for the IRB as well as within the Government of Canada as a whole, a second checklist is used to assess hearings involving persons of diverse sexual orientation, gender identity and expression (SOGIE). The checklist assesses the application of Chairperson’s Guideline 9: Proceedings before the IRB Involving Sexual Orientation and Gender Identity and Expression.

What was measured:

Target: 100% of hearings scored as compliant

General observations

• Almost all of the 16 cases assessed met the expectations for the indicators. While it is not possible to make a general conclusion based on this small sample, it can be said that in these cases, members were well aware of the guiding principles in Chairperson’s Guideline 9.

Strengths

• Name choice: In the 14 cases for which this indicator was relevant, members addressed and referred to claimants by their chosen name, terminology, and pronouns. Good practices included specifically asking the question to the claimant as to what their chosen names, pronouns, and preferred terminology for identity were. In 1 case, a member used the terms "suffers from" gender dysphoria. While such language might at first raise concerns and indicate an area for improvement, the member immediately corrected herself. This example shows that language might convey negative connotations, however, when members are trained to avoid negative connotations and biases, and to use appropriate language and terminology, they can address the issue immediately as this particular example illustrates.
• Questioning an individual: Members asked questions in a respectful manner in all the cases assessed. Good examples included asking open questions.
• Inconsistencies, vagueness / material omissions: This indicator was met in 6 cases. For example, in 1 case, the member considered the fact that for a SOGIE claimant, it was more difficult to have a witness, when sexual orientation is criminalized and taboo in a particular society. In another file, the member inquired into the fact that sexual orientation was not raised in the Basis of Claim form and whether this omission could be explained by psychological and cultural factors.
• Intersectionality: Intersectionality was raised in 4 cases only. Intersectional factors considered were subject to persecution as a result of being gay or trans and of a particular religion, as well as being a woman and a lesbian. In one case, the member made explicit in his reasons such interaction between the two intersectional factors. In another case, the member applied both Chairperson’s Guideline 4 and Chairperson’s Guideline 9.
• State protection, decriminalization and IFA: Members in the 15 cases assessed were fully aware of the problems involved with State Protection of SOGIE claimants and that there were no IFA available to them. When relevant (in 5 cases), members looked at the fact that the decriminalization of practices related to SOGIE was on paper only, and did not lead to concrete results to protect SOGIE claimants’ rights and ensure their safety.

Areas for Improvement

• Inconsistencies, vagueness / material omissions: In the file for which this indicator was not met, the member did not consider cultural factors in assessing a woman’s lack of knowledge of her visa application which was filled out by her brother.

Recommendations​

Continue to provide mandatory training to all members on the application of Chairperson’s Guideline 9.

Intersectionality: Provide guidance on when it is relevant and how to consider intersectional factors when making a decision for SOGIE claimants.

Annex A – Checklist

Annex B – SOGIE Quality Review Checklist:
Performance Indicators and Rating Guide