Quality performance in the Immigration Appeal Division - 2021 to 2022

On this page

• 1.0 Context
• Sample methodology
• Assessment methodology
• Considerations and limitations
• 2.0 Performance results
• 2.1 Timely and complete pre-proceeding readiness
• 2.2 Respectful proceedings
• 2.3 Focused proceedings
• 2.4 Reasons state conclusions on all the determinative issues
• 2.5 Decisions provide findings and analysis necessary to justify conclusions
• 2.6 Reasons are transparent and intelligible
• 2.7 Supplementary questions related to virtual hearings
• Annexe A – Checklist
• Annexe B - Proceedings involving sexual orientation, gender identity and expression, and sex characteristics (SOGIESC): Performance indicator and rating guide

1.0 Context

This report describes the results of the measurement of quality in decision-making in the Immigration Appeal Division (IAD).

Sample methodology

The study reviewed 70 randomly selected appeals decided on their merits, after a hearing held before a single member, finalized between April 1, 2021 and March 31, 2022. While the majority of the IAD’s finalizations are reserved for written decisions, the sample focused on oral decisions. Residency obligation cases are over-represented in the sample as they have a higher rate of oral decisions. The next quality review will have a greater focus on written decisions. The hearings were balanced in proportion to region and language. Hearings between 45 minutes and 6 hours were included.

The following diagrams illustrate the sample design:

Regional office

Appeal type

Language of appeal

Assessment methodology

This qualitative assessment was performed by an independent reviewer who is the University Research Chair in Administrative Law and Governance at the University of Ottawa and one of Canada’s leading experts in administrative law. The reviewer examined all evidentiary and administrative materials on file, listened to the complete audio recordings, and assessed these against qualitative indicators in a checklist developed by the Strategic Planning, Accountability and Reporting (SPAR) Directorate and approved by the Deputy Chairperson of the IAD (see Appendix A). Each indicator is assessed on a 1-to-3 rating scale. The standard checklist assesses thirty two indicators across six performance categories. Sixteen of the indicators are mandatory for assessment, and sixteen are assessed only when applicable. An additional four questions were added to the checklist that pertain to the virtual hearing process, all of which are assessed on an as applicable basis. The performance categories are:

1. Timely and complete pre-proceeding readiness
2. Fair and respectful proceedings
3. Focused proceedings
4. Reasons state conclusions on all determinative issues
5. Decisions provide findings and analysis necessary to justify conclusions
6. Reasons are transparent and intelligible
7. Virtual ​hearing processes

The 1-to-3 rating scale is as follows:

1=Does not meet expectations: The quality requirement was not met. The evidence showed one or more key instances where the proceeding or reasons would have markedly benefited had this requirement been met. There may have been an effort to apply the requirement but the level of achievement fell short of expectations.

2=Meets expectations: This is a level of acceptable achievement. On balance, the member satisfied this quality requirement though there is margin for minor improvement.

3=Exceeds expectations: This is a level of consistent, above-average achievement. The evidence shows a grasp of the quality requirement and an understanding of its importance to a high-quality proceeding or decision, as the case may be.

Results are also expressed as a percentage of hearings that meet expectations. A hearing is considered to meet high quality standards when 80% or more of the standard indicators for that hearing score a 2 or higher.

In addition, as part of Gender Based Analysis Plus (GBA+), a second checklist was developed for proceedings involving Sexual Orientation, Gender Identity and Expression, and Sex Characteristics (see Appendix B). The assessor received instructions to complete the supplementary checklist if it was applicable to the hearing or the member mentioned Guideline 9 in the proceeding of decision. No cases were identified. The checklist is appended for reference.

Considerations and limitations

The goal of the study was not to generate statistics but to identify areas of strength, areas for improvement, and patterns in decision-making quality. This study acknowledges the inherent limitations to qualitative research, which does not generate precise data as do quantitative metrics. To mitigate the inherent limitations of qualitative research, detailed performance indicators were provided to the assessor to help focus the assessment. To ensure quality and consistency in the assessment, a reviewer was selected based on their in-depth knowledge of the IRPA and their significant experience with immigration and refugee law. Moreover, the assessor was provided with orientation by the division, and a detailed training guide.

The findings of this report, including the strengths, areas for improvement and recommendations found in sections 2.1-2.7 are solely those of the assessor. The evaluation unit of SPAR developed the tables and statistics for each section, as well as the information contained in section 1.0 “Context” and 2.0 “Performance Results”. The observations of the assessor do not lend themselves to firm conclusions on legal matters such as the correct application of the law, the weighing of the evidence, or the fairness of the proceedings from a natural justice perspective. Only a court reviewing the case can arrive at such conclusions. This report aims to provide a perspective to improve the Division’s performance overall.  

2.0 Performance results

What was measured

Each performance result in sections (2.1 to 2.7) has a table representing the number of hearings assessed for each indicator, and the percentage of assessed hearings that scored a 2.0 or higher. The number of hearings assessed is provided for reference and context only.

The primary performance target for this assessment is for 75% of hearings to meet quality standards. The quality standard is that the hearing achieves a score of 2.0 or higher in at least 80% of all indicators assessed within that hearing. This target was achieved with 97% (68 out of 70) of hearings meeting the expectation. Although the scores are high and many hearings scored a 2.0 on most indicators, a score of 2.0 still allows room for improvement.

Where an indicator had many hearings that did not meet the target, it is addressed in the reviewer’s observations following the table (areas for Improvement).

2.1 Timely and complete pre-proceeding readiness

Why assess these indicators

The groundwork for quality is set before the hearing when the Registry prepares a timely, organized and complete file and the member understands the facts and key issues of the case.

Considerations

These indicators apply to all hearings.

General observations

The general standard of document preparation was very high. This is impressive because ensuring that all relevant documents are in the file can be challenging, especially with many self-represented appellants (i.e. appellants who have not retained legal counsel or an immigration consultant who is familiar with IRB procedures). The IAD E-File is a very useful device for gathering information, helping members to prepare on the one hand and facilitating the drafting of reasons on the other hand.

In addition, Members had invariably consulted the file beforehand and were familiar with the issues on the appeal as well as the principal pieces of evidence in the file and additional testimonial evidence likely to be added during the hearing.

Strengths

Internal processes for document management are robust. Relevant material is readily accessible.

Areas for improvement

The appellant and counsel for the Minister do not have access to the IAD’s file management system. This caused brief delays at the outset of hearing as the Member ensured that the appellant and counsel for the Minister had all relevant documents in their possession.

E-Files, which are PDFs of all electronic documents in a single file, were not available in all cases. This could be improved as accessibility of the E-File helps to ensure efficiency.

Recommendation

Consideration should be given to using a file-sharing portal where the E-file can be shared with the parties to the appeal in advance of the hearing.

E-Files should be made available in all matters proceeding to a hearing.

2.2 Respectful proceedings

Why assess these indicators

Individuals appearing before the IRB expect that they will be treated with sensitivity and respect. Any shortcoming in this regard potentially undermines tribunal integrity and public confidence.

Considerations

Indicators #4 and #5 are considered universal—they apply to all hearings. Indicators #6, #7, and #8 are assessed as applicable.

General observations

Members conducted respectful proceedings in almost 100% of appeals. Members were invariably patient, especially with self-represented appellants. At the outset of a hearing, the Member would typically advise the parties of how long the appeal was to take and that it might be necessary to reduce the number of witnesses in order to fit the time slot. However, I did not encounter any case in which a Member prevented an appellant from leading evidence or asking questions where the appellant wished to do so.

Members also engaged in active adjudication, that is, shaping the issues to be resolved, directing the parties to address certain issues and ensuring by asking their own questions that relevant issues have been addressed by the parties. Members were generally more willing to engage in active adjudication with self-represented appellants, whereas when the appellant was represented by counsel Members tended to be less active adjudicators. Even in cases with represented appellants, however, Members nonetheless shaped the issues, directed the parties where necessary and asked their own questions in order to fill evidentiary gaps.

Strengths

Members are familiar with the tenets of active adjudication.

Members are capable of dealing with self-represented appellants who may be unfamiliar with the IAD’s decision-making process.

Members ensure that proceedings are flexible and designed to permit appellants to make their case as effectively as possible.

Members strike an appropriate balance between active adjudication and the need to be patient with self-represented appellants.

Areas for improvement

If appellants provided brief written overviews of their grounds of appeal, these would assist in focusing the appeal hearing on relevant issues and giving the Member a sense, especially with self-represented appellants, of the appellant’s understanding of the issues.

Recommendation

Although the performance in this area met expectations, to maintain those skills, Members should continue to receive training on active adjudication, dealing with self-represented appellants and ensuring efficiency of hearings.

The IAD should consider offering appellants the opportunity to provide brief written overviews of their grounds of appeal.

2.3 Focused proceedings

Why assess these indicators

Proceedings that are efficient and well managed create favourable conditions for quality outcomes to emerge and support the IRB’s efforts to make the most effective use of its resources.

Considerations

Indicators #9 to #11 are considered universal—they apply to all hearings. Indicators #12 to #21 are assessed as applicable. In certain hearings, universal indicators could not be assessed, either because the appeal was withdrawn at the start of the hearing, or the appeal was allowed on consent.

General observations​

Ensuring focused proceedings must be understood in the context of the workload of the IAD.

In respect of the appeals in this particular sample – removal orders, sponsorship refusals and non-respect of residency obligations – the issues are typically not especially complex: the legal frameworks are well established and the Member’s task is to apply the legal frameworks to the facts they find.

In proceedings before the IAD, testimonial evidence is typically of central importance, as it will illuminate compliance or non-compliance with conditions imposed as part of the stay of a removal order, the genuineneness of a family relationship, or the extent of humanitarian and compassionate considerations in favour of granting permanent residence despite failure to comply with a residency obligation.Success in ensuring focused proceedings therefore turns to a large extent on the Member’s ability to preside over hearings in which relevant testimonial evidence is adduced.

Strengths

Members are very good at framing the issues at the outset of an appeal hearing. Given the importance of testimonial evidence, framing the relevant issues at the outset allows appellants to adduce important evidence on relevant points. This is especially useful where the appellant is not represented by counsel or an immigration consultant. This practice can also lead to appeals being allowed on consent of counsel for the Minister, which is an efficient way of making decisions. And this practice also ensures that appeals are completed in the time allotted.

Members are also very good at framing the issues when the time comes for the parties to make closing arguments about whether the appeal should be allowed. When the appellant is self represented most Members ask whether the appellant would prefer counsel for the Minister to go first, which allows the appellant to make submissions in response to counsel for the Minister. This is a useful practice as it contributes to fairness for self-represented appellants.

Time management in the sample was excellent. Hearings proceeded expeditiously and within the allotted time. Delays due to interpretation or technical difficulties were rare.

Areas for improvement

Members were less likely to actively frame the issues when the appellant had legal representation. However, the quality of legal representation in matters before the IAD is variable: some counsel are knowledgeable and competent, but others are less so. It is also difficult for Members to know whether counsel has explained the nature of the appeal proceeding to the appellant. Indeed, appellant’s counsel occasionally asked questions which the appellant seemed not to have anticipated.

Although appeals do not have a great deal of legal complexity, in some appeals it is difficult to make clear, convincing findings of fact because of the nature of the questions in issue. Such appeals may take longer than the allotted time. This was especially true of sponsorship appeals, where there tends to be a significant degree of testimonial evidence adduced to demonstrate the genuineness of the relationship in question. At the very least, in spousal or common-law partnership cases, both the appellant and the applicant will typically have to testify. These appeals could perhaps be handled more efficiently, perhaps by requiring written witness statements in some cases. However, it must be appreciated that issues relating to genuineness, especially in unconventional relationships, are often relatively hard to resolve due to the difficulty of making clear, convincing findings of fact. At a minimum more time should be allotted as matter of course for sponsorship appeals.

Appellants do not generally provide a written overview of their grounds of appeal in advance of a hearing. Providing an overview in advance might provide further efficiencies as it would guide the appeal hearing. However, the marginal efficiency gains here might be relatively low as the issues on IAD appeals tend not to be particularly complex in any event.

Recommendation

Members should always frame the issues at the outset of a hearing, even where the appellant is represented by counsel or an immigration consultant.

The IAD should allocate more time for the hearing of sponsorship appeals about spousal or common-law partnerships and consider whether the hearing of such appeals could be streamlined, perhaps by the preparation of written witness statements in advance of the hearing.

As with the same recommendation in section 2.2, the IAD should consider offering appellants the opportunity to provide brief written overviews of their grounds of appeal.

2.4 Reasons state conclusions on all the determinative issues

Why assess these indicators

The Supreme Court of Canada set the requirement for justifiability, intelligibility and transparency of the decisions of an administrative tribunal. Through indicators #22 and #23, this study applies the Court’s requirement in the context of IRB decision-making.

Considerations

Indicators #22 and #23 are considered universal—they apply to all hearings. In certain hearings, universal indicators could not be assessed, either because the appeal was withdrawn at the start of the hearing, or the appeal was allowed on consent.

General observations

The bare minimum in terms of justifiability, intelligibility and transparency is that issues identified as determinative be addressed in a decision-maker’s reasons, based on issues and evidence adduced during the proceedings. Where appeals are allowed on consent of counsel for the Minister, there are no reasons and, perforce, no written justification of the decision on the basis of issues and evidence adduced during the proceedings. This creates challenges in respect of justifiability, intelligibility and transparency.

Strengths

Members understand that decisions must addressed determinative issues.

Members understand that decisions must be based on evidence adduced during the proceedings.

Where appeals are allowed on consent, Members generally ensure that counsel for the Minister gives oral reasons which address the determinative issues in the appeal on the basis of evidence adduced during the proceedings. To the extent that different counsel for the Minister take different views of when an appeal should be allowed on consent, the requirement for counsel for the Minister to state their reasons helps to ensure the necessary level of justifiability, intelligibility and transparency.

Areas for improvement

Members performed very competently in respect of these indicators and, as such, there are no obvious areas for improvement.

Recommendation

Although the performance in this area met expectations, members should continue to receive training on their statutory mandate and the importance of writing reasons on determinative issues based on evidence adduced at hearings.

2.5 Decisions provide findings and analysis necessary to justify conclusions

Why assess these indicators

The Supreme Court of Canada set the requirement for justifiability, intelligibility and transparency of the decisions of an administrative tribunal. Through indicators #24 to #29, this study applies the Court’s requirement in the context of IRB decision-making.

Considerations

Indicators #24 to #26 are considered universal—they apply to all hearings. Indicators #27 to #29 are assessed as applicable. In certain hearings, universal indicators could not be assessed, either because the appeal was withdrawn at the start of the hearing, or the appeal was allowed on consent.

General observations

For IAD decision-making to be coherent and effective, Members must make clear findings of fact and support these with relevant evidence. Many of the matters Members must address are relatively straightforward in a factual sense. However, they must occasionally apply evaluative concepts – such as whether an appellant in a permanent residency matter had reasonable grounds for leaving Canada – and to do this effectively the evaluation must be grounded in clear factual findings.

Members must also engage in best-practice decision writing (developed further in the next section), such as writing for the losing side (indicator 27), which requires decision-makers to ensure that they have touched on all relevant issues raised by the unsuccessful party and clearly communicated the reasons for the conclusion.

The legitimacy of IAD decision-making also turns in part on the cultural competence of Members, in particular their respect for social and cultural contextual factors in assessing evidence.

Strengths

Members generally make clear findings of fact on issues before them.

Findings of fact are generally supported by evidence in the record which, in addition, has been found to be credible and trustworthy.

Members write for the losing side by identifying evidence or arguments that would support an alternative outcome to the one reached by the Member and explain why the evidence or arguments are not persuasive.

Members demonstrate cultural competence in their decision writing.

Areas for improvement

Findings of fact are most important when dealing with evaluative concepts such as reasonableness. In some appeals Members did not make clear findings of fact, which had an impact on the cogency of their evaluation of reasonableness.​

Recommendation

Members should continue to receive training on the following:

• making findings of fact supported by credible, trustworthy evidence in the record
• cultural competence
• best practices in administrative decision-making, including writing for the losing side (as further developed in the next section)

Training modules should be developed on the importance of making factual findings when an evaluative judgement is required by the Member – for example of reasonableness in the context of a permanent residency appeal. IAD leaders should also highlight, in continuing professional development seminars, decisions where the Member has done a particularly good job of linking factual findings to evidence.

2.6 Reasons are transparent and intelligible

Why assess these indicators

The Supreme Court of Canada set the requirement for justifiability, intelligibility and transparency of the decisions of an administrative tribunal. Through indicators #30 to #32, this study applies the Court’s requirement in the context of IRB decision-making.

Considerations

Indicators #30 to #32 are considered universal—they apply to all hearings. In certain hearings, universal indicators could not be assessed, either because the appeal was withdrawn at the start of the hearing, or the appeal was allowed on consent.

General observations

Reasons for decision are critical to the legitimacy of decision-makers who, like Members of the IAD, exercise public power. As the Supreme Court of Canada commented in the Vavilov decision, “reasoned decision-making is the lynchpin of institutional legitimacy”.

Best practices in terms of reasoned decision-making are plain-language expression, writing for the losing side, point-first writing and issue-driven analysis.

Plain language writing means writing in short sentences, using simple words and avoiding the passive voice. This is especially important given the context of the work of the IAD, where the appellant is often a non-native speaker of English or French.

Writing for the losing side was addressed in the previous section.

Point-first writing requires the writer to clearly state the points to be decided, with their analysis then flowing through the legal framework and their findings of fact to a clear, unambiguous conclusion. Issue-driven analysis involves the production of decisions which are thematic, rather than chronological. Taken together, point-first writing and issue-driven analysis in an administrative law context leads to decisions with the following form: (1) identify the relevant issues; (2) provide contextual factual information about the issue; (3) outline the applicable legal framework and standards; (4) discuss the evidence and arguments presented; and (5) set out the conclusions having applied the legal framework and standards to the facts. Engaging in point-first and issue-driven writing makes administrative decisions much easier to understand.

However, when Members render oral decisions there is a temptation to deal with the issues in chronological order. This results in decisions which are neither point-first nor issue-driven and thus more difficult to understand.

Strengths

Members engage very effectively in plain language writing, whether decisions are delivered orally or not.

Members appreciate the importance of writing for the losing party.

Members have clearly been well-trained in the art of point-first and issue-based writing and with some exceptions engage in this form of decision writing even when delivering oral decisions.

The “overview” sections of Members’ reasons are generally excellent.

Areas for improvement

In this sample, Members rendered a significant amount of oral decisions. Rendering oral decisions is an efficient way of bringing an appeal to a conclusion and gives an immediate answer to the appellant on an issue of great personal importance. However, some Members take a chronological approach when rendering oral decisions: there is no editing after the fact to give their decisions point-first and issue-drive form. As such, these decisions fall short when measured against these indicators.

Recommendation

Continue training on high-quality decision writing, with an emphasis on plain language expression, point-first writing, issue-based analysis and writing for the losing side.

Develop training modules on the advantages and disadvantages of rendering oral decisions.

At continuing professional development seminars, choose several decisions illustrative of best practices to provide benchmarks for Members in their own decision writing.

Consider the development of an ‘oral reasons policy’ which would support Members in organizing oral decisions to conform to best decision-writing practices.

2.7 Supplementary questions related to virtual hearings

Why measure this

During a time of transition from in person to virtual hearings, it is important to ensure that members adhere to guidelines prescribed for this new approach to adjudication. This includes ensuring that claimants are afforded accommodations when required, that the technology provides adequate quality for the hearing, that witnesses are participating with the same level of adherence to protocols as an in-person hearings, and that members follow practices in line with regional protocols. In conjunction with all other indicators, the below indicators will further support findings related to the quality of the virtual hearing process.

Considerations

Virtual hearings only very occasionally caused difficulties in terms of connections. Where these arose the Member was able to resolve them without significant effort. In general, virtual hearings ran excellently and I have no concerns about their use in this context.

Strengths

Virtual hearings only very occasionally caused difficulties in terms of connections. Where these arose the Member was able to resolve them without significant effort. In general, virtual hearings ran excellently and I have no concerns about their use in this context.

Areas for improvement

None identified.

Recommendations

N/A

Annexe A– Checklist

Annexe B- Proceedings involving sexual orientation, gender identity and expression, and sex characteristics (SOGIESC): Performance indicator and rating guide