Report on the sense of access to justice associated with virtual hearings held before the IRB using MS Teams

​​​​​​​​​​Pr. Nicolas Vermeys^{Note 1}
Valentin Callipel^{Note 2​}

Table of content

1. Tables
2. Foreword
3. Executive Summary
1. ​ Section 1 – Methodology
2. Section 2 – Data analysis
3. Section 3 – Literature review on best practices in implementing and holding virtual hearings using MS Teams
4. Section 4 – Findings and recommendations
4. Introduction
5. Section 1 – Methodology
1. ​ Access to justice survey
2. IRBCC questionnaire
3. IRB virtual hearings
4. Guidelines and Virtual Ready Tour
5. Literature review
6. Section 2 – Data Analysis
1. ​ MS Teams allowed me to present my case and/or tell the member what I wanted to say during my hearing
2. MS Teams allowed me to interact with my representative/client during the process
3. I/(my client) was treated with respect and dignity during the virtual hearing
4. The virtual hearing took place without significant technical issues
5. No issues with interpretation were experienced during my virtual hearing
6. The fact that the hearing was virtual helped me/(my client) save time and effort (time I needed to prepare for the virtual hearing, etc.)
7. The fact that the hearing was virtual helped me/(my client) save money (expenses related to travel for example)
8. The fact that the hearing was virtual made it less stressful for me/(my client) than having to go to an in-person hearing
9. I/(my client and I) was given sufficient support from the IRB to take part in the virtual hearing
7. Section 3 – Literature Review on Best Practices in Implementing and Holding Virtual Hearings Using MS Teams
1. ​ Documentation
   
2. Testing
3. Internet connection
4. Equipment
   
5. Interpreters
6. Changes to the process
7. MS Teams Configuration
8. Security
   
8. Section 4 – Findings and Recommendations
1. ​ Respondents who took part in virtual hearings using MS Teams experienced a strong sense of access to justice
2. Survey data does not always reflect the positions held by authors or by IRBCC members
3. More data is necessary should the IRB decide to continue offering virtual hearings after the pandemic
4. Virtual hearings can lead to a loss of important rituals and practices
5. Concerns regarding witnesses receiving information from off camera can easily be addressed
6. Concerns regarding the loss of non-verbal are not warranted
7. Concerns regarding the ​dehumanising aspect of virtual hearings should not be ignored
8. Concerns regarding the impact of technical difficulties on a hearing’s outcome should be addressed
9. Concerns regarding security should be addressed
9. Conclusion
10. References
1. ​ Laws
2. Caselaw
3. Books and Papers
4. Reports
5. Other references
11. Appendices
12. Notes

Tables

1. Table 1. Sense of access to justice – Overall results
2. Table 2. Sense of access to justice
3. Table 3. Sense of access to justice questions

Foreword

BVC Groupe Cyberjustice was mandated by the Immigration and Refugee Board of Canada (hereinafter: the “IRB”) to produce a report (1) presenting the results of a survey conducted with persons appearing before the IRB, their representatives,^{Note 3} as well as hearing officers from the Canada Border Services Agency (hereinafter: the “CBSA”) to measure the sense of access to justice granted by virtual hearings conducted via MS Teams, and (2) presenting a series of best practices to ensure that said hearings are optimally structured and conducted.

Executive Summary

During the COVID-19 pandemic, the IRB opted to conduct certain hearings virtually using the Microsoft Teams videoconferencing platform (MS Teams). To help identify the impact of IRB virtual hearings on access to justice, BVC Cyberjustice Group was mandated to assess stakeholders’ sense of access to justice during these virtual hearings, as well as to make recommendations aimed at improving them in the future.

This report contains our findings and conclusions.

Section 1 – Methodology

The Sense of Access to Justice Survey (the “Survey”) was used to collect data from persons appearing before the IRB, their representatives, as well as hearing officers from the Canada Border Services Agency (CBSA) who represent either the Minister of Public Safety or the Minister of Immigration, Refugees and Citizenship in matters before the IRB. These individuals were asked to complete a survey aimed at measuring whether virtual hearings using MS Teams were perceived as granting a sufficient level of access to justice.

To interpret and supplement the data obtained through the Survey, information was gathered using the following methods:

• Members of the Immigration and Refugee Board of Canada Consultative Committee (IRBCC) were asked to fill out a questionnaire presenting their organizations’ perspective on the use of MS Teams to conduct hearings.
• BVC Cyberjustice Group representatives viewed ten (10) IRB hearings of the Refugee Protection Division (RPD) to get a better understanding of the virtual hearing process and to identify which elements of said process have proven to be suitable and which could be improved upon.
• BVC Cyberjustice Group representatives reviewed IRB virtual hearings guidelines targeted at persons participating in an IRB hearing.
• A BVC Cyberjustice Group representative attended a “Virtual Ready Tour” to gather firsthand knowledge of what measures and best practices had already been put into place by the IRB to ensure that their virtual hearings proceed in the best possible way.
• A review of papers, studies, reports, and other documents pertaining to virtual and remote hearings was conducted to see if the collected data coincides with other similar studies and experiments and to establish best practices.

Section 2 – Data Analysis

Overall, 270 individuals (42 persons who appeared before the IRB, 120 representatives and 108 CBSA hearing officers) completed the Survey. Results suggest that stakeholders have a strong sense of access to justice when taking part in virtual hearings using MS Teams:

Table 1. Sense of access to justice – Overall results

These largely positive results were obtained by establishing a mean between the answers given to each of the following 9 questions (only five of which were asked to CBSA hearing officers since the others did not apply in their case):

The gathered data also allowed us to conclude that a strong majority of stakeholders had an overall positive experience with virtual hearings.

Section 3 – Literature Review on Best Practices in Implementing and Holding Virtual Hearings Using MS Teams

Our literature review allowed us to identify a series of best practices that the IRB has either already adopted or could adopt in the future to enhance the overall experience of participants in a virtual hearing and, incidentally, to increase their already strong sense of access to justice:

The fact that the IRB has already adopted and implemented most of these best practices in whole or in part can help explain stakeholders’ strong sense of access to justice when taking part in virtual hearings using MS Teams.

Section 4 – Findings and Recommendations

The collected data and literature review have allowed us to make the following series of findings and recommendations to further improve virtual hearings:

Finding 1: As noted in section 1, our main finding is that respondents who took part in virtual hearings using MS Teams experienced a strong sense of access to justice. The data further conveys a high level of satisfaction regarding virtual hearings. In this sense, subsequent findings should not be seen as an indictment of virtual hearings, but rather as ways to improve stakeholders’ experience.

Recommendation:

• Continue offering virtual hearings after the pandemic.

Finding 2: While the data collected through the Survey is very positive, it does not always correlate with the results of our literature review, or the positions adopted by some IRBCC members. While this could theoretically be explained by the size of our sample, negative biases that exist regarding virtual hearings undoubtedly caused many of these divergences.

Recommendations:

• Continue administering the Survey to generate more data and validate the conclusions of this report.
• Make the conclusions of this report available to the public to help challenge some inaccurate preconceived notions regarding virtual hearings.

Finding 3: Although virtual hearings have proven to be useful and effective during the pandemic, more data is necessary to establish how they should be utilised in the long term. This finding is supported both by the literature review and the data gathered from IRBCC members who consider that virtual hearings should not continue to be used in all circumstances after the pandemic.

Recommendations:

• Gather data on the sense of access to justice for participants of in-person hearings to compare with virtual hearings.
• Conduct periodic reviews of virtual hearings to establish what criteria should be used to decide in which cases they should be made available.

Finding 4: Although very few respondents to the Survey or IRBCC members seemed to consider it problematic, some of the important rituals and practices associated with in-person hearings are lost when a hearing is held virtually. This is corroborated by the literature review as well as our own research. While the data shows that the current process has been deemed suitable considering the present context, how persons appearing before the IRB perceive the seriousness with which their case is heard should remain a focus point.

Recommendations:

• If deemed useful, adopt new rituals and practices such as a pre-hearing online meeting to explain the process and its importance.
• Create better virtual backgrounds for Board members.
• Encourage Board members to make use of MS Teams’ Spotlight feature.
• Add participants’ full names and roles to their display names.

Finding 5: Virtual hearings can raise concerns regarding witnesses receiving information from a person off camera. While this issue is mentioned by a section of the literature review and some respondents to the Survey, most IRBCC members see it as being overblown.

Recommendation

• Adopt protocols for members on how to address concerns about participants either not being alone in the room, accessing scripted materials, or receiving information from off camera.

Finding 6: It is suggested by certain authors that virtual hearings affect the ability to read non-verbal cues and, therefore, the Board member’s capacity to assess credibility. While most studies have shown these concerns to be unfounded (a position shared by a majority of IRBCC members), some respondents to the Survey remain fearful that this might impact the outcome of a hearing.

• Adopt clear guidelines regarding camera placement, framing and lighting for virtual hearings.
• Offer alternative solutions to individuals who cannot abide by the guidelines (see Finding 8).
• Offer training to members and interpreters on how to address non-verbal cues.

Finding 7: While most respondents to the Survey felt they were treated with respect, concerns regarding the dehumanising aspect of virtual hearings are still underlined by certain authors and IRBCC members. However, these concerns are mostly raised by those who studied or took part in remote hearings where the person appearing before the IRB does so from police custody or jail where they may experience disorientation, not being able to hear or understand the proceedings due to their environment or the technology. They may also perceive a lack of fairness as they are visibly appearing from detention. There is no data supporting the fact that this could be an issue when a person appears from their home.

Recommendation

• Work with jails, detention centers, IRBCC members, and other partners to try and offer quiet and accessible areas for taking part in virtual hearings.

Finding 8: Technical difficulties are bound to happen during virtual hearings. This concern is supported by the data collected through the Survey and shared by both the literature and IRBCC members. However, our data also shows that, when they did happen, technical difficulties were mostly minor and did not adversely impact the outcome of hearings or participants’ sense of access to justice. Finally, while there is no way to guarantee that technical issues will not arise, there are ways to mitigate risk.

• Continue to offer and update documentation that will inform participants on the technical requirements associated with a virtual hearing, as well as on how to use the technology.
• Allow a space and time for participants to test their equipment.
• Ensure that all participants have access to adequate Internet connections and devices.
• Create spaces or organise equipment rental or lending services directly or through a third party.

Finding 9: Virtual hearings can introduce security concerns. In fact, many authors and a majority of IRBCC members have raised the issue of security with regards to the MS Teams platform, the devices on which it is accessed, as well as the network used by participants to connect to a virtual hearing. While these concerns are founded, there are ways to limit risk.

Recommendations:

• Continue to advise participants to install all security updates on their devices, and to use secure networks to log onto the hearing.
• Stay informed of privacy legislation and how the CLOUD Act is being enforced in Canada.

It should be reiterated that these findings and recommendations in no way negate the strong sense of access to justice felt by stakeholders who took part in virtual hearings using MS Teams. They simply serve to highlight key issues that the IRB may want to further address to enhance stakeholders’ levels of satisfaction with the process.

Introduction

The purpose of this report is twofold. First, it aims to evaluate the sense of access to justice felt by persons who have appeared virtually before the IRB during the pandemic. Second, it looks to find ways to increase said sense of access to justice should the IRB decide to maintain virtual hearings in a post-pandemic world.

In the past two decades, many authors have criticized the use of videoconferencing during proceedings and argued in favour of in-person hearings.^{Note 4} However, the COVID-19 pandemic has forced us to re-evaluate this stance. The question therefore is no longer “should the IRB allow virtual hearings”, but rather, how to ensure that these hearings are conducted in a manner that respects the tenets of the justice system and favours access to justice.

Furthermore, as more and more research and studies tend to show,^{Note 5} the type of technological antagonism adopted by those who oppose virtual hearings has proven to be as erroneous and flawed a position as that held by those who see technology as the solution to all problems. This report does not deal in absolutes. It starts with the premise that videoconferencing technology can be useful in certain cases to increase efficiency, lower costs, facilitate calendar management and increase access to justice.^{Note 6} In this sense it approaches virtual hearings not as a scourge or a necessity, but as a fact. To quote a passage from Arconti v. Smith:

“In my view, the simplest answer to this issue is, “It’s 2020”. We no longer record evidence using quill and ink. In fact, we apparently do not even teach children to use cursive writing in all schools anymore. We now have the technological ability to communicate remotely effectively. Using it is more efficient and far less costly than personal attendance. We should not be going back.”^{Note 7}

Once again, this statement should not be perceived as a blind endorsement of virtual hearings, but rather as an admission that “court users expect that if the courts can serve people equally or better remotely, the courts should have those options available.”^{Note 8} In the midst of the pandemic, the IRB has chosen to meet those expectations and to turn towards virtual hearings to ensure access to justice, a move that has been received positively by commenters:

“One of the success stories, in my view, of the pandemic, has been the way things have worked at the IRB, like in terms of online hearings. It was something that I had a lot of hesitation about, but those hearings are fast, they’re efficient, scheduling delays have really been nipped in the bud.”^{Note 9}

As noted above, to validate this view, the IRB mandated the authors of this report to evaluate how the main stakeholders (persons appearing before the IRB) have appreciated their virtual experience and how to make it better still should these hearings continue after the pandemic. The following pages will address these questions as follows:

1. In Section 1, we will explain our methodology and approach.
2. In Section 2, we will analyse the results of a survey administered to persons appearing before the IRB, their representatives, as well as hearing officers from the CBSA regarding access to justice through virtual hearings using MS Teams. The results of this survey will allow us to better understand which elements of the virtual hearing need to be addressed.
3. In Section 3, we will review some of the best practices put forth by the literature regarding virtual hearings using MS Teams. It should be stressed that the IRB has already implemented most of these best practices in whole or in part. However, modifications can still be made to improve the process.
4. Finally, in Section 4, we will present our general findings and make a series of recommendations that we believe could improve stakeholders’ sense of access to justice regarding virtual hearings held using MS Teams.

The current report contains numerous references to statements and comments made by persons appearing before the IRB, their representatives, hearing officers from the CBSA, Board members (hereinafter: “members”), as well as IRBCC members. To ensure respondents’ anonymity, these comments were not attributed and sometimes modified. Modifications were also made in certain cases to correct typos, but in no way have the quoted comments been modified in ways that could affect their meaning.

Section 1 – Methodology

In this section, we present the methodology used to gather the data that comprises this report. The data was collected using the following methods:

• Persons appearing before the IRB, their representatives, as well as hearing officers from the CBSA were asked to complete a survey aimed at measuring whether virtual hearings using MS Teams were perceived as granting a sufficient level of access to justice.
• Members of the IRBCC were asked to fill out a questionnaire presenting their organizations’ perspective on the use of MS Teams to conduct hearings.
• BVC Cyberjustice Group representatives viewed ten (10) IRB hearings of the Refugee Protection Division (RPD) to get a better understanding of the virtual hearing process and to identify which elements of said process have proven to be suitable and which could be improved upon.
• BVC Cyberjustice Group representatives reviewed IRB virtual hearings guidelines targeted at persons participating in an IRB hearing.
• A BVC Cyberjustice Group representative attended a “Virtual Ready Tour” to gather firsthand knowledge of what measures and best practices had already been put into place by the IRB to ensure that their virtual hearings proceed in the best possible way.
• A review of papers, studies, reports, and other documents pertaining to virtual and remote hearings was conducted to see if the collected data coincides with other similar studies and experiments and to establish best practices.

The methodology adopted for the drafting of this report relied on five (5) means of data-gathering:

1. Access to Justice Survey
2. IRBCC Questionnaire
   
3. Observation of IRB Virtual Hearings
4. Review of Guidelines and Virtual Ready Tour
5. Literature Review

Access to justice survey

Measuring access to justice through virtual hearings using MS Teams is somewhat difficult since the main stakeholders (persons appearing before the IRB) have no real reference point that allows them to give an informed answer. There are two main reasons for this. First, the very notion of access to justice remains difficult to properly circumscribe for legal experts, let alone laypeople.^{Note 10} For the purpose of this report, we choose to define access to justice in the particular context of persons appearing before the IRB as having three components:

1. having the ability to invoke and effectively participate in justice processes (procedural access);
2. obtaining a fair result when they do (substantive access); and,
3. having their unique circumstances and needs recognized and respected by the justice system (inclusive access).^{Note 11}

Second, as explained in a previous study cond​ucted at the behest of the IRB:

“[Persons appearing before the IRB] will presumably have very little experience with the use of video in a hearing – typically on only one or two occasions. They will also not have had their claim heard through both a video hearing and a normal hearing and so will typically have nothing to which to compare their video experience. In these circumstances, one would not know whether the negative experience (assuming that was a claimant's response) was attributable to the video, or merely to the style of the particular panel's conduct of the hearing, or to a natural first reaction to a formal hearing environment under stressful circumstances, or to the influence of their representative's views on video hearings, etc.”^{Note 12}

However, as our mandate was not to ascertain if and how virtual hearings are superior or inferior to in person hearings, but rather to assess the level of access to justice granted by virtual hearings using MS Teams, surveying persons appearing before the IRB as well as their representatives remained a necessity. In fact, this approach is recommended by the OECD, which suggests “implementing a survey for users and implement improvements to remote hearings based on layperson experience.”^{Note 13} Of course, the challenge then became to choose the right questions.

To this end, the methodology we devised is based on the work of Justice Jean-François Roberge.^{Note 14} Justice Roberge’s Sense of Access to Justice Index, which measures “the users’ self-assessed satisfaction with four aspects: quality of the outcome (fair, reparative, functional, transparent), quality of the process (fair, informational, interpersonal), cost-effectiveness (resource cost, psychological cost, opportunity cost) and the quality of the [member]’s actions”,^{Note 15} was reworked and adapted to be applicable to the context and reality of IRB hearings.

A proposed survey was workshopped with representatives from the IRB’s Refugee Protection Division (RPD), Immigration Division (ID), and Immigration Appeal Division (IAD). Following the workshop, an Assessment Guide (Appendix 1) identifying the questions that we believed should be submitted to respondents was drafted and submitted to the IRB for approval.

It was then agreed to reduce the number of contextual questions (i.e., questions that did not pertain directly to the sense of access to justice, but would allow us to identify factors that could explain certain scores) to the following:

• Did you connect to your virtual hearing using MS Teams videoconferencing technology?
• What was your hearing at the IRB about?
• Were you represented by a lawyer or an immigration consultant?
• Was the outcome in your favor?
• Did you receive the notice to appear?
• Did you read the notice to appear provided by the IRB?
• Did you find our notice to appear easy to understand?
• Did you receive any supporting information before your hearing, including the participant guide?
• Did you read any supporting information before your hearing, including the participant guide?
• Did you find the supporting information relating to your virtual hearing, including the participant guide, easy to understand?
• Before your hearing, did you understand what to expect and how to prepare for your hearing?
• When you entered the virtual hearing, was the procedure explained to you by someone from the IRB?
• What is your comfort level with technology?
• What device did you use to connect to the virtual hearing?
• From where did you connect to the hearing?

As for questions pertaining to the IRB process, only one was submitted to participants:

• To what extent do you agree with the following statement: At the end of the virtual hearing, I feel that, overall, my virtual hearing experience at the IRB was positive.

Finally, as mentioned in the Assessment Guide, only questions regarding the respondents’ experience were used to calculate their sense of access to justice. These questions were also reworked and then submitted to participants as follows:

• To what extent do you agree with the following statement: At the end of the virtual hearing, I feel that:
• MS Teams allowed me to present my case and/or tell the member what I wanted to say during my hearing.
• MS Teams allowed me to interact with my representative during the process.
• I was treated with respect and dignity during the virtual hearing.
• The virtual hearing took place without significant technical issues.
• No issues with interpretation were experienced during my virtual hearing.
• The fact that the hearing was virtual helped me save time and effort (time I needed to prepare for the virtual hearing, etc.).
• The fact that the hearing was virtual helped me save money (expenses related to travel for example).
• The fact that the hearing was virtual made it less stressful for me than having to go to an in-person hearing.
• I was given sufficient support from the IRB to take part in the virtual hearing.

We should also point out that the survey metrics were adjusted at the request of the IRB. The scale was modified from a scale ranging from 1 to 6 (as explained in Appendix 1) to a scale ranging from 1 to 5 where:

1 = strongly disagree
2 = disagree
3 = neither agree nor disagree
4 = agree
5 = strongly agree

The survey was administered to persons who appeared before the IRB during the summer of 2021, which yielded 42 responses. Adapted versions of the same survey were also sent to their representatives, yielding 120 responses; and to CBSA hearing officers, yielding 108 responses. This allowed us to analyse a total of 270 survey responses.

The collected data was then analysed using statistical analysis tools to establish access to justice scores and to pinpoint which factors could be seen as contributing to a higher (or lower) sense of access to justice.

IRBCC questionnaire

We drafted a short questionnaire based on preliminary answers to the survey described above. This questionnaire (Appendix II) was distributed to members of the IRBCC. In all, we received answers from seven (7) IRBCC members. The collected answers to the questionnaire were analysed and used to contextualize answers to the survey.

IRB virtual hearings

To get firsthand knowledge on how participants in online hearings interact with the technology and each other, we watched recordings of ten (10) virtual hearings of the Refugee Protection Division (RPD) held in 2021. These viewings allowed us to better understand the issues faced by IRB members, persons appearing before the IRB, their representatives, and other participants. It also allowed us to see how participants in virtual hearings adapted to the technology.

Guidelines and Virtual Ready Tour

We studied the documentation made available to persons appearing before the IRB regarding virtual hearings such as the “RPD virtual hearings – Guide for parties”, ^{Note 16} and attended a “Virtual Ready Tour”. These tours aim “to assist claimants in preparing for their hearing” ^{Note 17} . This allowed us to gather firsthand knowledge of what measures and best practices had already been put into place by the IRB to ensure that their virtual hearings function in the best possible way.

Literature review

A literature review was completed to identify articles, studies and reports conducted in Canada and abroad regarding the impact of remote or virtual hearings on access to justice or, more generally, on the judicial process.

The purpose of this literature review was to complete and build on the data collected through the survey to identify “best practices” in the field of virtual hearings. We focused on identifying sources that undertook to establish such “best practices”.

We observed that most of the literature pertaining to the use of tools such as MS Teams which was written before the pandemic focused on remote rather than virtual hearings. As a reminder, remote hearings usually imply that only one participant is accessing the hearing through videoconferencing, whereas in a virtual hearing all participants will connect remotely. While documents pertaining to remote hearings offer useful data and guidelines regarding things such as camera placement^{Note 18} and the need to maintain the “ritual elements of courtroom experiences”,^{Note 19} they mostly focus on issues such as:

• The legality of using videoconferencing tools to facilitate hearings;^{Note 20}
• The criteria set forth to justify allowing a witness to testify remotely ^{Note 21} when the law does allow it; and
• The ways of limiting the impact and possible negative inferences of having a witness testify remotely while the other participants are in the same room.^{Note 22}

While these elements can still be of interest, the legality of virtual hearings is no longer a point of contention.^{Note 23} Furthermore, since the purpose of this study is to gather data to help the IRB make an informed decision as to if and how virtual hearings should continue being used post-pandemic, questions that pertain to other forms of remote hearings were set aside. Finally, many of these studies relate to civil or criminal trials where the adversarial system is at play. As most IRB hearings do not follow that pattern, questions pertaining to how one party could benefit from an unfair advantage if they were present while the other party is testifying remotely do not apply. Therefore, although we have analysed over twenty pre-pandemic references and will refer to some of these, we have chosen to focus on pandemic-related research and studies as they remain more pertinent to our mandate.

Section 2 – Data Analysis

In this section, we analyse the results of a survey administered to persons appearing before the IRB, their representatives, as well as hearing officers from the CBSA regarding access to justice through virtual hearings using MS Teams. The results of said survey are quite compelling, as these categories of individuals rate their sense of access to justice at 86.4%, 85.9%, and 86.4% respectively.

As can be gleaned from Table 2, after analysing the pertinent data, we found that the general sense of access to justice of participants in virtual hearings held before the IRB using MS Teams ranges from 85.9% for representatives to 86.4% for persons appearing before the IRB. CBSA hearing officers have a general sense of access to justice of 86%, although that score comes with a caveat since only 5 of the 9 questions listed above were asked to these individuals; the others were deemed irrelevant for these participants as, for example, they do not have clients and, therefore, cannot interact with them.

Table 2. Sense of access to justice

These scores are comparable to – if not slightly higher than – those obtained by other procedures that the Sense of Justice Index was used to evaluate.^{Note 24} However, such a comparison remains unreliable since the questions and scale used in these other surveys were quite different than the ones used to evaluate virtual hearings held before the IRB using MS Teams. In no means does this negate or diminish our findings, but it does limit what we can draw from comparing our data to other similar experiments.

In our view, what is more interesting than the general sense of access to justice, however, is the relative score given to each question (see Table 3 below).

Table 3. Sense of access to justice questions

MS Teams allowed me to present my case and/or tell the member what I wanted to say during my hearing

Persons appearing before the IRB: 87.6%
Representatives: 93.8%
CBSA Hearing officers: 92.4%

The data shows that respondents overwhelmingly believe that the virtual medium does not hinder their capacity to express themselves.^{Note 25} This is a particularly encouraging statistic as many commenters claims that the screen limits one’s capacity to express themselves.^{Note 26}

As we will see in the third section of this report, this could have a lot to do with the format of some IRB hearings as, in most surveyed cases, there was very little reliance on documentary evidence during hearings, and little witness testimony or cross-examination. Also, closing pleadings were kept to a minimum, which could explain why representatives’ numbers are so high.^{Note 27}

MS Teams allowed me to interact with my representative/client during the process

Persons appearing before the IRB: 82.8%
Representatives: 80.4%

Overall satisfaction regarding interactions between persons appearing before the IRB and their representatives is quite high.^{Note 28} However, this question did receive the second lowest score, which would suggest that representative-client interactions are not as easy to replicate in a MS Teams environment. As one IRBCC member pointed out: “With regards to the ability to confer to clients, representatives found this hard as there was no interpreter present, whereas previously they could use the IRB appointed interpreter for simple communication with their client.”

Furthermore, regarding the confidentiality of client-representative interactions, while there is a possibility for breakout rooms, these do not seem to be used during proceedings. In fact, in the hearings we observed, members suggested that persons appearing before the IRB communicate with their representatives via telephone. This is also what is suggested on the IRB website.^{Note 29} As we see further on, one way to improve the satisfaction level regarding representative-client communications is to offer breaks to “allow for clients to consult their lawyers, parties to confer, judicial officers to consult associates or check other matters.”^{Note 30} Another way to improve communications between representatives and their clients is to offer “[c]onfidential breakout rooms for counsel and clients to be able to meet during hearing breaks, which would both reduce stress for the claimant and aid with hearing flow”. This suggestion was made by an IRBCC member and is supported by the literature.^{Note 31}

Although not directly related to the question, one IRBCC member commented that “Virtual hearings for detention reviews have increased access to counsel”. So, while interactions with a representative can be made more complicated by videoconferencing technology in certain cases, it clearly can have the opposite effect in others.

I/(my client) was treated with respect and dignity during the virtual hearing

Persons appearing before the IRB: 91.4%
Representatives: 94.4%
CBSA Hearing officers: 93.8%

A recuring criticism of virtual hearings is that they affect the dignity of the legal process.^{Note 32} In this sense, the fact that respondents consider that they were treated with respect and dignity bodes well for their overall sense of access to justice and appreciation of the online experience.^{Note 33} We should mention that these numbers are a direct result of the care taken by members to make persons appearing before them feel as comfortable as possible. While less than 1% of respondents criticized the seated member as being “a bit intense and unfriendly”, or “disrespectful and inpatient with both myself and my client”, the vast majority expressed a far different opinion.

These comments correspond to the answers gathered by the Canadian Council for Refugees for their survey.^{Note 34} They also concur with our own observations of IRB hearings, as we were quite impressed with the level of empathy and humanity displayed by members. They are also in line with comments taken from other similar surveys:

“a majority of responders in all other groups either were very satisfied or satisfied that the use of remote technology to conduct court proceedings promotes the traditional dignity and seriousness otherwise experienced during in-person court proceedings.”^{Note 35}

This is not to say that the technology is perfect. As one IRBCC member submits, “some claimants feel that their case is taken less seriously when it is not a decision maker in person but just someone on a screen.” Possible solutions to addressing this feeling will be presented later in this report.

The virtual hearing took place without significant technical issues

Persons appearing before the IRB: 84.2%
Representatives: 83.6%
CBSA Hearing officers: 81.8%

This question generated below average scores from all three categories of respondents. This is not surprising since technical difficulties come up in most studies on virtual hearings as one of the main impediments to access to justice.^{Note 36} What is reassuring, however, is that while the number of cases where there were technical difficulties remains quite high (69% for persons appearing before the IRB, 76.5% for representatives, and 71% for CBSA hearing officers),^{Note 37} these difficulties were not considered significant and did not impact the proceedings or the participants’ sense of access to justice.^{Note 38}

This sentiment was echoed by one IRBCC member who mentioned that “there are technological issues in about 1 of 5 of my hearings. They are usually very small issues, logging out and logging back in usually solves the problem.”

Again, these observations correspond to data collected in other jurisdictions.^{Note 39} As was the case in a study of the online experience for Chicago’s Immigration Court, “There did not appear to be any strong relationship between the occurrence of technical problems and the outcome of the hearings – that is, detained immigrants who experienced equipment difficulties were not more likely to be ordered removed than those who did not.”^{Note 40} In fact, IRB survey data shows a correlation of less than 10% between outcome and the presence of technical issues, which is of no statistical significance.

No issues with interpretation were experienced during my virtual hearing

Persons appearing before the IRB: 86.6%
Representatives: 83.4%

Most collected comments regarding interpretation related to the interpreter’s skills and competency, and not the environment. Therefore, the statistics regarding this question do not offer a clear picture of the issue. However, when technical issues did arise with interpretation (according to comments this only happened in 3% of cases), they mostly had to do with the interpreter’s Internet connection. As one IRBCC member explains: “Most participants stated that issues with interpretation were always present at the IRB, but virtual hearings have aggravated them. A lot of these issues had to do with technology and the interpreter closing their camera to ensure better internet speed.”

Finally, there is no statistically significant correlation (about 7%) between interpretation issues and outcome, which would suggest that the reported issues did not appear to influence the member in the respondents’ minds.^{Note 41}

The fact that the hearing was virtual helped me/(my client) save time and effort (time I needed to prepare for the virtual hearing, etc.)

Persons appearing before the IRB: 89%
Representatives: 84.2%
CBSA Hearing officers: 81.6%

The main argument in favour of virtual hearings is that they help save time and money.^{Note 42} In this sense, the data simply confirms what was already known: Virtual hearings are often more efficient than their in-person equivalent. In fact, this was underlined by several IRBCC members, as well as by several respondents.

The fact that the hearing was virtual helped me/(my client) save money (expenses related to travel for example)

Persons appearing before the IRB: 87.2%
Representatives: 86.6%

As with the previous question, the data simply confirms what was already known: lower costs (most notably travel costs) are among the main advantages associated with virtual hearings.^{Note 43}

This sentiment was echoed by more than half of IRBCC members, one of which pointed out that “Claimants living in remote locations are able to attend their hearings without incurring the cost and associated stress of long-distance travel”.

Of course, this is not to say that virtual hearings are cost neutral, as acquiring the proper equipment (computer, earphones, proper Internet connection) can be expensive. As one IRBCC member observed, however, “A lot of the cost of proceeding virtually is born by the lawyer when the client does not have an extra computer or proper internet at home to conduct the hearings. Lawyers have had to invest on another computer to help clients who cannot have their hearings at home.”

The fact that the hearing was virtual made it less stressful for me/(my client) than having to go to an in-person hearing

Persons appearing before the IRB: 82%
Representatives: 80%
CBSA Hearing officers: 80.4%

While these results are relatively low in comparison to others, one should keep in mind that, in a mock trial conducted on the use of videoconferencing in Australia, “The average score for ‘stressed’ was 3.5, just above the mid-point of the scale, a level not affected by whether the accused appeared on a screen or in-person.”^{Note 44} Therefore, any score above 50% should be seen as very positive. This implies that respondents are more fearful of going to an in-person hearing than to attend said hearing remotely, which studies suggest might be linked to the fact that virtual hearings are perceived as being less formal by participants.^{Note 45}

The positive impact of virtual hearings on the stress level of some persons appearing before the IRB was also pointed to by a majority of IRBCC members. To quote one of them: “For the majority of people participating in virtual hearings I would agree that there is a lower stress level. Participants can stay in their home; they don’t have to commute great distances to the hearing room or sit in an intimidating room.” These same IRBCC members, however, also underlined what some of the literature suggests^{Note 46} : Virtual hearings can have the opposite effect on participants. As one IRBCC member stated: “where people don’t know how to use technology or are unable to afford it, the idea of a virtual hearing can be more stressful. Especially in cases where participants may have mental health issues and designated representatives may be involved.”^{Note 47}

An interesting element regarding the stress level of respondents is that it seems to be higher prior to as opposed to during the hearing. As the Canadian Council for Refugees observed in its report, “Most claimants reported feeling tense and anxious in the days leading up to the hearing. Nerves were heightened by the online format.”^{Note 48} However, in the same report, respondents are quoted as claiming that the online medium helped relieve stress once the hearing had started,^{Note 49} which corresponds to the data we analysed. This suggests that representatives have a role to play in dedramatizing the online experience when preparing their clients for the hearing. In fact, as lawyers have strongly criticized online hearings in the past,^{Note 50} a criticism which is still very present in the answers gathered from some IRBCC members, we would posit that these positions might unjustly colour the perceptions of persons appearing virtually, and therefore, increase their stress level.

I/(my client and I) was given sufficient support from the IRB to take part in the virtual hearing

Persons appearing before the IRB: 86.6%
Representatives: 87.4%

There is really nothing to add to these numbers other than to point out that the pre-hearing information, including the participant’s guide, seem to be considered useful. In fact, there is somewhat of a correlation (among persons who appeared before the IRB) between those who have received (40%) and read (42%) the participant’s guide and the perception of having received proper support. As for support during the hearing, comments suggest that IRB staff are quite helpful.

Finally, it is interesting to juxtapose the sense of access to justice with the answers to the contextual questions asked of respondents. In doing so, we can identify relevant factors that could have impacted an individual’s answers to the survey.

For example, according to our data, there is a strong correlation between a person appearing before the IRB’s sense of access to justice and the outcome of the hearing (68.1%), while no such correlation can be established from the answers received from representatives or CBSA hearing officers. Therefore, it could be suggested that, for a person appearing before the IRB, outcome plays a more important role than the medium in establishing their sense of access to justice, while legal experts that are not personally affected by the results of the proceedings can more easily differentiate between the process and the outcome. As was mentioned in a previous study conducted for the IRB:

“It would be difficult, in my view, without extensive and careful questioning, to separate out a claimant's response to the use of video from his or her experience in winning or losing. The losers in particular may be counted on to be generally negative about the experience and their opinion of the use of video will be seen to be so influenced by that negativety (sic) (whether it is or not) as to destroy its weight as useful evidence. The opinion of winners might be more significant but in terms of value to an objective assessment it too will be questionable.”^{Note 51}

However, while the correlation between outcome and access to justice is strong, we should underline that our data set regarding outcome is somewhat limited, making it difficult to draw a clear conclusion. Out of the 42 respondents, only 13 had received the member’s decision at the time of filling out the survey, and of those, only two indicated receiving an unfavorable decision. Therefore, although the examined correlation reflects our instinctive belief, further data would be necessary to confirm the impact of outcome on one’s sense of access to justice.

Where we can draw a clear statistical link, however, is between a person’s sense of access to justice and whether their overall virtual hearing experience before the IRB was positive. When comparing those data sets, we unsurprisingly arrive to strong correlations (Persons appearing before the IRB: 79.7%; Representatives 75.2%; CBSA hearing officers: 85.6%). While this might seem like a foregone conclusion, it helps to confirm what we addressed in the introduction, and that is that the general satisfaction with virtual hearings held before the IRB using MS Teams is strong (Persons appearing before the IRB: 89.6%; Representatives 89.6%; CBSA hearing officers: 89.2%).^{Note 52} In fact, of the 63 comments received from respondents to the survey, most (40) were to convey satisfaction with virtual hearings and/or to express a wish that they be maintained post-pandemic. Here is a sample of some on the most informative comments:

It should also be mentioned that these observations are shared by those who conducted similar studies in other jurisdictions.^{Note 53}

As for IRBCC members, positions are more divided. While all would like to see virtual hearings continue, two are fundamentally opposed to their use except for exceptional cases and suggest that they should “not become the predominant refugee status determination mechanism”. Whatever the case, IRBCC members agree that the decision whether to be heard virtually or in person should belong to the person appearing before the IRB and that MS Teams should not become the de facto vehicle for hearings.

Section 3 – Literature Review on Best Practices in Implementing and Holding Virtual Hearings Using MS Teams

In this section, we review some of the best practices put forth by the relevant literature regarding virtual hearings using MS Teams. While we have found that the IRB has already implemented most of these best practices, some adjustments to how virtual hearings are conducted can still be made to enhance the experience of participants, to reinforce judicial legitimacy, and to increase the sense of access to justice.

The following suggestions and recommendations are made with an important caveat: They presume that the IRB will choose to pursue the exclusive use of MS Teams to hold virtual hearings. While MS Teams has many qualities, it can be difficult to use for certain people. ^{Note 54} As mentioned by the Canadian Council for refugees, “[i]t is not as user friendly as Zoom for instance, and requires a lot of battery for the computer or device to run.”^{Note 55} Furthermore, since MS Teams was not designed to support any type of judicial process, it remains limited in what it can do in the context of IRB hearings,^{Note 56} something that was underlined by one IRBCC member.

Finally, we should mention that, as we shall see, most of the best practices mentioned below have already been adopted in whole or in part by the IRB.^{Note 57}

1. Documentation

What the IRB offers:

• Virtual hearing guide (online)
• Information on the Notice to appear
• Participant guide

The most important element when discussing best practices is to generate documentation that will allow all participants to adopt said best practices, to acquire or borrow the proper equipment and software, and to familiarise themselves with these tools.^{Note 58} This documentation should also “set out what parties should do if technological issues arise during the hearing and how they will be addressed.”^{Note 59} Since the IRB has already published a series of guides^{Note 60} , the next step is to ensure that these documents are up to date and, more importantly, that they are read and followed. When analysing IRBCC comments, we were surprised at how many criticisms were already addressed in the IRB’s literature. There is therefore a disconnect between the IRB’s instructions and how participants behave. This disconnect can only be fixed through better information and guidance. Since the current documentation does not seem to be reaching all stakeholders, the IRB may want to adopt some of the recommendations formulated by one IRBCC member and “create an educational video and post it on IRB website”, or “provide the registry personnel and Members with screenshots of the process so they can walk people through what is going in with a visual aid they can share on the screen to help people having trouble to locate buttons”.

2. Testing

What the IRB website suggests:

“Test the link to the virtual hearing prior to the hearing to check that the microphone and webcam are being recognized by Microsoft Teams and see what your video will look like.”

While it cannot ensure success, assessing and testing the technology beforehand is a necessary step in holding a successful virtual hearing.^{Note 61} This is something that is already offered by the IRB through its Ready Tours. In fact, numerous IRBCC members, including the Canadian Council for Refugees,^{Note 62} mentioned that Ready Tours were deemed very useful for allowing persons appearing before the IRB to prepare for their hearing, and familiarise themselves with MS Teams.

3. Internet connection

What the IRB website suggests:

“Have a laptop, desktop, tablet computer, or smartphone with a camera (webcam) and access to high speed internet sufficient to reliably videoconference.”

According to one study, for MS Teams to run adequately during a group videocall requires a connection of 500kbps to 1mbps, while the required bandwidth is doubled for HD group video calling.^{Note 63} Therefore, prior to the hearing, participants should be invited to test their connection speed using sites such as www.fast.com or www.speedtest.net.^{Note 64} Furthermore, to ensure optimal speed, participants should use a hard-wired Internet connection rather than Wi-Fi^{Note 65} , and should disconnect from any VPN.^{Note 66} This last suggestion may be difficult to follow in some cases since, for what we suspect are security reasons, Government of Canada employees must connect through VPN including members of the IRB, and CBSA hearing officers.

Of course, reaching the types of speeds mentioned above may be problematic since “[m]any refugees may lack adequate internet access.”^{Note 67} , which could undermine the fairness of the proceedings.^{Note 68} As we will see below, the IRB and its partners should therefore make available installations that respect these criteria.

4. Equipment

Device

What the IRB website suggests:

“Have a laptop, desktop, tablet computer, or smartphone with a camera (webcam) and access to high-speed internet sufficient to reliably videoconference.”

“We do not recommend using a smartphone because of their small screens. If you do use a phone, do not hold it in your hand throughout the hearing, but instead position it in front of you in a stable position such as on a table.”

It goes without saying that all participants should use adequate devices to take part in a hearing.^{Note 69}

In our view, this should disqualify the use of smartphones. Unfortunately, while the IRB expressly mention that they do not recommend using smartphones, several respondents to both our survey and that of the Canadian Council for Refugees showed that “phones were among the devices typically used”^{Note 70} by persons appearing before the IRB. This is unfortunate as phone displays are simply “too small to view who and what is being displayed can also negatively affect the proceedings.”^{Note 71} Furthermore, the IRB members who oversaw the hearings we observed indicated to participants to keep their phones handy in case of a technical issue.^{Note 72} This creates a problem since participants may need to log out of the MS Teams meeting to take a call.

Of course, we are quite aware that not all persons appearing before the IRB have access to a computer. Therefore, if one does use a smart phone, the device should “be placed on a stable and stationary surface and not held”, as “[m]ovement requires more bandwidth and reduces both video and audio quality.”^{Note 73} Furthermore, in one hearing we viewed, the fact that the person appearing before the IRB held their cellphone in their hand created constant movement and regularly cut off portions of their face.

The device used to take part in a virtual hearing using MS Teams should be a Windows computer with 4 GB of RAM, and a processor speed of 2.0 GHz, or a Mac computer with 4 GB of RAM and an Intel processor^{Note 74} or better. To ensure its proper function, “participants should close all applications on their computer that are not needed for the hearing”,^{Note 75} and ensure that the device “has completed any outstanding system updates before the hearing”.^{Note 76} To limit issues and distractions, all notifications should be disactivated during the hearing^{Note 77} , as should any automatic updates.^{Note 78}

Screen size

As larger video screens make it easier for participants to see each other,^{Note 79} and “produce an increased sense of involvement and perceptual realism”,^{Note 80} it is recommended for all participants to use high-definition and wider screens. In fact, when possible, it is recommended to use two large screens^{Note 81} , one for viewing the other participants, and the other to consult documents (although this undoubtedly will have an impact on eye contact).^{Note 82}

Camera

What the IRB website suggests:

“Ensure you are in a quiet, distraction-free location, with good lighting, and minimal background noise. Use an appropriate camera angle and background for your video.”

Camera placement

Given that “past studies dealing with the effect of camera angle in television production suggest that camera angle can affect credibility”^{Note 83} , The camera should be positioned at eye level or slightly above eye level^{Note 84} Furthermore, considering the importance attributed to eye contact by some commenters^{Note 85} (lack of eye contact is sometimes construed “as a sign that the speaker is being uncertain or even dishonest” ^{Note 86} ), participants should be reminded to look directly at the camera when speaking rather than at the image of their interlocutor or, worst of all, their own image. In fact, seeing one’s own image in MS Teams, while useful to see “what the judge can see” such as pets in the background,^{Note 88} is an important source of anxiety , and distraction.^{Note 90} While other videoconferencing platforms such as Zoom allow for individuals to disactivate self-view, MS Teams has yet to incorporate that feature. Until they do so, an admittedly artisanal option would be to put a post-it on one’s own image.

Finally, if, as suggested in the previous section (Device), participants are encouraged to use more than one screen, the camera should be installed “on the screen with the video feed”^{Note 91} to ensure eye contact with the other participants.

Camera framing

Participants should always be instructed to face the camera^{Note 92} , as it is preferable than seeing someone’s profile,^{Note 93} and “strives to negate any effects induced by the interposed videoconferencing medium”.^{Note 94} Furthermore, although ideal placement might differ depending on the scenario^{Note 95} , the camera should always be positioned “so that the judge and parties can clearly see the witness, particularly their whole face and their hands if possible.”^{Note 96} Showing a participant's hands serves two purposes: 1) it facilitates communication when a participant talks with their hands, and 2) it allows the Member to make sure that participants are not manipulating documents outside of the camera’s eye.

As one respondent to our survey submitted, “[a]fter the hearing, my wife told me that the image of my face had been too big in screen.” This can be problematic as it is shown that being too close or too far from the camera can have an adverse effect on one’s perceived credibility.^{Note 97}

Angles are also important as “[s]tandard filmmaking texts teach that high angle shots tend to make the person depicted appear smaller or weaker, while low angle shots make the person seem more significant and powerful, and experimental studies have found that faces seen from below are perceived more positively than faces seen from above.”^{Note 98}

Proper framing is also important for the member. Studies show that “the framing of the judge may compromise their standing and the authority of the bench”.^{Note 99} Therefore, like other participants, members should be invited to “[s]it or stand close enough to the camera so that [their] face appears clearly”,^{Note 100} but not so close as to overwhelm the other participants.

Lighting

The main element when it comes to lighting is to ensure that participants are clearly lit from the front^{Note 101} . Backlighting (e.g., sitting in front of a window) might blind the camera^{Note 102} and make the participant appear too dark and difficult to see,^{Note 103} while overhead lighting can cast shadows^{Note 104} The quantity of light should also be carefully chosen. As one author put it: “Too little light and a participant will be in shadow, possibly affecting her credibility. Too much light may cause strange effects, especially if the person is using a virtual backdrop.”^{Note 105}

Lighting issues were a recurring issue in the hearings we observed, and they had an impact on our capacity to follow the proceedings. We can therefore surmise that this was also an issue for participants.

Another element linked to lighting has to do with glasses. The lighting source should always be positioned “so as to avoid reflections off of your glasses.”^{Note 106} It should also be positioned to limit the reflection of the screen in your glasses. In two of the hearings we observed, we could see a reflection of the member’s screen in their glasses, which was not only distracting, but can cause confidentiality concerns when the other participants have a large screen.

Microphone and earphones

What the IRB website suggests:

“Have access to a private and distraction-free location with good lighting and minimal background noise.”

“Use headphones that have a microphone to avoid echo and increase audio quality.”

Poor audio quality is not only an issue regarding one’s understanding of the proceedings. Studies have shown that individuals who have difficulties with audio quality “may be inclined to misattribute their negative feelings […] to the witness himself and to evaluate him less favorably”.^{Note 107} For this reason, participants should not rely on their computer’s built-in microphone and speakers but should rather use noise-canceling headsets with an integrated microphone.^{Note 108} As we observed in a number of the hearings we viewed, this will reduce the impact of ambient noise, as well as enhance the confidentiality of hearings.^{Note 109}

It goes without saying that it would be preferable to advise participants “to locate themselves in a quiet private space where they will not be interrupted”^{Note 110} . However, as explained by the Canadian Council for Refugees, this might not be possible for some individuals who share their house with several people or live next to noisy neighbours.^{Note 111} In these cases, proper noise-canceling headsets present the best option.

Of course, we are quite aware that many of the technological recommendations listed above can be inaccessible to persons appearing before the IRB for different reasons, such as them being on the wrong side of the digital divide.^{Note 112} Therefore, as IRBCC members underlined, it is important to keep and expend upon programs providing technological support, “loaner” laptops^{Note 113} , as well as giving persons appearing before the IRB access to “dedicated hearing rooms at IRB to provide distanced hearing options.” This is best summarized by one IRBCC member’s comment on current accommodations: “We commend the IRB for providing spaces at its offices for claimants to participate in hearings and encourage the IRB to seek further opportunities to expand this possibility in community settings with community organizations”. ^{Note 114}

As for IRB members and interpreters, as was underlined by one IRBCC member, resources need to be put into getting quality equipment, as well as robust Internet connections.

5. Interpreters

What the IRB website suggests:

“Witnesses should ensure that the interpreter has enough time to fully interpret their responses to questions. To do this, witnesses should pause often to let the interpreter translate.”

As all other participants, interpreters need to have a fast Internet connection, a proper computer and a headset with an integrated microphone, as well as a camera (interpreters should be visible).^{Note 115} Although most interpreters we observed seemed to follow these basic rules, comments from IRBCC members as well as participants seem to indicate that this remains a major issue. As one IRBCC member explained: “A lot of the technical difficulties raised were with interpreters and them having poor quality microphones, or internet connection issues.”

Also, while some could argue that interpreters need not be visible, as one IRBCC member suggests, “[w]ith a sightline, the interpreter is able to gesture the speaker to break their answer up, rather than cut them off (in many cases). This facilitates communication.” Another best practice is to establish a verbal cue that the interpreter can use to address the person they are translating and vice-versa.^{Note 116} This will help make communication easier if, for example, a word was missed by the interpreter.

One best practice that is debated is whether interpretation should be simultaneous or consecutive. It is suggested that simultaneous interpretation could help make the process more efficient and fluid. ^{Note 117} In the same vein, one IRBCC member suggested that incorporating simultaneous captions could facilitate translation and understanding. However, as neither of these options are currently offered by MS Teams (unlike other platforms such as Zoom which do offer simultaneous translation), the question is moot.

Therefore, other than to instruct persons appearing before the IRB to keep their answers short to facilitate interpretation, something that the members we observed already do, there is very little that can be done to enhance the interpretation considering the current limitations of MS Teams.

6. Changes to the process

Journey from home

What the IRB website suggests:

“Registry staff will go online approximately 15 minutes before the hearing start time to greet the parties, answer any logistical questions, and ensure that the connection is working.”

As some experts explain:

“An important element of the court experience involves the journey from home (or detention) through the court building and into the courtroom. This journey also needs reimagining for a virtual space. In a virtual court, the journey begins with technical checks to ensure that participants’ equipment meets minimum standards for effective participation, including support for the conferencing software being used, adequate internet speed and computer capacity. Ideally, their environment should also be assessed for suitability in terms of aural and visual privacy, lighting, acoustics, camera angle, amenity and physical size.”^{Note 118}

Such a journey already exists as the IRB website indicates that “Registry staff will go online approximately 15 minutes before the hearing start time to greet the parties, answer any logistical questions, and ensure that the connection is working.”^{Note 119} While this is an important first step, it lacks the sense of ceremony and ritual that accompany walking into a hearing room, ^{Note 120} and could be improved by incorporating a series of steps such as explaining the oath, the position of the parties, etc. to “allow a participant to be fully informed about their role and help prepare them participate effectively.”^{Note 121}

Hearing

What the IRB website suggests:

“It is expected that counsel and their clients will normally communicate with each other by phone during hearing breaks if they are not located together. The Microsoft Teams software indicates when it is recording, and parties should confirm that the Member has stopped the recording during any break when they are having a private conversation. Where a participant is accessing their hearing from an IRB office, the microphone connected to the computer has an off switch that they can press to ensure that their conversation during a break is private.”

“Registry staff will go online approximately 15 minutes before the hearing start time to greet the parties, answer any logistical questions, and ensure that the connection is working.”

As some experts suggest: “[s]mall modifications to the court ritual, such as expanded introductions, acknowledgements, and breaks, can help orientate participants and promote effective participation.”^{Note 122}

Regarding breaks, we observed two somewhat problematic issues. First, although members advise participants to turn off their cameras, in two of the hearings we viewed, this was not done. In one case, the person appearing before the IRB could be heard sobbing in the background, which – although not on the record – could have been heard and considered by the member. The second issue we observed is that because participants were asked to log on at a given time, it happened on more than one occasion that the person appearing before the IRB and the member were on MS Teams while the person’s representative was still absent. While this wasn’t cause for concern in the cases we viewed, it would be preferable for members to wait an extra minute to enter the room only once all other participants are present.

Simply adding extended introductions, acknowledgements and breaks remains insufficient to reproduce the “in person experience”. As some authors put it: “[t]he substantiality of the courtroom in the courthouse, the formality that the configuration of the room encourages, and the state authority that the building’s and room’s symbolism convey all tend to make participants in proceedings feel that they have had the opportunity to be heard. They feel this not just in a technical legal sense but in a way that is vivid, dignified, and resonant.”^{Note 123} Because online hearings occur from one’s home using the same computer as they use to watch videos or shop, this feeling may be lost^{Note 124} and therefore needs to be otherwise recreated.

Of course, “the design, framing, and ritual of a virtual court need not recreate the formality of a traditional court”; authority can be communicated in other manners, including “gestures of respect and inclusion.”^{Note 125} For example, during the Ready Tour, we were informed that persons appearing before the IRB would be asked to stand to take the oath, but this wasn’t the case in the hearings we viewed, which is completely understandable as asking someone to stand online implies losing eye contact. However, we felt that those members who took the time to explain why the oath would be administered differently did a better job of framing the importance of said oath.

In the same vein, we noticed that, while members were given a script that covers elements such as whether the person appearing before the IRB understands the risks associated with a virtual hearing, reminding participants that it is forbidden to take screenshots or record the proceedings, and asking all involved to confirm they are alone in the room and cannot be heard by others, in more than half of the hearings we viewed this script was not followed and was sometimes even ignored. Furthermore, regarding this last indication (asking all involved to confirm they are alone in the room), we noticed that while participants always answered that they were alone and could not be heard, on more than one occasion, someone would show up during the hearing at the opportune time, which suggests that they could hear what was being said. Should members have concerns regarding this, since adding a second camera that would allow them to see the entire room is not feasible in most cases,^{Note 126} two best practices that we observed in pre-COVID situations are to ask the person to pick-up their laptop or camera and go around the room (something that was even suggested by an IRBCC member); or to ask them to reposition themselves in front of the door to ensure that nobody enters during the hearing.

One element that goes a long way in recreating the formality of an in-person hearing is attire.^{Note 127} Most observers agree that proper business attire should be required for virtual hearings^{Note 128} , although the use of a camera makes it a little more constraining since patterns and stripes do not film well.^{Note 129} While viewing hearings, we were surprised to see that most participants, including members, were dressed rather casually.

7. MS Teams Configuration

Image configuration

What the IRB website suggests:

“The web browser version of Microsoft Teams only displays the video of the person speaking. To prevent this, pin those you want to see by clicking on ‘show participants’, clicking on the ellipsis next to the participant you want to pin, and selecting ‘Pin’.”

As one expert observes, “[v]ideo meetings with their grid appearances are decidedly informal. Assuming that the court wishes to convey that virtual trials and hearings are as important as in-person ones, it must consider how to convey that formality.”^{Note 130}

Although IRB hearings are less formal than trials, an element of formality is still necessary. In a virtual setting, this can be done using the Spotlight function as well as adequate backdrops.

For example, during the Ready tour, we were told that the screen layout during hearings would be as follows:^{Note 131}

Because of the way MS Teams is configured, this setup is not automatic. In fact, none of the hearings we viewed offered this same display. This was somewhat problematic in certain scenarios where the representative or the interpreter were in bigger windows than the member or the person appearing before the IRB.

This therefore creates a hierarchy of roles that differs from that of in person hearings where the member is in the command position.^{Note 132} Furthermore, while experts advocate “for a set tile placement that [appears] on all participants’ screens”^{Note 133} to ensure that all participants share the same view, this is simply not possible using MS Teams. Quite the opposite, views may shift as participants log on or when you go in and out of screenshare mode, which can be quite distracting. As one expert put it “[t]his increased mental effort that [members] must allocate to what they are doing in the virtual courtroom may itself bias their impressions of witnesses’ and parties’ demeanors.” ^{Note 134}

One workaround that is offered by MS Teams is the Spotlight feature, which allows the member to “choose anyone's video (including their own) to be the main video people see.”^{Note 135} Members should use this function to impose their own image^{Note 136} or that of the person appearing before the IRB depending on the context of the hearing and what they deem to be most important at a given time.^{Note 137}

Another way to increase formality is to adopt a virtual background. As some experts explain, “[i]n a virtual hearing, judicial dress and background may be the strongest symbol that participants have available to represent a justice proceeding. In these circumstances, it is particularly important to have a background that distinguishes the court”. ^{Note 138} For this reason, other experts suggest that members use a virtual background which shows, for example, a coat of arms, as “[t]he inclusion of this symbol, which is one of the best-known symbols of State, would undoubtedly assist in elevating and formalising the tone of the proceedings.”^{Note 139} While we understand the IRB may prefer using a less imposing background, we suggest the background that is currently used be changed. In the hearings we observed, while all members used a background which incorporated the IRB logo, said logo became impossible to read because of resizing. Furthermore, the background appears to be of plain wood tables bookended by two Canadian flags, which – in our opinion – does not convey the seriousness of the hearing.

As for the other participants, a virtual background might also be recommended as they “do not always have access to a private space or a neutral background”, which “is a potential barrier to equal access to justice.”^{Note 140} Of course, as virtual backgrounds “are designed to obscure the room and therefore the presence of other people”,^{Note 141} some believe that they should not be used by persons appearing before the IRB.^{Note 142} If the decision is made to disallow the use of virtual backgrounds, participants should be instructed to attend the hearing from a space that offers a neutral physical background^{Note 143} since “the various backgrounds that appear behind different witnesses and parties may affect [the member’s] interpretations of their demeanors differently”.^{Note 144} This is important because “the visibility of each participant’s home or office environment can exert a distracting centrifugal force, fragmenting the virtual courtroom into a mere juxtaposition of personal settings filled with window treatments, bookshelves, and tchotchkes.”^{Note 145}

Identification of participants

In observing the recordings of hearings, we were surprised to notice that the participants were not identified. This may seem trivial but adding the names and roles of participants on screen could help them better situate themselves, especially at the beginning of the hearing.^{Note 146} While it is possible to see someone’s name when you move your cursor onto their picture, the name that appears is not always that of the individual behind the camera and that person’s title isn’t necessarily indicated. For example, during one of the hearings we viewed, the translator was using their spouse’s computer and MS Teams account. The name on the screen was therefore that of their spouse.

Adding names would also help correct mistakes such as persons appearing before the IRB and their representatives addressing the member as “judge”. It will also help eliminate the confusion brought by the fact that tiles in MS Teams tend to reorganise themselves whenever a participant logs on or off the platform. We noticed this in most of the hearings we observed when a break was taken. Depending on which participant returned first, windows would change positions.

Screen sharing

What the IRB website suggests:

“Participants can share their screens during the hearing using Microsoft Teams. For example, during submissions, counsel may wish to share their screen to point the Member to a part of a case they are looking at. Similarly, the Member may wish to show a document on the screen to the claimant. Participants can do this using the share icon in Microsoft Teams. The help article on ‘showing your screen during a meeting’ on the Microsoft Teams video training website describes the process for doing this.”

While the IRB website states that “[p]articipants can share their screens during the hearing using Microsoft Teams”,^{Note 147} that function was only used once in the hearings we observed. In most cases, documents were simply shown to the camera and persons appearing before the IRB were asked to confirm these were their documents without being able to properly see them. On one occurrence, it was the member that showed a document to the camera, but the virtual backdrop made this impossible. He therefore had to temporarily disactivate his backdrop, which could have had security implications.

Of course, as one IRBCC member submitted, screen-sharing is not optimal, especially when hearings are document-heavy. Furthermore, the screen-sharing function has the negative side-effect of reducing the size of participants’ windows, which is problematic.^{Note 148} For this reason screen sharing should never be used before getting prior authorization from the member.^{Note 149} In fact, for security reasons, screen-sharing should be disallowed for any participant unless authorised by the member.^{Note 150}

A better approach than screen sharing would be to acquire third-party software that allows participants to share a view of certain documents. Of course, this would require either a second screen or, at the very least, a very large screen that can hold both the MS Teams window and that of this other application.

8. ​Security

What the IRB website suggests:

“Have a home/office technology setup that is sufficiently secure and private. See the Government of Canada’s “Cyber Safe” recommendations.”

There is little that can be done to address security issues that the IRB has not already put into its Notice to appear (see below).^{Note 151} All participants should be advised to install all security updates on their devices, and to use secure networks to log onto the hearing.^{Note 152} We should underline that this goes against a previous best practice (not using VPNs). In this sense, as is often the case, there is an obvious trade-off between security and accessibility.^{Note 153} Members will need to decide on a case-by-case basis which should prevail while advising participants of the consequences to allow for an informed decision.

Section 4 – Findings and Recommendations

In this section we submit a list of 9 findings that could help guide the IRB should they chose to pursue virtual hearings post-pandemic. These findings are accompanied by a series of recommendations to help address them. The findings are as follows:

1. Respondents who took part in virtual hearings using MS Teams experienced a strong sense of access to justice
2. Survey data does not always reflect the positions held by authors or by IRBCC members
3. More data is necessary should the IRB decide to continue offering virtual hearings after the pandemic
4. Virtual hearings can lead to a loss of important rituals and practices
5. Concerns regarding witnesses receiving information from off camera can easily be addressed
6. Concerns regarding the loss of non-verbal cues are not warranted
7. Concerns regarding the dehumanising aspect of virtual hearings should not be ignored
8. Concerns regarding the impact of technical difficulties on a hearing’s outcome should be addressed
9. Concerns regarding security should be addressed

Considering the data gathered through the survey, the literature review, as well as the other methods identified in section 1 of this report, we have made the following findings:

1. Respondents who took part in virtual hearings using MS Teams experienced a strong sense of access to justice

As described extensively in section 2 of this report, the sense of access to justice of persons appearing before the IRB is quite strong. Furthermore, our data shows an overall high satisfaction level with the virtual hearing process. This is not surprising. After all, many of the IRB cases we viewed or surveyed fit most of the criteria similar studies conducted in the United Kingdom^{Note 154} have identified as favoring virtual hearings:

• Cases where evidence is not examined
• Cases that involved represented parties and/or professionals
• Parties who are confident with technology
• Short hearings
• One sided applications

Furthermore, the benefits of virtual hearings are numerous. They:

• “Allow hearings to take place when safety, security, public health or logistical difficulties present otherwise insurmountable obstacles.
• May reduce processing/wait times for refugee claimants where they cannot be conducted in person as a result of security risks, unavailability of resources, including specialized staff, and/or other obstacles.
• May reduce processing/wait times for refugee claimants when used for the purpose of facilitating interpretation. For example, where interpretation services in specific languages are not readily available at the location of the hearing.”^{Note 155}

This analysis was shared by at least one IRBCC member. In fact, the Canadian Council for Refugees came up with a similar list of criteria, but adding one important item: Vulnerable persons sometimes feel safer staying at home.^{Note 156}

Recommendation:

• Continue offering virtual hearings after the pandemic.

2. Survey data does not always reflect the positions held by authors or by IRBCC members

While the survey data is quite promising, no author has suggested that virtual hearings should permanently replace all in person hearings, and many hold positions that cannot be conciliated with the data we collected. While none of the literature is steadfast against virtual hearings, it is widely believed that these hearings are not necessarily adapted to all types of cases and that members should be given sufficient leeway to decide if a case – given its nature and context – should proceed virtually or in person.^{Note 157} Of course, as underlined in a 2004 study, this is also the conclusion the IRB arrived at regarding remote hearings:

“The Board does not consider all refugee claims to be appropriate for videoconferencing. The policy statement specifically identifies the following cases as inappropriate: ‘those involving unaccompanied minors or other persons who, in the opinion of the RPD, may be unable to appreciate the nature of the proceedings; those involving detained persons (priority processing); and those likely to present a significant degree of case complexity.’”^{Note 158}

As we saw earlier, this position was shared by all IRBCC members who responded to our questionnaire. In their opinion, persons appearing before the IRB post-pandemic should be given a choice between an in-person and virtual hearing. Furthermore, they believe that “there should not be a default to virtual hearings, and scheduling must defer to client preference.”

While one IRBCC member suggested that hearings should be held in person “In all cases but refugee hearings and perhaps IAD appeals”, others had a more nuanced view and identified specific contexts where virtual hearings should not take place. In fact, as one IRBCC member submitted, it’s not necessarily the type of hearing that should establish whether to use MS Teams, but rather the context of each hearing. As they explained, “I could see instances where there was interactive evidence that needed to be shown to a member where an in-person hearing would be preferable.” Two other IRBCC members suggested that the profile of the claimant^{Note 159} (for example if they experience mental health or cognitive issues) and not the type of claim should dictate whether a hearing can be held virtually.

However, as each IRBCC member offered their own list of situations where they deemed virtual hearings to be inappropriate, a compounded list would be too long and cover almost all possible scenarios. In other words, while each individual list seems reasonable and limited, combining all lists would relegate virtual hearings to extremely rare occurrences justified by emergencies such as the current pandemic.

As for the collected data, it does not allow us to establish a list of possible criteria on which to base such a decision. At most, our analysis of recorded hearings suggests that lack of decent equipment or of an adequate environment could impact the proceedings,^{Note 160} but this isn’t an argument for in person hearings as much as for participants to better plan that aspect of the hearing by gaining access to better equipment or reserving a space at a public library or other adapted facility. This is not to say that there are no concerns regarding virtual hearings. However, as we found, these concerns are often exaggerated and sometimes uninformed or based on faulty data.

Recommendations:

• Continue administering the survey to generate more data and validate the conclusions of this report.
• Make the conclusions of this report available to the public to help challenge some inaccurate preconceived notions regarding virtual hearings.

3. More data is necessary should the IRB decide to continue offering virtual hearings after the pandemic

While the Sense of Access to Justice score of virtual hearings held before the IRB using MS Teams is high, it remains difficult to establish how virtual hearings compare to traditional hearings without an established baseline. Therefore, we would suggest that, should in person hearings return at a more substantial rate, the participants in said hearings be asked to answer similar survey questions to the ones that were analysed in this report. This will allow the IRB to compare the data and see if an individual’s sense of access to justice is impacted by the medium.

Furthermore, as the survey was administered during a relatively limited period, it is advised to continue administering it in the next few years to gather more data^{Note 161} (most notably regarding cases where the outcome is not the one the person appearing before the IRB was hoping for^{Note 162} ). It will also serve to see, as we would posit considering previous data,^{Note 163} if the general sense of access to justice rises as participants become more familiar with virtual hearings and as the technology itself improves. This is particularly important if the IRB choses to follow any of the recommendations put forth in this report, as they should improve the process and, therefore, how it is viewed by different participants and stakeholders.

We should underline that this recommendation is shared by at least half of IRBCC members, which suggest the IRB hold “periodic review[s] of virtual hearings so as to be in a position to make changes as more information comes to light regarding the impact of these new procedures.”^{Note 164}

Finally, periodic reviews will allow the IRB to see if the popularity of virtual hearings will stand the test of time or if it is simply linked to current circumstances.^{Note 165}

Recommendations:

• Gather data on the sense of access to justice for participants of in-person hearings to compare with virtual hearings.
• Conduct periodic reviews of virtual hearings to establish what criteria should be used to decide in which cases they should be made available.

4. Virtual hearings can lead to a loss of important rituals and practices

As some authors have explained:

“why is physical presence a recurring theme among those who seem to fear the establishment of cyberjustice? Beyond immediate and contingent arguments, such as the importance of cross-examination in the common law, a plausible explanation lies in the deep ritualization of the legal process in general. If the parties are absent, there is a loss of theatricality, and this troubles some lawyers. Law remains today “one of the most ritualized functions of social life”.”^{Note 166}

It is well established that a shift to virtual hearings forces us to abandon some of the rituals associated with in-person hearings.^{Note 167} For example, as we addressed earlier, persons appearing before the IRB no longer stand when taking the oath online since this would cut sightlines. While the loss of these rituals does not need to be seen as an obstacle to virtual hearings, rituality does hold some importance and should not simply be ignored:

“ritual, particularly through its symbolic aspect, contributes to the social order. The challenge for cyberjustice is thus to re-invent appropriate rituals that are, of course, based on those of the past, or at least to adapt rituals to new technology […] Cyberjustice cannot be exempt from rituals that assure continuity with the more traditional rituals of law”^{Note 168}

While IRB hearings do not rely on rituals as much as court hearings, they do still rely on certain rituals. Therefore, efforts should be made to 1) identify current rituals and practices; 2) establish which rituals are necessary to the process; and 3) see how these rituals could be adapted to virtual hearings.^{Note 169}

Recommendations:

• If deemed useful, adopt new rituals and practices such as a mandatory pre-hearing online meeting to explain the process and its importance.
• Create better virtual backgrounds for Board members.
• Encourage Board members to make use of MS Teams’ Spotlight feature.
• Add participants’ full names and roles to their display names.

5. Concerns regarding witnesses receiving information from off camera can easily be addressed

One of the risks associated with virtual hearings is that we do not know what can happen outside the camera view, which leads people to make statements such as “[w]hen the clients are home, I am not sure who else is around and who else could be listening to their testimony, and that could impact what they share with the member.”^{Note 170} These types of concerns are shared by a number of authors^{Note 171} and were proven to be somewhat valid by the recent case of Kaushal v. Vasudeva et al.^{Note 172} . However, these concerns are seen by IRBCC members to be overblown in the context of IRB hearings,^{Note 173} especially if the person is represented by an attorney bound by a code of ethics.^{Note 174} Another member suggested that “these concerns can be addressed to a large extent by the presiding member and counsels' careful observation of the witness, such as where the witness’ eyes are focusing, body language, etc. and ensuring a clear and well-lit video of witness during the virtual hearing will help.”

To this end, protocols can be put in place that offer a structured approach to the problem should there be concerns:

1. Board members should have participants swear that they are alone in the room. While this was a common occurrence with the hearings we observed, it was not done automatically in all instances;
2. If there is still doubt, Board members should ask participants to “sweep the room” with their camera and/or to sit in front of the door to ensure that no one enters the room during the hearing;
3. If necessary, board members could also ask participants to place a second camera in the room to allow them to see what is on the other side of participants’ screens (this is however unlikely to be possible in most cases);
   
4. ​In the extremely rare cases where there is still serious doubt, the hearing would have to be held in person.

Recommendation:

• Adopt protocols for members on how to address concerns about participants either not being alone in the room, accessing scripted materials, or receiving information from off camera.

6. Concerns regarding the loss of non-verbal are not warranted

Many articles in the literature review, as well as two IRBCC members have expressed concerns that – because participants to a virtual trial are not in the same room – it becomes difficult to pick up on non-verbal cues and therefore more difficult to assess credibility.^{Note 175} This is seen as particularly relevant in immigration proceedings since a person appearing before the IRB’s credibility in retelling their life story is pivotal.^{Note 176}

However, data shows that such a concern is unfounded for two reasons. First, the belief that demeanor evidence allows individuals to assess credibility is flawed and has been “tested and rejected by social scientists”^{Note 177} since it is often based on “stereotypical expectations for how parties and witnesses should express themselves.”^{Note 178}

Second, it is simply untrue that non-verbal cues cannot be perceived through videoconferencing. In fact, participants will often see each other better than in a physical hearing.^{Note 179} As one author explains: “videoconferencing may even enhance the ability to assess credibility because of high quality sound reproduction, action replays, and the ability of the judge to view the witness from different angles.”^{Note 180} This sentiment was echoed by at least one IRBCC member.

Furthermore, even if we chose to ignore these arguments, two IRBCC members pointed out that non-verbal communications are not on the record. They should therefore not be considered to determine a person’s credibility.^{Note 181} In fact, one IRBCC member suggested that members and interpreters be given proper training in how to address these non-verbal cues, while another suggested that the IRB publish guidelines instructing “members not to make credibility determinations based on demeanour.”

Finally, some IRBCC members, while suggesting that it may be more difficult to read non-verbal cues when watching a person on screen, added that this implies that the member taking part in an in-person hearing is paying attention to non-verbal cues, which isn’t always possible as they are taking notes and reviewing documents.

Recommendations:

• Adopt clear guidelines regarding camera placement, framing and lighting for virtual hearings.
• Offer alternative solutions to individuals who cannot abide by the guidelines (see Finding 8).
• Offer training to members and interpreters on how to address non-verbal cues.

7. Concerns regarding the dehumanising aspect of virtual hearings should not be ignored

As explained by Amy Salyzyn, some researchers have pointed to the potentially “dehumanizing” effect of using technology to replace in-person courtroom presence.^{Note 182} However, most studies that mention the dehumanising effect of videoconferencing have analysed remote hearings from a detention area^{Note 183} or another room within the courthouse^{Note 184} rather than virtual hearings. In such cases, “some defendants have reported feeling disorientated, not being able to hear or understand the proceedings and lacking confidence in the fairness of the hearing.”^{Note 185} This has notably been observed in asylum cases in the United States where a study showed that “asylum seekers appearing remotely from detention are less likely to actively participate in their tribunal hearing and more likely to be deported.”^{Note 186} However, as one respondent to the survey pointed out, in the case of IRB hearings, this seems to be more of an issue with the available technology in certain prisons, and not with the “dehumanising aspect” of videoconferencing from a detention area.

That being said, while “defendants who appear remotely from police custody or jail are more likely to have a higher bail set, plead guilty and receive longer sentences than those who appear in person”,^{Note 187} and while “pre-pandemic experience with partly virtual hearings—for example, parole or asylum hearings in which the applicant appears via videoconference—suggests that witnesses and litigants will be evaluated more negatively in virtual courts than in in-person hearings or trials”,^{Note 188} other studies find no correlation between convictions (in criminal cases) or refusals (in refugee claims) and the type of hearing.^{Note 189} In fact, one such study found that “there was no statistically significant evidence that judges adjudicated deportation cases more harshly over a video screen”.^{Note 190}

In other words, while we agree that we should be weary of the potential dehumanizing effect of virtual hearings, we should not presume it is prevalent in all cases. As one observer suggests: “[t]he presence of all key participants in the trial on a screen just a few centimetres away for all those involved generated a sense of intimacy which promoted a sense of close engagement with the process.”^{Note 191} This “sense of closeness” can be enhanced by adopting clear guidelines regarding backgrounds for persons appearing before the IRB remotely, including, if possible, the use of virtual backgrounds. This will ensure a certain uniformity between participants.

Furthermore, when a person appearing before the IRB does so from a jail, detention area, or any other form of custody, effort should be put into creating an environment that is quiet, inviting, and accessible. This can be done using architecture (creating dedicated rooms)^{Note 192} , or with headsets that cancel background noise when such rooms are unavailable.

Recommendation:

• Work with jails, detention centers, IRBCC members, and other partners to try and offer quiet and accessible areas for taking part in virtual hearings.

8. Concerns regarding the impact of technical difficulties on a hearing’s outcome should be addressed

As the survey data shows, there remain important concerns regarding the impact of technical difficulties on proceedings. In fact, the literature review shows that technical difficulties are one of the main impediments to access to justice.^{Note 193} Although technical difficulties are not seen as having had an impact on proceedings according to the survey,^{Note 194} the IRB should remain vigilant regarding how these difficulties are perceived by participants. As we observed earlier, studies have shown that members who experience technical difficulties “may be inclined to misattribute their negative feelings […] to the witness himself and to evaluate him less favorably”.^{Note 195} In the same vein, persons appearing before the IRB may falsely attribute a negative decision to technical difficulties rather than the merit of their case. As an IRBCC member summarized: “The impact of the technical problems were identified as follows: wasted time, flow is interrupted, claimants feel powerless, which could impact their testimony, members seem frustrated which can impact their assessment of the claim, and access to justice in general for those claimants who lack technological skills or reliable internet.” While there is little the IRB can do to directly mitigate technical difficulties on the user’s end, it should continue offering information on technical requirements and updating this information as new versions of MS Teams are made available. For persons appearing before the IRB that do not have financial of physical access to the appropriate technology, it may be necessary to offer spaces and equipment either directly or through a third party. While this is already possible in certain circumstances, some IRBCC members have underlined the need for more resources in certain areas of the country.

Recommendation:

• Continue to offer and update documentation that will inform participants on the technical requirements associated with a virtual hearing, as well as how to use the technology.
• Allow a space and time for participants to test their equipment.
• Ensure that all participants have access to adequate Internet connections and devices.
• Create spaces or organise equipment rental or lending services directly or through a third party.

9. Concerns regarding security should be addressed

As expressed by most IRBCC members, security is obviously a major concern with any web-based videoconferencing platform,^{Note 196} and while MS Teams appears to be a more secure platform than many of its competitors,^{Note 197} it remains subject to a certain number of security issues.^{Note 198} One such issue is “Zoom bombing”, where third parties will log onto the platform to simply disrupt proceedings.^{Note 199} However, while Zoom bombing is irritating and can cause stress for the person appearing before the IRB, it won’t affect the confidentiality of data or affect attorney-client privilege, and disruptive parties can always be ejected from the hearing as they would in an in-person context.

As Ebner and Zeleznikow explain, there are three levels of security that must be addressed when using a platform such as MS Teams:

• Informational security (“which protects parties’ information from being shared by outsiders.”)
• Data security (“which includes protections around the communication channels, software, servers and any hardware used.”)
• System security (“the degree to which users feel confident that the service they are using – the technological platform or its human operators – is not utilising their information, participation, behaviour or data in any way.”)^{Note 200}

Regarding informational security, since Microsoft is known for investing massively in security, risks are well controlled. However, regarding data security, as one IRBCC member observes, there are important risks linked to the fact that the IRB does not check if participants are using a secure connection to log onto the hearing. Furthermore, as another IRBCC member explains: “Some claimants have also expressed concern about confidentiality, and the possibility of monitoring from their country of origin when hearings take place virtually. This is especially the case for claimants from countries where the internet is heavily monitored”.^{Note 201} This risk is increased by the fact that certain legislation, most notably the American CLOUD Act, allows the American government to request access to information stored on Microsoft servers, wherever they may be situated.^{Note 202} While Microsoft refuses to comply with these requests,^{Note 203} a risk remains that they may eventually be forced to do so. In this sense, while Microsoft offers the proper assurances, system security remains uncertain.

Recommendation:

• Continue to advise participants to install all security updates on their devices, and to use secure networks to log onto the hearing.
• Stay informed of privacy legislation and how the CLOUD Act is being enforced in Canada.

To conclude this section, the concerns regarding virtual hearings can best be summarized by this comment made by one IRBCC member: “the IRB should ensure that the technology used for virtual hearings permits clear, reliable, and uninterrupted audio and video transmission, as well as confidential and secure communication. This technology should be adequate to minimize the risks of having gaps in the communication and/or unrecoverable speech in the audio and/or video recording, where applicable. Further, the Board Member must ensure that the quality of the audio and video transmission is adequate throughout the hearing for all participants, should ask participants to signal any technological problems that arise during the hearing and seek to address them immediately.”

While some of these elements can be easily addressed, others remain outside the scope of possibility for different reasons, most notably the limitations of MS Teams (or any other videoconferencing platform).

Conclusion

As one IRBCC member suggests “The IRB should […] actively ensure that claimants are aware of the advantages and disadvantages of both in-person hearing and virtual hearings, so that claimants can make the best choice for themselves.” Of course, for them to be able to do so, the IRB must first itself know what these are. After all, while it is undeniable that “[t]he ability of technology to increase access to justice is profound”,^{Note 204} it can also be said, as was posited by one IRBCC member, that “safeguards must exist to ensure that procedural fairness is followed when using remote hearings.”

This report was a first step in better understanding how virtual hearings using MS Teams are perceived by different stakeholders. However, as we suggested in another publication, “[a]ny potential long-term solution must be the product of serious contemplation to ensure that the values essential to upholding our democratic society are not undermined.”^{Note 205} In this sense, more research will be needed to fully grasp the impact of virtual hearings, their unintended effects,^{Note 206} and whether the perceived advantages and inconveniences of the format hold true. As one author explains:

“The impact of videoconferencing on the justice process, however, is hard to assess. The impact of technology is often subtle. Videoconferencing is unlikely to produce measurable changes in the outcome of proceedings. Nevertheless, we cannot and should not assume that videoconferencing technology is entirely benign”^{Note 207}

As it stands, however, we can be satisfied that current data suggests that virtual hearings should not be abandoned post-pandemic. For example, according to the Canadian Council for Refugees, 78% of persons appearing before the IRB through virtual hearings that were surveyed claimed that they would pick that format again should they have to go through another hearing.^{Note 208} Of course, what this also implies is that 22% of respondents would rather go back to in-person hearings. This confirms that virtual hearings still need to be perfected and that there may be cases that cannot and should not be heard virtually.

We opened this report with a quote from the Arconti v. Smith case, so we feel it apt only to close with one as well: “technology is a tool, not an answer”.^{Note 209} We therefore must see how to best utilise this tool and not simply do so without understanding the consequences.

References

Laws

Clarifying Lawful Overseas Use of Data Act, H.R. 4943.

Immigration and Refugee Protection Act, SC 2001, c 27.

Caselaw

Arconti v. Smith, 2020 ONSC 2782 (CanLII).

Gomez Posada v. Canada (Citizenship and Immigration), 2007 FC 216 (CanLII).

Kaushal v. Vasudeva et al., 2021 ONSC 440.

Books and Papers

Appendices

Appendix 1 - Assessment guide

Pr. Nicolas Vermeys^{Note 210}

Valentin Callipel^{Note 211}

Table of Contents

1. Foreword
   

1. ​ Methodology
   
2. Index and indicators
3. Survey Metrics
4. Survey Questions (claimant.appellant
5. Survey Questions (representative)
6. Survey Questions (hearing officers)
7. References

Foreword

BVC Groupe Cyberjustice was mandated by the Immigration and Refugee Board of Canada (hereinafter: the “IRB”) to build an Assessment guide that proposes an index, indicators, as well as survey questions to measure access to justice when a person appearing before the IRB does so during a virtual hearing using Microsoft Teams. While this mandate appears to be relatively narrow, it raises certain difficulties pertaining to the concept of access to justice, a notion that is very much in the eye of the beholder and, therefore, difficult to measure using a binary approach.

Furthermore, while it is possible to establish whether or not individuals feel or believe that they have been granted sufficient access to justice when appearing via MS Teams, whether the quality of this access is equal, inferior or superior to in-person hearings is a separate question. As we have no baseline to compare the two forms of hearings, and as our mandate does not allow for the establishment of such a baseline, the proposed framework cannot offer any clear indication as to the relative quality of virtual hearings when compared to their in-person equivalent.

Methodology

As Access to justice remains a fluid concept that has come to be defined in different ways by researchers, our first task was to analyze and consider existing definitions^{Note 212} in order to select one that satisfies the IRB. Following a workshop held on May 26th, 2021, with IRB representatives (hereinafter: “the workshop”), we have come to understand that a narrower definition of access to justice should be used to structure our index, one relying on factual and measurable indicators. While this approach does have the advantage of offering an almost binary answer to potential questions, it lacks the capacity to evaluate whether persons appearing before the IRB perceive that they have been granted access to justice. For this reason, we chose to adopt the following middle ground definition proposed by Emily Bates, Jennifer Bond & David Wiseman. According to these authors, access to justice is composed of three elements:

1. [claimants/appellants] having the ability to invoke and effectively participate in justice processes (procedural access);
2. obtaining a fair result when they do (substantive access); and,
3. having their unique circumstances and needs recognized and respected by the justice system (inclusive access).^{Note 213}

The next step in developing our index was to analyze existing indexes and to establish which one could best serve our purpose, i.e.: measure access to justice through MS Teams hearings. To this end, the following indexes were considered:

• A strategic framework for access to justice in the federal civil justice system^{Note 214} ;
• National Legal Assistance Data Standards Manual^{Note 215} ;
• Recommended Access to Justice Indicators for Implementation of Goal 16 of the UN 2030 SDG^{Note 216} ;
• Measuring Online Legal Resources: A framework Inspired by the Drake Equation^{Note 217} ;
• Paths to Justice: A Past, Present and Future Roadmap^{Note 218} ;
• Assisted Digital Support for Civil Justice System Users: Demand, Design and Implementation^{Note 219} ;
• Measuring the costs and quality of Access to Justice^{Note 220} ;
• Access to Justice Well-being Index^{Note 221} ;
• Development of an Access to Justice Index for Federal Administrative Bodies^{Note 222} ;
• Global Insights on Access to Justice^{Note 223} ;
• Sense of Access to Justice^{Note 224} .

Most of these tools shared two main drawbacks:

1. They were developed mostly for civil matters and, therefore, are not adapted to the types of cases heard by the IRB since they mostly focus on the capacity to help parties settle a dispute. This is not to say that we cannot pull methodological cues from these tools, but applying them to IRB processes would generate incoherent answers since many proposed questions (for example, questions regarding choice of dispute resolution mechanism) are simply unapplicable to administrative matters;
2. They were mostly self-assessment tools rather than indexes to measure the satisfaction of claimants or appellants. In this sense, they relate how a given court or tribunal evaluates the quality of its own processes, and not how the main stakeholders (claimants or appellants) perceive the quality of access to justice they received. This is not to say that self-assessment tools are not pertinent or useful, but, as stated above, access to justice is in the eye of the beholder. This is why it is important to measure how those affected by IRB decisions feel about the process.

For these reasons, we chose to use, as a starting off point, the “Sense of Access to Justice” index, as it was the only proposed index to address the question of access to justice from the claimant/appellant’s point of view and to be flexible enough to be adapted to IRB hearings. The Sense of Access to Justice index was adapted and only the actual questions that were assessed are included in the survey guides found in sections 4, 5, and 6 of this appendix.

Index and Indicators

The original sense of Access to Justice index measures a claimant/appellant’s feeling regarding the fairness of the process, and its efficiency. In its original form, it is composed of twelve (12) factors separated into four (4) categories and given equal weight. Each factor is evaluated individually and given a score between 1 and 6. The factors are represented in the following table:

Given the particular context of our mandate, we have chosen to rework and refine these factors so that they would better reflect the answers that external stakeholders require. In this sense, most questions relating to the process rather than the technology were set aside. This leaves us with seven (7) rather than twelve (12) questions.

While this approach offers a more succinct and direct tool to measure how using MS Teams for hearings affects access to justice, it does not allow for a broader appreciation of IRB processes. Only those questions that were actually used in the IRB access to justice survey are listed below in sections 4-6.

Finally, as the survey can be filled out by individual claimants/appellants, their representatives, or CBSA hearing officers, we believed it to be preferable to draft questions differently depending on the respondent. To this end, three different surveys were used, although they contain similar questions.

The surveys used for this assessment contain three distinct sections. Section 1 contains contextual questions aimed at better understanding the background of the individuals filling out the survey. This information will then allow researchers to generate statistics according to certain indicators such as technological sophistication prior to the hearing. These questions are not necessary to measure the sense of access to justice. They only serve to better inform the IRB on the background of respondents.

Section 2 contains the questions that will allow us to measure the sense of access to justice. Each question is crafted to measure one of the seven factors described in section 2 of this Guide, which, based on the scientific research used to build this chosen model, were established as being key to measuring an individual’s sense of access to justice. To this end, this section generates essential data to feed our model.

Section 3 is simply added to allow the IRB to gather general comments. This section is not necessary to feed our index.

Survey Metrics

An individual’s sense of access to justice will be measured by adding the value attributed to each question referenced in section 3 (Your experience). Every question will be granted equal weight (n = 1) and be scored by the respondent according to a scale from 1 to 5. The score given for every question will then be added and transferred to a percentage. For example, should the IRB decide to limit itself to the suggested 7 questions, and a respondent answers each question as follows:

• Q1 = 4/5
• Q2 = 5/5
• Q3 = 5/5
• Q4 = 3/5
• Q5 = 4/5
• Q6 = 5/5
• Q7 = 4/5

The total would equal 30/35, which gives us a sense of access to justice of 85.7%. If a respondent refuses to answer one of the questions or selects “not applicable”, that question will be eliminated from the final calculation.

As with any test or survey, there is no pre-established passing grade. In other words, the hope is to attain as high a percentage as possible. While it is custom to establish that a passing grade is situated at 60%, the goal of the IRB should not be to reach a given number, but rather to build on the initial results to tweak how virtual hearings are conducted in order to obtain constantly higher scores during future evaluations. The final percentage could also be compared to an eventual access to justice score for in-person hearings to establish whether virtual hearings are perceived as granting better, equivalent or worse access to justice as their in-person equivalents.

Survey questions (claimant/appellant)

As stated above, only those questions that were actually used in the surveys are listed below.

Section 1 - Context

Section 4 - Comments

Section 2 - Your experience

Setion 3 - Comments

Survey questions (CBSA hearing officers)

Section 1 - Context

Section 2 - Your experience

Section 3 - Comments

References

1. Australian Government Attorney-General's Department “A strategic framework for access to justice in the federal civil justice system”, (2009) Access to Justice Taskforce, online: https://www.ag.gov.au/LegalSystem/Documents/A%20Strategic%20Framework%20for%20Access%20to%20Justice%20in%20the%20 Federal%20Civil%20Justice%20System.pdf.
2. Australian Government Attorney-General's Department, “National Legal Assistance Data Standards Manual” (2015), online: https://www.ag.gov.au/LegalSystem/Legalaidprogrammes/Pages/National-Legal-Assistance-Data-Standards.aspx.
3. Bates, E., J. Bond & D. Wiseman, “Troubling Signs: Mapping Access to Justice in Canada’s Refugee System Reform”, (2016) 47(1) Ottawa Law Review 39.
4. Centre for Access to Justice at Fordham Law School, “Recommended Access to Justice Indicators for Implementation of Goal 16 of the UN 2030 Sustainable Development Agenda in the United States” (2016) National Centre for Access to Justice at Fordham Law School, online: http://ncforaj.org/wp-content/uploads/2016/12/NCAJ-CHRI-9-15-16-Recommended-AtJ-National-Indicators-12-1-16- final.pdf.
5. Denvir, C., “Assisted Digital Support for Civil Justice System Users: Demand, Design and Implementation” (2018) U.K. Courts and Tribunals Judiciary, online: https://www.judiciary.uk/wp-content/uploads/2018/06/cjc-report-on-assisted-digital-support.pdf.
6. Gramatikov, M. et al., “A Handbook for Measuring the Costs and Quality of Access to Justice”, (2010) Law for life, online: http://lawforlife.org.uk/wp-content/uploads/2013/05/handbook-for-measuring-the-costs-and-quality-of-access-to-justice-271.pdf.
7. Gramatikov, M.A. & M. Laxminarayan, Weighting Justice: Constructing an Index of Access to Justice (2009), Tilburg University Legal Studies Working Paper No. 18/2008, online: https://ssrn.com/abstract=1344418 or http://dx.doi.org/10.2139/ssrn.1344418.
8. McDonald, S., “Development of an Access to Justice Index for Federal Administrative Bodies” (2017) Department of Justice Canada, online: https://www.justice.gc.ca/eng/rp-pr/jr/fab-eaf/p3.html.
9. Norman, D., Things That Make Us Smart: De-fending Human Attributes in the Age of the Machine, (Boston:Addison-Wesley Longman, 1993).
10. Pleasence, P., N.J. Balmer & R.L. Sandefur, “Paths to Justice: A Past, Present and Future Roadmap”, (2013) Nuffield Foundation, online: http://www.nuffieldfoundation.org/sites/default/files/files/PTJ%20Roadmap%20NUFFIELD%20Published.pdf.
11. Quinn, L., and J. Raby, “Measuring Online Legal Resources: A Framework Inspired by the Drake Equation”, (2018) Florida Legal Technology Center, online: https://www.srln.org/system/files/attachments/Drake-Equation-for-Online-Legal-v7.pdf.
12. Roberge, J-F, “"Sense of Access to Justice" as a Framework for Civil Procedure Justice Reform: an Empirical Assessment of Judicial Settlement Conferences in Quebec (Canada)”, (2015) 17:23 Cardozo J. Of Conflict Resolution 323.
13. Roberge, J-F, “Le sentiment de justice. Un concept pertinent pour évaluer la qualité du règlement des différends en ligne ?”, (2020) 1 Revue juridique de la Sorbonne 5.
14. Roberge, J-F, La justice participative, Cowansville, Éditions Yvon Blais, 2011.
15. Roberts, T. & Associates Consulting, “Access to Justice Well-being Index: Final report and Index” (2018) Department of Justice Canada, Evaluation Division. (not as yet published).
16. Sossin, L., “Chapter 7 : Access to administrative justice and other worries”, in C. Flood & L. Sossin, eds, Administrative Law in Context, 2nd ed., Toronto, Emond Montgomery Publishing, 2013.
17. Winner, L. The Whale and the Reactor, 2nd ed., Chicago, University of Chicago Press, 2020.
18. World Justice Project, “Global Insights on Access to Justice” (2019) online : https://worldjusticeproject.org/sites/default/files/documents/WJP-A2J-2019.pdf.

Appendix 2 - Immigration and Refugee Board of Canada Consultative Committee Questionnaire

Pr. Nicolas Vermeys^{Note 225}
Valentin Callipel^{Note 226}

Table of Contents

1. Foreword

1. Context
2. Questionnaire

Foreword

On behalf of the IRB, as part of fulfilling its mandate to assess the impact of access to justice on virtual hearings, BVC Cyberjustice Group would like to engage the Immigration and Refugee Board of Canada Consultative Committee (IRBCC) members on their organizations’ perspectives on the use of Microsoft Teams to conduct hearings.

IRBCC members are asked to answer the following questions in writing as representatives of their respective organizations and not based on individual experiences in a hearing. We ask that you please provide one response per IRBCC member organization. Please note that the answers to this questionnaire will be treated confidentially, and while certain answers may find their way into our final report, they will not be attributed. Your feedback will help in the review of the IRB’s virtual hearing model and inform the development of best practices.

Please send responses to the questionnaire to nicolas.vermeys@groupecyberjustice.com by October 5, 2021.

Context

In the context of BVC Cyberjustice Group’s previously stated mandate, persons appearing in front of the Immigration and Refugee Board of Canada, their representatives, as well as representatives from the Canada Border Security Agency were asked to fill out an access to justice survey regarding their experience in taking part in a virtual hearing using Microsoft Teams (hereinafter: “The Access to Justice Survey”). The survey contains a series of questions pertaining to, among other things:

• The object of the hearing
• Whether the outcome was in their favour
• Their satisfaction with the level of information and support received
• Whether the technology allowed them to fully engage in the hearing
• Whether they felt they were treated with respect during the hearing
• Whether they experienced technological issues
• Whether the hearing helped them save time and effort
• Whether the technology had an impact to the stress level associated with a hearing
• Whether their overall experience was positive

While answers are still being collected, a preliminary analysis of the data we have already received allows us to draw certain initial conclusions as to the perceptions that individuals appearing before the IRB or representatives hold with regards to hearings held using Microsolf TeaTeams. These are the conclusions on which we wish to obtain IRBCC members’ observations.

Questionnaire

For the purpose of this questionnaire, a virtual hearing is defined as “a hearing where participants use the Microsoft Teams application to connect to the hearing using their own devices from a private place like their home or counsel’s office”.

1. Preliminary findings from the Access to Justice Survey show that a large majority of respondents have indicated that virtual hearings save them time, effort, as well as money. These findings also show that most respondents believe that virtual hearings have a positive effect on the stress level associated with the hearing process. What is your organization’s view of the advantages of virtual hearings from an access to justice perspective and do they reflect those listed above?
2. While most respondents to the Access to Justice Survey reported no significant technology issues, some did state that they experienced technical difficulties during their hearing (dropped connection, frozen images, synchronization issues between sound and image, etc.). While this does not seem to have affected most respondents’ overall level of satisfaction with virtual hearings, some have raised concerns that these technical difficulties could have an impact on the process and its outcome. What is your organization’s view of the impact that technical difficulties can have on the process and outcome of virtual hearings and how should the IRB go about mitigating the underlying risks?
3. A few respondents to the Access to Justice Survey raised concerns regarding the risk that witnesses appearing remotely can be coached by someone standing outside of the camera’s view or can otherwise access information while they are testifying. What is your organization’s position on the validity of these concerns and, should you consider them valid, how could this issue be addressed?
4. Some respondents to the Access to Justice Survey have raised concerns regarding the fact that non-verbal cues cannot be properly taken into account when using Microsoft Teams. What is your organization’s view on the validity of these concerns and how could this issue be addressed?
5. Preliminary findings from the Access to Justice Survey do not allow us to establish any correlation between the type of hearing (e.g.: admissibility hearing, refugee appeal, Immigration appeal, etc.) and the respondents’ sense of access to justice. Has your organisation come to the same conclusion or are there certain types of hearings or situations in which you have established should not be held virtually?
6. Although not identified in the preliminary findings from the Access to Justice Survey, other literature on the topic seems to suggest that there could be some correlation between computer literacy and one’s sense of access to justice regarding virtual hearings. What is your organisation’s position on how this could be addressed? Should those who lack computer literacy skills be given access to certain resources, and if so, would your organization have any to recommend?
7. On top of the items addressed in the previous questions, respondents to the Access to Justice Survey also pointed to issues with interpretation, the ability for counsel to confer with their clients, and participants talking over one another. What are the issues your organization has identified with regards to virtual hearings and do they correspond to the ones mentioned in this questionnaire? Finally, how have you chosen to address these issues?
8. Overall, while the majority of respondents seem favourable to maintaining virtual hearings post-pandemic, as pointed out in previous questions, some concerns were raised with the process and, therefore, a few respondents favour maintaining in-person hearings as an option. What is your organization’s overall view on maintaining virtual hearings at the IRB?
9. Do you have any other comments or suggestions as to how to improve the quality of virtual hearings?

We thank you for taking the time to complete this questionnaire. Should you have any further comments or suggestions, please feel free to contact BVC Groupe Cyberjustice. Any information received will be treated confidentially.

Notes

Pr. Vermeys is a Director at BVC Groupe Cyberjustice. He is also a full Professor, and the Associate Dean of Programs at the University of Montreal’s Faculty of Law and the Associate Director of the Cyberjustice​ Laboratory.

Return to note 1 referrer

Mr. Callipel is a Director of BVC Groupe Cyberjustice, and the Head of mission at the Cyberjustice Laboratory.

Return to note 2 referrer

The term “representative” is used in this report to encompass lawyers, consultants, as well as other representatives such as a family member, a friend, or a volunteer.

Return to note 3 referrer

See e.g., SOSSIN 2007.

Return to note 4 referrer

See e.g., SALYZYN, 2012.

Return to note 5 referrer

OHIO 2021, p. 21-22; SALYZYN, 2012, p. 443; and LEDERER 1999, p. 830.

Return to note 6 referrer

ARCONTI V. SMITH, 2020 ONSC 2782 (CanLII), par. 19.

Return to note 7 referrer

CALIFORNIA 2021, p. 7.

Return to note 8 referrer

Borderlines 2021.

Return to note 9 referrer

See BATES 2016.

Return to note 10 referrer

This definition is derived from BATES 2016.

Return to note 11 referrer

ELLIS 2004.

Return to note 12 referrer

OECD 2020.

Return to note 13 referrer

See ROBERGE 2015; and ROBERGE 2020.

Return to note 14 referrer

ROBERGE 2015, p. 349.

Return to note 15 referrer

IRB Website 2021.

Return to note 16 referrer

IRB-RPD PowerPoint 2021.

Return to note 17 referrer

LOCKE 2009, p. 36; and HAAS, 2006, p. 87.

Return to note 18 referrer

ROWDEN 2013, p. 7.

Return to note 19 referrer

See e.g., UNITAR 2020, p. 8. One of the main elements that is covered by the literature is how virtual trials could impact public access to hearings. See e.g., SALYZYN, 2020. However, since many IRB hearings are not open to the public and since a protocol was put into place for members of the public to “observe a public hearing held virtually before the Immigration Division (ID) or the Immigration Appeal Division (IAD), the question is moot.

Return to note 20 referrer

See e.g., LEDERER 1999.

Return to note 21 referrer

As explained by Amy Salyzyn: “much of the past research, as Rossner and Tait note, focusses on hearings where “the judge, prosecutor and often defence counsel, as well as court staff and members of the public, are all grouped together in the actual courtroom, while the defendant is alone on a screen”, resulting in an inherent imbalance that does not exist if everyone is appearing remotely in a virtual hearing”. See SALYZYN, 2020.

Return to note 22 referrer

Immigration and Refugee Protection Act, SC 2001, c 27, section 164: “Where a hearing is held by a Division, it may, in the Division’s discretion, be conducted in the presence of, or by a means of live telecommunication with, the person who is the subject of the proceedings.” An in-depth analysis of the legal arguments permitting the use of videoconferencing for IRB hearings can be found in ELLIS 2004.

Return to note 23 referrer

See VERMEYS 2019, p. 238.

Return to note 24 referrer

The Canadian Council for Refugees arrived at the same conclusion. See CCR 2021, p. 11.

Return to note 25 referrer

See e.g., ROWDEN 2013, p. 32.

Return to note 26 referrer

We should however specify that data collected in Arizona demonstrates that most attorneys don’t consider that the use of videoconferencing impacts how they prepare or present oral arguments. See ARIZONA 2021, pp. 17-18.

Return to note 27 referrer

The Canadian Council for Refugees arrives at the same conclusion. See CCR 2021, p. 10.

Return to note 28 referrer

“It is expected that counsel and their clients will normally communicate with each other by phone during hearing breaks if they are not located together. The Microsoft Teams software indicates when it is recording, and parties should confirm that the Member has stopped the recording during any break when they are having a private conversation. Where a participant is accessing their hearing from an IRB office, the microphone connected to the computer has an off switch that they can press to ensure that their conversation during a break is private.” See IRB Website 2021.

Return to note 29 referrer

TAIT 2021, p. 106.

Return to note 30 referrer

See HAAS, 2006, p. 86.

Return to note 31 referrer

See e.g., TAIT 2021, p. 97.

Return to note 32 referrer

EAGLY 2015, pp. 982-983, “social scientists demonstrated that participants’ perceptions of fairness are influenced by factors such as respectful treatment by the judge and the opportunity to speak in court.”

Return to note 33 referrer

CCR 2021, p. 11.

Return to note 34 referrer

OHIO 2021, p. 16.

Return to note 35 referrer

See SALYZYN, 2020; SOURDIN 2020; CCR 2021, p. 9; and HAAS, 2006, p. 77. This issue was also echoed by most IRBCC members.

Return to note 36 referrer

It should be mentioned that these statistics are higher than those reported by other surveys. For example, a survey out of Arizona claims that “about 60 percent had experienced technical disruptions frequently or occasionally”, while one relating to experience of the Chicago Immigration Court stated that “nearly 45% of the observed cases had one or more problems”. However, as these surveys related to remote rather than virtual hearings, the potential for technical issues was much lower, which can explain the statistical differences. See ARIZONA 2021, p. 16; and CHICAGO 2005, p. 6.

Return to note 37 referrer

Studies conducted in the UK came to a similar conclusion. See MCCURDY 2020, p. 29.

Return to note 38 referrer

“When such technical difficulties occurred, more than 80 percent of respondents said that it took no more than several minutes to resolve the problem, and the proceeding then resumed.” See ARIZONA 2021, p. 16. CHICAGO 2005, p. 37.

Return to note 39 referrer

CHICAGO 2005, p. 37.

Return to note 40 referrer

In this case, we cannot compare our data to that of the Chicago Immigration Court as they “did not have enough data to make a full assessment of the relationship between interpretation problems and removal orders.” CHICAGO 2005, p. 43.

Return to note 41 referrer

OHIO 2021, p. 21-22; ROTH 2000, p. 190; and MCCURDY 2020, pp. 2 and 33. See also Arconti v. Smith, 2020 ONSC 2782, par. 19.

Return to note 42 referrer

T. JOHNSON 2006, pp. 211-212; OHIO 2021, p. 21-22; SOSSIN 2007, p. 258; and ROTH 2000, p. 190. See also Arconti v. Smith, 2020 ONSC 2782, par. 19.

Return to note 43 referrer

ROSSNER 2021, p. 12.

Return to note 44 referrer

TAIT 2021, p. 106: “Several participants in the UK video hearings pilot indicated that their hearing felt less formal from (how they imagined) a physical court, but that this helped to reduce their anxiety and stress.” See also MCCURDY 2020, p. 36. MULCAHY 2020, p. 4: “Lay participants appeared to be comfortable with the technology once trained and there were suggestions that in some instances it caused less stress than going to a physical courthouse”. See also CCR 2021, p. 12. BANDES 2020, p. 1297: “Witnesses who might be intimidated by the formality of the courtroom or the physically co-present judge may testify more confidently and coherently from the comfort of their homes.” We should underline that one IRBCC member views things quite differently: “First, for some vulnerable refugee claimants, the ability to sit in an informal setting with the board member is important and needs to be maintained. A casual setting is not reproduced in a virtual environment, as there is a level of formality to that environment still. Additionally, there can be technical difficulties compounding the stress for those involved.”

Return to note 45 referrer

ELLIS 2004. It should be noted, however, that this report addressed remote hearings rather than virtual hearings.

Return to note 46 referrer

The OECD has made similar observations: “some evidence suggests that participants involved in video conference proceedings face increased isolation and emotional distress”. See OECD 2020.

Return to note 47 referrer

CCR 2021, p. 8.

Return to note 48 referrer

CCR 2021, p. 8.

Return to note 49 referrer

ELLIS 2004.

Return to note 50 referrer

ELLIS 2004.

Return to note 51 referrer

These numbers seem to be higher than those recorded in other jurisdictions for courts (82% for legal representatives and 86% for parties), which would suggest that IRB hearings are especially well adapted to virtual environments. See e.g., OHIO 2021, pp. 15 and 18.

Return to note 52 referrer

ARIZONA 2021, p. 3: “Allowing parties to appear through virtual platforms has significantly increased appearance rates. This practice should continue, where appropriate, post- pandemic.”; CALIFORNIA 2021, p. 2: “California courts should expand and maximize remote access on a permanent basis for most proceedings and should not default to pre-pandemic levels of in-person operations.”; OHIO 2021, p. 16: “Clear majorities of responders in all groups indicated that courts should continue conducting remote proceedings in the future”, which leads the comity to recommend that courts “continue the use of remote technology to conduct court proceedings” following the pandemic (p. 22).

Return to note 53 referrer

HOPKINS 2020, p. 13.

Return to note 54 referrer

CCR 2021, p. 15.

Return to note 55 referrer

BENYEKHLEF 2020.

Return to note 56 referrer

IRB Website 2021.

Return to note 57 referrer

OECD 2020.

Return to note 58 referrer

ONTARIO 2021, p. 7.

Return to note 59 referrer

See e.g., IRB Website 2021.

Return to note 60 referrer

ONTARIO 2021, p. 8; CALIFORNIA 2020, p. 14; CCR 2021, p. 14; OECD 2020; and MCCURDY 2020, p. 22.

Return to note 61 referrer

CCR 2021, p. 6.

Return to note 62 referrer

ONTARIO 2021, p. 27.

Return to note 63 referrer

HOPKINS 2020, p. 20.

Return to note 64 referrer

ONTARIO 2021, p. 8.

Return to note 65 referrer

HOPKINS 2020, p. 20.

Return to note 66 referrer

CCR 2021, p. 14.

Return to note 67 referrer

TAIT 2021, p. 97.

Return to note 68 referrer

HOPKINS 2020, p. 20.

Return to note 69 referrer

CCR 2021, p. 16.

Return to note 70 referrer

LEDERER 2021, p. 333. See also FEDERMAN 2006, p. 441.

Return to note 71 referrer

This is a practice used in other jurisdictions as well. See ONTARIO 2021, p. 10.

Return to note 72 referrer

ONTARIO 2021, p. 10.

Return to note 73 referrer

ONTARIO 2021, p. 28.

Return to note 74 referrer

ONTARIO 2021, p. 10.

Return to note 75 referrer

ONTARIO 2021, p. 10.

Return to note 76 referrer

ONTARIO 2021, p. 10.

Return to note 77 referrer

ONTARIO 2021, p. 10.

Return to note 78 referrer

CHICAGO 2005, p. 57.

Return to note 79 referrer

TAIT 2017, p. 20.

Return to note 80 referrer

ONTARIO 2021, p. 9; and HOPKINS 2020, p. 21.

Return to note 81 referrer

ROWDEN 2013, p. 30.

Return to note 82 referrer

LEDERER 2021, p. 333. See also LOCKE 2009, p. 45: “Researchers have hypothesized that shooting an already dominant subject from a high camera angle increases that subject’s credibility because it brings that subject down to the audience’s level, whereas a low camera angle separates the subject from the audience even further.”

Return to note 83 referrer

CALIFORNIA 2020, p. 15; HOPKINS 2020, p. 24; and ONTARIO 2021, p. 23.

Return to note 84 referrer

ROSSNER 2021, p. 15; TAIT 2017, p. 20; B. POULIN 2004, p. 1125; HAAS, 2006, p. 61; and ROWDEN 2013, p. 9.

Return to note 85 referrer

BANDES 2020, p. 1294. See also HAAS, 2006, p. 76.

Return to note 86 referrer

LEDERER 2021, p. 333; CALIFORNIA 2020, p. 15; BANDES 2020, p. 1294; and B. POULIN 2004, p. 1122.

Return to note 87 referrer

MCCURDY 2020, p. 35.

Return to note 88 referrer

BANDES 2020, p. 1297; and B. POULIN 2004, p. 1137.

Return to note 89 referrer

BAILENSON 2021; and BANDES 2020, pp. 1301-1302.

Return to note 90 referrer

ONTARIO 2021, p. 9.

Return to note 91 referrer

LOCKE 2009, p. 46: “A camera shooting straight at the subject generally seems to have a neutral effect on audience perception.”

Return to note 92 referrer

TAIT 2017, p. 20.

Return to note 93 referrer

LOCKE 2009, p. 53.

Return to note 94 referrer

HOPKINS 2020, p. 24.

Return to note 95 referrer

ONTARIO 2021, pp. 16 and 23. See also MCCURDY 2020, p. 25.

Return to note 96 referrer

BANDES 2020, p. 1302: “faces that appear too close may seem to be occupying observers’ personal space, which observers may regard as inappropriate and annoying or even threatening. Conversely, witnesses whose faces appear much smaller than others may be granted less importance.” See also ROTH 2000, p. 203.

Return to note 97 referrer

BANDES 2020, pp. 1302-1303.

Return to note 98 referrer

TAIT 2021, p. 97.

Return to note 99 referrer

ONTARIO 2021, p. 23.

Return to note 100 referrer

MULCAHY 2020, p. 26; and ONTARIO 2021, p. 23.

Return to note 101 referrer

ONTARIO 2021, p. 23.

Return to note 102 referrer

CALIFORNIA 2020, p. 14; ONTARIO 2021, p. 23; and HOPKINS 2020, p. 24.

Return to note 103 referrer

CALIFORNIA 2020, p. 14; and ONTARIO 2021, p. 23.

Return to note 104 referrer

LEDERER 2021, p. 333.

Return to note 105 referrer

ONTARIO 2021, p. 23.

Return to note 106 referrer

BANDES 2020, p. 1302.

Return to note 107 referrer

HOPKINS 2020, p. 21; and ONTARIO 2021, p. 10.

Return to note 108 referrer

HOPKINS 2020, p. 21; and ONTARIO 2021, p. 10.

Return to note 109 referrer

MULCAHY 2020, p. 26.

Return to note 110 referrer

CCR 2021, p. 9.

Return to note 111 referrer

OECD 2020; ARIZONA 2021, p. 20; CALIFORNIA 2021, p. 7; CALIFORNIA 2020, p. 4; and PUDDISTER 2020.

Return to note 112 referrer

See also CCR 2021, p. 7.

Return to note 113 referrer

For a similar recommendation in other jurisdictions, see ARIZONA 2021, p. 20.

Return to note 114 referrer

CCR 2021, p. 10. This position is shared by at least one IRBCC member: “Participants stated the need to have the interpreter’s camera on and them not just be on the phone.”

Return to note 115 referrer

LICOPPE 2015, p. 127.

Return to note 116 referrer

CHICAGO 2005, p. 54.

Return to note 117 referrer

TAIT 2021, p. 104. See also MCCURDY 2020, p. 24; and ROWDEN 2013, p. 25.

Return to note 118 referrer

IRB Website 2021.

Return to note 119 referrer

MULCAHY 2020, p. 30. See also BANDES 2020, p. 1319.

Return to note 120 referrer

TAIT 2021, p. 107.

Return to note 121 referrer

TAIT 2021, p. 105.

Return to note 122 referrer

BANDES 2020, p. 1319. See also SOSSIN 2007, pp. 255 and 262.

Return to note 123 referrer

BANDES 2020, p. 1316.

Return to note 124 referrer

TAIT 2021, p. 106.

Return to note 125 referrer

Some commenters “advocated for the use of two cameras for witnesses: the computer camera and then another video camera positioned to the side and further away from the witness that showed the entire room including the witness and computer screen.” See HOPKINS 2020, p. 24. See also ROWDEN 2013, p. 11.

Return to note 126 referrer

SALYZYN, 2012, pp. 455 and 458.

Return to note 127 referrer

ONTARIO 2021, p. 21; and ROWDEN 2013, p. 54.

Return to note 128 referrer

CALIFORNIA 2020, p. 14.

Return to note 129 referrer

LEDERER 2021, p. 335. See also BANDES 2020, p. 1325.

Return to note 130 referrer

IRB PowerPoint 2021.

Return to note 131 referrer

BANDES 2020, p. 1325.

Return to note 132 referrer

HOPKINS 2020, p. 25.

Return to note 133 referrer

BANDES 2020, pp. 1301-1302.

Return to note 134 referrer

https://support.microsoft.com/en-us/office/using-video-in-microsoft-teams-3647fc29-7b92-4c26-8c2d-8a596904cdae#bkmk_videolayout

Return to note 135 referrer

MCCURDY 2020, p. 35: “Two users noted that the judge was at the top of their screen, and another thought that the judge had a bigger frame than the other parties. Several users also commented on the formality of the judge’s video frame.”

Return to note 136 referrer

LEDERER 2021, p. 336.

Return to note 137 referrer

TAIT 2021, p. 102. See also TAIT 2018; and MCCURDY 2020, p. 4.

Return to note 138 referrer

MULCAHY 2020, p. 29.

Return to note 139 referrer

TAIT 2021, p. 103.

Return to note 140 referrer

HOPKINS 2020, p. 25.

Return to note 141 referrer

ONTARIO 2021, pp. 16 and 21.

Return to note 142 referrer

ONTARIO 2021, pp. 21 and 23; MULCAHY 2020, p. 26; and CALIFORNIA 2020, p. 14.

Return to note 143 referrer

BANDES 2020, p. 1303.

Return to note 144 referrer

BANDES 2020, p. 1325. See also: LEDERER 2021, p. 334; and B. POULIN 2004, p. 1108.

Return to note 145 referrer

TAIT 2021, p. 101; and ONTARIO 2021, p. 12.

Return to note 146 referrer

IRB Website 2021.

Return to note 147 referrer

ONTARIO 2021, p. 19.

Return to note 148 referrer

ONTARIO 2021, p. 19.

Return to note 149 referrer

CALIFORNIA 2020, pp. 14-15.

Return to note 150 referrer

IRB PowerPoint 2021.

Return to note 151 referrer

VERMEYS 2010, p. 127.

Return to note 152 referrer

VERMEYS 2010, p. 25.

Return to note 153 referrer

MCCURDY 2020, p. 42.

Return to note 154 referrer

See also OECD 2020: “Moving some processes towards the online sphere when shaping the ‘new normal’ of justice systems in the medium term could improve accessibility of justice, in particular for vulnerable groups”.

Return to note 155 referrer

CCR 2021, p. 18.

Return to note 156 referrer

For example, in Gomez Posada v. Canada (Citizenship and Immigration), 2007 FC 216 (CanLII), the court came to the conclusion that the Board needed to make “every effort to see that the redetermination hearing is not held by video conference” as different criteria such as the person appearing before the IRB’s “nervousness, her age, the dynamic of the hearing, the presence of the interpreter and the frequent interventions by both the presiding member and the interpreter” justified such a finding.

Return to note 157 referrer

ELLIS 2004.

Return to note 158 referrer

As one IRBCC member explains, in person hearings should be preferred for: Claimants with Vulnerable Persons status before the IRB; Unaccompanied and separated minors; Gender-related persecution asylum claims; SOGIE asylum claims; Individuals with disabilities that would make remote participation more difficult, such as visual and hearing impairments, cognitive impairments and mental health illnesses.

Return to note 159 referrer

In one occurrence, the representative of the person seeking refugee status had such bad audio (which seemed to be linked to the equipment and not the connection) that the commissioner was forced to put an end to their pleadings. As the decision was favorable to the person seeking refugee status, this did not impact the outcome of the hearing, but does demonstrate the possible consequences of using bad equipment.

Return to note 160 referrer

This recommendation is also one made by others who are studying the impact of virtual hearings. See e.g., OHIO 2021, p. 12 “[courts should] collect and analyze data to determine whether remote proceedings result in disparities or unintended consequences”.

Return to note 161 referrer

This finding is shared by the Canadian Council for Refugees, which had similar difficulties getting access to respondents that had received a negative decision. See CCR 2021, p. 5.

Return to note 162 referrer

As we observed previously, the data we collected shows that while representatives were generally quite critical of the use of videoconferencing according to a 2004 study (see ELLIS 2004), they now appear to be much more enthusiastic about its use.

Return to note 163 referrer

See also CCR 2021, p. 20.

Return to note 164 referrer

CCR 2021, p. 3: “In addition, many claimants had been waiting for a long time for a refugee hearing, as a result of the accumulated backlog at the IRB, exacerbated by the cancellation of hearings at the beginning of the pandemic. As a result, many are anxious to have a hearing whatever the format, and welcome a virtual hearing because it is more immediately available.”

Return to note 165 referrer

BENYEKHLEF 2005, p. 6.

Return to note 166 referrer

OECD 2020.

Return to note 167 referrer

BENYEKHLEF 2005, p. 8.

Return to note 168 referrer

BENYEKHLEF 2010.

Return to note 169 referrer

CCR 2021, p. 17.

Return to note 170 referrer

ELLIS 2004.

Return to note 171 referrer

2021 ONSC 440.

Return to note 172 referrer

As one IRBCC member stated, “Overall, I do not think this is a great concern, most participants aren’t capable of pulling something like that off.”

Return to note 173 referrer

As one IRBCC member stated, “If a claimant is appearing with a lawyer licensed by the Law Society, it would be contrary to ethics and law society rules to encourage their client to lie or permit someone to encourage the client to lie. Therefore, this is not a concern in such hearings.”

Return to note 174 referrer

BANDES 2020, p. 1281; ROWDEN 2013, p. 59; HAAS, 2006, p. 70; A SALYZYN, 2012, p. 445; FEDERMAN 2006, p. 444; B. POULIN 2004, p. 1110; A HAAS, 2006, p. 72; SOSSIN 2007, p. 260; and WALSH 2008, p. 272.

Return to note 175 referrer

See e.g., SOURDIN 2020; SALYZYN, 2020; and B. POULIN 2004, p. 1115.

Return to note 176 referrer

BANDES 2020, p. 1284; ROTH 2000, p. 208; and SALYZYN, 2012, p. 452.

Return to note 177 referrer

BANDES 2020, p. 1328. See also ROWDEN 2013, p. 32.

Return to note 178 referrer

BANDES 2020, p. 1298.

Return to note 179 referrer

SOSSIN 2007, p. 258.

Return to note 180 referrer

A quote collected by S. Ronald Ellis offers a similar observation: “Finally, I think that hearings by video-conference may, to a degree, help the Board move away from troublesome 'demeanour' credibility findings in some cases, (see, for example, Valtchev v. Canada (Minister of Citizenship and Immigration) 2001 FCT 776) as a claimant's demeanour in delivering testimony is slightly more remote in doing hearings by video-conference, forcing the Board to focus on examining the substance of testimony, where it properly should in my opinion.” See S ELLIS 2004.

Return to note 181 referrer

SALYZYN, 2020. See also SALYZYN, 2012, p. 447; and WALSH 2008, p. 269.

Return to note 182 referrer

See e.g., EAGLY 2015.

Return to note 183 referrer

See e.g., ROWDEN 2013.

Return to note 184 referrer

ROSSNER 2020.

Return to note 185 referrer

ROSSNER 2020. See also SALYZYN, 2012, p. 447.

Return to note 186 referrer

ROSSNER 2020.

Return to note 187 referrer

BANDES 2020, p. 1292. See also B. POULIN 2004, p. 1118.

Return to note 188 referrer

ROSSNER 2020; and EAGLY 2015, p. 976.

Return to note 189 referrer

EAGLY 2015, p. 937.

Return to note 190 referrer

MULCAHY 2020, p. 20.

Return to note 191 referrer

Best practices regarding the design of such rooms can be found in ROWDEN, 2013.

Return to note 192 referrer

See SALYZYN, 2020; SOURDIN 2020; CCR 2021, p. 9; and HAAS, 2006, p. 77. This issue was also echoed by most IRBCC members.

Return to note 193 referrer

Studies conducted in the UK came to a similar conclusion. See MCCURDY 2020, p. 29.

Return to note 194 referrer

BANDES 2020, p. 1302.

Return to note 195 referrer

BENYEKHLEF 2020; UNITAR 2020, p. 23; CALIFORNIA 2020, pp. 14-15; LEDERER 2021, p. 331; SOURDIN 2020; and OECD 2020.

Return to note 196 referrer

HOPKINS 2020, p. 13.

Return to note 197 referrer

See KOVACS 2020; and SOURDIN 2020.

Return to note 198 referrer

CALIFORNIA 2020, pp. 14-15; LEDERER 2021, p. 331.

Return to note 199 referrer

SOURDIN 2020.

Return to note 200 referrer

See also ELLIS 2004; and LEDERER 1999, p. 839.

Return to note 201 referrer

Clarifying Lawful Overseas Use of Data Act, H.R. 4943.

Return to note 202 referrer

MICROSOFT 2021.

Return to note 203 referrer

ARIZONA 2021, p. 9.

Return to note 204 referrer

BENYEKHLEF 2020.

Return to note 205 referrer

SALYZYN, 2020.

Return to note 206 referrer

B. POULIN 2004, p. 1157.

Return to note 207 referrer

CCR 2021, p. 11.

Return to note 208 referrer

Arconti v. Smith, 2020 ONSC 2782 (CanLII), par. 20.

Return to note 209 referrer

Pr. Vermeys is an Associate Director at BVC Groupe Cyberjustice. He is also a full Professor, and the Associate Dean of Programs at the University of Montreal’s Faculty of Law and the Associate Director of the Cyberjustice Laboratory.

Return to note 210 referrer

Mr. Callipel is the vice-president of BVC Groupe Cyberjustice, and the Head of mission at the Cyberjustice Laboratory.

Return to note 211 referrer

For this purpose definitions from the following authors and reports were considered: Access to Justice Toolbox, Internal document, Ottawa: Department of Justice Canada, 2012; Lorne Sossin, “Chapter 7 : Access to administrative justice and other worries”, in Colleen Flood & Lorne Sossin, eds, Administrative Law in Context, 2nd ed., Toronto, Emond Montgomery Publishing, p. 203; Jean-François Roberge, La justice participative, Cowansville, Éditions Yvon Blais, 2011.

Return to note 212 referrer

Emily Bates, Jennifer Bond & David Wiseman, “Troubling Signs: Mapping Access to Justice in Canada’s Refugee System Reform”, (2016) 47(1) Ottawa Law Review 39.

Return to note 213 referrer

Australian Government Attorney-General's Department, “A strategic framework for access to justice in the federal civil justice system”, (2009) Report by the Access to Justice Taskforce, online: https://www.ag.gov.au/LegalSystem/Documents/A%20Strategic%20Framework%20for%20Access%20to%20Justice%20in%20the%20Federal%20Civil%20Justice%2 0System.pdf.

Return to note 214 referrer

Australian Government Attorney-General's Department, “National Legal Assistance Data Standards Manual” (2015) online: https://www.ag.gov.au/LegalSystem/Legalaidprogrammes/Pages/National-Legal-Assistance-Data-Standards.aspx.

Return to note 215 referrer

Centre for Access to Justice at Fordham Law School, “Recommended Access to Justice Indicators for Implementation of Goal 16 of the UN 2030 Sustainable Development Agenda in the United States”, (2016) National Centre for Access to Justice at Fordham Law School, online: http://ncforaj.org/wp- content/uploads/2016/12/NCAJ-CHRI-9-15-16-Recommended-AtJ-National-Indicators-12-1-16-final.pdf.

Return to note 216 referrer

Laura Quinn and Joyce Raby, “Measuring Online Legal Resources: A Framework Inspired by the Drake Equation”, (2018) Florida Legal Technology Center, online: https://www.srln.org/system/files/attachments/Drake-Equation-for-Online-Legal-v7.pdf .

Return to note 217 referrer

Pascoe Pleasence, Nigel Balmer & Rebecca Sandefur, “Paths to Justice: A Past, Present and Future Roadmap”, (2013) Nuffield Foundation, online: http://www.nuffieldfoundation.org/sites/default/files/files/PTJ%20Roadmap%20NUFFIELD%20Published.pdf.

Return to note 218 referrer

Catrina Denvir, “Assisted Digital Support for Civil Justice System Users: Demand, Design and Implementation”, (2018) U.K. Courts and Tribunals Judiciary, online: https://www.judiciary.uk/wp-content/uploads/2018/06/cjc-report-on-assisted-digital-support.pdf.

Return to note 219 referrer

Martin Gramatikov et al., “A Handbook for Measuring the Costs and Quality of Access to Justice”, (2010) Law for life, online: http://lawforlife.org.uk/wp- content/uploads/2013/05/handbook-for-measuring-the-costs-and-quality-of-access-to-justice-271.pdf.

Return to note 220 referrer

Tim Roberts & Associates Consulting, “Access to Justice Well-being Index: Final report and Index”, (2018), Department of Justice Canada, Evaluation Division. (not as yet published).

Return to note 221 referrer

Susan McDonald, “Development of an Access to Justice Index for Federal Administrative Bodies”, (2017) Department of Justice Canada, online: https://www.justice.gc.ca/eng/rp-pr/jr/fab-eaf/p3.html.

Return to note 222 referrer

World Justice Project, “Global Insights on Access to Justice”, (2019) online : https://worldjusticeproject.org/sites/default/files/documents/WJP-A2J-2019.pdf.

Return to note 223 referrer

Jean-François Roberge, “Sense of Access to Justice” as a Framework for Civil Procedure Justice Reform: an Empirical Assessment of Judicial Settlement Conferences in Quebec (Canada)”(2015) 17(23) Cardozo J. Of Conflict Resolution 323; Jean-François Roberge, “Le sentiment de justice. Un concept pertinent pour évaluer la qualité du règlement des différends en ligne ? ”, (2020), 1 Revue juridique de la Sorbonne 5.

Return to note 224 referrer

Pr. Vermeys is an Associate Director at BVC Groupe Cyberjustice. He is also a full Professor, and the Associate Dean of Programs at the University of Montreal’s Faculty of Law and the Associate Director of the Cyberjustice Laboratory.

Return to note 225 referrer

Mr. Callipel is the vice-president of BVC Groupe Cyberjustice, and the Head of mission at the Cyberjustice Laboratory.

Return to note 226 referrer