Iqra Aziz  ( Department of Medical Education, Shalamar Medical and Dental College, Lahore, Pakistan )
Syed Hasan Shoaib  ( Department of Medical Education, Shalamar Medical and Dental College, Lahore, Pakistan )
Sundus Iftikhar  ( Department of Medical Education, Shalamar Medical and Dental College, Lahore, Pakistan )
Muhammad Zahid Bashir  ( Department of Forensic Medicine, Shalamar Medical and Dental College, Lahore, Pakistan )
Akbar Ashraf  ( Department of Information Technology, Shalamar Medical and Dental College, Lahore, Pakistan )
Munawar Shahzad  ( Department of Information Technology, Shalamar Medical and Dental College, Lahore, Pakistan )

Selection of medical students requires both cognitive and soft skills assessment. Shalamar Medical and Dental College (SMDC) has been using on-campus multiple mini interviews to assess the latter but due to Covid-19 pandemic it became imperative that an alternative be found. The aim of this communication is to share the process SMDC went through to plan, design, and ultimately conduct WhatsApp-based multiple mini interviews (wMMI) in a low risk method as an entry criteria for undergraduate medical students. The process involved designing scenarios appropriate for online interviews, training the faculty members regarding conducting MMI as well as the use of technology, and designing an online webpage for enrolling, scheduling and assessing candidates. We were able to successfully complete wMMI process for 522 candidates within one week in a low risk setting using WhatsApp as communication medium with strong IT and administrative support.

 

Keywords: Medical student; Admission; Selection; Multiple mini interviews; Technology.

DOI: https://doi.org/10.47391/JPMA.5325

 

Background

 

The aim of selecting students for medical school is to identify, from a large number of applicants, the candidates who will be successful in medical training and ultimately become competent clinicians. Selecting the best candidates is typically on the basis of both cognitive and soft skills assessment.1 Academic ability in Pakistan is assessed using the scores obtained in the FSC/Matriculation exams and medical and dental colleges’ admission test (MDCAT). The format for assessing soft skills is mostly the multiple mini interviews (MMI) due to its high reliability and feasibility.2

MMI process includes multiple stations each assessing a different soft skill domain.3 Generally, 6-10 situational interviews are designed with professionalism, communication skills and teamwork being the top three most frequently assessed areas.4 Each candidate spends 7-10 minutes at individual station where they give face to face interviews which are then scored. Candidates, assessors, and support staff thus occupy the same physical space. This format, while resource intensive, does allow for more candidates to be assessed at one time and reduces individual assessor bias as the overall score is calculated by summing up individual interview scores.

Due to the Covid-19 pandemic it became imperative that an alternative to the on-site process be identified in order to minimise the spread of infection while maintaining integrity of admission process. Universities of Indonesia and Singapore have reported similar ideas where they successfully conducted remote MMI using Zoom as the medium.5,6 Shalamar Medical and Dental College (SMDC), a private medical institute in Lahore, Pakistan, has always strived to stay relevant and adaptable amidst all challenges. SMDC enrols 150 medical students each year by aggregating scores of FSC/Matric, MDCAT and MMI. MMI has been used as an enrolment criterion in SMDC since 2012 but has always been conducted face-to-face, on-campus. The Department of Medical Education (DME) at SMDC in collaboration with Information Technology (IT) Department was able to conduct fully remote, no-contact MMI for the cohort of applicants to the undergraduate medical education programme in 2020 using WhatsApp video call feature (wMMI).

The purpose of this special communication is to share the process SMDC went through to plan, design, and ultimately conduct wMMI in a low risk, budget-friendly method and to provide guidance to other medical schools faced with the similar challenge of shifting from on-site to online MMI.

 

Planning and Execution: The decision to shift MMI from face-to-face to online was made well in advance so the preparation phase lasted two months. This phase predominantly focused on finalising scenarios, training faculty members, selecting the appropriate app and designing online web page with linked scheduler.

Six domains were finalised on the basis of popularity in literature review and keeping in mind the ease of conducting interviews in an online format. The finalised domains were communication skills and counselling, critical and analytical thinking, leadership and decision making skills, empathy skills, motivation and preparation to study medicine and ethics. A pool of scenarios was created for each domain from which two scenarios were finalised after in-depth discussions and review process (see Table for sample scenario). The review process included focus group discussion with the fellow medical educationists in SMDC who have completed certificate programme or above in medical education. The focus group had 10 members. Active discussion regarding scenarios was conducted and feedback was then incorporated accordingly.

All scenarios had a scoring checklist based on global rating scale and 4-5 probing questions shared only with the assessor. The latter was to aid the interview process in case the discussion lagged or the candidate needed prompting. Each scoring list had five criteria points each with a range of scores of 1 to 10 where 1-2 denoted unsuitable; 3-4, less suitable; 5-6, satisfactory; 7-8, above average; 9-10, outstanding. A total of 132 faculty members from basic and clinical departments were trained over 14 training sessions with an average of 9-10 participants in each session. Each training session lasted two hours and catered to two domains only so as not to overwhelm the faculty with information. Mock interviews with discussions regarding the scenario, utility and use of probing questions along with scoring criteria and technique made up the bulk of the training session.

The next step included assigning one faculty member from each domain to a panel. A panel consisted of six faculty members, each trained in a different domain. A total of 22 panels were thus identified. Six of the trained members were unavailable during the interview days and hence the number was reduced to 21 working panels.

The preparation phase also involved discussions on the selection of an appropriate online application to employ for the interviews. Two top contenders were Zoom (zMMI) and WhatsApp (wMMI). The former was primarily chosen for its breakout rooms feature, whereas the latter was the popular choice by the faculty for its familiarity and ease. Two plans were created where plan A was wMMI and plan B was zMMI. Pros and cons of both the plans were discussed (Table) and Plan A was finalised.

 

Table: [Comparison of WhatsApp and Zoom plan] The IT department designed an online webpage linked with WhatsApp software and Integre Informatica (i2) software. A module for wMMI scheduling and candidate registrations was created in i2. This enabled the IT to systematically enter all trained faculty members and registered candidates in i2. DME then linked the faculty with candidates and created interview schedules. Double blinding technique was used where both the faculty and candidates were kept unaware regarding schedules. Close collaboration with Student Affairs Department ensured that candidates were emailed their schedule two days in advance. DME was responsible for intimating the faculty members their schedules via both email and WhatsApp groups. Mock runs were conducted to identify maximum possible issues that can arise on the interview days and steps were taken to mitigate majority of them. Training session for all faculty members regarding use of webpage and WhatsApp was conducted by IT department. Informative flow charts (Figure-1) and instructional videos for both the faculty and candidates were also disseminated through email, WhatsApp and college website to better acquaint the involved personnel with the technology being used.

 

Figure-1: [Instructional flowchart for the faculty members] The wMMI was conducted over eight days with registration of 570 candidates, out of which 522 candidates successfully completed the entire process. The first four days were used to schedule interviews for all 570 registrations, whereas the last four days were reserved for candidates who requested rescheduling at least 12 hours prior to their interview time. Day 1 till day 4 were further divided into three sessions, i.e. morning, afternoon and evening, and the faculty was given the liberty to choose one session each day. The duration of each session was 1.5 to 2 hours. A single interview ran for eight minutes and four extra minutes were provided for scoring and connectivity issues. By the end of day 4, 454 candidates had completed interviews whereas 68 candidates requested rescheduling and were accommodated on days five to eight. Forty-eight candidates failed to complete the MMI process as they had secured admission in other medical colleges and were not included in the final lists. Only candidates who had successfully completed all six wMMI interviews were considered eligible for admission.

Faculty members were given the option of conducting MMI interviews either through their offices or college’s IT Lab. Eighty-three (63%) faculty members felt comfortable with the use of technology and conducted interviews from their offices on campus, while the remaining 49 (37%) members chose IT lab because of readily available help there. Support staff was present in the college’s IT lab all day for any faculty member needing assistance. All faculty members logged on to the web page 30 minutes prior to the first scheduled candidate. This enabled support staff to answer any last minute queries. First and second merit list were displayed at the end of 4th and 8th day respectively. The wMMI process concluded with obtaining short feedback from participating candidates via Google docs. There were four questions with response choices of strongly agree, agree, disagree, and strongly disagree. A total of 197 candidates responded and the results are shared in Figure-2.

 

Figure-2: [wMMI candidates’ feedback result] Overall both the faculty members and the candidates seemed to enjoy the process, which is evident by the emails received, such as:

‘Congratulations to Principal SMDC, Medical Education and IT department for conducting such successful MMI. It was wonderful experience.’ (Faculty)

‘It’s a great initiative in this pandemic where the jury can judge the candidate's potential online. I have only seen Shalamar Medical College practicing these standards of interviews. Appreciation-worthy efforts.’ (Candidate)

 

Discussion

 

Shalamar Medical and Dental College (SMDC) adopted MMI as the selection criteria in 2012. The interviews were conducted on campus where the candidates, assessors and support staff would occupy the same space. A lot of arrangements had to be made such as booking of venues, refreshment, etc. and there was huge paper load due to printing of scenarios, result sheets, feedback forms, etc. Due to the Covid-19 pandemic and risk of infection spread, the traditional method was no longer an option and a shift to online MMI promised feasible yet diverse for the selection process of the candidates.7 The reliability and validity of an online MMI is also comparable to onsite MMI.6,8

The choice between wMMI and zMMI was made after realising that not only zMMI required more support staff for its operation, e.g., to act as moderators, etc., but during mock runs a lot of mistakes were made while moving either faculty or candidate from one breakout room to other.5 The wMMI did not face this issue as the faculty themselves was in charge of managing and calling their interviewees.

Some of the otherwise commonly assessed domains like team work and those requiring simulated actors (e.g. communication skills) had to be dropped or modified where the assessor played the role of an actor. This was in part due to limited human resources but also due to the complexity faced while planning execution. This can be addressed in future possibly by the use of virtual patients.9

Integrity of the entire process was maintained by matching the identification photo of the candidate submitted at the time of registration with the face in video call. Candidates were also asked to show 360 degree view of their surroundings in order to confirm that no helping material or person was present. This could have been further improved by recording all interviews but unfortunately the feature was not available in WhatsApp.

The execution of wMMI faced its fair share of challenges. Training faculty members regarding conduction of MMI and use of technology was a lengthy process. The most commonly encountered issue was that the faculty members after failing to contact their scheduled candidate would skip and call the next available candidate immediately instead of strictly following the scheduled times, thereby disrupting the entire schedule and causing confusion amongst the panel members. Some candidates declined participation in wMMI after the first scheduled faculty member called them. A few of the registered candidates had given incomplete or wrong WhatsApp numbers which only came to light after the interview process had started. This was fixed by close working with IT department but resulted in excessive anxiety for the candidates. The process can be further improved with the addition of call and video recording feature to WhatsApp application. This would allow for both monitoring and evaluation of the process. The reliability and validity of conducting MMI using Zoom and Skype has been established to be comparable to onsite process but the use of WhatsApp for MMI is an unexplored area. The interviews can be analysed for internal consistency and the reliability of the online process can thus be established. Reliability of face-to-face MMI process at SMDC was calculated in a previous study to be equal to 0.7.10

 

Conclusion

 

It is possible to conduct a fully remote, low risk MMI process using WhatsApp video-call feature with strong IT and administrative support. This is an especially suitable alternative for institutions that are facing challenges in shifting from onsite to online forum due to limited support staff. Training a large number of faculty members is resource intensive but overall allows for an independent interviewing process requiring minimum supervision.

 

Disclaimer: None.

 

Conflict of interest: None.

 

Funding disclosure: None.

 

References

 

1.      Patterson F, Knight A, Dowell J, Nicholson M, Cousans F, Cleland J. How Effective are selection methods in medical education? A systematic review. Med Educ 2016;50:36–60. doi: 10.1111/medu.12817

2.      Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD. The Multiple Mini-Interview (MMI) for student selection in health professions training - a systematic review. Med Teach 2013;35:1027-41. doi: 10.3109/0142159X.2013.829912.

3.      Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ 2004;38:314-26. doi: 10.1046/j.1365-2923.2004.01776.x.

4.      Yusoff MSB. Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review. J Taibah Univ Med Sci 2019;14:e203-40. doi: 10.1016/j.jtumed.2019.03.006.

5.      Yolanda S, Wisnu W, Wahjudi JM, Findyartini A. Adaptation of internet-based multiple mini-interviews in a limited-resource medical school during the coronavirus disease 2019 pandemic. Korean J Med Educ 2020;32:281-9. doi: 10.3946/kjme.2020.175.

6.      Kok KY, Chen L, Idris FI, Mumin NH, Ghani H, Zulkipli IN, et al. Conducting multiple mini-interviews in the midst of COVID-19 pandemic. Med Educ Online 2021;26:1891610. doi: 10.1080/ 10872981.2021.1891610.

7.      Ballejos MP, Oglesbee S, Hettema J, Sapien R. An equivalence study of interview platform: Does videoconference technology impact medical school acceptance rates of different groups? Adv Health Sci Educ Theory Pract 2018;23:601-10. doi: 10.1007/s10459-018-9817-2.

8.      Tiller D, O'Mara D, Rothnie I, Dunn S, Lee L, Roberts C. Internet-based multiple mini-interviews for candidate selection for graduate entry programmes. Med Educ 2013;47:801-10. doi: 10.1111/medu.12224.

9.      Berman NB, Durning SJ, Fischer MR, Huwendiek S, Triola MM. The Role for Virtual Patients in the Future of Medical Education. Acad Med 2016;91:1217-22. doi: 10.1097/ACM.0000000000001146.

10.    Haider SI, Bari MF, Ijaz S. Using Multiple Mini-Interviews for Students' Admissions in Pakistan: A Pilot Study. Adv Med Educ Pract 2020;11:179-85. doi: 10.2147/AMEP.S246285.