Sakina Sadiq Malik  ( Department of Dermatology, Combined Military Hospital, Rawalpindi, Pakistan )
Rahila Yasmeen  ( Riphah Academy of Research and Education, Riphah International University, Islamabad,Pakistan )
Rehan Ahmed Khan  ( Department of Surgery, Riphah International University, Islamabad, Pakistan )

Objective: To assess the effectiveness of flipped classroom on dermatology residents’ learning of procedural skills, and to identify their perceptions about it.

Methods: The case-control study was conducted at Dermatology department of Pak Emirates Military and Combined Military Hospital Rawalpindi from 1/3/2018 to 2/7/2018, and comprised post-graduate dermatology residents training at Pak Emirates Military and Combined Military Hospital, Rawalpindi, Pakistan. The residents were randomised into two equal groups. A pre-test consisting of multiple-choice questions was taken. Group A was emailed the related teaching material 1 week prior to the session. The in the subsequent week, both groups attended a 3-hour hands-on training session. In the first hour, Group A had small group discussion, while Group B attended interactive lecture utilising the same teaching material that was posted to Group A earlier. Both groups then participated in hands-on session with the same facilitators. The process was repeated next week for another topic. Finally, a post-test comprising multiple-choice questions and objectively structured clinical exam at 16 stations was conducted. Group A answered the flipped classroom perception instrument and three open-ended questions. Quantitative data was analysed using SPSS 21, while manual thematic analysis was done for qualitative data.

Results: Of the 40 subjects, there were 40(50%) in each of the two groups. There was a significant difference between median pre-test and post-test scores in both groups (p<0.001).. With respect to objectively structured clinical exam, the median score of Group A was significantly different from Group B (p=0.001).

Conclusion: Flipped classroom was found to be an effective teaching strategy for procedural skills in dermatology residents.

Keywords: Flipped classroom, Procedural skills, Post-graduate residents.

(JPMA 70: 1547; 2020)

DOI: https://doi.org/10.5455/JPMA.27173

 

Introduction

 

Flipped learning combines asynchronous lectures (videos, tutorials, online teaching material) outside the class with in-class student-driven, active, problem-based, group learning facilitated by the teacher.¹ The former is based on direct instructional methods driven by behaviourism, and the latter is based on constructivism with social scaffolding in group learning. Keeping Bloom’s revised taxonomy of cognitive domain of learning in mind, pre-class activities help students understand and remember, while in-class activities help in application and analysis of their knowledge, and post-class assessment of higher order synthesis of key concepts focusses on creating, evaluating and reflecting on what has been learned.² While planning these flipped classrooms, cognitive load management considerations can align learning process effectively with human cognitive architecture (Figure-1).

By decreasing the extraneous load and optimising the germane load, a well-designed flipped classroom can be an efficient teaching approach.³ It can also be tailored according to the expertise of the learners. Novices can rewind the videos as per their requirements and experts can skip certain parts, thus learning at their own pace.⁴ Cognitive load theory of multimedia learning which is primarily based on cognitive load theory can form the basis for pre-class technology-driven activities of a flipped classroom.⁵

Learning of procedural skills is based on Dave’s taxonomy,⁶ starting from imitations followed by manipulation, acquiring precision, then articulation and finally naturalisation; culminating step by step into climbing the learning curve.  Scientifically-tailored teaching methods for procedural skills, like flipped learning, can steepen the learning curve.⁷

In Pakistan there are 40 units recognised for post-graduate training in dermatology, including Fellowship and Membership of the College of Physicians and Surgeons (FCPS, MCPS) and Medicinae Doctor (MD) training. There is a wide variation in learning resources, training methods and opportunities for exposure to different procedural skills across these teaching units. Acquiring proficiency in procedural skills in clinical settings requires learning opportunities, and supervision of these opportunities along with timely constructive feedback.⁸ All these cannot be provided to every resident on rotation who is required to endorse a large number of procedural skills in their e-logbooks.

According to a recent survey, 49% dermatology residents in Pakistan can do shave excisions of moles and 35% can perform phototherapy.⁹ Most residents reported deficiency in training in dermatoscopy, patch testing, iontophoresis, mesotherapy, lasers, botox and dermal fillers.

The current study was planned to see how a flipped class room strategy can be used efficiently.

 

Subjects and Methods

 

The case-control study was conducted at Dermatology department of Pak Emirates Military and Combined Military Hospital Rawalpindi from 1/3/2018 to 2/7/2018, and comprised post-graduate dermatology residents training at Pak Emirates Military and Combined Military Hospital, Rawalpindi, Pakistan. After approval from the ethics review committee of Riphah University, Islamabad, the sample size was calculated using the World Health Organisation (WHO) sample size calculator¹⁰ while keeping the alpha value as 5%, power 80%, population standard deviation (SD) as 5 and difference between the two populations as 5.

The sample was raised using random sampling from among the post-graduate dermatology residents undergoing FCPS or MCPS training. Those who consented were included, while residents either not willing or not present were excluded.

Those included were randomised into two intervention Group A and control Group B. A pre-test consisting of 30 multiple-choice questions (MCQs) was conducted for both the groups. The same 30 questions were included in the post-test.

Students were assessed objectively at the end of both flipped and traditional method classrooms by objectively structured clinical exam (OSCE). As two flipped classrooms were being assessed, the students had the OSCE at 16 stations of 5-10-minute duration each. The OSCE stations were constructed as per the Association for Medical Education in Europe (AMEE) Guide 81 A and B.¹¹

Flipped classroom perception instrument (FCPI),¹² a validated tool, was used to record the perceptions of residents about the intervention. Instrument for qualitative data collection were three open-ended questions constructed in accordance with the AMEE Guide 87.¹³

Triangulation was done for ensuring the validity of the study. Post flipped classroom results were checked both by post-test MCQs and the OSCE. Similarly, perception of students about the intervention was measured both by FCPI and also by open-ended questions. Unknown confounders were limited by randomisation (Figure-2).

Previous knowledge or expertise of residents could serve as a bias but pre-test was done to limit this possibility. Group equivalence was maintained by using the same teaching material, giving equal practice time after workshop and same assessment for both the groups. We couldn’t ensure that the group exposed to flipped classroom did not share their teaching material before the flipped class with the other group, but we tried to minimise this bias by educating the study group about the educational importance of the concept while taking consent. FCPI reliability was established by introducing the questionnaire to five participants of flipped classroom after phototherapy class and Cronbach alpha was calculated to be 0.91. It was later used after the second flipped classroom for the entire interventional group.

Quality assurance measures were done for the qualitative aspect of the study as well. For credibility, responses to open-ended questions were subjected to member checking when thematic analysis was done. Furthermore, peer debriefing during formulation and expert validation of open-ended questions was done. Again, the responses to questions and the generated themes were analysed by Dermatology Department faculty to point out over- or under-emphasised points, general errors and biases at any stage. Triangulation in the form of checking students’ perceptions by FCPI Likert-scale also ensured confirmability and dependability besides credibility. As far as transferability was concerned, purposive sampling, i.e. only exposing the flipped group to open-ended questions, was done.

Quantitative data was analysed using SPSS 21. Shapiro-Wilk significance or p<0.05 for the flipped and non-flipped group rejected the null hypothesis for the test of normality. As such, non-parametric tests were used. Mann-Whitney U test was used for comparison of the OSCE results of the interventional and control groups. Wilcoxon signed rank test for pre- and post-intervention MCQ score of both the groups was used. Reliability of the OSCE stations and the FCPI tool were assessed independently using Cronbach alpha, and the values were 0.769 and 0.914 respectively, indicating acceptability. Responses to each item in FCPI were analysed in terms of frequency and percentages.

Open-ended questions were analysed using an inductive coding process consistent with content analysis. Initial codes were narrowed down to 9 codes for advantages, 5 for disadvantages and 3 for limitations of the implemented flipped classroom. Themes were generated finally by aggregating similar codes to highlight the major ideas in the data, leading to 3 each for advantages and disadvantages, and 2 for limitations. Coding analysis was performed manually as there was a small database to analyse.

 

Results

 

Of the 40 subjects, there were 40(50%) in each of the two groups. Median pre-test score for the flipped class was 47 (interquartile range [IQR]: 24.5) which increased in post-test to 97 (IQR: 9.25) (p<0.001). For the non-flipped group, median pre-test score was 37 (IQR: 12.25) which improved to 90 (IQR: 13) (p<0.001). In the flipped group, median OSCE score was 83.75 (IQR: 3.75). In the non-flipped group, the median score was 73.13 (IQR: 11.57) (p<0.001).

Overall mean score for FCPI items was 4.613±0.043

(Figure-3).

Results of open-ended questions revealed advantages which can be categorised as general (promoting deep learning, speeding up learning, student centred-approach and improved learning abilities of students), related to pre-class activities (self-paced learning, engaging study material and re-usable study material) and in-class activities (efficient use of time and improved student engagement). Limitations were categorised as general (resource intensiveness), in-class activity (loss of student’s interest) and pre-class (time consuming and uselessness for unmotivated students). Students recommended improvement in videos by making them short and concise, and giving more time to students for pre-class activity.

 

Discussion

 

Comparison of pre- and post-intervention MCQs’ result of flipped and non-flipped classrooms showed significant improvement in post-test results. But percentage increase in non-flipped (143.2%) was more than the flipped classroom (106.38%). The reason for this could be that the randomly selected students for flipped classroom intervention performed better in their pre-test compared to the control group because of their better pre-knowledge. And although their scores improved in post-tests, the improvement was not more than in the control group whose performance was poorer in the pre-test. Another study carried out in 2015 during Gynae/Obstetric clerkship revealed similar unexpected situation when the MCQ scores in traditionally-taught group were better than the flipped group.¹⁴ It may be inferred from our results that assessing knowledge acquisition by checking students’ scores in MCQs was not sufficient enough. It only addressed the information accumulation aspect. The other dimensions of knowledge acquisition, like information accessibility, information stimulation, information communication and, finally, information interaction, need also to be addressed before making inferences.¹⁵

In a study using flipped classroom intervention for ocular trauma module there was no difference between the pre-test MCQ scores of the interventional and control group, but there was a significant increase in the post-test result of the flipped group.¹⁶

While comparing flipped team-based learning course for advanced cardiac support with traditional course, two of the three tests showed statistical improvement for the flipped format.¹⁷

With respect to OSCE results, there was significant difference between the groups, indicating that flipped classroom can be good option for teaching procedural skills as OSCE is a better assessment tool for procedural skills in which the flipped group performed better. However, the percentage improvement in the non-flipped group in MCQs was better.

In a study carried out in dental undergraduates in India, competency in denture treatment showed statistically significant result in students trained by flipped classroom intervention (p<0.001) compared to those trained by traditional methods.¹⁸

An important aspect of quality control of the OSCE is its reliability which is a source of internal structure validity. Cronbach alpha estimates one facet of measurement errors in the OSCE, i.e. scores on stations,¹⁹ which normally range from 0 to 1. The closer the value is to 1, the better is the internal consistency of results of the OSCE at different stations. Reliability measured as Cronbach’s alpha for the 16 OSCE stations with 5 marks each, in which 40 candidates were examined, was 0.769, which is acceptable.

Cronbach’s alpha reliability coefficient was 0.914 during pilot testing the FCPI, an already validated tool containing 8 items. It was an excellent reliability showing good internal consistency of items used in the questionnaire.

Mean score was 4.613 for 8 items in the FCPI in our study. It was 3.94 in quality improvement classroom in residency education as checked across 143 residents at Mayo clinic in 2014.²⁰ It was 3.94 after implementing flipped continuing medical education (CME) module for 167 participants in Mayo clinic 2015 Internal Medicine Board Review (IMBR) course.¹² Our mean score was better, showing better satisfaction level.

As a result of qualitative analysis, 34 advantages came as a response from 20 candidates. Promotion of deep learning was the most perceived advantage of the intervention. Other general advantages were speeding up the learning curve, student-centred approach and improvement in learning abilities of the students. Advantages of pre-class activities were evident in 7 responses focussing on self-paced learning, engaging process and re-usability of study material. Advantages related to in-class activities were better student participation and efficient use of class time by the teacher. A study showed that students value interactivity, creativity and application in the flipped classroom.¹⁷ Teacher gets flexibility for teaching methods and content delivery, gets more time for interaction with students and give timely feedback and instructions.²¹

In response to question regarding limitations, most students pointed out the resource intensiveness of the intervention as the major pitfall followed by time requirement for pre-class activities. Some pointed out that it cannot help unmotivated students, does not cater to different learning styles and leads to monotonous effect of classroom due to repetition and re-enforcement of the pre-class teaching activity. The last-mentioned limitation resembles the one mentioned in another study which pointed out that students can get off the topic due to repetition.²² Another study pointed out technological glitches as the major limitation of the process.¹⁵ This comes under the same umbrella of resource intensiveness as pointed out in our study.

Student’s recommendations about improvement in the flipped classroom included improvement in quality of videos, short and concise pre-class study material, and more time to go through the pre-class material. Our findings matched those in another study where students recommended replacement of traditional power-point videos with videos and demanded periodic revision of practice questions and clinical reasoning cases.²³

The current study has a few limitations, like a small sample size from a single centre. Besides, teachers’ perspectives have not been incorporated. More open-ended questions could have been added to capture the full breadth of residents’ attitude. Also, one group was kept as the control groups throughout both the flipped classes. There are many variables related to a student’s abilities, like motivation level etc., which could have been accommodated with a cross-over in second flipped learning session. Finally, long-term effects of the process could have been recorded by more authentic work-place-based assessments, like direct observation of procedural skills.

Despite the limitations, however, the study will give residency programme directors and tutors better insight into the mechanism of flipped classroom for procedural skills, and will encourage them to utilise this strategy to its full potential.

 

Conclusion

 

Flipped classroom was found to be an effective teaching strategy for procedural skills in dermatology residents. The same can be applied in other fields of medicine and surgery, and both for post-graduate and undergraduate courses.

 

Disclaimer: The text is based on a Master’s in Health Professional Education thesis.

Conflict of interest: None.

Source of Funding: None.

 

References

 

1.      O’Flaherty J, Phillips C. The use of flipped classrooms in higher education: A scoping review.[Online] 2015 [Cited 2017 January 21]. Available from: URL: http://dx.doi.org/10.1016/j.iheduc.2015.02.002.

2.      Armstrong, P. Bloom’s taxonomy, CFT teaching guides. Vanderbilt University Center for Teaching. [Online] [Cited 2018 August 17]. Available from: URL:  https:// cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/.

3.      Turan Z, Goktas Y. The Flipped Classroom: Instructional Efficiency and Impact on Achievement and Cognitive Load Levels. [Online] [Cited 2018 December 12]. Available from: URL: https://www.learntechlib.org/p/173672/.

4.      Abeysekera L, Dawson P. Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. High Educ Res Dev. 2015; 34:1–14.

5.      Sorden SD. The cognitive theory of multimedia learning. Handb Educ Theories. 2012; 2:1-31.

6.      Pešaković D, Šafhalter A. Unified taxonomy of competences for verification of student's skills. Problem Educ 21st Century.2016; 72: 89-99.  

7.      Awidi IT, Paynter M. The impact of a flipped classroom approach on student learning experience. Comput Educ. 2019: 128: 269-83.

8.      Touchie C, Humphrey-Murto S, Varpio L. Teaching and assessing procedural skills: a qualitative study. BMC Med Educ. 2013; 13:69.

9.      Azfar NA, Malik LM, Aslam S, Jahangir M, Rashid T. Exposure of dermatology postgraduate trainees to different learning resources and procedural skills in Pakistan. JPAD.2017; 27:54–8.

10.    Lwanga S, Lemeshow S.  Sample size determination in health studies:  a practical manual.  England:  World Health Organization, 1991.

11.    Khan KZ, Ramachandran S, Gaunt K, Pushkar P. The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part I: An historical and theoretical perspective. Med Teach. 2013; 35:e1437–46.

12.    Stephenson CR, Wang AT, Szostek JH, Bonnes SL, Ratelle JT, Mahapatra S, et al. Flipping the Continuing Medical Education Classroom. J Contin Educ Health Prof. 2016; 36:256–62.

13.    Artino AR, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE Guide No. 87. Med Teach. 2014; 36:463–74.

14.    Gillispie V. Using the Flipped Classroom to Bridge the Gap to Generation Y. Ochsner J. 2016; 16:32–6.

15.    Newman G, Kim J-H, Lee RJ, Brown BA, Huston S. The Perceived Effects of Flipped Teaching on Knowledge Acquisition Background and Literature Review. J Eff Teach. 2016; 16:52–71.

16.    Tang F, Chen C, Zhu Y, Zuo C, Zhong Y, Wang N, et al. Comparison between flipped classroom and lecture-based classroom in ophthalmology clerkship. Med Educ Online. 2017; 22:1395679.

17.    Boysen-osborn M, Anderson CL, Navarro R, Yanuck J, Strom S, Mccoy CE, et al. Flipping the advanced cardiac life support classroom with team- based learning : comparison of cognitive testing performance for medical students at the University of California , Irvine , United State. J Educ Eval Health Prof. 2016; 13:11.

18.    K A, K KM, Vijayaraghavan P, Kandaswamy D. Flipped clinical training: a structured training method for undergraduates in complete denture prosthesis. Korean J Med Educ. 2016; 28:333-42.

19.    Zainuddin Z, Halili SH. Flipped classroom research and trends from different fields of study. Int Rev Res Open Distance Learn. 2016; 17: 1-28.

20.    Bonnes SL, Ratelle JT, Halvorsen AJ, Carter KJ, Hafdahl LT, Wang AT, et al. Flipping the Quality Improvement Classroom in Residency Education. Acad Med. 2017; 92:101–7.

21.    Moraros J, Islam A, Yu S, Banow R, Schindelka B. Flipping for success: evaluating the effectiveness of a novel teaching approach in a graduate level setting. BMC Med Educ. 2015; 15:27.

22.    Young JE, Williamson MI, Egan TG. Students reflections on the relationships between safe learning environments, learning challenge and positive experiences of learning in a simulated GP clinic. Adv Health Sci Educ Theory Pract. 2016; 21:63-77.

23.    Liebert CA, Mazer L, Bereknyei Merrell S, Lin DT, Lau JN. Student perceptions of a simulation-based flipped classroom for the surgery clerkship: A mixed-methods study. Surgery. 2016; 160:591-8.