Kinza Aslam  ( University College of Medicine and Dentistry, Lahore )
Attia Bari  ( Peadiatric Medicine, Children Institute, Lahore )
Ayesha Ayub  ( Govt City Dispensary, Faisalabad, Pakistan )

Objective: To evaluate the perception of undergraduate dentistry medical students about the assessment environment at their institution.
Methods: The cross-sectional descriptive study was conducted at the University of Lahore, Lahore, Pakistan, from April 2017 to September 2017, and comprised undergraduate dentistry medical students who were asked to complete the Assessment Environment Questionnaire. Students' individual perception scores were calculated and the means of both individual domain and global score were compared in terms of different academic years. SPSS 20 was used for statistical analysis.
Results: Of the100 subjects, 50(50%) each were from 1st year and 2nd year. Overall, the students perceived their assessment environment positive yielding a global mean score of 49.84}10.84 out of the maximum 80. Second year students scored significantly higher compared to first year students in all domains except feedback mechanism (p=0.0017), learning and performance (p=0.0014), information on assessment (p=0.0064), and total mean score of 53.5}8.24 compared to 46.12}11.7 (p=0.0004).
Conclusion: The students' perception of the assessment environment was more positive than negative.
Keywords: Undergraduate medical student, Dentistry, Assessment, Feedback. (JPMA 69: 1834; 2019) DOI:10.5455/JPMA.293664

Introduction

The provision of high-quality healthcare services is a major and crucial goal for patient safety. Among the major aims of health system, an important target is the training of doctors who are the ultimate service providers.¹ There is a significant change in medical education overtime due to educational technology, advancement in medical education, educational concepts and curricula. ² Educational environment plays a very vital role in the learning process and this may have a profound effect on the students' behaviour, their performance, and ultimately the outcome of curriculum. ³ For targeting improvement in the training programmes, many validated instruments are developed in various countries. These are Dundee Ready Educational Environmental Measure (DREEM) for undergraduate health professional education, ⁴ Surgical Theatre Educational Environment Measure (STEEM), ⁵ Anaesthetic Theatre Educational Environment Measure (ATEEM), ⁶ and Postgraduate Hospital Educational Environment measure (PHEEM) ⁷ etc. Assessment of students is an integral part of all educational programmes. Both teaching and administrative faculty pay careful attention to its different part as it derives students' learning. ⁸ Assessment is about testing, measuring, collecting and combing information and finally providing a feedback. Assessment in medical education is multifaceted which drives learning, provides information on efficacy of teaching and protects patients. Examinees should know the purpose of assessment which they take, scoring system/ standard setting process, and receive feedback that foster their ongoing learning. ⁹ For assessment of educational environment, Assessment Educational Questionnaire (AEQ) is used. ¹⁰The AEQ instrument, a validated 20-item questionnaire, has been used to rate various aspects of the assessment environment for medical students. A significantly high level of reliability of AEQ is documented with Cronbach's alpha coefficient being 0.89. ¹⁰ Literature search reveals scarcity of studies on the assessment aspect of educational environment from Pakistan. The current study was planned to explore how undergraduate dentistry students perceived their assessment environment, and to investigate the association of their perception with different years of medical education.

Subjects and Methods

The university-based cross-sectional descriptive study was conducted at the University of Lahore (UoL), Lahore, Pakistan, from April 2017 to September 2017, and comprised undergraduate first year and second year dentistry students as only they were exposed to the new assessment system. After approval was obtained from the institutional review board and from the author of AEQ inventory, ¹⁰ the questionnaire was adapted to Pakistani context. Census sampling was used and the questionnaire was distributed among the subjects. Students were asked to anonymously complete the questionnaire in four domains; feedback mechanism, learning and performance, information on assessment, and assessment system/procedure. A 4-point Likert scale was used to assess the response; 4= Strongly agree (SA), 3= Agree (A), 2= Disagree (D), 1=Strongly disagree (SD). Higher level of agreement was correlated with more effective assessment educational environment. The scores ranged from a minimum 20 to maximum 80. Demographic data included student's year of education and gender. Data was analysed using SPSS20. Descriptive statistics included frequencies and percentages for categorical variables and mean and standard deviations for continuous variables. T-test was used to compare the mean AEQ scores of 1st and 2nd year students. P<0.05 was considered statistically significant.

Results

Of the 140 subjects approached, 100(71%) responded. Of them, 66(66%) were females. The total mean score of the entire sample was 49.84}10.84, with second year students scoring higher than the 1st year students (Table-1).



The lowest rated score (2.16}0.93) was from the feedback mechanism domain, while the highest score was (2.68}0.76) related to the perception of learning and performance domain (Table-2).



The perception of assessment system/procedure was the only domain where the result was not significant (p=0.31). All the other results were statistically significant (p<0.001) (Table-3).



Discussion

The study, evaluating the applicability of AEQ in a Pakistani setup, showed that all students, irrespective of the academic year, had a favourable perception of the assessment environment with respect to the level of feedback mechanism, learning and performance, provision of information of assessment as well as perception of assessment system. The overall mean AEQ score of 48.84/80 indicated that they were comparable with the study conducted by the developer of the questionnaire. ¹⁰ The mean scores of the subscales suggested that the questionnaire could be used to identify areas of strength and weaknesses within the assessment environment/ system in an undergraduate programme. While all items in the study were rated positively with a mean score of >2, no item was scored in true positive range with a mean score of 3 or 4, pointing towards problem areas within the assessment environment. Item 1.1, "I received feedback on my performance for continuous assessment", had the lowest rated mean score (2.16), followed by item 1.4, "feedback is given promptly after an assessment", (2.28) and item 1.3, "feedback from assessors about my performance is adequate", (2.34). Since all these items were from the feedback mechanism domain, the mean scores could ring a possible alarm for the faculty to re-examine the area and take improvement measures. The results of the current study are consistent with a research which showed that the feedback provided to medical students, including its timeliness, was not optimal.11 Another study showed similar results in which about 66% students had rarely received feedback from clinical teachers, and corrective feedback was only around 18%.¹² A study showed that senior students, during clinical years, valued immediate informal feedback as a result of observed clinical assessment and considered it to be important for their personal development. ¹³ As a contrast, students in early academic years perceived the receiving of feedback merely as a passive activity. ¹⁴ Another study showed that majority of the students agreed that assessment at their medical school was fair, but they were not satisfied on the fairness of feedback on assessment provided by the supervisor on their clinical rotations. A large number of medical students expressed the desire for the provision of more feedback on performance for future learning. ¹⁵Generally, medical students seem dissatisfied with the process of giving and receiving feedback ¹⁷. The area of feedback mechanism, hence, requires attention if the assessment environment is to be improved as a whole. Some items from the assessment domain did not perform very well either. For instance, Item 3.2, "I received information about what is expected of me in any exam/assessment", had a mean score of 2.46, followed by Item 3.3, "students receive clear information about assessment", with a mean score of 2.46. These perceived areas of weakness point to the fact that students need to be conveyed all information regarding assessments clearly and in a timely manner. All items in the sub-domain of assessment procedure had almost similar mean score of 2.57-2.62. Most of the weaknesses identified in the current study were consistent with the study done by Sim et al.10 The most important variable considered as an outcome of perceived classroom environment is achievement. There are numerous studies demonstrating that the students' perception of learning environment is significantly related to achievement. In the current study, perception of learning and performance was the highest scored domain. These results are consistent with the findings of numerous studies.^16-18 Students of different academic years had significantly different perceptions of the assessment environment, as there were significant differences among their AEQ scores. The mean AEQ score of 2nd year students were significantly higher compared to students in the 1st year, showing that 2nd year students perceived their assessment environment more favourably. Possibly this difference was due to the fact that 1st year students were trying to adjust to the new assessment environment which was different from their previous experiences and 2nd year students had adjusted to the new system. The items, which scored low in the current study, point towards issues within the assessment system. These issues, however, are not difficult to solve if due importance is given to faculty training. Additionally, provision of timely and constructive feedback, coupled with counselling for students who are wary of the assessment systems and procedures can also help in improving the assessment environment The results of the current study might have been influenced by inclusion of only dentistry students who are rotated to different specialties during their academic year, and, as such, one of the confounding factors may be the student's rotation in a specialty with less supportive assessment environment at the time of data collection.

Conclusion

AEQ inventory was found to be a reliable and valid tool for measuring the assessment environment of an undergraduate medical programme. The overall assessment educational environment of our medical school was perceived as more positive than negative but with plentiful room for improvement, especially in feedback mechanism.

Disclaimer: The study was part of a Poster presentation at the OTTAWA/ICME conference in Abu Dhabi on March 11,2018.

Conflict of Interest: None.

Source of Funding: None.

References

1. Verma S. Medical Science Awareness of Patient Safety Issues and Expectations Among Undergraduate Medical Students in a Government Medical College. IJSR Inter J Sci Res. 2016; 461:22-43.
2. Qiao YQ, Shen J, Liang X, Ding S, Chen FY, Shao L, et al. Using cognitive theory to facilitate medical education. BMC Med Educ. Bio Med Central; 2014; 14:79.

3. Al-Mohaimeed A. Perceptions of the educational environment of a new medical school, Saudi Arabia. Int J Health Sci (Qassim). 2013; 7:150–9.

4. Roff S. The Dundee Ready Educational Environment Measure (DREEM)--a generic instrument for measuring students’ perceptions of undergraduate health professions curricula. Med Teach. 2005; 27:322–5.

5. Dimoliatis ID, Jelastopulu E. Surgical Theatre (Operating Room) Measure STEEM (OREEM) Scoring Overestimates Educational Environment: the 1-to-L Bias. Univers J Educ Res. 2013; 1:247–54.

6. Holt MC, Roff S. Development and validation of the Anaesthetic Theatre Educational Environment Measure (ATEEM). Med Teach. 2004; 26:553–8.

7. Al-Shiekh MH, Ismail MH, Al-Khater SA. Validation of the postgraduate hospital educational environment measure at a Saudi university medical school. Saudi Med J. 2014; 35:734–8.

8. Mortaz Hejri S, Jalili M. Standard setting in medical education: fundamental concepts and emerging challenges. Med J Islam Repub Iran. 2014; 28:34.
9. Norcini JJ, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, et al. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach. 2011; 33:206–14.

10. Hiong Sim J, Ting Tong W, Hong WH, Vadivelu J, Hassan H. Development of an instrument to measure medical students’ perceptions of the assessment environment: initial validation. Med Educ Online. 2015; 20:28612.

11. Naekashri Jothi MSBYD. Knowledge and Perception of Medical Students on Feedback. Educ Med J. 2015; 7:44–55.

12. Al-Mously N, Nabil NM, Al-Babtain SA, Fouad Abbas MA. Undergraduate medical students’ perceptions on the quality of feedback received during clinical rotations. Med Teach. 2014; 36:S17–23.

13. Murdoch-Eaton D, Sargeant J. Maturational differences in undergraduate medical students’ perceptions about feedback. Med Educ. 2012; 46:711–21.

14. Kohoulat N, Hayat AA, Dehghani MR, Kojuri J, Amini M. Medical students’ academic emotions: the role of perceived learning environment. J Adv Med Educ Prof. 2017; 5:78–83.

15. Todres M, Tsimtsiou Z, Sidhu K, Stephenson A, Jones R. Medical students’ perceptions of the factors influencing their academic performance: An exploratory interview study with high-achieving and re-sitting medical students Med Teach. 2012; 34:325–31.

16. Iram Khursheed LB. Students’ perceptions of educational environment of a private medical school in Pakistan. J Pak Med Assoc. 2014; 64:1244–9.

17. Al Kadri HM, Al-Moamary MS, Magzoub ME, Roberts C, van der Vleuten CPM. Students’ perceptions of the impact of assessment on approaches to learning: a comparison between two medical schools with similar curricula. Int J Med Educ. IJME; 2011; 2:44–52.
18. Duffield KE, Spencer JA. A survey of medical students’ views about the purposes and fairness of assessment. Med Educ. 2002; 36:879–86.