Fatima Zehra  ( Department of physiology, Bahria University Medical and Dental College, Karachi, Pakistan )
Sonia Ijaz Haider  ( Department of Educational Development, Aga Khan University, Karachi, Pakistan )

Objective: To develop an instrument to assess professional behaviour among medical students, and to determine its validity, reliability, feasibility and acceptability.

Method: This cross-sectional study was conducted at Bahria University Medical and Dental College, Karachi, from July to September 2017, and comprised of medical students in years 1 and 2 of their academic programme. The instrument was developed for the Physiology laboratory sessions using nominal group technique and was tested for validity, reliability, feasibility and acceptability by two assessors through validity and reliability statistics. Data was analysed using SPSS 23.

Results: Of the 250 students enrolled, 232(92.8%) had their assessment completed. Applicability of the scale was indicated by exploratory factor analysis. Two factors were isolated which explained 57.65% of the variation. The first factor, comprising of 7 sub-factors, was labelled as 'practical performance', and the second factor, consisting of 5 sub-factors, as 'personal interactions'. A final 12-item instrument was retained. Cronbach's alpha was determined to be high at 0.863. Intra class correlation was 0.863 for both years 1 and 2 combined.

Conclusion: It was possible to develop an instrument for professional behaviour assessment in Physiology laboratory skills session which could indicate valid, reliable, feasible and acceptable results.

Keywords: Professionalism, Professional behaviour, Unprofessional behaviour, Instrument, Physiology laboratory, Existing tools for professionalism. (JPMA 70: 1398; 2020). https://doi.org/10.5455/JPMA.37124

 

Introduction

 

During the course of undergraduate medical studies, Physiology laboratory skills sessions provide hands-on experiential learning to students through clinical responses, clinical skills, laboratory investigations and experiments.¹ Medical theory and practice are closely connected. The laboratory skills sessions provide experience to the students for handling of instruments and specimens. This greatly assists in the correct interpretation of laboratory tests with normal reference values.² This forms the groundwork of patient prognosis, clinical efficacy of treatment and health outcomes.³ Thus, assessment in the Physiology laboratory is closely associated with bedside teaching and assessment in the wards and clinical rotations, and constitutes the foundation of medical practice.^1,2 The terms 'professionalism' and 'professional behaviour' are interchangeably used in literature.⁴ However, for our purpose, the term 'professional behaviour' was defined as comprising of dependability, effective interaction with patients and patient respect, integrity, self-reflection, team work and regard for peers. It is also defined as dutifulness, uprightness, selfimprovement, collaborated team effort, reflective practice and concern for peers.⁵ There is no globally accepted core definition of 'professionalism'.⁶ The elements of 'professionalism' are categorised into three competencies of ethics, clinical proficiency and professionalism.⁵ The American Board of Internal Medicine (ABIM) considers clinical competence an integral domain of professionalism, whereby cognitive and clinical excellence is essential for registration in ABIM. It comprises of moral principles, code of behaviour and characteristics present in an individual belonging to a particular profession.⁷ Thus, 'professional behaviour' is an integral component of 'professionalism'. A study conducted in Pakistan reported academic misconduct through the prevalence of literary theft, dishonesty, cheating in examinations and fabrication of documents.⁸ Another study reported marking of proxy attendance, using unfair means during examinations and approaching the faculty for unjust grading of results.⁹ No instrument for assessment of professional behaviour and attitudes during Physiology laboratory skills sessions by faculty for undergraduate medical students was found on Google Scholar, PubMed and PakMediNet.^10,11 The current study was designed to develop an instrument to assess professional behaviour in the Physiology laboratory sessions of undergraduate medical students, and to determine its validity, reliability, feasibility and acceptability.

 

Subjects and Methods

 

This cross-sectional study was conducted at Bahria University Medical and Dental College (BUMDC), Karachi, from July to September 2017, and comprised of medical students in years 1 and 2 of their academic programme. All medical students in years 1 and 2 of their academic programme were included, and different references suggest the sample size was adequate.¹² For each item of the scale, 10 students were considered and this was considered sufficient for factor analysis. A pilot study comprising of year 2 medical students was conducted to ascertain the face validity of instrument and feasibility of research. Systematic sampling was used by selecting every fifth student.¹³ The results of pilot study were not included in the final assessment and the students in the pilot study were excluded from the final sample. Data was collected over a period of two weeks in the Physiology laboratory at BUMDC for year 1 and 2, after taking informed consent from all the students, prior to assessment. Two items 10 and 11 were found not applicable to both the years and the assessors were asked not to rate these items for the relevant year 1 and 2, respectively. The 15-item revised instrument was administered for the final study after approval was obtained from the ethics review committees of BUMDC and Aga Khan University (AKU), Karachi. A new instrument was developed using a nominal group technique (NGT) by 5 content experts focusing upon essential and desirable traits as observable professional behaviours. In NGT, recommended number of participants to reach consensus is 5-9.^14,15 In the present study the participants comprised of a team consisting of two physiologists, one internist, one surgeon and one anatomist who was also a medical educationist. For NGT, a list of items was prepared on the basis of literature review.^16,17 After introduction of the topic of professional behaviour, a question was put forth to members. They were inquired about the observable professional behaviours that must be observed by medical students during the Physiology laboratory skills session. Each member documented the items separately without discussion. Each participant's written comments about observable behaviour were collected. The step of round robin was excluded. Nearly all the items developed by the content experts were included in the list prepared on the basis of literature review.^16,17 These items were distributed to the content experts for their consensus. After incorporating their comments, the final list was distributed for review and feedback. These were collected in the first round. Additions were incorporated in the list of items and given back individually to them for review and feedback. Items which were left without consensus or where there was a difference of opinion were discussed and revised in the third round. There was a final compilation of items in the instrument.^14,15 For scaling of the instrument, the experts agreed upon a visual analogue scale (VAS) as it provides more choice of response and thus prevents central tendency.¹⁸ A scale was devised ranging from 1 (poor) to 10 (excellent), with 13 items related to laboratory skills. After the completion of pilot study, two more items that were found to be missing were included in the instrument after approval by the participants of NGT. These were identified based upon the practical performance, whereby no item was found on clinical examination and the presence of students throughout the session was considered important after discussion with assessors. An instrument with descriptors was provided to 2 assessors for inter-rater reliability. Rater training was not conducted. Folders containing instrument forms were provided to the assessors. Demonstration of practical was delivered by the facilitator followed by practical performance by the facilitator and later by students who were divided into five nearly equal groups. Precordium examination and blood glucose estimation by glucometer were facilitated for the first and second year students respectively. The first-year practicals comprised of blood, respiratory and cardiovascular physiology, while the second-year practicals consisted of gastrointestinal, renal, endocrine and neurophysiology components. Students performed the practical upon their peers. The faculty assessed students of respective years, that is year 1 faculty assessed students of year 1, and year 2 faculty assessed students of year 2.17 Two female junior lecturers assessed students of year 1, and two female senior lecturers assessed students of year 2. Year 1 assessors examined students of practical groups A, B and C, while year 2 assessors examined students of practical groups D and E. Each session lasted for two hours and assessment was completed in 90 minutes each. Total time taken for the assessment of students of 5 practical groups was seven hours and 30 minutes spread over 5 sessions. Collective feedback was given to students after data analysis and interpretation of results. Data was stored and analysed using SPSS 23. It was taken as interval data. Independent sample t-test was used to see the mean differences of responses between male and female samples. Mean comparison between the two batches was determined. One-way analysis of variance (ANOVA) was performed to compare the item means from group A to group E, where p<0.01 was considered statistically significant. Cronbach's alpha was used to determine internal consistency of the instrument,¹⁹ mean and standard deviation (SD) of items were calculated. Intraclass correlation (ICC) was also done with 95% confidence interval (CI). Inter-item correlations and item total statistics were conducted. Exploratory Factor Analysis (EFA) was done to identify the dimensions of the instrument. An Eigenvalue >1.5 was used for extraction of factors. Varimax rotation was done to obtain the factor loadings of each item in the extracted factors. A scree plot representing the graphical presentation of extracted factors was plotted (Figure).

Two studies based upon teacher assessment of students were employed for factor analysis.^20,21 Kaiser Meyer Olkin (KMO) test and Bartlett's test suggested that EFA was applicable and appropriate.^12,20-23 Acceptability of the instrument was determined by conducting separate focused group discussions (FGDs) for feedback of 7 and 6 randomly selected students of year 1 and 2, respectively. Feasibility of using the instrument was assessed by asking the faculty to complete a feedback questionnaire.

 

Results

 

Of the total 300 students, 50(16.6%) were part of the pilot study and were excluded from the final sample. Of the remaining 250(83.3%), 232(92.8%) had their assessment completed; 142(61.2%) of year 1 in practical groups A, B and C, and 90(38.8%) of year 2 in practical groups D and E. Overall, there were 150(64.65%) females and 82(35.34%) males. There was no significant difference between the performance of the two genders except that female students scored higher on item 1, while male students on item 13 (Table 1).

Comparison of mean scores across all practical groups revealed significant differences on all the items across all groups (p=0.01) except on items 4 and 5. Inter-item correlation for year 1 was 0.157 and for year 2 it was 0.355, while combined for both years it was 0.331. The combined mean of year 1 and 2 was 8.04 with a variance of 2.47 SD. Cronbach's alpha for year 1 was 0.73 and for year 2 it was 0.89, while combined for both years it was 0.863. ICC was 0.311 for single measures and 0.863 for average measures with 95% CI. In EFA, Bartlett's test value was 0.000, suggestive of applicability of this test. The KMO value was 0.887, which revealed the adequacy of the test. At Eigenvalue >1.5, two factors were isolated which explained 57.65% of the variation. The magnitude of communalities for retained items ranged from 0.3 to 0.8. Items 6, 8, 9, 11, 12, 13 and 14 were loaded on factor 1 with values in 0.5- 0.9 range, while items 1, 2, 4, 5 and 7 were loaded on factor 2 in the range between 0.5 and 0.8. Seven items were loaded on factor 1 and five items on factor 2. The first factor was labelled as 'practical performance', while the second factor was labelled as 'personal interactions'. The items which did not load on any factor and were excluded were 3, 10 and 15. Therefore, 12 items were retained. No cross-loadings were found between the extracted items (Table 2).

Summary item statistics revealed item means of 8.04, with a variance of 2.47 SD. A 12-item self-designed questionnaire was distributed to the faculty to ascertain feasibility of the study. Most assessors agreed that the directions for scoring the students were clear and it was easy to score a student's professional behaviour. The items were clear to understand and the item descriptors did not have confusing words. They did not face time constraints while assessing students and were able to assess each student on each item, except those found inapplicable. Students were randomly selected and verbal group feedback was taken separately from the group of seven year-1 and six year-2 medical students through two FGDs. Most agreed that there should be an assessment for professional behaviour and it should be incorporated in the medical curriculum. Two students were of the opinion that students could possibly have been at their best behaviour because of being informed prior to assessment.

 

Discussion

 

The instrument devised was named the Instrument for Professional Behaviour in Laboratories (IPBL). The items of the instrument were found relevant to the construct of professional behaviour and closely aligned with the theoretical framework of situated learning. In accordance with the 'Situativity Theory', learning is embedded within experience, which in this study comprised of an interplay between medical students and the Physiology laboratory.²⁴ Interpretation of factors is subjective and hypothetical.²⁵ The first factor of 'practical performance' was labelled based upon cognition and performance during laboratory skills session, and the second as interpersonal communication. Practical performance comprises of development of laboratory skills which include investigations and clinical examination which comprises the basis of patient diagnosis and eventually successful patient management. It includes team work and assuming responsibility for personal learning. Personal communication encompasses interactions with faculty, peers, laboratory assistants, nursing staff and patients. This leads to development of effective bedside manners, empathetic attitude towards patients, maintaining respectful doctor patient relationship, communicating information of terminal illness or death to the patient's family. This further enhances history taking skills, effective management and patient compliance with therapeutic treatment and better health outcomes.²⁶ One of the items included on the basis of pilot testing was 'performed the steps of relevant clinical examination', although this item was based both upon performance of skills and cognition. This could possibly be explained by the fact that General Medical Council (GMC) includes academic integrity and competence as a fundamental component of professional behaviour.²⁷ According to Hafferty, professionalism encompasses knowledge, skills, attitudes and dedication to healthcare service.⁶ To the best of our knowledge, the current study is the first of its kind. A similar intervention has not been reported nationally or internationally in literature. This research shall open new horizons of the paradigm of professional behaviour development as an instrument has been introduced for the first time in laboratory skills sessions. It is anticipated that this shall consequently result in an instrument which may be incorporated for medical and dental, physical therapy, medical technology and nursing students in Physiology laboratory skills sessions as formative assessment of professional behaviour competence.¹⁷ It could also be possibly administered during other laboratory skills sessions, as most items are applicable to all basic health sciences laboratories. In terms of limitations, the current study had a crosssectional design, with a single observation during one module. There were a large number of students for each laboratory skills session i.e. 50 students. The sample size is small as the study was conducted in a single institute, therefore the results may not be generalisable. Formative assessment may not contribute towards the same impact on students as summative does, although formative assessment provided with feedback leads to better results.¹⁷ It also does not demonstrate cross-cultural validity, criterion validity and responsiveness as no previous instrument was present for comparison.¹¹ Besides, students were briefed about the purpose of the study for ethical considerations, which could possibly have led them to be at their best behaviour. Students are likely to be more attentive to a facilitator with better teaching skills. A possible bias could have been introduced by the difference in designation between the senior and junior lecturers. Faculty may have been biased from their previous experience with the students which could probably have affected their grading, and produced a 'halo' effect. However, due to time constraints it would have been difficult for faculty to identify and assess unfamiliar students during ongoing sessions. Another implication is that one faculty assessment of professional competence is unlikely to have a high impact on medical students. Single measures Intra-class correlation(ICC) was found to be low in comparison with average measures, which was high. Rater training could have yielded higher inter-rater reliability.²⁸ The purpose of assessment was formative, and, therefore, we did not take cut-off scores into consideration. Other dimensions of aptitude, including academic scores, were also not compared with professional behaviour scores in the study. The number of FGDs may have been fewer than the required number for the determination of acceptability of an instrument. Experts on professionalism could have had a greater impact on the development of the instrument. Future multi-institutional studies are recommended over a longer duration with observations across the entire academic year. Repeated faculty assessments and summative format may enhance student performance.¹⁷

 

Conclusion

 

Findings suggested that it was possible to develop an instrument for professional behaviour assessment in Physiology laboratory skills sessions which could indicate valid, reliable, feasible and acceptable results. Therefore, the new instrument could be incorporated in the curriculum for formative and summative assessment of professional behaviour in medical, dental, physiotherapy, medical technology and nursing students.

 

Acknowledgement: I acknowledge Dr Naghma Naeem, who inspired me with the conception of my research instrument. I am grateful to Dr Syeda Kauser Ali for her guidance. I am thankful to the content experts who contributed in the development of my instrument. I wish to acknowledge the efforts of Dr Rabia Siddiqui, Dr Rabiya Ali, Dr Lalarukh Munawar and Dr Haya Masud, who contributed in the data collection. I also wish to recognize Mr Syed Adnan Ali, for his statistical support.

Disclaimer: The text is based on a thesis for the degree of Masters in Health Professions Education, Aga Khan University, Karachi.

Conflict of Interest: None.

Source of Funding: None.

 

References

 

1. Dohn NB, Fago A, Overgaard J, Madsen PT, Malte H. Students' motivation toward laboratory work in physiology teaching. Adv Physiol Educ. 2016; 40:313-8.

2. Quinn J, Tansey E, Johnson C, Roe S, Montgomery L. Blood: tests used to assess the physiological and immunological properties of blood. J Med Ethics. 2010; 36:250-4.

3. Wians FH. Clinical laboratory tests: which, why, and what do the results mean? Lab Med. 2009; 40:105-13.

4. O'Sullivan H, van Mook W, Fewtrell R, Wass V. Integrating professionalism into the curriculum: AMEE Guide No. 61. Med Teach. 2012; 34:e64-e77.

5. Rogers W, Ballantyne A. Towards a practical definition of professional behaviour. J Med Ethics. 2010; 36:250-4.

6. Birden H, Glass N, Wilson I, Harrison M, Usherwood T, Nass D. Defining professionalism in medical education: a systematic review. Med Teach. 2014; 36:47-61.

7. Akhund S, Shaikh ZA, Ali SA. Attitudes of Pakistani and Pakistani heritage medical students regarding professionalism at a medical college in Karachi, Pakistan. BMC Res Notes. 2014; 7:150.

8. Ghias K, Lakho GR, Asim H, Azam IS, Saeed SA. Self-reported attitudes and behaviours of medical students in Pakistan regarding academic misconduct: a cross-sectional study. BMC Med Ethics. 2014; 15:43.

9. Sobani ZA, Mohyuddin MM, Saeed SA, Farooq F, Qaiser KN, Gani F, et al. Professionalism in medical students at a private medical college in Karachi, Pakistan. J Pak Med Assoc. 2013; 63:935.

10. Davidova R, Mushatova P, Yochkova S, Iliev M. Professionalism dissected: The role of anatomy faculty in promoting medical professionalism. J Biomed Clin Res. 2010; 3:12-8.

11. Li H, Ding N, Zhang Y, Liu Y, Wen D. Assessing medical professionalism: A systematic review of instruments and their measurement proper ties. PloS one. 2017; 12:e0177321.

12. Taherdoost H, Sahibuddin S, Jalaliyoon N. Exploratory factor analysis: Concepts and theory. Advances in Pure and Applied Mathematics. 2014.

13. Creswell JW. Educational research: Planning, conducting, and evaluating quantitative. USA: Pearson Education Limited, 2014; pp-161.

14. Council GM. Medical students: professional values and fitness to practise: General Medical Council; 2009.

15. Committee GMCE. Tomorrow's doctors: recommendations on undergraduate medical education. London: General Medical Council London, 1993.

16. Dannefer EF, Henson LC, Bierer SB, Grady-Weliky TA, Meldrum S, Nofziger AC, et al. Peer assessment of professional competence. Med Educ. 2005; 39:713-22.

17. Schönrock-Adema J, Heijne-Penninga M, Van Duijn MA, Geertsma J, Cohen-Schotanus J. Assessment of professional behaviour in undergraduate medical education: peer assessment enhances performance. Med Educ. 2007; 41:836-42.

18. Kuhlmann T, Dantlgraber M, Reips UD. Investigating measurement equivalence of visual analogue scales and Likert-type scales in Internet-based personality questionnaires. Behav Res Methods. 2017; 49:2173-81.

19. Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use: Oxford University Press: Great Britain, 2015; p 88.

20. Nelson AT, Lopata C, Volker MA, Thomeer ML, Toomey JA, Dua E. Exploratory factor analysis of SRS-2 teacher ratings for youth with ASD. J Autism Dev Disord. 2016; 46:2905-15.

21. Besnoy KD, Dantzler J, Besnoy LR, Byrne C. Using exploratory and confirmatory factor analysis to measure construct validity of the Traits, Aptitudes, and Behaviors Scale (TABS). J Educ Gifted. 2016; 39:3-22.

22. Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003; 37:830-7.

23. Thompson B, Daniel LG. Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Sage Publications Sage CA: Thousand Oaks, CA, 1996.

24. Durning SJ, Artino AR. Situativity theory: A perspective on how participants and the environment can interact: AMEE Guide no. 52. Med Teach. 2011; 33:188-99.

25. Williams B, Onsman A, Brown T. Exploratory factor analysis: A fivestep guid e for novices. Aust J Paramed. 2010;8:1-13.

26. Gilligan C, James EL, Snow P, Outram S, Ward BM, Powell M, et al. Interventions for improving medical students' interpersonal communication in medical consultations. JMIR Mhealth Uhealth. 2018; 6:e65.

27. Roff S, Chandratilake M, McAleer S, Gibson J. Medical student rankings of proposed sanction for unprofessional behaviours relating to academic integrity: results from a Scottish medical school. Scott Med J. 2012; 57:76-9.

28. Downing SM. Reliability: on the reproducibility of assessment data. Med Educ. 2004; 38:1006-12.