Triad of rubric assessment, constructive feedback and video recorded surgeries key assessment for competent ophthalmic surgeon

Introduction

Small-incision cataract surgery (SICS) is a widely accepted, appropriate, and affordable procedure that can deliver high-quality visual outcomes.^[1], ^[2], ^[3], ^[4]

Despite this need, concerns remain in several regions over the safety, quality, and efficiency of surgical training for cataract surgery.^[5], ^[6]

Ophthalmic surgery involves very precise surgical skills which are difficult to teach. The surgical technique of a resident is also difficult to assess. Currently, teaching ophthalmic surgical skill is not structured. Every surgeon have their own way of imparting surgical training, which creates confusion amongst the residents, thus hampering their learning skills.

Hence there is need for education system to come up with some teaching and assessment tool for evaluation of surgical performance of the post graduate students.

It is also desirable for any assessment method to be objective, thus removing bias and favoritism.

For assessing performances in clinical settings Miller developed a theoretical framework known as Millers Pyramid clinical assessment where bottom levels begins as ‘Knows’, then ‘Knows How’, ‘Shows how’ and at top ‘Does’. The top level ‘Does’ has been considered as gold standard of assessment in surgical training measured by direct faculty observation. There may be some barriers for assessment of surgical trainings of trainees e.g. fear of being labelled for strict and harsh teacher, it may be challenging for supervisor to assess progress of underperforming trainees because of poor documentation, lack of opportunities to observe performance, or fear of legal action, hence the Master of surgery degree is conferred without assessing actual operative skill of surgeon which may put health of society at risk. Hence there is need for education system to come up with some assessment tool for evaluation of surgical performance of post graduate student.

In practice, ICO has developed and implemented various assessment tool to evaluate the different level of millers pyramid in form of task specific checklist called Ophthalmic surgery clinical assessment Rubric(OSCAR).^[1], ^[7]

The ICO OSCAR for SICS was developed by experts at the ICO using a modified Dreyfus scale (novice, beginner, advanced beginner, and competent).^[8]

Significance of the study:

Formative Assessment and constructive feedback in surgical training will improve the competency of new ophthalmic surgeons.

Structured surgical training will be relatively easy to observe and perform, as trainee learns what is required to be competent.

Ultimately improve patient care.

Aim & Objective

To develop more standardize surgical training.

To assess efficacy of new tool which can improve surgical skill of PG student.

To know the effect of constructive feedback on surgical performance.

Materials and Methods

This study was prospective observational study, carried out in Department of Ophthalmology of Medical College and Research Centre between January 2019 to February 2020 after institutional ethical clearance.

Seven final year Post Graduate (PG) residents were chosen for the study with convenience sample technique Small incision cataract surgery training assessment was done by Rubric designed by tool called ICO-Ophthalmic Surgery Competency Assessment Rubric. The same tool was used to assess video recorded cataract surgery of residents by 2 different faculties and the response was noted. Assessor independently assessed the surgical technique without any prior influence and knowledge about previous surgical assessment, PG resident’s surgical video were numbered as PG 1 to PG 7 to maintain anonymity and prevent information bias. The assessor simply circled the observed performance description at each step of the procedure. The ICO-OSCAR score was completed. Observations of each resident were recorded.

At the end of each case assessor immediately discussed case with student to provide timely, structured, specific performance feedback. Surgical performances were recorded and their assessment were done by different faculties with same tool. Post-operative patient’s record were reviewed for outcome of surgery and complications if any. Rubric Score were analysed with inter rater agreement. Post assessment feedback was collected from faculties as well as resident about the assessment technique. Patients were assessed in post-operative period for visual recovery and any other complications. The recorded Data was entered in Master Chart on Word Excel document.

Result

Research analysis requirement: statistical analysis (descriptive and inferential), graphical representation Specific statistical test applied: kappa statistics Statistical packages used: SPPS and graph pad.

Final Master Table: [Table 2] Interclass correlation coefficient on chronbachs alpha analysis shows is 0.961 in 95% confidence interval it shows there was a very good agreement in all 3 raters.

Two-way random effects model where both people effects and measures effects are random.

Discussion

Continuous professional development relies on the link between performance and an educational process aimed at improving knowledge and skill, one of the most broadly used frameworks for assessing skills is Miller’s Pyramid. This Pyramid has a series of levels of achievement beginning with knowledge (at the base) and ending with routine application in the clinical setting.^[9] Intraoperative evaluation is the gold standard. Intraoperative assessment achieves highest level evaluate ‘DOES’ in Miller’s pyramid. In a study of the Objective Structured Assessment of Technical Skills, Goff et al. (2002) had two faculty members complete checklists to rate the performance of OB/GYN trainees; ratings increased with amount of training.^[10]

While standardized patient examinations have been successfully implemented in medical education, certification, and licensure, they cannot be used to assess all aspects of competence.^[7], ^[11]

In our study we have seen it has very good inter rater agreement similar findings were noted by Golnic KC et al.^[8], ^[12]

In our studies we have taken descriptive feedback on this assessment method by students as well as assessor and we found out it is reliable tool, adding educational value, it is feasible and acceptable by all.

As it is time tested, skills sharpen with multiple attempts or practice. Similarly if multiple assessment with constructive & time specific feedback is given it will lead to better surgical performance. Hence multiple assessment improves abilities of trainee in the surgical performance. This type of assessment tool serves three distinct purposes.^[13], ^[14] It minimizes subjectivity by precise instruction, the rubric states what is expected of the trainee to attain competence at each level. Self-Assessment by reflective practice is possible when the trainee’s surgical procedures are recorded. This ICO-Oscar tool was easy to use for assessor as well as learner. ICO-Oscar Tool in our study has been precise and valid for PG student’s surgical assessment. This tool can be used by Training institutes as bench mark for trainers to decide competency and necessary certification for Small Incision Cataract Surgery proficiency.

Conclusion

Good quality assessment not only satisfies the needs of accreditation but also contributes to student’s learning. Assessment methods should match the competencies being learnt and the teaching formats being used.

Source of Funding

None.

Conflict of Interest

The authors declare no conflict of interest.