EVALUATIONS
Consistent with ACGME guidelines our program is designed to train residents to gain competencies in the following six general areas:
- Patient Care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health.
- Medical Knowledge about established and evolving biomedical, clinical and cognitive (e.g. epidemiological and social-behavioral) sciences and the application of this knowledge to patient care.
- Practice-Based Learning and Improvement that involves investigation and evaluation of their own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care.
- Interpersonal and Communication Skills that result in effective information exchange and teaming with patients, their families, and other health professionals.
- Professionalism , as manifested through a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population.
- Systems-Based Practice, as manifested by actions that demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value.
The residency program assesses resident competencies throughout the training and utilizes the assessment results to improve resident performance. Assessment methods include written and oral examinations, regular written evaluations by supervisors and key staff, portfolios, 360-degree feedback, and regular record audits. The program provides regular and timely performance feedback to residents, thus achieving progressive improvements in residents’ competence and performance. We also regularly assess the residency program and use resident performance, outcome assessments and various program evaluation methods to improve the residency program.
Resident Evaluations:
Individual and clinic supervisors are asked to provide written evaluations of child and adolescent psychiatry residents every six months. These evaluations provide a vehicle through which child psychiatry residents obtain vital feedback about their strengths and weaknesses as developing child and adolescent psychiatrists. They also provide the training directors with helpful information about each of the residents, their participation in and development during their training, and the quality of the training program.
This information is supplemented by residents’ monthly patient logs, which detail all of the cases the child psychiatry residents see each week. These logs are turned into the training office on a monthly basis. Every 6 months the child and adolescent psychiatry residents meet one-on-one with the training directors for their reviews. During this time, the residents and training directors discuss the residents’ progress in the program, the evaluations provided by their supervisors, any needed remediation for the remainder of their training and early career goals. The supervisory evaluations provide the residents and the training directors with a detailed and topic-specific evaluation of the residents’ competencies 1 through 6. Every 6 months, the Residency Training Committee will review each trainee’s progress and make recommendations for advancement in the program or for any needed remediation. The results of this review will be integrated into the training director’s summary evaluation letter.
Resident Evaluation Methods :
Written Evaluations By Faculty/Supervisors:
The faculty involved in the training and supervision of residents evaluate the residents they supervised every six months. The evaluation forms used for this purpose assesses resident skills in all the six competencies. The faculty identifies strengths and weaknesses of the supervised resident and suggests a remediation plan if needed. The training directors review these evaluations with residents in their six-monthly meetings.
Resident Portfolios:
A portfolio is a collection of products prepared by the resident that provides evidence of learning and achievement related to learning plan. Portfolios are useful for evaluating mastery of competencies that are difficult to evaluate in other ways such as practice-based improvement, use of scientific evidence in patient care, professional behaviors and patient advocacy. Portfolio allows credit for some activities that otherwise would have gone un-noticed. The following items have been included in the portfolios for residents in Child & Adolescent Psychiatry at the University of Florida covering the two year training period:
- Patient Logs: It is imperative that all residents keep an ongoing record of the cases in which they have participated. This record is essential for the individual’s ability to take the Boards and for the program to remain accredited by the RRC.
- Supervision Logs
- Two videotaped interviews/presentations.
- List of 20 research articles utilized in patient care and a brief summary of each article
- List of 2 ethical dilemmas and how they were handled
- Description of your quality improvement plan and how this plan will improve your performance as a resident
- Summary of activities demonstrating patient advocacy
- Four complete psychiatric evaluations including biopsychosocial formulations and treatment plans
- Summary of what has been learned during this academic year and perceived areas of strength and weakness
- Copy of one referral letter sent to another physician
- Summary of two cases presented in the case conferences along with a list of five research articles used for each case.
Written and Oral Examinations :
All child and adolescent psychiatry residents participate in the annual nationally administered Child/Adolescent Psychiatric Residents-in-Training Examination (Child PRITE), which is modeled after Part I of the ABPN. All PGY4s also take the General PRITE exam. PGY-5 and 6 trainees have the option of not taking the General PRITE.
Oral Board Examinations (modeled after Part II of the ABPN) are required for completion of the Child and Adolescent Psychiatry Training Program and are conducted on yearly basis. First- and second-year child trainees complete two types of examinations, one based on a videotape interview and one based on written vignettes. Furthermore, during their inpatient psychiatry rotation residents complete an Oral Board Examination style adolescent interview administered by two faculty members. The PRITE Exams and the Oral Exams provide the residents and the training directors with a detailed and topic-specific evaluation of the residents’ knowledge, clinical skills, and competencies 1 through 6.
Record Reviews:
The Division conducts regular chart audits to assure compliance with required documentation standards. Residents’ record keeping practices are thus routinely reviewed in a standardized manner, specifically assessing competencies 1 and 5.
Patient Satisfaction Surveys:
Clinic patients and their families are asked to provide satisfaction assessments. This provides feedback on several areas of resident competencies, especially competencies 1, 4 and 5. These results of these surveys are provided to the training directors who integrate them into the 6-monthly feedback session with individual residents.
360-Degree Evaluations:
Key divisional and faculty group practice staff are asked to provide regular evaluations of the trainees’ professionalism. Staff provides feedback on trainees’ availability, promptness in returning pages and phone calls, courteous behavior with patients and families, sensitivity to cultural variations, waiting room behavior, interactions with staff, and adherence to required record keeping procedures. These evaluations are provided to the training directors who integrate them into the 6-monthly feedback session to specifically address competency 5.
Promotion of Residents:
Promotion to the next level of training is determined by the faculty’s assessment of the resident’s ability to assume the responsibilities of the new level. This is based in part on the resident’s demonstrated achievement of the milestones listed in the goals and objectives for each rotation and each year, in part on the resident’s performance on the Child Psychiatry Residency In-Training examination, and in part on the overall assessment of the resident’s progress in relation to the expected core competencies at any given year level. Promotion and retention are also dependent on continued appropriate moral, ethical and professional conduct by the resident.
Program Evaluations :
General Program Assessment Methods:
Program evaluations are conducted in various formats and time frames. Ongoing verbal feedback is obtained in the monthly meeting of trainees and training directors. Furthermore, individual verbal feedback is ascertained from each trainee during their six-monthly feedback meetings with the training directors. In addition, regular opportunities for regular verbal feedback are built into the program. For example, the child psychiatry residents meet with the chief child psychiatry residents each month to discuss any upcoming events, changes and to troubleshoot any issues of concern. The chief residents discuss these issues with the training directors at their meetings. Furthermore, to insure that child and adolescent psychiatry residents have the opportunity to anonymously evaluate each of their instructors, supervisors, courses, and rotations, the training program obtains annual online evaluations from all trainees. This provides the training program with valuable information about its didactic and clinical components, and its teaching staff. The results of the evaluations get tabulated and reviewed in several formats. The training director provides a personal evaluation summary to each faculty member. Furthermore, a general review of the evaluation results occurs during the Residency Training Committee meeting, and plans for program improvement are developed. Another avenue for evaluation, feedback and program improvement is the annual retreat
Annual Educational Retreat
The annual retreat provides another opportunity for the residents and faculty to review the program, make improvement plans and get to know more about each other. In addition to combined discussions, the residents and faculty meet in separate groups as well. The training directors update the faculty and residents about any upcoming educational issues. The chief residents prepare written feedback from residents to aid with continuous program improvement.
