Program Evaluation Specialist

General summary/purpose:

The mission of the Office of Assessment and Evaluation (OAE) is to enhance the educational programs in the Johns Hopkins University School of Medicine (JHUSOM) through evidence-based best practices in the assessment of learners and the evaluation of programs. OAE partners with JHUSOM faculty and staff members, administrators, and learners to design, launch, and maintain assessment and evaluation initiatives in graduate biomedical education, postdoctoral education, undergraduate medical education, graduate medical education, and continuing medical education. OAE also supports educational activities initiated by the Simulation Center, the Institute for Excellence in Education, the Office of Student Affairs, the Office of Faculty Development, and the Professional Development and Career Office.

The position reports to the Director of the Office of Assessment and Evaluation, and works closely with the OAE team and other stakeholders, including faculty and staff.

Specific duties & responsibilities:

• Assist the OAE Director with strategic planning initiatives related to program evaluation
• Serve as lead OAE project manager on development of evaluation plans and design of comprehensive evaluation systems
• Serve as lead OAE writer on evaluation plans for grant proposals
• Serve as lead OAE project manager on development, implementation, and use of program evaluation instruments/data collection
• Use program evaluation models including Logic Models to build strategic program evaluation plans
• Manage process to determine data necessary for program evaluation; collect and/or oversee collection of such data for all SOM educational programs
• Support course revisions and curriculum planning from evidence-based program evaluation results
• Provide support to faculty for education research involving program evaluation
• Serve as an active member of various committees as needed
• Supervise staff and/or serve as team leader for program evaluation initiatives
• Advise faculty members, staff members, and learners in best practices related to program evaluation
• Assist the Director of OAE with systemic program evaluation process improvement strategies to advance day-to-day operations
• Advise faculty members, staff members, and learners in the interpretation of program evaluation reports to facilitate data-driven decision making and action planning
• Assist the Director of OAE with the production of recurring OAE newsletters
• Maintain professional and effective working relationships with JHUSOM educational program stakeholders and with partnering units, including the JHUSOM Office of Information Technology, Office of Medical Student Affairs, Office of Medical Student Admissions, Office of Graduate Biomedical Education, Office of Graduate Medical Education, Office of Postdoctoral Affairs, and Office of the Registrar
• Coordinate with the Precision Education Data Governance Specialist on the collection, management, and access of program evaluation data stored in a data lake environment
• Assist the OAE Director with annual program evaluation of the office

Consultation and Service

• Provide support to faculty members, staff members, and learners for medical and biomedical education research or scholarly activities, especially as they relate to program evaluation, quantitative and qualitative data analysis
• Assist OAE Director with the planning and conduct of professional development activities related to program evaluation

Other Program Evaluation Activities

• Assist the Director of OAE and other OAE team members, as appropriate, in facilitating program evaluation-related activities with faculty members, staff members, and learners (e.g., evaluation planning, survey design, focus group moderation, qualitative data analysis)
• Other duties as assigned

Minimum qualifications (mandatory):

• Master’s degree in Education, Evaluation, Education Psychology, Social Sciences or related field
• Four years related work experience, or PhD
• Experience should include a combination of the following;
• Direct experience with data systems and reporting software
• Experience in program evaluation activities in an educational setting
• Experience in the use of institutional data systems and reporting software
• Experience writing scholarly reports
• Experience using data to drive decision making and action planning with stakeholders

Preferred qualifications:

• Experience working with learners, faculty, and staff in a health sciences environment preferred.
• Knowledge of medical and biomedical education programs, and familiarity with the academic medicine environment, preferred
• Knowledge of learning management systems, including those that support life-long learning, preferred
• Knowledge of Logic Models to build program evaluation systems preferred

Special knowledge, skills, and abilities:

• Demonstrated ability to work independently and as a member of a team
• Demonstrated ability to work with a diverse group of stakeholders
• Demonstrated ability to prioritize tasks, manage time, and meet deadlines effectively
• Strong organizational skills and keen attention to detail
• Strong written and verbal communication skills
• Effective use of diplomacy, tact, and judgment
• Proficiency with the use of Qualtrics, Microsoft Word, Excel, and PowerPoint
• Intermediate to advanced skills in the use of statistical analysis software (e.g., SAS, SPSS)
• Knowledge of inferential statistical analyses, including independent and dependent samplest-test, chi-squared test, ANOVA, ANCOVA, Factor Analysis, simple and multiple linear regression, and simple and multiple logistic regression

• Strong data visualization skills
• Knowledge of standards and policies related to the MSCHE, LCME, ACGME, NIH, or other accrediting bodies and funding agencies, preferred

• Knowledge of systems related to program evaluation; data management, analysis, and reporting; and survey design and dissemination (e.g., Qualtrics, B-Line, Blackboard, New Innovations)

Classified Title:Monitoring & Evaluation Officer
Working Title: Program Evaluation Specialist
Role/Level/Range: ACRP/04/MF
Starting Salary Range:$70,805 - $97,435
Employee group: Full Time
Schedule: M-F 8:30am-5pm
Exempt Status: Exempt
Location: 04-MD:School of Medicine Campus
Department name: 10003320-SOM Admin Registrar General Admin
Personnel area: School of Medicine

The successful candidate(s) for this position will be subject to a pre-employment background check.

If you are interested in applying for employment with The Johns Hopkins University and require special assistance or accommodation during any part of the pre-employment process, please contact the HR Business Services Office atjhurecruitment@jhu.edu. For TTY users, call via Maryland Relay or dial 711.

The following additional provisions may apply depending on which campus you will work. Your recruiter will advise accordingly.

During the Influenza ("the flu") season, as a condition of employment, The Johns Hopkins Institutions require all employees who provide ongoing services to patients or work in patient care or clinical care areas to have an annual influenza vaccination or possess an approved medical or religious exception. Failure to meet this requirement may result in termination of employment.

The pre-employment physical for positions in clinical areas, laboratories, working with research subjects, or involving community contact requires documentation of immune status against Rubella (German measles), Rubeola (Measles), Mumps, Varicella (chickenpox), Hepatitis B and documentation of having received the Tdap (Tetanus, diphtheria, pertussis) vaccination. This may include documentation of having two (2) MMR vaccines; two (2) Varicella vaccines; or antibody status to these diseases from laboratory testing. Blood tests for immunities to these diseases are ordinarily included in the pre-employment physical exam except for those employees who provide results of blood tests or immunization documentation from their own health care providers. Any vaccinations required for these diseases will be given at no cost in our Occupational Health office.

Equal Opportunity Employer
Note: Job Postings are updated daily and remain online until filled.

EEO is the Law
Learn more:
https://www.eeoc.gov/sites/default/files/migrated_files/employers/poster_screen_reader_optimized.pdf