Location: Hybrid 

Department: Assessment 

Reports to: Director, Test Development 

Exemption Status: Exempt 

Job Summary:  

The Manager of Test Development serves three major roles: provides leadership and supervision for the Test Development Specialists within the Assessment Department; oversees and takes the lead on the planning and operational aspects of test development projects; serves as the primary Test Development Specialist for two of the subspecialty examination programs and backup Test Development Specialist for multiple examination programs. As a supervising manager, this position is accountable for the accurate and timely development of over 40 high-stakes examinations: 16 in-training examinations, 8-9 initial certification examinations, and 16 longitudinal assessments each year. Additionally, this position is responsible for managing test development project plans, defining and tracking tasks, assessing work efforts, providing feedback, and monitoring department goals. This position is also responsible for general supervisory duties such as approving timesheets, conducting performance reviews, and providing mentorship. 

Essential Duties and Responsibilities:  

• Manage and provide leadership to a team of Test Development Specialists, including but not limited to completing performance reviews, approving timesheets, training, prioritizing projects, and overseeing general workflow.
• Mentor all Test Development Specialists in industry best practices, assist with their professional development, and ensure redundancy within the department.
• Identify opportunities for innovation and process improvement within test development operations, proactively designing and implementing enhancements to increase efficiency, quality, and scalability
• Foster a culture of collaboration and continuous improvement through team-based workflows.

• Lead and manage multiple concurrent and overlapping test development projects, partnering with department members, the psychometric team, and other business units to ensure time completion.
• Lead and participate in the implementation of strategic initiatives of the ABP related to testing.
• Build and maintain strong relationships with committee members and staff to achieve desired results.
• Serve as program manager and test development specialist for two assigned examination programs, including but not limited to managing the item bank, determining content needs, building examination forms, leading the review of forms, building exam delivery packages, transferring examinations to delivery vendors, and archiving examination content.
• Develop, update, and coordinate assignments and project schedules with staff and committee members for assigned examination programs.
• Facilitate annual in-person examination committee meetings and remote meetings throughout the year for assigned examination programs. 
• Oversee the creation and maintenance of standard policy and procedure documents relating to test development activities. 

Other Functions: 

• Serve as the primary liaison for test development vendors (eg, item banking, assessment delivery).
• Serve as the backup Test Development Specialist for assigned examination programs.
• Ensure test development processes and policies are documented and in compliance with industry best practice. 
• Utilize emerging technologies, such as artificial intelligence, to increase efficiency and productivity.
• Contribute test development expertise to the department and committee members.
• Participate in testing research as needed.
• Provide technical support for staff and subject-matter experts for testing applications.
• Support the recruitment efforts of subject-matter experts.
• Represent the ABP when making presentations at industry events (eg, ATP, ICE, ABMS Conference) and when interacting with individuals from other organizations.
• Initiate, lead, and participate in special projects outside of standard role and responsibilities.
• Performs other duties as assigned.

Supervisory Responsibilities: 

This role supervises a team of Test Development Specialists, currently three. 

Required Qualifications: 

• Four-year college degree and at least 7 years of professional testing experience or related experience, masters preferred but relevant experience may be considered in its place.
• Demonstrated experience managing or leading personnel and providing feedback
• Demonstrated experience planning, maintaining, managing, and monitoring overlapping projects with ability to independently, efficiently, and proactively implement mission-critical activities
• Strong facilitation skills with the ability to lead meetings, guide group discussions, and foster effective collaboration among diverse stakeholders
• Experience working with external vendors, including managing contracts, service-level agreements, and vendor performance to ensure high-quality deliverables and service
• Knowledge of change management principles
• Ability to problem solve, using logic and reasoning to identify the strengths and weaknesses of alternative solutions, conclusions, or approaches
• Ability to adopt and adapt to new processes and technologies
• Willingness to take initiative and strive for continuous improvement
• Ability to manage, collaborate with, and develop strong relationships with highly skilled and well-educated staff as well as physician subject matter experts
• Ability to be creative and innovative
• Ability to apply and incorporate the testing portions of national accreditation standards such as the National Commission for Certifying Agencies (NCCA) Standards or ISO 17024 Standards
• Proficiency with or ability to learn item banking software (eg, ExamDeveloper) and project management software (eg, Wrike)
• Proficiency with Microsoft 365 (eg, Excel, Word, PowerPoint, SharePoint)

Preferred Qualifications: 

• Advanced degree education, measurement, psychology, statistics, or related field
• Experience in the examination development industry with large-scale, high stakes examinations
• Formal project management experience

• Demonstrated ability to incorporate change management principles resulting in positive results
• Excellent writing skills, impeccable grammar, and attention to detail when revising and proofreading
• Advanced ability with item banking software (eg, ExamDeveloper) and project management software (eg, Wrike)
• Advanced ability with Microsoft 365 (eg, Excel, Word, PowerPoint, SharePoint)

Travel Required 

Travel is expected to be between 0-5%. 

Work Environment: 

The noise level in the work environment is usually moderate. 

The setting for this role will include an office setting, either at location or the home environment. Exposure to controlled temperature environments, both hot and cold, is to be expected. 

This position will involve a combination of in-office and remote work. Remote work will be available on Mondays and Fridays, with on-campus presence required from Tuesday through Thursday, beginning June 3rd.

Physical Demands: 

While performing the duties of this job, the employee is regularly required to communicate with others. The employee is frequently required to stand, walk, sit, demonstrate hand and finger dexterity in the course of their work, handle or feel, and reach with hands and arms.  

Reasonable accommodations may be made to enable individuals with disabilities to perform the essential duties and responsibilities. 

The expected physical workload is expected to be: 

Sedentary Work: Exerting up to 10 pounds of force occasionally and/or a negligible amount of force frequently or constantly to lift, carry, push, pull or otherwise move objects, including the human body. Sedentary work involves sitting most of the time. Jobs are sedentary if walking and standing are required only occasionally and all other sedentary criteria are met.  

This role is expected to use the following machines, tools, and equipment: 

• Laptop computer, external monitors, keyboard, mouse, printer, meeting room AV systems.