Staff Machine Learning Scientist

Chicago, NYC, SF, Remote

Focus: This role will contribute to the fundamental architectural design of large multimodal models (LMMs) and the development of the scalable infrastructure required for their training.

Are you driven by the potential of artificial intelligence to revolutionize healthcare?

Recent technological breakthroughs have paved the way for AI to make a significant impact on clinical care. Our partner is at the forefront of this revolution, operating a unique platform that integrates a vast ecosystem of real-world evidence. This platform delivers timely, actionable insights to medical professionals, empowering them with crucial information to guide treatment decisions and ensure patients receive the most appropriate care.

Responsibilities: As a key member of the team, you will be involved in:

• Designing and defining the core architecture of LMMs, exploring various fusion strategies and modality-specific processing techniques.
• Implementing, refining, benchmarking, and optimizing model architectures utilizing deep learning frameworks such as PyTorch or TensorFlow.
• Developing and managing comprehensive end-to-end training pipelines, encompassing data loading, preprocessing, and model training. Architecting and deploying distributed training workflows, optimizing performance across cloud GPU resources.
• Implementing distributed training methodologies to effectively handle extensive datasets and complex models.
• Designing and implementing approaches to integrate knowledge with multimodal representations within the LMM.
• Experimenting with diverse strategies to enhance the model's comprehension and reasoning capabilities through knowledge integration.
• Monitoring and debugging training processes, proactively identifying and resolving performance limitations.
• Collaborating closely with the knowledge integration engineer to ensure the architectural design seamlessly supports knowledge injection mechanisms.

Skills and Experience:

• In-depth understanding of deep learning principles and architectures, with a strong emphasis on transformer networks.
• Significant experience with multimodal machine learning concepts and methodologies, including various fusion techniques for text and images. A solid grasp of optimization strategies for large-scale models is essential.
• Strong proficiency in Python and deep learning frameworks (PyTorch/TensorFlow), along with experience using model management libraries like Hugging Face Transformers.
• Demonstrated experience in training large multimodal models utilizing distributed training frameworks (e.g., Horovod, MosaicML) and managing GPU resources in a cloud environment.
• A strong understanding of knowledge representation concepts, such as knowledge graphs and ontologies.
• Experience with distributed training frameworks and cloud computing platforms (e.g., GCP, Azure).