Principal Research Scientist - AI Scaling & Optimization

As a Principal Research Scientist – Scaling, you will lead a team of world-class researchers and engineers to advance the state of the art in large-scale machine learning, focusing on post-training, RL and inference efficiency, optimization, and scaling.

You will define and execute a research roadmap that advances the Databricks AI platform and delivers tangible improvements to how customers train, serve, and adapt LLMs at scale, working closely with product, data, and engineering leaders to bring cutting-edge methods into production.

Key responsibilities:

• Lead and grow a multidisciplinary research team focused on foundational and applied AI problems, with a particular emphasis on LLM scaling, efficiency, and systems performance.
• Define the scaling research roadmap in alignment with Databricks’ strategic objectives, prioritizing advances in foundation model efficiency and large-scale training and inference.
• Drive algorithmic innovations for large-scale neural network training and inference, including novel optimizers, low-precision techniques, and model adaptation methods.
• Optimize end-to-end ML systems for distributed training and RL, memory efficiency, and compute efficiency through close collaboration with core systems and platform teams.
• Partner with product and engineering to translate research breakthroughs into customer-impacting capabilities in the Databricks AI platform.

Requirements:

• Proven ability to lead a research team to develop novel techniques for foundation model efficiency and related topics, with a strong track record of industry impact.
• Deep expertise in at least one of: generative AI, LLMs, distributed ML systems, model optimization, or responsible AI, with a strong emphasis on scaling and efficiency for large-scale neural networks.
• Hands-on leadership - strong programming skills and demonstrated ability to write high-quality, efficient code in Python and PyTorch for research implementation and experimentation.

Nice to have:

• Prior work at the intersection of systems and ML, such as distributed training frameworks, compiler and kernel optimization for deep learning workloads, or memory-/compute-efficient model design.
• Strong industry and academic network in large-scale ML, with ongoing collaborations or service at top conferences in ML and systems.