Principal Machine Learning Engineer, Accelerated Apache Spark

We are looking for a Principal Machine Learning Engineer to join our GPU accelerated Apache Spark team. As a Principal Machine Learning Engineer, you will design and implement machine learning solutions for performance prediction and optimization of GPU accelerated enterprise Apache Spark workloads.

Your responsibilities will include:

• Designing and implementing machine learning solutions for performance prediction and optimization of GPU accelerated enterprise Apache Spark workloads.
• Developing advanced algorithms and adaptive systems to continuously improve the performance of Apache Spark workloads on GPUs.
• Developing AI-based agents and tools to assist with fixing system issues and application optimization.
• Collaborating with key partners and customers on the deployment of complex machine learning solutions in various environments.
• Maintaining deep domain expertise by knowing the latest published advances in ML systems and algorithms.
• Providing technical mentorship and leadership in data science and machine learning to a team of engineers.

Requirements include:

• A Bachelor's degree in Machine Learning, Data Science, Computer Science or a closely related field.
• 12+ years of professional experience in designing, implementing, and productionizing high-quality ML/DL solutions.
• 5+ years of experience as a technical lead in ML model development.
• Proven hands-on experience with large-scale data processing platforms, such as Apache Spark.
• Proven ability to employ modern tooling and sound techniques for all aspects of crafting, deploying, and maintaining machine learning models.
• Excellent programming skills in Python and Python data science related libraries like NumPy, Pandas, Scikit-learn, SciPy, PyTorch, and TensorFlow.
• Deep experience with sophisticated ML methodologies, including LLM/GenAI, reinforcement learning, and adaptive, on-line ML systems.
• Strong expertise in feature engineering, feature importance assessment, and developing boosted tree model solutions (e.g., XGBoost).

Preferred qualifications include:

• Understanding of the internal workings and architecture related to Apache Spark.
• Familiarity with NVIDIA GPUs and CUDA.
• Experience coding in Scala, Java, and/or C++.

If you are passionate about machine learning and have a strong background in software development, we encourage you to apply for this exciting opportunity.