# AI Data Engineer

**Company**: S&P Global
**Location**: New York, NY
**Work arrangement**: hybrid
**Experience**: senior
**Job type**: full-time
**Salary**: $90,000 USD – $150,000 USD
**Category**: Engineering
**Industry**: Finance

**Apply**: https://spgi.wd5.myworkdayjobs.com/en-US/SPGI_Careers/job/New-York-NY/Data-Scientist_332760-1?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/job/ai-data-engineer-e029affc-713/

## Description

The AI Data Engineer will specialize in building and optimizing machine learning data pipelines, focusing on AI model tracking, lifecycle management, and integration with AI governance systems.

**Key Responsibilities:**

- Design and implement specialized data pipelines for AI model metadata, training data lineage, and model performance metrics tracking.

- Build data infrastructure on Databricks leveraging Spark for large-scale distributed dataset processing.

- Develop MCP servers and enable AI data distribution via MCP.

- Develop feature engineering pipelines and data preprocessing workflows for AI model training and inference.

- Implement model versioning, experiment tracking, and model registry integration using MLflow or similar tools.

- Create automated workflows for AI agent discovery, classification, and inventory management across the enterprise.

- Design and maintain knowledge graph structures for representing AI model relationships, dependencies, and data lineage.

- Build real-time data pipelines for AI model monitoring, drift detection, and performance tracking.

- Develop data quality frameworks specific to AI training datasets and validation data.

- Collaborate with data scientists to optimize data access patterns and feature store implementations.

- Implement security and compliance controls for sensitive AI training data and model artifacts.

- Create comprehensive documentation for AI data architectures, schemas, and integration patterns.

**Required Skills and Qualifications:**

- 5+ years of hands-on experience in data engineering, with at least 2 years focused on AI/ML workloads.

- Expert proficiency in Python and experience with ML frameworks such as PyTorch, TensorFlow, or scikit-learn.

- Strong experience with Databricks, Apache Spark, and distributed computing for ML workflows.

- Deep understanding of the machine learning lifecycle, including model training, deployment, and monitoring processes.

- Experience with feature engineering, data preprocessing techniques, and ML data pipelines.

- Knowledge of vector databases, embeddings, and similarity search for AI applications.

- Proficiency in SQL for structured and unstructured data management.

- Understanding of data governance, model governance, and AI ethics principles.

- Strong analytical and problem-solving capabilities with attention to data quality.

- Excellent collaboration skills for working with data scientists, ML engineers, and architects.

- Bachelor's or Master's degree in Computer Science, Data Science, Machine Learning, or related field.

**Preferred/Nice-to-Have Skills:**

- Experience with generative AI applications, including RAG (Retrieval-Augmented Generation) and fine-tuning.

- Knowledge of LangChain, HuggingFace, or other GenAI frameworks.

- Familiarity with Azure ML, AWS SageMaker, or Google Vertex AI platforms.

- Experience with graph databases (Neo4j, Amazon Neptune) for knowledge graph implementation.

- Understanding of AI model explainability and interpretability techniques.

- Experience with A/B testing frameworks for ML model evaluation.

- Certification in Databricks, AWS, Azure, or GCP AI/ML services.

- Publications or contributions to open-source ML projects.

**Compensation/Benefits Information:** The anticipated base salary range for this position is $90,000 USD – $150,000 USD.

**Benefits:**

- Health & Wellness: Health care coverage designed for the mind and body.

- Flexible Downtime: Generous time off helps keep you energized for your time on.

- Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.

- Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.

- Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in-class benefits for families.

- Beyond the Basics: From retail discounts to referral incentive awards,small perks can make a big difference.

For more fresh new jobs, check out [yubhub.co](https://yubhub.co/).

## Skills

### Required
- Python
- Databricks
- Apache Spark
- MLflow
- PyTorch
- TensorFlow
- scikit-learn
- SQL

### Nice to have
- LangChain
- HuggingFace
- Azure ML
- AWS SageMaker
- Google Vertex AI
- graph databases
- Neo4j
- Amazon Neptune

---

Source: [Apply at spgi.wd5.myworkdayjobs.com](https://spgi.wd5.myworkdayjobs.com/en-US/SPGI_Careers/job/New-York-NY/Data-Scientist_332760-1?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)