Data Engineer

We are seeking a Data Engineer to join our team at S&P Dow Jones Indices. The successful candidate will partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform, provide hands-on engineering support, implement repeatable engineering patterns, and support secure AWS deployment patterns.

Key Responsibilities:

• Partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform.
• Provide hands-on engineering support to stabilize and optimize pipelines post-transition, focusing on performance, scalability, maintainability, and operational reliability.
• Implement and follow repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring.
• Support secure AWS deployment patterns and governed data access using AWS IAM, encryption, and platform guardrails.
• Work with lakehouse technologies such as Delta Lake, Apache Iceberg, and Databricks Unity Catalog, and support metadata-driven pipeline design.
• Assist with establishing and maintaining observability practices (logging, telemetry, alerts, dashboards) for production pipelines.
• Support asset-agnostic onboarding by applying standard ingestion and transformation frameworks to new datasets.
• Help offshore teams adopt common onboarding standards, delivery practices, and technical templates.
• Contribute to reusable technical assets such as reference implementations, onboarding templates, documentation, and operational runbooks.
• Support both batch and streaming onboarding patterns using AWS services (e.g., AWS Glue, AWS Lambda, Amazon Kinesis) or event-driven integrations as appropriate.
• Support integration of platform datasets with the enterprise data mastering platform.
• Help establish and/or maintain data quality controls, reconciliation, and metadata alignment for trusted mastered data.
• Provide technical guidance to engineering peers through code reviews, design reviews, and implementation planning.
• Contribute to engineering standards and delivery rigor across CI/CD, infrastructure automation, testing practices, and operational readiness.
• Use AI tools (Claude, GitHub CoPilot, LLMs) to accelerate engineering tasks such as drafting code, refactoring, writing documentation, generating unit tests, and troubleshooting.

Requirements:

• 5–7+ years of experience in data engineering, data platforms, or related technical domains.
• Hands-on experience building and operating batch and/or streaming data pipelines.
• Experience working with Databricks-based platforms and lakehouse architectures.
• Strong familiarity with AWS data engineering services such as Amazon S3, AWS Glue, AWS Lambda, AWS Lake Formation, Amazon Kinesis, and AWS IAM.
• Working knowledge of Delta Lake, Apache Iceberg, and Databricks Unity Catalog (or equivalent governed lakehouse concepts).
• Solid understanding of data quality, schema management, lineage/metadata basics, access control, encryption, and operational support.
• Ability to provide technical guidance without people management responsibility.
• Experience collaborating with global teams across time zones and working through ambiguous delivery problems.
• Strong communication skills and ability to translate technical requirements into implementable tasks.
• Practical experience using AI-assisted engineering tools such as Claude, GitHub CoPilot, or LLM-based development features.

Benefits:

• Health & Wellness: Health care coverage designed for the mind and body.
• Flexible Downtime: Generous time off helps keep you energized for your time on.
• Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
• Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
• Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
• Beyond the Basics: From retail discounts to referral incentive awards,small perks can make a big difference.