# Data Engineer 4, Real-time Data Platform

**Company**: Capital One
**Location**: Plano, TX
**Experience**: senior
**Job type**: full-time
**Salary**: $179,400 - $204,700
**Category**: Engineering
**Industry**: Finance

**Apply**: https://capitalone.wd12.myworkdayjobs.com/en-US/Capital_One/job/Plano-TX/Data-Engineer-4--Real-time-Data-Platform_R1003879-1?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_4cbf473f-b5a

## Description

We are seeking experienced Data Engineers to join our team, who are passionate about marrying data with emerging technologies. As a Capital One Data Engineer, you'll have the opportunity to be on the forefront of driving a major transformation within Capital One.

**Responsibilities:**

- Lead design and build Enterprise Level scalable, low-latency, fault-tolerant streaming data platform that provides meaningful and timely insights

- Build the next generation Distributed Streaming Data Pipelines and Analytics Data Stores using streaming frameworks (Flink, Spark Streaming) using programming languages like Java, Scala, Python

- Lead a group of engineers building data pipelines using big data technologies (Spark, Flink, Kafka, Snowflake, AWS Big Data Services, Snowflake, Redshift) on medium to large scale datasets

- Influence best practices for Data Pipeline design, Data architecture and processing of structured and unstructured data

- Work in a creative & collaborative environment driven by agile methodologies with focus on CI/CD, Application Resiliency Standards, and partnership with Cyber & Security teams

- Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community

- Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies

- Work with a team of developers with deep experience in machine learning, distributed microservices, and full stack systems

- Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment

- Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance

**Basic Qualifications:**

- Bachelor's Degree or higher in Computer Science or a related quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Engineering)

- At least 4 years of experience in application development

- At least 2 years of experience in distributed data

- At least 2 years of experience with SQL

- At least 2 years of experience with one programming language (Python, Java, or Scala)

- At least 2 years of experience in data pipeline design and development

- At least 1 year of experience in data modeling and designing end-to-end data solutions using both relational and non-relational database systems

**Preferred Qualifications:**

- 7+ years of experience in application development with demonstrated proficiency in Python, SQL, Scala, or Java

- 4+ years of hands-on experience designing, deploying and operating data workloads in at least one public cloud environment (AWS, Microsoft Azure, or Google Cloud)

- 4+ years of experience building or supporting distributed data or compute workloads using tools such as EMR, Spark, Glue, or Databricks

- 4+ years of experience designing, implementing, and operating real-time or streaming data pipelines

- 2+ years of experience working on data observability or data orchestration tools

- 4+ years of experience working with unstructured or semi-structured data using NoSQL databases

- 4+ years of experience designing and supporting data warehousing solutions

- 2+ years of experience working in an Agile development environment

- 2+ years of experience developing user-centric reusable data products

**Benefits:**

- Comprehensive, competitive, and inclusive set of health, financial and other benefits that support your total well-being

## Skills

### Required
- Java
- Scala
- Python
- SQL
- Spark
- Flink
- Kafka
- Snowflake
- Redshift
- AWS Big Data Services

### Nice to have
- EMR
- Glue
- Databricks
- Airflow
- Dagster
- Monte Carlo
- Splunk
- MongoDB
- Cassandra
- DynamoDB

---

Source: [Apply at capitalone.wd12.myworkdayjobs.com](https://capitalone.wd12.myworkdayjobs.com/en-US/Capital_One/job/Plano-TX/Data-Engineer-4--Real-time-Data-Platform_R1003879-1?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)