Description
Electronic Arts creates incredible entertainment experiences that inspire players and fans worldwide. Here, everyone is part of the story, part of a community that connects people globally. As a Principal ML Engineer, Training Pipelines, you will own the systems that prepare and deliver multimodal game data for large-scale model training.
You will architect and build shared training input pipelines from certified game capture to accelerators, covering storage layout, decoding, sharding, sampling, packing, batching, caching, prefetching, and host-to-device delivery across multiple training frameworks.
Your responsibilities will initially focus on hands-on individual contribution and technical leadership, with potential to take on people-management responsibilities as the team grows. You will partner with researchers and central data and compute teams to turn evolving model requirements into reusable training capabilities.
Key responsibilities:
- Architect and build shared training input pipelines
- Define schemas, data contracts, encodings, compression, and standards for dataset versioning
- Build deterministic loaders and checkpoint-safe iteration
- Design reproducible dataset mixtures and balanced batches
- Benchmark and enforce throughput, data-wait time, step time, and accelerator utilization
- Diagnose bottlenecks and intermittent failures
- Integrate compilers, runtimes, optimized kernels, low-precision training, and hardware-accelerated media processing
- Lead training-pipeline architecture reviews, mentor engineers, and align priorities with research leads
Qualifications:
- 20+ years of development experience, including owning production systems for distributed-training input
- Expertise building custom PyTorch training and ingestion systems or adapting TorchTitan, JAX/XLA with MaxText, Megatron Core/NeMo
- Expertise in FSDP2/HSDP, tensor/context/pipeline parallelism, mixed-precision training, and optimization on one or more accelerator platforms
- Pre-training experience and an understanding of how model architecture, data representation, batching, and orchestration affect training-system design and efficiency
- Expertise building large-scale multimodal streaming or hardware-accelerated ingestion
- Deep knowledge of sharding, sampling, packing, caching, prefetching, deterministic data mixing, checkpoint-safe iteration, and elastic restart
Benefits:
- Comprehensive benefit packages, including physical, emotional, financial, professional, and community well-being support
- Personalized packages to meet local needs, including medical insurance, mental wellness support, pension plans, paid time off, family leaves, free games, and more