Research Engineer / Research Scientist, RL Frontiers

Anthropic's mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole.

The RL Scaling team works on how RL scales: what happens to throughput, stability, and learning efficiency as models get larger, episodes get longer, and compute grows by orders of magnitude. You will develop next-generation architectures and RL algorithms, take them from small-scale results to frontier-scale runs, and understand every place they behave differently.

Key Responsibilities:

• Study how RL training and sampling scale with model size, context length, and compute.
• Develop next-generation model architectures and RL algorithms.
• Take promising small-scale results to frontier-scale runs and diagnose issues.
• Build experimental infrastructure for research velocity.
• Own end-to-end performance of large RL runs.
• Build performance and cost models for proposed changes.
• Investigate training dynamics at scale.

Minimum Qualifications:

• Deep familiarity with modern transformer language models.
• Hands-on experience training large models in a distributed setting.
• Track record of original technical work in ML training or systems.
• Ability to design rigorous experiments at scale.
• Strong programming skills in Python and JAX or PyTorch.

Preferred Qualifications:

• Research experience in reinforcement learning, optimization, or large-scale training.
• Experience developing RL algorithms for language models.
• Experience with scaling laws or quantitative models of training efficiency.
• Experience designing or modifying transformer architectures.
• Experience scaling training to large fleets of accelerators.

Representative Projects:

• Characterize how a new RL algorithm's throughput and learning efficiency change at scale.
• Develop a new attention variant and measure its quality and throughput.
• Prepare large-scale RL runs and fix issues.
• Trace loss instabilities to root cause.
• Build models predicting throughput and cost of proposed architecture changes.

The annual compensation range for this role is $500,000-$850,000 USD.