Description
The Behavior AI team at Datadog builds AI-based anomaly detection models that learn what normal looks like across billions of logs, events, and telemetry records flowing through the platform every second. These models flag behavior that does not fit, on every record, in real time, at a cost that makes sense at our scale.
We are hiring a Senior Applied Scientist to build these models from start to finish. You will contribute to designing the architecture, training at scale, and optimizing work that takes a model from training to running efficiently on production traffic.
Responsibilities:
- Design and build custom mid-size models for high-throughput stream processing, and train them at scale.
- Optimize these models from start to finish, working across both the mathematics and the systems.
- Build the training data pipelines the work depends on when they do not already exist.
- Work with engineering to integrate models into production, with a clear focus on GPU utilization, latency, and cost per record on live traffic.
- Plan the roadmap of model and system improvements.
- Build an agentic layer on top of the models to analyze, validate, and act on their outputs.
- Build lightweight interpretability tools that make model behavior easier to explain.
- Maintain and monitor the models, services, and infrastructure your team owns.
Requirements:
- BS/MS/PhD in Computer Science, Engineering, Machine Learning, Applied Mathematics, or a related scientific field.
- Hands-on experience training and fine-tuning models at scale, and deploying them into production systems.
- Working knowledge of how GPUs work, and a track record of making models run efficiently within real hardware constraints.
- Strong applied-mathematics fundamentals.
- Passion for applied mathematics, software design, and implementation.
- Experience with efficient sequence architectures, model interpretability, or large-scale streaming systems is a bonus.
Benefits:
- New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
- Continuous professional development, product training, and career pathing
- Opportunity to attend and present at conferences and meetups
- Intra-departmental mentor and buddy program
- Inclusive company culture
- Competitive global benefits