Description
Thomson Reuters is seeking a Senior Inference Engineer, AI to collaborate with platform teams to enhance capacity forecasting for AI workloads and work with Product, Data Science, Architecture, and Enterprise AI teams to onboard new research models into production.
As a Senior Inference Engineer, AI, you will be responsible for productionizing, optimizing, and scaling AI and LLM workloads that power TR's AI-driven products. Your tasks will include:
- Optimizing LLMs and ML models for high-performance inference using techniques such as quantization, pruning, distillation, and hardware-specific tuning
- Deploying and scaling inference workloads on GPUs across AWS, Azure, GCP, and internal Kubernetes clusters, ensuring predictable performance during peak traffic hours
- Implementing routing and failover strategies for OpenAI/Anthropic/Vertex AI traffic
- Integrating models into production-grade APIs supporting TR products and enterprise workflows
- Developing highly optimized environments and eliminating performance bottlenecks to reduce latency
- Collaborating with Platform Engineering teams to ensure inference workloads align with TR's cloud-native patterns
- Building and optimizing containerized inference pipelines using Kubernetes for large-scale distributed workloads
- Ensuring compliance with TR's AI standards for deployment, monitoring, governance, and drift detection
- Profiling inference performance, identifying GPU/CPU bottlenecks, and optimizing compute utilization across heterogeneous hardware
- Implementing observability and health monitoring for inference pipelines, ensuring reliability of enterprise AI services
- Collaborating with AI engineers to invent new quantization techniques, improve numerical precision, and explore non-standard architectures
- Partnering with Cloud Engineers to develop guardrails and automation that support inference workloads
To be a potential fit for this role, you should have:
- 5+ years of relevant experience
- Strong understanding of ML/LLM fundamentals and inference optimization techniques
- Hands-on experience with GPU programming (CUDA preferred), inference runtimes (TensorRT, ONNX Runtime), and deep learning frameworks (PyTorch/TensorFlow)
- Proficiency in Python and at least one systems language (C++ strongly preferred for performance-critical inference paths)
- Experience deploying AI workloads to AWS/GCP/Azure and Kubernetes
- Familiarity with vector search systems (OpenSearch vectors) and retrieval-augmented generation pipelines
- Knowledge of distributed systems, microservices, CI/CD, and cloud-native architecture
In terms of benefits, you can expect:
- Hybrid work model with flexible working environment
- Flexibility and work-life balance with supportive workplace policies
- Career development and growth opportunities
- Industry-competitive benefits, including comprehensive benefit plans, retirement savings, tuition reimbursement, and employee incentive programs
- Access to mental, physical, and financial wellbeing resources
- Opportunities for social impact and community involvement
The base compensation range for this role is $110,000 USD - $204,200 USD for eligible US locations, and $100,000 CAD - $145,000 CAD for Ontario, Canada.