Description
We're seeking an Inference Infrastructure Architect to join our founding China team. As an Inference Infrastructure Architect, you will operate and expand our B300 fleet efficiently, maximizing useful inference throughput per GPU-dollar while meeting latency and reliability SLOs. You will also make the product run on it, including serverless inference for the open-weight catalog and dedicated, tuned model deployments for enterprises.
Responsibilities:
- Operate and expand the fleet efficiently
- Make the product run on it
- Serverless serving pools
- The fleet layer
- The platform under it
- Weight logistics and elasticity
- The dedicated tier
- Observability and economics
Requirements:
- Experience with production LLM serving under meaningful traffic and latency constraints
- Knowledge of Kubernetes on GPU fleets
- Deep operational command of vLLM or SGLang
- Performance engineering at the system level
- Python and Go for automation
- Experience working with the community in English and Chinese
Nice to have:
- Experience with fine-tuning / RL infrastructure
- Real-time voice latency work
- Multi-region deployments and data-residency requirements