Senior System Software Engineer - Local AI
NVIDIA is seeking a Senior Systems Software Engineer to build efficient on-device AI software for RTX and DGX-class systems, focusing on high-performance local inference, low latency, and efficient memory use. The LocalAI team is working on delivering AI models locally, closer to the source of data, to reduce latency, improve real-time processing, and address privacy concerns. Responsibilities: Partner with NVIDIA's software, research, architecture, and product teams to align strategies and technical needs for AI on RTX and DGX PCs. Build and optimize local AI inference stack for RTX, RTX Pro, and DGX GPUs, focusing on performance, stability, and scalability. Develop modern inference runtimes and execution stacks for frameworks like Llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX. Perform end-to-end optimization of AI models, data pipelines, and inference runtimes to enhance performance across GPU architectures. Apply model optimization techniques such as quantization, pruning, sparsity, and distillation to enable efficient deployment of large models. Perform system-level debugging, performance optimization, and performance-accuracy trade-off analysis. Requirements: 5+ years of experience with Bachelor's, Master's, or PhD in Computer Science, Software Engineering, Mathematics, or a related field. Excellent C++ programming and debugging skills, with a strong understanding of data structures, algorithms, and machine learning. Proven experience working with AI inferencing pipelines and applications using ML/DL frameworks. Deep interest in inference backends and runtime internals, including scheduling, memory management, and hardware-aware optimization. Strong analytical and problem-solving abilities, with outstanding written and oral communication skills. Benefits: Competitive salaries Generous benefits package Opportunities for growth and innovation in a top employer known for innovation and growth