Senior System Software Engineer - Local AI

NVIDIA is seeking a Senior Systems Software Engineer to build efficient on-device AI software for RTX and DGX-class systems, focusing on high-performance local inference, low latency, and efficient memory use.

The LocalAI team is working on delivering AI models locally, closer to the source of data, to reduce latency, improve real-time processing, and address privacy concerns.

Responsibilities:

• Partner with NVIDIA's software, research, architecture, and product teams to align strategies and technical needs for AI on RTX and DGX PCs.
• Build and optimize local AI inference stack for RTX, RTX Pro, and DGX GPUs, focusing on performance, stability, and scalability.
• Develop modern inference runtimes and execution stacks for frameworks like Llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX.
• Perform end-to-end optimization of AI models, data pipelines, and inference runtimes to enhance performance across GPU architectures.
• Apply model optimization techniques such as quantization, pruning, sparsity, and distillation to enable efficient deployment of large models.
• Perform system-level debugging, performance optimization, and performance-accuracy trade-off analysis.

Requirements:

• 5+ years of experience with Bachelor's, Master's, or PhD in Computer Science, Software Engineering, Mathematics, or a related field.
• Excellent C++ programming and debugging skills, with a strong understanding of data structures, algorithms, and machine learning.
• Proven experience working with AI inferencing pipelines and applications using ML/DL frameworks.
• Deep interest in inference backends and runtime internals, including scheduling, memory management, and hardware-aware optimization.
• Strong analytical and problem-solving abilities, with outstanding written and oral communication skills.

Benefits:

• Competitive salaries
• Generous benefits package
• Opportunities for growth and innovation in a top employer known for innovation and growth