Description
Cisco is seeking an experienced and innovative Control Plane Engineer to develop the control plane for the next-generation AI infrastructure. This role focuses on designing and implementing scalable, reliable, and efficient control plane components to manage AI workloads.
As AI Control Plane Engineer your work will have a significant impact on:
- Enabling seamless orchestration and management of AI workloads across distributed environments.
- Improving the reliability, scalability, and performance of AI control plane services.
- Ensuring operational simplicity and ease of debugging for control plane services.
- Driving innovation in control plane architecture to improve efficiency and performance of AI infrastructure.
Key Responsibilities:
- Design and implement control plane components using Golang and Python.
- Leverage Kubernetes (K8s) concepts, CRDs, and operator patterns (e.g., Kubebuilder) to build control plane services.
- Develop scalable and highly available (HA) microservices that span across regions, ensuring reliability at scale.
- Build and maintain GRPC, REST APIs, and CLI tools for seamless integration and control of AI infrastructure.
- Address operational challenges of running applications as SaaS, focusing on ease of deployment and lifecycle management.
- Establish and follow best practices for release management, including CI/CD pipelines and version control hygiene.
- Design and implement strategies for live upgrades to minimize downtime and ensure service continuity.
- Develop and implement telemetry collection mechanisms using eBPF and other tools to provide insights into system performance and health.
- Define and monitor SLA/SLO metrics to ensure the reliability of control plane services.
- Design and manage stateful applications, ensuring high performance and reliability of underlying databases.
- Build systems with debuggability in mind, simplifying troubleshooting and remediation for operational teams.
- Collaborate with other teams to ensure control plane compatibility with GPU/CUDA-related technologies.
Minimum Qualifications:
- Proficiency in Golang and Python.
- Strong expertise in Kubernetes (K8s), including CRDs, the operator pattern, and tools like Kubebuilder and Helm.
- Experience with API design and implementation (GRPC, REST APIs, and CLI).
- Proven track record in designing and scaling highly available microservices across regions. Strong understanding of distributed systems architecture and fundamentals.
- Familiarity with telemetry tools and SLA/SLO monitoring for large-scale systems.
- Strong debugging skills and experience building systems with easy remediation capabilities.
- Passion for learning and staying updated on the latest trends in AI infrastructure and cloud-native technologies.
- Bachelor’s degree+ and relevant 7+ years of Engineering work experience.