Description
Some careers have more impact than others. If you're looking for a career where you can make a real impression, join HSBC and discover how valued you'll be.
We are currently seeking an experienced professional to join our team in the role of Associate Director, Software Engineering (AI Guardrail).
As an Associate Director, Software Engineering (AI Guardrail), you will be responsible for building high-concurrency async services in Python (asyncio) and/or Go for AI/LLM guardrail runtimes. You will also design scalable sync/async execution pipelines, including batching, routing, and inference orchestration.
Key responsibilities include:
- Building high-concurrency async services in Python (asyncio) and/or Go for AI/LLM guardrail runtimes
- Designing scalable sync/async execution pipelines, including batching, routing, and inference orchestration
- Implementing queueing and flow-control (backpressure, retries, timeouts, cancellation, overload protection, rate limiting)
- Delivering streaming and real-time processing via WebSocket, SSE, gRPC streaming, and event-driven patterns
- Building distributed messaging architectures using Kafka/PubSub to support chunked/streaming inference and detection
- Instrumenting end-to-end observability (metrics, tracing, telemetry) with OpenTelemetry/OpenInference and actionable dashboards/alerts
- Leading reliability and performance engineering: profiling, concurrency tuning, bottleneck/latency/overload remediation, SLO/SLA practices
- Enabling governance-ready controls: config-driven policies/detectors (versioning/rollout/rollback), multi-tenancy (isolation/quotas), audit trails and traceable runtime signals
Requirements include:
- Bachelor's degree or above in CS/Software Engineering (or equivalent experience) with 8+ years' engineering experience, including 5+ years building and running distributed, cloud-native backend systems in production
- Strong hands-on Python (asyncio/FastAPI) and/or Go, with deep knowledge of non-blocking I/O, async programming, and high-concurrency service design
- Proven runtime control and performance engineering: queueing/backpressure, rate limiting/overload protection, retries/timeouts/cancellation, latency and throughput optimisation under load
- Experience with event-driven and streaming systems plus API platforms: WebSocket/SSE/message streaming, REST/gRPC, and API gateway/platform components; strong observability (metrics/logs/traces, OpenTelemetry) and production troubleshooting (profiling, RCA)
- Cloud and platform operations: Kubernetes/Docker on AWS/GCP/Azure; operational controls such as RBAC/authN/authZ, configuration management, and audit logging; AI-native mindset and effective use of coding assistants
You'll achieve more when you join HSBC. HSBC is an equal opportunity employer committed to building a culture where all employees are valued, respected and opinions count. We take pride in providing a workplace that fosters continuous professional development, flexible working and, opportunities to grow within an inclusive and diverse environment.