Description
At Gong, you will join a company built on innovative products, ambitious goals, and passionate people.
As a Software Architect, AI Applications, you will lead the architecture and technical direction of the Revenue Harness, the agentic framework that powers an AI teammate for every revenue professional.
Responsibilities
What You'll Build
- Agentic Core: Design agent orchestration, planning and reasoning loops, multi-step execution, and multi-agent coordination for long-running revenue workflows.
- Tools & Integrations: Develop a secure, extensible tool ecosystem, including MCP-based tools and connectors, that lets agents read from and act on Gong data, CRMs, email, and calendars.
- Context & Memory: Create retrieval pipelines over calls, emails, deals, and accounts, using RAG, hybrid search, and knowledge graphs, plus short- and long-term memory.
- Quality & Trust: Implement offline and online evaluation of accuracy, groundedness, and task success, along with guardrails, permissions, and human-in-the-loop controls.
- Model Platform: Design multi-provider model routing, streaming, LLM observability, and cost and latency management.
- Runtime & Execution: Develop high-concurrency infrastructure for stateful and stateless agents, including lifecycle management, isolated workers, task queues, and distributed state persistence.
- Deployment & CI/CD: Manage cloud-native Kubernetes deployments with multi-region rollout, zero-downtime updates, canary releases for non-deterministic workflows, and strict tenant sandboxing.
Key Responsibilities
- Revenue Harness Architecture: Serve as the group's central technical leader. Define the reference architecture and the contracts between agents, tools, context, and models, so that teams build on a shared, scalable foundation.
- Hands-On Engineering: Embed in teams across domains and contribute directly to design and code.
- Quality & Reliability Standards: Set the group's eval methodology, regression gates, and production monitoring for non-deterministic systems.
- Mentorship & Engineering Standards: Raise the bar on design quality and best practices across the group.
- AI-Assisted Development: Drive adoption of coding agents, AI code review, and agentic workflows to improve velocity and quality.
Challenges
- Reliability on Non-Deterministic Foundations: Make LLM behavior trustworthy through structured outputs, tool-calls validation, fallbacks, and well-defined failure modes.
- Context at Enterprise Scale: Get the right information into the context window efficiently when the data spans millions of conversations, deals, and accounts.
- Secure, Permission-Aware Agents: Enforce data boundaries, user permissions, and tenant isolation, and defend against prompt injection and unintended actions.
- Quality, Cost & Latency Trade-offs: Balance them through model selection, caching, and routing.
- Ambiguity & Divergence Across Teams: Bring clarity to loosely defined cross-team problems, surface blind spots, and steer teams toward aligned, reusable patterns.
- Speed vs. Long-Term Evolution: Catch architectural gaps early and make pragmatic trade-offs, so that systems keep pace with rapidly changing models and tooling.
Impact
- The Future of Revenue Work: Help sales professionals shift their time from busywork to selling.
- A Higher Engineering Bar: Build stronger, more autonomous teams through better design practices and mentorship.
- Consistent, Fast Delivery: Reduce duplication and accelerate execution without compromising long-term platform health.
Requirements
- 10+ years of backend development, with strong expertise in Java and modern frameworks such as Spring Boot.
- Proven Staff or Principal-level impact across multiple teams, with the ability to lead through influence rather than authority.
- Hands-on experience shipping production LLM applications, such as agents, tool calling, RAG, or structured generation, and an understanding of their failure modes.
- Deep expertise in designing large-scale distributed systems, including asynchronous, event-driven, and streaming architectures.
- Versatility in diving into unfamiliar codebases, switching context across teams, and spotting high-impact opportunities.
- A passion for mentoring, and the ability to communicate complex ideas clearly to diverse stakeholders.
- Genuine curiosity about the AI landscape, and a track record of driving adoption of AI-assisted development.
Nice to Have
- Experience with agent frameworks and protocols (e.g., MCP, Spring AI, LangChain/LangGraph) or with building an in-house equivalent.
- Experience with LLM evaluation (LLM-as-judge, golden datasets, online monitoring), observability, and multi-provider routing.
- Experience with vector databases, search infrastructure (e.g., Elasticsearch/OpenSearch), or knowledge graphs.
- Familiarity with AI security concerns, including prompt injection, data leakage, and agent permission models.
- Working knowledge of Python alongside a JVM stack.
- Background in B2B SaaS, CRM, or revenue technology.