# Software Architect, AI Applications **Company**: Gong **Location**: Tel Aviv **Work arrangement**: hybrid **Experience**: senior **Job type**: full-time **Category**: Engineering **Industry**: Technology **Apply**: https://job-boards.greenhouse.io/gongio/jobs/4712289006?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply **Canonical**: https://yubhub.co/jobs/job_2bde1d41-d0a ## Description At Gong, you will join a company built on innovative products, ambitious goals, and passionate people. As a Software Architect, AI Applications, you will lead the architecture and technical direction of the Revenue Harness, the agentic framework that powers an AI teammate for every revenue professional. ## Responsibilities ### What You'll Build - Agentic Core: Design agent orchestration, planning and reasoning loops, multi-step execution, and multi-agent coordination for long-running revenue workflows. - Tools & Integrations: Develop a secure, extensible tool ecosystem, including MCP-based tools and connectors, that lets agents read from and act on Gong data, CRMs, email, and calendars. - Context & Memory: Create retrieval pipelines over calls, emails, deals, and accounts, using RAG, hybrid search, and knowledge graphs, plus short- and long-term memory. - Quality & Trust: Implement offline and online evaluation of accuracy, groundedness, and task success, along with guardrails, permissions, and human-in-the-loop controls. - Model Platform: Design multi-provider model routing, streaming, LLM observability, and cost and latency management. - Runtime & Execution: Develop high-concurrency infrastructure for stateful and stateless agents, including lifecycle management, isolated workers, task queues, and distributed state persistence. - Deployment & CI/CD: Manage cloud-native Kubernetes deployments with multi-region rollout, zero-downtime updates, canary releases for non-deterministic workflows, and strict tenant sandboxing. ## Key Responsibilities - Revenue Harness Architecture: Serve as the group's central technical leader. Define the reference architecture and the contracts between agents, tools, context, and models, so that teams build on a shared, scalable foundation. - Hands-On Engineering: Embed in teams across domains and contribute directly to design and code. - Quality & Reliability Standards: Set the group's eval methodology, regression gates, and production monitoring for non-deterministic systems. - Mentorship & Engineering Standards: Raise the bar on design quality and best practices across the group. - AI-Assisted Development: Drive adoption of coding agents, AI code review, and agentic workflows to improve velocity and quality. ## Challenges - Reliability on Non-Deterministic Foundations: Make LLM behavior trustworthy through structured outputs, tool-calls validation, fallbacks, and well-defined failure modes. - Context at Enterprise Scale: Get the right information into the context window efficiently when the data spans millions of conversations, deals, and accounts. - Secure, Permission-Aware Agents: Enforce data boundaries, user permissions, and tenant isolation, and defend against prompt injection and unintended actions. - Quality, Cost & Latency Trade-offs: Balance them through model selection, caching, and routing. - Ambiguity & Divergence Across Teams: Bring clarity to loosely defined cross-team problems, surface blind spots, and steer teams toward aligned, reusable patterns. - Speed vs. Long-Term Evolution: Catch architectural gaps early and make pragmatic trade-offs, so that systems keep pace with rapidly changing models and tooling. ## Impact - The Future of Revenue Work: Help sales professionals shift their time from busywork to selling. - A Higher Engineering Bar: Build stronger, more autonomous teams through better design practices and mentorship. - Consistent, Fast Delivery: Reduce duplication and accelerate execution without compromising long-term platform health. ## Requirements - 10+ years of backend development, with strong expertise in Java and modern frameworks such as Spring Boot. - Proven Staff or Principal-level impact across multiple teams, with the ability to lead through influence rather than authority. - Hands-on experience shipping production LLM applications, such as agents, tool calling, RAG, or structured generation, and an understanding of their failure modes. - Deep expertise in designing large-scale distributed systems, including asynchronous, event-driven, and streaming architectures. - Versatility in diving into unfamiliar codebases, switching context across teams, and spotting high-impact opportunities. - A passion for mentoring, and the ability to communicate complex ideas clearly to diverse stakeholders. - Genuine curiosity about the AI landscape, and a track record of driving adoption of AI-assisted development. ## Nice to Have - Experience with agent frameworks and protocols (e.g., MCP, Spring AI, LangChain/LangGraph) or with building an in-house equivalent. - Experience with LLM evaluation (LLM-as-judge, golden datasets, online monitoring), observability, and multi-provider routing. - Experience with vector databases, search infrastructure (e.g., Elasticsearch/OpenSearch), or knowledge graphs. - Familiarity with AI security concerns, including prompt injection, data leakage, and agent permission models. - Working knowledge of Python alongside a JVM stack. - Background in B2B SaaS, CRM, or revenue technology. ## Skills ### Required - Java - Spring Boot - LLM applications - distributed systems - asynchronous architectures - event-driven architectures - streaming architectures - mentoring ### Nice to have - agent frameworks - LLM evaluation - vector databases - search infrastructure - knowledge graphs - AI security - Python --- Source: [Apply at job-boards.greenhouse.io](https://job-boards.greenhouse.io/gongio/jobs/4712289006?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)