# Senior AI Software Engineer **Company**: GDIT **Location**: Sterling, VA **Work arrangement**: onsite **Experience**: senior **Job type**: full-time **Salary**: $128,039 - $173,229 **Category**: Engineering **Industry**: Technology **Apply**: https://gdit.wd5.myworkdayjobs.com/en-US/External_Career_Site/job/USA-VA-Sterling/Senior-AI-Software-Engineer_RQ230390?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply **Canonical**: https://yubhub.co/jobs/job_17fe0c18-a9a ## Description GDIT is seeking a highly motivated and experienced Senior AI Software Engineer to support the development, modernization, integration, and sustainment of an on-premises, AI-powered enterprise chatbot platform supporting the U.S. Department of State in Sterling, VA. The selected candidate will design, develop, and integrate software solutions leveraging Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), vector databases, document processing technologies, and modern AI inference frameworks. **Key Responsibilities** **1. AI Application Development & Integration** - Design, develop, test, and maintain Python-based applications, microservices, and APIs supporting an on-prem chatbot. - Extend, configure, and integrate OpenWebUI with enterprise AI services and backend systems. - Develop custom OpenWebUI Functions, Tools, and integrations to support mission-specific use cases. - Integrate approved LLMs into chatbot using OpenAI-compatible APIs and inference frameworks such as vLLM and Ollama. - Develop reusable software components for AI model access, prompt management, response processing, and application workflows. - Collaborate with AI/ML engineers to improve model inference performance, response quality, and application reliability. - Support multimodal AI use cases, including document understanding, optical character recognition, and vision-language model integration. - Develop backend capabilities to support secure enterprise AI workflows. **2. Retrieval-Augmented Generation (RAG) & Document Processing** - Design, implement, and optimize RAG workflows for enterprise document search and question answering. - Integrate document processing technologies such as Docling, Tesseract, and vision-language models into automated ingestion pipelines. - Develop document chunking, metadata extraction, embedding generation, and retrieval workflows. - Integrate embedding models and reranking models to improve retrieval accuracy and relevance. - Work with PostgreSQL and pgvector to support vector search, semantic retrieval, and document indexing. - Develop APIs and background processing services for document ingestion and knowledge base management. - Collaborate with data engineers to ensure accurate storage, indexing, and retrieval of enterprise information. - Implement testing and evaluation approaches to measure RAG accuracy, retrieval relevance, and answer quality. **3. AI Model Serving & Inference Integration** - Integrate and maintain AI inference services using vLLM, Ollama, and compatible model-serving frameworks. - Support the deployment and configuration of approved open-weight LLMs, embedding models, rerankers, and vision-language models. - Develop software integrations using OpenAI-compatible REST APIs. - Assist in optimizing inference configurations, including context length, GPU memory utilization, concurrency, and model serving performance. - Support the transition from existing inference frameworks to improved architectures as technologies mature. - Collaborate with AI/ML engineers to troubleshoot inference errors, model loading issues, and integration failures. - Develop benchmarking and performance-testing utilities for AI workloads. **4. Containerization & Platform Modernization** - Support the migration of DS Chat components from traditional deployments into containerized services. - Develop, configure, and maintain container images using Podman or Docker. - Create and maintain Podman Compose configurations defining application services, networking, volumes, and environment variables. - Integrate services such as OpenWebUI, vLLM, Docling, PostgreSQL, pgvector, PgBouncer, and Redis. - Develop scripts and automation for repeatable installation, deployment, upgrades, and recovery. - Implement health checks, readiness checks, logging, and service dependency management. - Troubleshoot container networking, authentication, storage, GPU access, and service connectivity. - Collaborate with infrastructure and cybersecurity teams to improve deployment consistency and security. **5. Agentic AI & Future Capabilities** - Research, prototype, and develop agentic AI capabilities for future DS Chat releases. - Implement controlled tool-calling and multi-step AI workflows. - Develop integrations that allow approved AI models to interact with authorized enterprise APIs and services. - Explore emerging technologies involving AI orchestration, workflow automation, and Model Context Protocol (MCP). - Design secure execution boundaries, permissions, audit logging, and approval mechanisms for AI-driven actions. - Collaborate with stakeholders to evaluate new AI capabilities for mission relevance, security, and operational feasibility. - Implement safeguards to support human oversight and prevent unauthorized actions by AI-enabled systems. **6. Software Engineering, Testing & DevSecOps** - Apply software engineering best practices, including modular architecture, code reviews, documentation, and version control. - Develop automated unit, integration, regression, and performance tests using frameworks such as pytest. - Build integration tests validating communication between OpenWebUI, inference services, document processing components, and databases. - Support CI/CD processes appropriate for disconnected and classified environments. - Troubleshoot production issues through application logs, metrics, traces, and system diagnostics. - Integrate operational logging and monitoring capabilities with tools such as Splunk. - Assist with vulnerability remediation, dependency management, software supply chain reviews, and container security assessments. - Produce technical documentation, deployment procedures, architecture diagrams, and operational runbooks. - Participate in Agile development activities, including sprint planning, technical design reviews, and demonstrations. **7. Security & Classified Environment Support** - Design software solutions suitable for secure, on-premises, air-gapped government environments. - Implement secure authentication integration, role-based access controls, API authorization, and data protection measures. - Ensure AI services communicate only through authorized internal network paths. - Support software packaging and offline deployment using preapproved dependencies, container images, and model artifacts. - Collaborate with cybersecurity personnel to address security findings and support Risk Management Framework (RMF) activities. - Assist with technical documentation and evidence required for TRB/CCB reviews and Authority to Operate (ATO) processes. - Follow approved procedures for handling sensitive and classified information. **Required Qualifications** - Minimum Secret Clearance, no sponsorship - Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Computer Engineering, or a related technical field, or equivalent practical experience. - 5+ years of professional software engineering experience, including backend application development. - Strong proficiency in Python and experience developing production-quality software. - Experience developing RESTful APIs and integrating distributed application services. - Hands-on experience integrating LLMs or generative AI capabilities into software applications. - Understanding of transformer-based models, inference APIs, prompt engineering, and LLM application development. - Experience with RAG concepts, including document ingestion, embeddings, vector search, and retrieval pipelines. - Experience working with relational databases such as PostgreSQL. - Familiarity with container technologies such as Podman or Docker. - Experience with Linux-based development and application deployment. - Proficiency with Git, automated testing, debugging, and software development best practices. - Experience troubleshooting applications across multiple interconnected services. - Strong analytical, problem-solving, communication, and technical documentation skills. - Ability to work effectively with cross-functional teams and government stakeholders. **Preferred Qualifications** - Experience with OpenWebUI, including configuration, integration, custom Functions, Tools, or application extensions. - Hands-on experience with vLLM, Ollama, Hugging Face Transformers, or other LLM inference technologies. - Experience serving or integrating open-weight LLMs in on-premises environments. - Experience with NVIDIA GPUs, CUDA, GPU memory management, and AI inference optimization. - Experience with PostgreSQL pgvector, vector similarity search, and semantic retrieval. - Familiarity with embedding models, rerankers, and cross-encoder architectures. - Experience with Docling, Tesseract, or document intelligence technologies. - Experience implementing production RAG pipelines and evaluation frameworks. - Familiarity with agentic AI frameworks, tool calls, and Model Context Protocol (MCP). - Experience developing or maintaining containerized microservice architectures. - Familiarity with Redis, PgBouncer, and asynchronous background processing. - Experience with Red Hat Enterprise Linux and Windows Server. - Experience with FastAPI, Pydantic, pytest, and related Python technologies. - Experience using Splunk for application monitoring and troubleshooting. - Familiarity with software supply chain security, vulnerability scanning, and container hardening. - Experience supporting federal government systems, classified networks, or air-gapped deployments. - Knowledge of federal cybersecurity requirements, RMF, and ATO processes. ## Skills ### Required - Python - Agile Software Development - Back-End Development - Microservices Architecture - Large Language Models (LLMs) - Retrieval-Augmented Generation (RAG) - vector databases - document processing technologies - modern AI inference frameworks - OpenWebUI - OpenAI-compatible APIs - vLLM - Ollama - PostgreSQL - pgvector - Podman - Docker - Linux-based development - Git - automated testing - debugging - software development best practices ### Nice to have - Hugging Face Transformers - NVIDIA GPUs - CUDA - GPU memory management - AI inference optimization - PostgreSQL pgvector - vector similarity search - semantic retrieval - embedding models - rerankers - cross-encoder architectures - Docling - Tesseract - document intelligence technologies - agentic AI frameworks - tool calls - Model Context Protocol (MCP) - Redis - PgBouncer - asynchronous background processing - Red Hat Enterprise Linux - Windows Server - FastAPI - Pydantic - pytest - Splunk --- Source: [Apply at gdit.wd5.myworkdayjobs.com](https://gdit.wd5.myworkdayjobs.com/en-US/External_Career_Site/job/USA-VA-Sterling/Senior-AI-Software-Engineer_RQ230390?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)