# Senior Software Engineer, Trust & Safety **Company**: Replit **Location**: Foster City, CA **Work arrangement**: hybrid **Experience**: senior **Job type**: Full time **Salary**: $210K - $265K **Category**: Engineering **Industry**: Technology **Wikidata**: https://www.wikidata.org/wiki/Q60768699 **Apply**: https://jobs.ashbyhq.com/replit/75477d6d-82e2-413c-b355-ec9426e56555?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply **Canonical**: https://yubhub.co/jobs/job_5ce46296-48c ## Description The Trust & Safety team is the front line defending Replit's platform from exploitation. We detect and shut down phishing deployments, prevent cryptomining on free-tier infrastructure, stop LLM token farming, and keep bad actors from weaponizing the platform against our users. This is adversarial work: attackers adapt constantly, and we build the detection systems, heuristics, and automated responses that stay ahead of them. What makes this role unique is the AI-native nature of Replit's platform. You'll work on problems that barely exist elsewhere: building guardrails for AI-generated code, detecting prompt injection attacks at scale, and using LLMs as a defensive tool against abuse. If you want hands-on experience applying AI to security problems, this is one of the few places you can do it in production with real attackers. You'll own problems end-to-end, from identifying emerging abuse patterns to shipping the systems that stop them at scale. ### Responsibilities - Design and implement LLM guardrails that detect abuse scenarios in AI-generated code and agent interactions - Build AI-powered detection systems that use LLMs to identify malicious patterns, classify threats, and automate response decisions - Build and operate abuse detection systems that identify phishing, cryptomining, account takeover, and financial fraud across millions of daily user actions - Design automated response mechanisms that enforce platform policies without manual intervention - Own the full abuse response lifecycle: detection, investigation, enforcement, and handling appeals alongside Support and Legal - Analyze attack patterns using BigQuery and Hex, turning investigation findings into new detection rules - Maintain and extend internal detection tools (Slurper, Netwatch) that continuously monitor user activity - Integrate and tune security scanners (SAST, SCA) in CI pipelines with tight performance SLAs - Track abuse trends, measure detection effectiveness, and adapt defenses as attack patterns evolve ### Requirements - 4+ years of experience in security engineering, anti-abuse, trust & safety, or fraud detection - Strong programming skills in Python and/or TypeScript for building detection systems and automation - Experience with SQL and data analysis at scale (BigQuery, Snowflake, or similar) - Experience building or fine-tuning ML/LLM-based classifiers for security or abuse detection - Familiarity with prompt injection, jailbreaking, and other LLM-specific attack vectors - Ability to investigate complex abuse patterns and translate findings into automated defenses - Familiarity with common attack patterns: phishing infrastructure, account takeover, credential stuffing, resource abuse - Clear communication skills for working across Security, Support, Legal, and Engineering teams ### Nice to Have - Experience at a platform company dealing with user-generated content or compute abuse (hosting providers, cloud platforms, developer tools) - Background in fraud detection, payment abuse, or financial crime - Familiarity with device fingerprinting, IP reputation, and email validation services - Experience with CI/CD security tooling (SAST, SCA, Dependabot, Snyk) - Knowledge of container security, Linux internals, or cloud infrastructure (GCP preferred) - Prior work with abuse reporting pipelines, trust & safety tooling, or content moderation systems ### Tools + Tech Stack for this role - Languages: Python, TypeScript, Go, SQL - Data: BigQuery, Hex - Detection tools: Slurper, Netwatch, Stytch (device fingerprint); ClearOut (email reputation) - CI/CD Security: Dependabot, Snyk, SAST/SCA scanners - Infrastructure: GCP, Kubernetes - Collaboration: Linear, Slack, Zendesk (for abuse reports) ### This role may not be a fit if - You prefer deep security research over building operational detection systems - You want to focus on vulnerability management, pentesting, or bug bounty triage (that's our Security team) - You're looking for a role with predictable, well-defined problems rather than constantly adapting to adversarial behavior - You prefer working in isolation rather than partnering closely with Support, Legal, and cross-functional teams - You're uncomfortable making enforcement decisions that affect real users ### Full-Time Employee Benefits Include - Competitive Salary & Equity - 401(k) Program with a 4% match (US Only) - Health, Dental, Vision and Life Insurance - Short Term and Long Term Disability - Paid Parental, Medical, Caregiver Leave - Flexible Time Off (FTO) + Holidays - Commuter Benefits (In-Office Only) - Monthly Wellness Stipend - Autonomous Work Environment - In Office Set-Up Reimbursement (In-Office Only) - Quarterly Team Gatherings - In Office Amenities (In-Office Only) ## Skills ### Required - security engineering - anti-abuse - trust & safety - fraud detection - Python - TypeScript - SQL - BigQuery - Hex - ML/LLM-based classifiers - prompt injection - jailbreaking - common attack patterns - phishing infrastructure - account takeover - credential stuffing - resource abuse ### Nice to have - experience at a platform company - background in fraud detection - familiarity with device fingerprinting - IP reputation - email validation services - CI/CD security tooling - container security - Linux internals - cloud infrastructure - abuse reporting pipelines - trust & safety tooling - content moderation systems --- Source: [Apply at jobs.ashbyhq.com](https://jobs.ashbyhq.com/replit/75477d6d-82e2-413c-b355-ec9426e56555?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)