Description
GuidePoint Security is seeking an AI & Data Science Intern to support its Northeast region. As part of the GPSU program, you will work with the NE AI and AppSec teams as part of the NE DevSecOps team, contributing to the evaluation and governance infrastructure that supports GuidePoint's agentic AI harnesses.
Responsibilities:
- Define and implement evaluation frameworks to measure the accuracy, reliability, and consistency of agentic AI harnesses
- Design and run structured evals against harness outputs to detect regressions, output drift, and behavioral degradation over time
- Build and maintain benchmark datasets and ground-truth corpora used to evaluate harness quality
- Analyze eval results, identify failure modes, and collaborate with the NE AI and AppSec teams to drive iterative improvements
- Research and apply prompt evaluation techniques and LLM benchmarking methodologies relevant to security-specific harnesses
- Instrument agentic harnesses with telemetry to capture latency, token usage, error rates, and output quality signals
- Analyze telemetry data to identify patterns, surface anomalies, and generate actionable insights for harness tuning
- Support model and prompt tuning efforts based on eval and telemetry findings, documenting changes and measuring their impact
- Contribute to building an internal AI governance pipeline: defining standards, logging outputs, flagging quality issues, and maintaining an audit trail for harness behavior
Qualifications:
- Currently enrolled, or recently graduated with a degree or substantial coursework in data science, statistics, computer science, engineering, or a related field
- Proficiency or strong interest in Python, with exposure to data manipulation libraries (pandas, numpy) or LLM APIs
- Genuine curiosity about large language models, agentic systems, and how AI performs (and fails) in applied contexts
- Strong analytical and written communication skills, with an ability to translate data findings into clear, actionable observations
- Technical aptitude and comfort working with ambiguity in a fast-moving, practitioner-led environment
Internship Details and Expectations:
- Part-time, remote paid internship ($20/hr) that runs for 12 weeks, with potential for full-time employment at the conclusion of the internship
- Expected to work 25-29hrs/wk Monday-Friday supporting the Eastern Time Zone
- Must be physically located within the United States during the entire duration of the internship