Shaun McIlroy

Help Scout — Remote

• → Lead evaluation and reliability efforts for AI-driven features, designing frameworks that measure LLM behaviour and help teams understand where AI responses succeed, fail, or require safeguards.
• → Investigate real-world AI system behaviour using customer sessions and product data, translating failure patterns into prompt improvements and clearer quality signals.
• → Establish feedback loops that allow teams to monitor AI behaviour over time and evaluate the impact of prompt, model, and product changes.

Help Scout — Remote

• → Established and led the Advanced Triage rotation program, designing the onboarding structure, investigation process, and coaching approach for complex technical investigations.
• → Served as a senior escalation point for critical customer incidents, identifying recurring product gaps and translating findings into structured engineering insights.

Help Scout — Remote

• → Investigated complex product behaviour and translated real customer usage into actionable insights for Product, Engineering, and the Experience organisation.
• → Led deep investigations into defects, SDK issues, and ambiguous product behaviour, producing clear, reproducible findings that guided engineering decisions.
• → Partnered with Platform and Mobile teams on HIPAA workflow redesign, permission model improvements, and SDK behaviour clarity.

Help Scout — Remote

• → Joined the Triage team, developing root-cause analysis and deep product investigation skills handling increasingly complex and ambiguous cases.
• → Identified recurring patterns across customer reports and surfaced systemic issues that informed early product prioritisation discussions.

Vox Media — Remote

• → Supported editorial teams across The Verge, Polygon, SB Nation, and Vox.com through responsive technical support and operational coordination.
• → Designed and executed the onboarding process that enrolled 8,000+ contributors into mandatory two-factor authentication.

BSc. Computer Games Technology