Description
As an Adversarial AI Evaluation Scientist, you will review and validate adversarial agent tasks: indirect prompt-injection (IPI), data-exfiltration, tool-misuse, and related attack scenarios to ensure each one is realistic, fair, deterministic, reproducible, and correctly graded. You will decide whether a scenario genuinely demonstrates a model vulnerability or merely an artifact of how the task was built, and you will combine adversarial-security judgment with statistical analysis and LLM-assisted tooling to find quality issues, analyze attack-success patterns, and recommend or automate improvements.
Responsibilities
- Review adversarial tasks for realism and threat-model soundness: is the attacker’s capability plausible, can the attacker actually reach the poisoned content, and is the target system a realistic one?
- Assess attack fairness and attribution: confirm the harm is driven by the adversary (not pre-baked into the system prompt, vouched for by a trusted layer, or otherwise rigged), so a careful agent could plausibly pass.
- Validate grading correctness: verify that success/failure criteria, reward logic, and deterministic checks detect the intended harm, with no reward hacks, token-collapse, or evadable gates.
- Measure and interpret attack success rate (ASR@K) and its variance; verify deterministic, reproducible behavior across repeated executions.
- Judge attack quality - novelty vs. known patterns, sophistication, stealth, and coverage across harm types and target surfaces.
- Identify anomalies and systemic issues across large sets of tasks (e.g. scenarios that succeed ~100% of the time against every model - a red flag for an unfair task, not a win).
- Define validation guidelines, acceptance criteria, quality thresholds, and release gates for adversarial tasks.
- Use LLMs and AI models to analyze attack failures, identify root causes, and assist with task or grader fixes.
- Build automated validation, regression, and quality-monitoring pipelines for adversarial task sets.
- Work closely with red-teamers, researchers, and task authors to improve attack design, fairness, and reliability.
Requirements
Must Have
- 4+ years in an analytical, ML, security-research, or applied-research role, including hands-on work with adversarial ML, LLM security, red-teaming, or AI safety evaluation.
- Bachelor’s or Master’s in Computer Science, Machine Learning, Statistics, a related quantitative field, or equivalent practical experience.
- Working knowledge of prompt injection / indirect prompt injection, jailbreaks, data exfiltration, or tool-misuse attacks against LLM-based systems or agents.
- Strong Python and data-analysis skills.
- Solid grounding in statistics and experiment design: distributions, variance analysis, hypothesis testing, and anomaly detection: applied to evaluation and attack-success measurement.
- Experience building or auditing data-driven validation, evaluation, or quality frameworks.
- Experience using LLMs for analysis, evaluation, code generation, or automated remediation.
- Strong understanding of reproducibility, deterministic systems, and automated testing.
- Sharp adversarial thinking and the ability to spot subtle ways a test can be unfair, gameable, or mis-graded.
- Excellent analytical, problem-solving, and communication skills.
Preferred Skills That Set You Apart
- Certifications in offensive cybersecurity (e.g., OSWA, OSWE, OSCE3, SEC542, SEC522)
- Experience designing or evaluating AI agents, LLM-powered applications, or agentic workflows (e.g. email/doc assistants, tool-using agents).
- Familiarity with RL Gym-style task formats, Gymnasium/OpenAI Gym, or agent-evaluation harnesses.
- Experience red-teaming production AI systems or building adversarial benchmark datasets.
- Experience with automated evaluation pipelines, regression testing, and quality gates.
- Familiarity with safety/security taxonomies (harm types, attack patterns) and threat modeling.
About Alice
Alice is a trust, safety, and security company built for the AI era. We safeguard the communicative technologies people use to create, collaborate, and interact- whether with each other or with machines.
In a world where AI has fundamentally changed the nature of risk, Alice provides end-to-end coverage across the entire AI lifecycle. We support frontier model labs, enterprises, and UGC platforms with a comprehensive suite of solutions: from model hardening evaluations and pre-deployment red-teaming to runtime guardrails and ongoing drift detection.
Alice is widely considered a global leader in online safety and AI security. We have some of the most forward-thinking and passionate minds in the world working to safeguard over 3 billion users across the largest AI and tech platforms.
If you're creative and driven to secure the future of AI, we want to hear from you!
