Description
Cybersecurity & Trust and Safety | RLGym
We are looking for a Director of AI Research & Data Operations to lead and scale RLGym's research and data generation efforts, delivering high-quality training and evaluation data to frontier AI models at scale.
This role combines research leadership, data science, and large-scale operational management. You will build and lead multidisciplinary teams of researchers, data scientists, domain experts, and operations specialists, transforming complex research challenges into scalable, high-quality data production systems.
The ideal candidate brings experience managing large research and data operations, a strong understanding of modern AI and LLM development, and the ability to deliver high-quality datasets that advance frontier model capabilities, with a particular focus on cybersecurity and trust & safety.
What you'll be doing
- Lead and scale large AI research and data operations, managing multidisciplinary teams of researchers, data scientists, domain experts, and operational leaders.
- Own the end-to-end generation and delivery of high-quality training and evaluation datasets for frontier AI models, ensuring scalability, consistency, and research value.
- Define the research strategy, operational roadmap, and data generation priorities across cybersecurity, trust & safety, and emerging AI capabilities.
- Build and optimize large-scale data production pipelines, combining human expertise, automated workflows, synthetic data generation, and AI-assisted research.
- Establish rigorous data quality frameworks, including validation methodologies, quality assurance processes, benchmarking, and continuous improvement.
- Lead research initiatives focused on generating realistic, challenging, and diverse datasets for LLMs, GenAI applications, and agentic AI systems.
- Work closely with research, product, engineering, and data science teams to translate frontier model requirements into actionable research programs and scalable data deliverables.
- Develop scalable systems for expert-driven data generation, evaluation, annotation, and human feedback, including workflows supporting supervised fine-tuning and reinforcement learning.
- Define and track operational and research KPIs, balancing data quality, throughput, cost efficiency, and delivery timelines.
- Build partnerships with external researchers, domain experts, and research organizations to expand data generation capabilities and domain expertise.
- Drive innovation in data generation methodologies, evaluation benchmarks, and research infrastructure to continuously improve the quality and impact of RLGym's data.
Requirements
What we need to see
- Proven experience leading and scaling large research, data science, AI/ML, or data operations teams.
- Strong experience managing complex, high-volume data generation or research operations, with measurable quality and delivery outcomes.
- Deep understanding of AI/ML development, LLMs, GenAI, and the role of training and evaluation data in improving model performance.
- Experience designing and implementing data quality frameworks, evaluation methodologies, and scalable research workflows.
- Ability to translate complex research objectives into structured execution plans and deliver results at significant scale.
- Strong analytical and data-driven decision-making skills, with experience defining operational metrics and optimizing research processes.
- Experience managing multidisciplinary teams and collaborating with research scientists, engineers, and domain experts.
- Strong strategic leadership, operational execution, and cross-functional communication skills.
- Familiarity with cybersecurity, AI security, or trust & safety challenges and their relevance to frontier AI model development.
- 8+ years of relevant experience in AI/ML, data science, research leadership, large-scale data operations, or related technical fields.
Ways to stand out
- Experience delivering large-scale training or evaluation datasets to leading frontier AI labs.
- Background in human feedback, RLHF, reinforcement learning environments, synthetic data generation, or expert-driven annotation programs.
- Experience building data generation platforms, automated evaluation pipelines, or large-scale AI research infrastructure.
- Familiarity with cybersecurity and trust & safety datasets, adversarial evaluations, or AI red-teaming methodologies.
- Experience developing data quality measurement systems, including automated grading, expert calibration, and statistical quality control.
About Alice
Alice is a trust, safety, and security company built for the AI era. We safeguard the communicative technologies people use to create, collaborate, and interact- whether with each other or with machines.
In a world where AI has fundamentally changed the nature of risk, Alice provides end-to-end coverage across the entire AI lifecycle. We support frontier model labs, enterprises, and UGC platforms with a comprehensive suite of solutions: from model hardening evaluations and pre-deployment red-teaming to runtime guardrails and ongoing drift detection.
Alice is widely considered a global leader in online safety and AI security. We have some of the most forward-thinking and passionate minds in the world working to safeguard over 3 billion users across the largest AI and tech platforms.
If you're creative and driven to secure the future of AI, we want to hear from you!
