Most AI ethics testing works backward. Researchers define a set of known failure modes, build benchmarks around them, and check whether a system passes. The problem is obvious: you only find the problems you already know to look for.
A new framework from MIT, called SEED-SET, takes the opposite approach. Instead of testing against a fixed checklist, it actively searches for the ethical failures nobody thought to test — the unknown unknowns that surface only when autonomous systems meet the messy complexity of the real world.
How SEED-SET Works
SEED-SET — Scalable Experimental Design for System-level Ethical Testing — was developed by a team led by Chuchu Fan, an associate professor in MIT's Department of Aeronautics and Astronautics, with Anjali Parashar as lead author. The paper is being presented at ICLR (International Conference on Learning Representations) this week.
The framework rests on a key architectural decision: it separates objective, measurable performance metrics from subjective, user-defined human values. An autonomous traffic system, for instance, might be optimized for throughput (objective) while also being expected to avoid disproportionately routing heavy traffic through low-income neighborhoods (a human value). These two dimensions are evaluated independently.
To assess the human values component, SEED-SET uses a large language model as a proxy for human stakeholder preferences. The LLM is prompted with descriptions of stakeholder groups — residents near a highway, commuters, emergency responders — and asked to evaluate whether a given system recommendation aligns with their values. This is not a replacement for real stakeholder input, but a scalable way to approximate it across thousands of simulated scenarios.
The adaptive experimental-design loop is what makes SEED-SET more than a static evaluation tool. Rather than running a fixed set of test cases, the framework iteratively identifies the most informative scenarios to test next, focusing its computational budget on the boundary regions where ethical alignment is most uncertain. This allows it to discover failure modes that uniform sampling would miss.
Demonstrated on High-Stakes Simulations
The MIT team validated SEED-SET on two simulation domains: power-grid management and urban-traffic control. Both are areas where autonomous decision-making has real consequences — where to route electricity during peak demand, or how to time traffic signals across a city.
In the power-grid simulation, SEED-SET identified scenarios where an AI system's cost-optimization recommendations would disproportionately burden certain grid regions during outages — a pattern that static testing had not flagged. In the traffic simulation, it discovered edge cases where throughput-optimized signal timing created unsafe pedestrian crossing conditions at specific intersections during school hours.
These are precisely the kinds of failures that emerge from the interaction between system-level optimization and localized human impact — difficult to predict in advance, but discoverable through systematic search.
Why This Matters Now
The timing of this research is significant. Autonomous systems are moving from controlled environments into public infrastructure at an accelerating pace. AI-managed power grids, traffic systems, and supply chains are no longer theoretical — they are being deployed. And the ethical evaluation tools available to the organizations deploying them have not kept pace.
Current approaches to AI ethics testing tend to fall into two categories: high-level principles documents that lack technical specificity, and narrow benchmark suites that test for known biases but miss systemic effects. SEED-SET occupies a middle ground — technically rigorous enough to integrate into a deployment pipeline, but flexible enough to accommodate different stakeholders' definitions of what "ethical" means in context.
The use of LLMs as stakeholder proxies will draw scrutiny. Language models carry their own biases, and using one AI system to evaluate another raises legitimate questions about circular reasoning. The MIT team acknowledges this limitation, positioning the LLM proxy as a screening tool rather than a final arbiter — a way to flag potential issues for human review rather than to certify a system as ethically sound.
The Larger Implication
SEED-SET's most valuable contribution may be conceptual rather than technical. By formalizing the separation between objective performance and subjective values — and by building a system that actively hunts for conflicts between the two — it provides a template for how ethical evaluation could work at scale.
The framework does not answer the question of what is ethical. It answers a more practical question: where should we look? And in a landscape where autonomous systems are being deployed faster than they can be manually reviewed, knowing where to look may be the most important capability of all.
