What happened
Microsoft launched ASSERT, an open-source framework converting natural-language descriptions of AI goals and policies into thorough, scored tests. This tool generates problem scenarios, runs them against target systems, and scores results, recording AI system paths for inspection. Developers can specify system context, tools, and constraints, for example, preventing an AI agent from emailing external contacts or restricting confidential information to C-level executives. Microsoft states ASSERT fills a gap in application-specific AI behaviour evaluation, enabling continuous monitoring and evaluation during and after deployment.
Why it matters
Application-specific AI behaviour enforcement becomes automated, reducing manual testing overhead for platform engineers and security architects. ASSERT's mechanism converts high-level policy descriptions into verifiable test cases, providing continuous monitoring of AI systems against defined constraints. This capability addresses the gap where general AI evaluations fall short, ensuring models adhere to product-specific rules and organisational policies. Procurement teams evaluating AI solutions gain a tool to validate vendor claims against internal compliance requirements. This release follows a broader industry shift towards repeatable testing and regression checks, as seen with Stanford's HELM and MLCommons' AILuminate.
