|
Getting your Trinity Audio player ready...
|
During the ATxAI conference on the second day of Asia Tech x Singapore (ATxSG) 2024, Singapore’s Minister for Communications and Information, Mrs. Josephine Teo, introduced AI Verify Project Moonshot, a pioneering testing toolkit aimed at addressing the security and safety challenges of large language models (LLMs).
As a cornerstone of many AI-driven solutions, LLMs require robust testing, and Moonshot stands out as one of the world’s first open-sourced tools to integrate red-teaming, benchmarking, and baseline testing into a user-friendly platform. This initiative underscores Singapore’s commitment to leveraging the global open-source community to mitigate AI risks.
Project Moonshot is currently in its open beta phase, designed to provide clear, intuitive results regarding the quality and safety of a model or application, making it accessible even to non-technical users. Developed in collaboration with industry partners such as DataRobot, IBM, Singtel, and Temasek, the toolkit ensures relevance and alignment with industry needs.
Anup Kumar, Distinguished Engineer, CTO Data & AI, Head Client Engineering Asia Pacific at IBM, remarked, “IBM is pleased to be a design partner and contributor to AI Verify Moonshot. The provision of this new tool is significant as it aims to help developers and data scientists test their LLM applications against a baseline of risks, thereby accelerating the adoption of AI. We look forward to working closely with IMDA to develop appropriate open standards through our contributions.”
Similarly, Jay Schuren, Chief Customer Officer at DataRobot, expressed enthusiasm about the partnership: “DataRobot is thrilled to partner with IMDA to make LLM evaluation more accessible. The integration of Project Moonshot into the DataRobot AI Platform allows AI builders to confidently and responsibly scale generative AI within their organisations.”
Project Moonshot represents a significant step towards establishing global testing standards. Leading AI testing organisations, AI Verify Foundation and MLCommons, have joined forces to develop a common safety benchmark suite.
Peter Mattson, President of MLCommons and Co-chair of the AI Safety working group, highlighted this collaboration: “We are proud that leading AI testing organisations AI Verify Foundation and MLCommons are coming together to support common testing benchmarks for AI safety, building on innovative efforts like Project Moonshot. To further the collaboration, AI Verify Foundation and MLCommons have signed a memorandum of intent (MOI) on 29th May, intending to positively impact AI safety by providing model and application developers with a globally accepted approach to safety testing for generative AI.”
The AI Verify Foundation (AIVF) marked its first anniversary at ATxSG. Since its launch in June 2023, AIVF has doubled its membership to over 120, with new premier members including Amazon Web Services (AWS) and Dell. The foundation’s scope has expanded from AI testing tools to developing trust-enhancing AI safety products such as the Model AI Governance Framework for Generative AI.
At the foundation’s first-year anniversary celebration, Dr Ong Chen Hui, Chair of the Governing Committee at AIVF, shared insights from user feedback, stressing the need for AI Verify to align with emerging standards and cater to sector-specific testing. In response, she announced the completion of AI Verify’s mapping with ISO 42001 and plans to integrate AI Verify with the Monetary Authority of Singapore’s (MAS) Veritas toolkit.
The United Nations Development Programme (UNDP), in collaboration with IMDA, also played a significant role at the ATxSummit by convening a high-level roundtable and launching a 100-day Global Consultation and Digital Inclusion Playbook. This playbook includes IMDA’s Digital Skills for Life (DSL) framework, which supports the national Digital for Life (DfL) movement, among other global case studies. The roundtable initiated a consultation process with a productive dialogue among senior government officials, international organisations, private sector representatives, and civil society.
The launch of Project Moonshot and the first anniversary of the AI Verify Foundation highlight Singapore’s proactive approach to ensuring AI safety and security. By fostering collaborations with industry leaders and international organisations, Singapore is paving the way for the responsible and effective use of AI technologies globally. The initiatives presented at ATxSG 2024 reflect a commitment to innovation, safety, and inclusivity in the digital age.
