|
Getting your Trinity Audio player ready...
|
Generative AI has become a focal point due to its capacity to produce text and images. Its impact is profound and multifaceted, shaping a wide array of scenarios, such as when a patient undergoes medical treatment, a storm disrupts flight schedules or an individual interacts with software applications.
Leveraging generative AI to create realistic synthetic data in these scenarios can help organisations effectively address challenges like patient care, flight rerouting, or software platform improvement, particularly in cases where real-world data are scarce or sensitive.
For the past three years, MIT spinout DataCebo has offered the Synthetic Data Vault (SDV), a generative software system designed to assist organisations in creating synthetic data for software testing and machine learning model training.
SDV is an open-source library for generating synthetic tabular data that has been downloaded over 1 million times, with more than 10,000 data scientists utilising it. The founders, Principal Research Scientist Kalyan Veeramachaneni and alumna Neha Patki ’15, SM ’16, attribute the company’s success to SDV’s transformative impact on software testing.
In 2016, Veeramachaneni’s group at the Data to AI Lab introduced a suite of open-source generative AI tools to help organisations create synthetic data that mirrors the statistical properties of real data. Using synthetic data allows companies to protect sensitive information while maintaining the statistical relationships between data points. Additionally, synthetic data can be used to simulate new software performance before its public release.
The inspiration for SDV came from the group’s work with companies willing to share their data for research purposes. This diverse exposure across industries helped them realise the versatility of their tools.
In 2020, DataCebo was founded to enhance SDV features for larger organisations. Since then, the range of applications has been diverse. For instance, DataCebo’s flight simulator enables airlines to plan for rare weather events in ways previously impossible using only historical data. In another application, SDV users synthesised medical records to predict health outcomes for cystic fibrosis patients.
In 2021, Kaggle hosted a competition for data scientists using SDV to create synthetic datasets, attracting roughly 30,000 participants who built solutions and predicted outcomes based on the company’s realistic data.
Despite their tools being used for various purposes, DataCebo primarily focuses on expanding its presence in software testing. Veeramachaneni explained, “People need data to test these software applications. Traditionally, developers manually write scripts to create synthetic data. With generative models created using SDV, you can learn from a sample of data collected and then sample a large volume of synthetic data (which has the same properties as real data) or create specific scenarios and edge cases and use the data to test your application.”
Patki added, “It is common for industries to have sensitive data in some capacity. Hence, synthetic data is always better from a privacy perspective.”
DataCebo believes it is advancing the “synthetic enterprise data” field generated from user behaviour on large companies’ software applications.
Veeramachaneni emphasises that “Enterprise data of this kind is complex, and there is no universal availability, unlike language data. When folks use our publicly available software and report back if it works on a certain pattern, we learn many of these unique patterns, allowing us to improve our algorithms. From one perspective, we are building a corpus of these complex patterns, which for language and images is readily available.”
DataCebo has also released new features to enhance SDV’s utility, including the SDMetrics library to assess the “realism” of generated data and the SDGym to compare models’ performances.
Their tools aim to ensure organisations trust this new data. Veeramachaneni stated, “The tools over programmable synthetic data allow enterprises to insert their specific insight and intuition to build more transparent models.”
As companies across industries embrace AI and other data science tools, DataCebo is helping them do so transparently and responsibly. “In the next few years, synthetic data from generative models will transform all data work,” Veeramachaneni asserted. “We believe 90% of enterprise operations can be done with synthetic data.”
