Computer-made data for AI training is now the norm in many industries.

We have all heard of platforms like ChatGPT that are trained on massive ‘real world’ data – but did you know that there is a growing trend for training AI using data that has itself been computer-made?

I don’t know about you, but that feels like being stuck in an infinite loop that gets more and more ‘not real’ as it goes along…

[If this concept is completely new to you, you might like to read this post first:  https://bellastjohn.com/harnessing-synthetic-data-advancing-ai-training-while-navigating-challenges/]

What “Miracle on the Hudson” taught us about relying on computer-made data for AI training

In the world of aviation, Captain Chesley “Sully” Sullenberger became a household name for his miraculous emergency landing of US Airways Flight 1549 on the Hudson River in 2009, saving the lives of all 155 people on board. His exceptional skills and quick decision-making showcased the vital role of human judgment in critical situations. However, an often-overlooked aspect of this remarkable incident lies in the lessons it offers about the potential pitfalls of relying too heavily on computer-made data to train AI models.

In the aftermath of the “Miracle on the Hudson,” investigators conducted simulations to understand the events leading up to the water landing. While these simulations provided valuable insights into the sequence of events and the performance of the aircraft systems, they failed to accurately capture the human element—the split-second decisions, instincts, and experience that Sullenberger and his co-pilot, Jeffrey Skiles, brought to the table.

This critical omission underscores a fundamental challenge in the development and training of AI models: the danger of overlooking human reactions and response times when relying on computer-made data. Here are some important lessons we can draw from this incident:

1. The Complexity of Human Decision-Making: The quick thinking and sound judgment exhibited by Captain Sullenberger and his crew in an emergency situation like Flight 1549 cannot be replicated solely through computer-generated scenarios. Human decision-making involves intuition, experience, and a deep understanding of context—a complexity that is challenging to simulate accurately.

2. Real-World Factors and Uncertainty: Simulations are typically based on known variables and data, but real-life emergencies often involve unexpected factors and uncertainties. Human operators have the ability to adapt and make critical decisions in response to evolving situations, which is difficult to capture in a controlled synthetic environment.

3. The Role of Stress and Pressure: In high-stress situations, human performance can be significantly affected. Stress and pressure can lead to heightened awareness and focus, but they can also impair decision-making. Understanding how humans react under pressure is essential for designing AI systems that can complement human actions effectively.

4. The Need for Hybrid Models: The limitations of computer-made data in replicating human decision-making suggest the importance of hybrid models that integrate both synthetic and real-world data. Such models can better prepare AI systems for handling complex, high-stakes scenarios by accounting for the unpredictable nature of human responses.

5. Ethical Considerations: The reliance on computer-made data alone can have ethical implications, especially in fields like aviation, healthcare, and autonomous vehicles, where human lives are at stake. Ensuring that AI systems are trained to make ethical decisions in critical situations requires a nuanced understanding of human values and moral judgment.

Why use computer-made data?

I get that training AI models takes a huge amount of data, and it is obvious how using ‘fake data’ to test the security systems of financial institutions, for instance, might make sense.

…but it is not stopping there.

Examples of computer-made data for AI training

I asked ChatGPT to provide a list of 20, along with some concerns:

1. “Healthcare Technology Companies:
   • Concern: Privacy and security concerns when generating synthetic medical data to protect patient information.
2. Financial Services and Banking Institutions:
   • Concern: Ensuring that synthetic data for fraud detection and risk assessment doesn’t introduce biases or ethical issues.
3. Autonomous Vehicle Manufacturers:
   • Concern: The accuracy of synthetic data in simulating real-world driving scenarios and safety implications.
4. Insurance Companies:
   • Concern: The fairness and accuracy of synthetic data used for pricing, underwriting, and claims assessment.
5. Retail and E-commerce Platforms:
   • Concern: The potential for synthetic data to affect personalized recommendations and customer profiling.
6. Manufacturing and Supply Chain Management Companies:
   • Concern: The reliability of synthetic data for optimizing production processes and logistics.
7. Agriculture Technology Firms:
   • Concern: The effectiveness of synthetic data in modeling crop yield predictions and pest management.
8. Energy and Utilities Providers:
   • Concern: The accuracy of synthetic data for grid optimization and energy demand forecasting.
9. Aerospace and Defense Contractors:
   • Concern: The realism of synthetic data for simulating aircraft performance and military scenarios.
10. Environmental Monitoring Organizations:
    • Concern: The validity of synthetic data in climate modeling and predicting environmental changes.
11. Education Technology Startups:
    • Concern: Ensuring the relevance and fairness of synthetic data for personalized learning and educational assessments.
12. Entertainment and Gaming Companies:
    • Concern: The authenticity of synthetic data used in video game environments and virtual reality experiences.
13. Human Resources and Talent Management Firms:
    • Concern: Bias and fairness issues in synthetic data used for recruitment and workforce analytics.
14. Real Estate and Property Management Companies:
    • Concern: The reliability of synthetic data for property valuation and market analysis.
15. Telecommunications Providers:
    • Concern: Privacy considerations when generating synthetic data for network optimization and customer behavior analysis.
16. Government Agencies (e.g., Census Bureau):
    • Concern: Ensuring the accuracy and fairness of synthetic data for policy-making and resource allocation.
17. Logistics and Delivery Services:
    • Concern: The realism of synthetic data used to optimize delivery routes and predict package delivery times.
18. Social Media Platforms:
    • Concern: The potential impact of synthetic data on content recommendation algorithms and user experiences.
19. Pharmaceutical and Biotech Companies:
    • Concern: Ethical and regulatory issues when using synthetic data in drug discovery and clinical trials.
20. Legal Services and LegalTech Startups:
    • Concern: Ensuring the validity and fairness of synthetic data used for legal research and predictive analytics.”

It is also interesting that a few days ago when I asked a couple of friends who are in real estate whether they use AI, they both replied ‘no’ due to lack of perceived reliability of data (also referenced above).

Incorporating human factors into AI training is not a straightforward task, but it is essential for creating AI systems that can operate effectively and safely in the real world. While simulations and synthetic data have their place in training models, they should complement, not replace, the rich insights gained from real-world experience.

The “Miracle on the Hudson” serves as a powerful reminder that AI, no matter how advanced, cannot entirely replace the judgment, instincts, and adaptability of humans in high-stakes situations. As we continue to develop AI systems for various applications, it is crucial that we remain cognizant of the limitations of computer-made data and strive for a more holistic approach that honours the irreplaceable role of human expertise and experience in complex decision-making processes.

~ Bella

In the rapidly evolving landscape of artificial intelligence (AI) and machine learning, the quest for data remains paramount. However, as the demand for high-quality training data outpaces its supply, a powerful alternative has emerged: computer-made data, or synthetic data. This article explores the concept of synthetic data, the concerns associated with its use in AI training, how companies employ it, and the compelling reasons behind this adoption.

What is Synthetic Data?

Synthetic data is computer-generated information designed to mimic real-world data. It is born within algorithms, mathematical models, or software systems rather than originating from actual observations or measurements. The primary goal is to create data that closely resembles real data, facilitating the training of AI models.

Concerns for Using Synthetic Data in AI Training

While synthetic data offers numerous advantages, it is not without its challenges and concerns:

1. Quality Assurance: Ensuring the fidelity and authenticity of synthetic data can be challenging. Poor-quality synthetic data may lead to models that struggle to generalize to real-world scenarios.
2. Overfitting: Models trained extensively on synthetic data can become overly tailored to the synthetic dataset, diminishing their ability to perform well in diverse real-world situations.
3. Bias and Fairness: Biases embedded in the algorithms generating synthetic data can perpetuate or even introduce biases into AI models, leading to unfair or discriminatory outcomes.
4. Privacy Risks: Although synthetic data is generated, it can still inadvertently expose sensitive information if not meticulously designed and validated.

How Companies Utilize Synthetic Data for AI Training

Companies across various industries leverage synthetic data for a multitude of purposes:

1. Data Scarcity Mitigation: In sectors where obtaining sufficient real-world data is challenging or costly, synthetic data supplements the available dataset, enabling more robust AI training.
2. Privacy Enhancement: Synthetic data allows organizations to create datasets that retain the statistical properties of real data without exposing sensitive or private information, crucial in fields like healthcare and finance.
3. Robustness and Diversity: Synthetic data introduces controlled variations and edge cases to improve model robustness and adaptability to different scenarios.
4. Cost-Efficiency: Generating synthetic data is often more cost-effective than collecting, cleaning, and annotating large volumes of real-world data.
5. Testing and Validation: Companies create standardized testing and evaluation datasets using synthetic data to assess AI model performance in controlled environments, ensuring fair comparisons.

Why Companies Embrace Synthetic Data

Companies adopt synthetic data for several compelling reasons:

1. Data Limitations: In many cases, real-world data is insufficient or unavailable, hindering AI model development and testing. Synthetic data addresses this data scarcity issue.
2. Privacy Preservation: Synthetic data enables organizations to work with data while safeguarding sensitive information, ensuring compliance with data protection regulations.
3. Robust Model Training: By introducing synthetic data alongside real data, companies enhance model robustness and resilience, leading to more reliable AI systems.
4. Cost Savings: The cost-effective nature of synthetic data generation appeals to organizations looking to optimize their AI development budget.
5. Ethical Considerations: Synthetic data can help organizations reduce biases in AI models and ensure fairness and equity in decision-making processes.

In summary, synthetic data has emerged as a crucial tool in the arsenal of companies seeking to train AI models effectively while navigating data limitations and privacy concerns. While challenges exist, careful validation, ethical considerations, and a thoughtful integration of synthetic and real data can result in AI systems that perform robustly and ethically in diverse real-world scenarios. As AI continues to transform industries, synthetic data is poised to play a pivotal role in its ongoing advancement.

[Please note:  this post was written by ChatGPT as an informative background and pre-reading if desired for the post written by Ms Bella St John: https://bellastjohn.com/the-danger-of-computer-made-data-for-ai-training]