The Threat of ‘Knowledge Saturation’ to AI Advancement and Research

Introduction

Solving data storage, manipulation, availability and manipulation were the earliest motivators to computing advancements. Many decades before the first computing device was invented people still managed information across multiple domains in the form of hardcopy documents and files. As technology advanced and computing devices became a reliable form of data storage knowledge became more organized and available for further analysis. The reliance on computers for storage gave rise to other topics of interest such as information security and data visualization. The foundation of information technology is built on how data is extracted, processed, stored and made accessible for decision making. Millions of terabytes of data are produced every day and this figure is projected to rise steadily in the next few years. The large volume of data means that AI researchers have a large pool of datasets to train high performance models. This paper will answer the question whether AI advancement is at the threat of a sharp decline if we solely rely on AI generated outputs at the expense of more original human generated content.

Current Status of AI Advancement

In the past five years AI models and assistants have become very popular for accomplishing routine tasks and even accomplishing deep reasoning for complex work. Many organizations are already replacing their workforce with AI in repetitive and automatable jobs. The most affected are tech related roles where AI agents are already taking center stage in generating production level code. When text generation models gained prominence about three years ago they were only able to provide information up to a specified time frame. This is because they only relied on a knowledge base that already had the training data verified only for that timeframe. However, most Large Language Models(LLM) , which are the most popularly used tools, are now able to scan the internet in real-time and gain access to a wider range of information. The LLMs have also advanced to include support for audio and visual content generation at high precision. Structured textual data has always been more readily available for training machine learning models compared to audio and visual data. Most databases already store textual data in a format that algorithms can easily recognize and it's also easy to extract this data in structured datasets for use in AI training.

The quality of outputs from AI tools is as good as the data used to train the model. An AI model can only learn to easily recognize patterns or generate accurate content when exposed to a large volume of preexisting data. To achieve the high level of performance and accuracy witnessed in popular LLMs the relevant organizations have invested a lot in data cleaning and annotation; to a level of hiring dedicated teams to evaluate the data for accuracy and organize it in the correct format before feeding it to the model. Three years ago when using the initial version of the LLM tools it was common to get a response like; “I do not have capacity to assist you with that task” or “I do have access to that information”. These drawbacks have largely been resolved as AI models become exposed to more data and gained capacity to query more data sources. Additionally, AI has become embedded in almost every software tool imaginable. Be it your favorite IDE to help autocomplete code, a photo editing software or even a messaging tool to remind you of that email you sent out five days ago and haven’t received a response or the way that message should be best composed for different use cases. In addition to the structured training data sets AI is now exposed to personal information in our devices where it continues to learn for optimized responses.

Knowledge Saturation

Access to a wide range of information is very good for AI. It means that it has access to all knowledge imaginable thus increasing its capacity to give us accurate responses that are relevant to what a human actor would produce. But are we producing enough original knowledge to keep up with the training pace that AI anticipates? We are now relying on AI for all our repetitive tasks and taking the content it provides to our workflows with very minimal modifications. The phenomenon of ‘deep fakes’ where in many occasions individuals are sometimes unable to differentiate between fake and real content. There are also scenarios where originally written content is marked as AI generated content whenever passed through plagiarism checker tools. These examples go to show that AI already has access to all information and thought patterns that human beings utilize when producing their work. Therefore, there is no creativity and fresh ideas when human beings consistently rely on AI generated content and pass the same autogenerated content to train AI models.

The core tenet of AI is that we were supposed to pass human intelligence to machines and make them mimic human decision making capacity and precision. Even before the AI revolution computers were still very useful to us for tasks that required preprogrammed logic. However, the extent of information that could be drawn through programs was highly limited especially for tasks that required human level intelligence. For instance, such programs did not have the ability to memorize user’s routines and help them complete their tasks autonomously.

Impacts on AI Advancement

In reality, AI cannot be relied upon to lead to novel ideas and discoveries. New discoveries and inventions are a result of intense research with access to realworld scenarios and extensive information access. Knowledge saturation means that AI models get stuck in their current level and can only excel in generating repetitive and recycled information. The main offsets of knowledge saturation is the decrease in research productivity especially because people will rely on AI for research purposes. Ultimately there will be little incentive to invest in AI advancement since it is not giving any new knowledge or something we do not know how to do yet. The degradation in output quality due to saturated information will also make AI less reliable in fields where cognitive thinking is essential such as medicine.

Conclusion

There is no denying that AI is thriving and has definitely made it easy for humans to complete routine tasks. However, if we consistently rely on AI as the sole information source and recycle the same data in training new models then the quality of outputs is highly degraded. As we continue to reap the benefits of AI it is imperative to invest highly in quality data and encourage avenues for novel research and discoveries that do not rely on AI outputs.