The Rise of AI Products: The Pre-ChatGPT Journey
Over the past seventy-five years, leading up to the emergence of ChatGPT, artificial intelligence has evolved remarkably. Starting with the early days of symbolic AI, supercomputers, and expert systems, artificial intelligence has advanced to achieve the groundbreaking capabilities seen in deep learning and large language models (LLMs). This post aims to underscore the pivotal advancements and technological milestones that have shaped today's AI applications. It is not intended as an exhaustive catalog of technologies, but rather to spotlight key developments in the field over time:
First Wave: 1950s-1970s. The initial wave of AI development spanned from the 1950s to the 1970s, concentrating on symbolic AI and rules-based systems. In 1951, Marvin Minsky and Dean Edmonds at MIT pioneered the first neural network simulator, and subsequently, Minsky created SNARC (Stochastic Neural Analog Reinforcement Calculator), an early attempt to model the brain's learning processes using neural networks. "Logic Theorist," introduced in 1956 for an audience of scientists, mathematicians, and computer scientists, was designed to prove theorems in symbolic logic. In 1967, "Logo" was introduced as an educational programming language, notable for its graphical turtle features. Logo’s main audience was educators and researchers in computer science, but it was also used to teach programming and computational thinking to children. During this period, expert systems like “DENDRAL”, aimed at automating chemical analysis and determining the molecular structure of organic compounds, and “MYCIN”, a medical diagnostic system for identifying bacterial infections and recommending antibiotics. At the Department of Defense several expert systems were developed and deployed as Cold War ICBM and war simulation technologies. Backpropagation was invented in the 1970s as a general optimization method for automatically differentiating complex nested functions. However, limited computational power and unmet expectations led to declining enthusiasm and funding for AI research.
Second Wave: 1980s-2000s. By the late 1980s and 1990s, methods were developed for dealing with uncertain or incomplete information, employing concepts from probability and economics. Also, during this period, machine learning (ML) and statistical approaches began to rise, even though the industry was experiencing an "AI Winter" marked by reduced interest and funding from government and academic circles. Despite these challenges, neural networks and ML techniques gained traction, driving advancements in data-driven learning and pattern recognition. Progress was made despite ongoing difficulties regarding limited data availability and computational constraints.
In 1983, Sheryl Handler and Danny Hillis founded Thinking Machines Corporation (TMC) as a supercomputer and AI company, based on their MIT doctorate in massively parallel computing architectures. Over the years, many of TMC's employees transitioned into key roles within the hardware and software industry – including Digital Equipment Corporation (DEC) which was a business and scientific computing company, leading to further innovations in AI and supercomputing.
In 1997, IBM's Deep Blue, a chess-playing computer, defeated the reigning world champion Garry Kasparov in a six-game match, marking a significant milestone in the history of AI. According to IBM, Deep Blue wasn't just a breakthrough for the world of chess. Its underlying technology advanced the ability of supercomputers to tackle the complex calculations needed to discover new pharmaceuticals, assess financial risk, uncover patterns in massive databases, and explore the inner workings of human genes.
During the early Web 1.0 period from 2000 to 2010, there were many attempts and effective integrations of AI into web apps, partly because of the plethora of data produced by web page interactions and online customer feedback and purchases.
And the general view that broad ai wasn’t feasible but “expert systems” like medical transcription — was where the practical apps would be.
Third Wave: 2010-2023. The past decade-plus has seen significant growth in AI products, driven by advancements in deep learning and big data.
2010-2015: During this period, Siri (2011) and Google Now (2012) were early AI assistants that were introduced, using basic natural language processing (NLP) to perform simple tasks. Deep learning algorithms began personalizing recommendations on platforms like Netflix and Amazon, while products like Picasa pioneered image recognition.
2015-2020 – the Deep Learning and Big Data Boom: AI assistants such as Alexa and Google Assistant became more sophisticated, used more advanced natural language, and performed more complex tasks. Chatbots powered by deep learning improved customer service and tools like Google Translate achieved near-human accuracy in machine translation. Early generative AI models also helped advance natural-sounding text-to-speech synthesis for audiobooks and voice assistants.
2020-2023: The Age of Powerhouse Models was ushered in with the release of GPT-3 in 2020 marking a new era. Large language models represented a distinctive game changer by generating realistic text to inquiries, translating languages, and answering questions informatively. AI tools like DALL-E 2 in 2022 began creating high-quality images from text descriptions. AI further enhanced personalization across platforms, tailoring user experiences with vast datasets.
Today's generative AIs represent significant advancements in scale, deep learning, NLP, adaptability, and user interaction, distinguishing them from earlier technologies and making them powerful tools in today's AI landscape. It's important to note that there are many generative AIs, each with unique features, offering a range of choices and rapid innovation. Unlike past efforts, such as Deep Blue, the diversity and pace of development in AI today are remarkable. Over the past 75 years, AI products, combined with developments in open-source and commercial software, Big Data analytics, neural networks, large datasets, and advancements in image recognition, speech recognition, computational power, and cloud services, have revolutionized AI technology, paving the way for the versatile applications we see today.