| | {{date|2026-10-12}} |
| | |
| | # History of AI (Artificial Intelligence) |
| | |
| | [[File:alan-turing-1951.jpg|right|200px|Alan Turing, 1951. Public domain.]] |
| | |
| | Artificial intelligence — the effort to build machines that can perceive, reason, learn, and act — is one of the defining scientific undertakings of the modern era. Its history spans seven decades of ambition, breakthrough, disillusionment, and reinvention, from a handful of mathematicians asking whether machines could think to a global industry reshaping every sector of human activity. |
| | |
| | This article traces that history from Alan Turing's foundational 1950 paper through the AI winters, the expert systems boom and bust, the deep learning revolution, and the frontier models of the mid-2020s. |
| | |
| | --- |
| | |
| | # 1950–1956: the question and the name |
| | |
| | ## Turing's paper |
| | |
| - | On October 1, 1950, the British mathematician Alan Turing published "Computing Machinery and Intelligence" in the journal *Mind*. Rather than attempt to define the word "think," Turing replaced the question "Can machines think?" with a behavioural test: the **Imitation Game**. In the game, a human interrogator communicates via text with a machine and a human. If the interrogator cannot reliably distinguish the machine from the human, the machine passes the test[1]. |
| + | On October 1, 1950, the British mathematician Alan Turing published "Computing Machinery and Intelligence" in the journal *Mind*. Rather than attempt to define the word "think," Turing replaced the question "Can machines think?" with a behavioural test: the **Imitation Game**. In the game, a human interrogator communicates via text with a machine and a human. If the interrogator cannot reliably distinguish the machine from the human, the machine passes the test[[cite: A. M. Turing, "Computing Machinery and Intelligence," *Mind* LIX(236), October 1950, pp. 433–460 — [doi:10.1093/mind/LIX.236.433](https://doi.org/10.1093/mind/LIX.236.433)]]. |
| | |
| | Turing's paper did not propose a specific program or architecture. It proposed a *criterion* — a way of measuring whether a machine exhibits intelligent behaviour without requiring a philosophical definition of intelligence. The paper remains one of the most cited in all of computer science. |
| | |
| | ```fact |
| | Turing predicted in his 1950 paper that by the year 2000, machines with 128 MB of memory would be able to fool 30% of human judges in a five-minute conversation. He was remarkably close — modern chatbots achieved this benchmark in the 2010s, and large language models surpassed it decisively by 2023. |
| | ``` |
| | |
| | ## The Dartmouth workshop |
| | |
| | [[File:dartmouth-college.jpg|right|250px|Dartmouth College, Hanover, New Hampshire. Photo by Gavin Huang, CC BY-SA 3.0.]] |
| | |
| | In the summer of 1956, a group of ten researchers gathered at Dartmouth College in Hanover, New Hampshire, for an eight-week workshop that would become the founding event of artificial intelligence as a formal field of study. |
| | |
| | The workshop was proposed on August 31, 1955, by four organizers: **John McCarthy** (Dartmouth), **Marvin Minsky** (Harvard), **Nathaniel Rochester** (IBM), and **Claude Shannon** (Bell Telephone Laboratories). Their proposal stated: |
| | |
| | ```quote |
| | "We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it." — Dartmouth AI Proposal, August 31, 1955 |
| | ``` |
| | |
| - | The term "artificial intelligence" was coined by McCarthy for this proposal. He chose it deliberately — avoiding "cybernetics" (which was associated with Norbert Wiener and analog feedback systems) and "automata theory" (which was too narrow). The name stuck[2]. |
| + | The term "artificial intelligence" was coined by McCarthy for this proposal. He chose it deliberately — avoiding "cybernetics" (which was associated with Norbert Wiener and analog feedback systems) and "automata theory" (which was too narrow). The name stuck[[cite: J. McCarthy, M. L. Minsky, N. Rochester, and C. E. Shannon, ["A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence"](http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html), August 31, 1955; reprinted in *AI Magazine* 27(4), 2006]]. |
| | |
| - | The Dartmouth workshop did not produce a single breakthrough result. What it produced was a **community** — a group of researchers who now shared a name, a goal, and a set of problems. The attendees included Allen Newell and Herbert A. Simon, who had already built the Logic Theorist (a program that proved theorems from Whitehead and Russell's *Principia Mathematica*), and Arthur Samuel, who was developing a checkers-playing program at IBM that would become one of the first demonstrations of [[Machine_Learning|machine learning]][2]. |
| + | The Dartmouth workshop did not produce a single breakthrough result. What it produced was a **community** — a group of researchers who now shared a name, a goal, and a set of problems. The attendees included Allen Newell and Herbert A. Simon, who had already built the Logic Theorist (a program that proved theorems from Whitehead and Russell's *Principia Mathematica*), and Arthur Samuel, who was developing a checkers-playing program at IBM that would become one of the first demonstrations of [[Machine_Learning|machine learning]][[cite: J. McCarthy, M. L. Minsky, N. Rochester, and C. E. Shannon, ["A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence"](http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html), August 31, 1955; reprinted in *AI Magazine* 27(4), 2006]]. |
| | |
| | --- |
| | |
| | # 1956–1974: the first summer |
| | |
| | The years following the Dartmouth workshop were marked by extraordinary optimism. Researchers built programs that could prove mathematical theorems, play games, solve word problems, and understand simple English sentences. Funding flowed from DARPA (then ARPA), which saw AI as potentially transformative for military applications. |
| | |
| | ## Key achievements |
| | |
| | | Year | System | Creator(s) | Achievement | |
| | |---|---|---|---| |
| | | 1956 | Logic Theorist | Newell, Shaw, Simon | Proved 38 of the first 52 theorems in *Principia Mathematica* | |
| | | 1958 | Perceptron | Frank Rosenblatt | First neural network with a learning algorithm | |
| | | 1958 | LISP | John McCarthy | Programming language that would dominate AI for three decades | |
| | | 1961 | Unimate | George Devol | First industrial robot, installed at GM | |
| | | 1966 | ELIZA | Joseph Weizenbaum | First chatbot; simulated a Rogerian psychotherapist | |
| | | 1970 | SHRDLU | Terry Winograd | Understood English commands about a simulated blocks world | |
| | | 1969 | Shakey | SRI International | First mobile robot to reason about its actions | |
| | |
| | ## The perceptron |
| | |
| | [[File:perceptron-diagram.png|right|250px|Diagram of a perceptron. Public domain.]] |
| | |
| - | In 1958, Frank Rosenblatt at the Cornell Aeronautical Laboratory built the **Mark I Perceptron**, the first hardware implementation of a [[Neural_Networks|neural network]] with adjustable weights and a learning algorithm. The perceptron could learn to classify simple patterns — a remarkable demonstration that machines could learn from data rather than being explicitly programmed[3]. |
| + | In 1958, Frank Rosenblatt at the Cornell Aeronautical Laboratory built the **Mark I Perceptron**, the first hardware implementation of a [[Neural_Networks|neural network]] with adjustable weights and a learning algorithm. The perceptron could learn to classify simple patterns — a remarkable demonstration that machines could learn from data rather than being explicitly programmed[[cite: F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," *Psychological Review* 65(6), 1958 — [doi:10.1037/h0042519](https://doi.org/10.1037/h0042519)]]. |
| | |
| | The perceptron generated enormous excitement. The *New York Times* ran a story in 1958 with the headline "New Navy Device Learns By Doing." Rosenblatt himself made bold claims about the perceptron's future potential. |
| | |
| - | But the perceptron had a fundamental limitation: it could only learn **linearly separable** functions. In 1969, Marvin Minsky and Seymour Papert published *Perceptrons*, a mathematical analysis proving this limitation and showing that a single-layer perceptron could not learn the XOR function. The book was widely interpreted as a proof that neural networks were a dead end — a conclusion Minsky and Papert did not explicitly make, but one that had devastating consequences for neural network research funding[3]. |
| + | But the perceptron had a fundamental limitation: it could only learn **linearly separable** functions. In 1969, Marvin Minsky and Seymour Papert published *Perceptrons*, a mathematical analysis proving this limitation and showing that a single-layer perceptron could not learn the XOR function. The book was widely interpreted as a proof that neural networks were a dead end — a conclusion Minsky and Papert did not explicitly make, but one that had devastating consequences for neural network research funding[[cite: M. Minsky and S. Papert, *Perceptrons*, MIT Press, 1969 — [mitpress.mit.edu](https://mitpress.mit.edu/9780262534771/perceptrons/)]]. |
| | |
| | ## ELIZA and the illusion of understanding |
| | |
| | In 1966, Joseph Weizenbaum at MIT created **ELIZA**, a program that simulated a Rogerian psychotherapist by applying simple pattern-matching rules to the user's input. ELIZA would rephrase the user's statements as questions ("I feel sad" → "Why do you feel sad?") and offer generic reflective prompts when it had no pattern to match. |
| | |
| | ELIZA was intended as a demonstration of the superficiality of human-machine communication. Weizenbaum was shocked to discover that many users — including his own secretary — formed emotional attachments to the program, confiding in it and attributing understanding to it. Some psychiatrists even suggested that ELIZA could serve as a genuine therapeutic tool. |
| | |
| | Weizenbaum, horrified, spent the rest of his career warning against the tendency to anthropomorphize machines. His 1976 book *Computer Power and Human Reason* argued that certain applications of computing — even if technically possible — should be forbidden on ethical grounds. In retrospect, ELIZA anticipated the debates that would erupt sixty years later around ChatGPT, Claude, and other conversational AI systems: the human instinct to attribute understanding to anything that produces fluent language is powerful, persistent, and potentially dangerous. |
| | |
| | ## The optimism problem |
| | |
| | The early researchers made predictions that seem almost comically overconfident in retrospect: |
| | |
| | - Herbert Simon (1957): "Within ten years a digital computer will be the world's chess champion." (It took 40 years.) |
| | - Marvin Minsky (1967): "Within a generation the problem of creating 'artificial intelligence' will substantially be solved." (The generation came and went; the problem remains open.) |
| | - The ALPAC report (1966): A committee convened under the National Academy of Sciences concluded that machine translation was not worth pursuing, leading to a near-complete cut in MT funding. The field would not recover for decades. |
| | |
| - | These predictions were not made out of ignorance. The researchers had genuine reasons to be optimistic — their programs were solving problems that had seemed impossible just years before. What they underestimated was the gap between solving toy problems and solving real-world problems at scale. The phenomenon would later be formalized as **Moravec's Paradox**: tasks that are easy for humans (recognizing faces, walking, understanding context) are extraordinarily hard for computers, while tasks that are hard for humans (chess, mathematics, logic) are comparatively easy to automate[4]. |
| + | These predictions were not made out of ignorance. The researchers had genuine reasons to be optimistic — their programs were solving problems that had seemed impossible just years before. What they underestimated was the gap between solving toy problems and solving real-world problems at scale. The phenomenon would later be formalized as **Moravec's Paradox**: tasks that are easy for humans (recognizing faces, walking, understanding context) are extraordinarily hard for computers, while tasks that are hard for humans (chess, mathematics, logic) are comparatively easy to automate[[cite: Actuaries Institute, ["A history of AI winters"](https://actuaries.digital/2023/03/20/a-history-of-ai-winters/), actuaries.digital]]. |
| | |
| | ```quote |
| | "It is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, but difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility." — Hans Moravec, *Mind Children*, 1988 |
| | ``` |
| | |
| | --- |
| | |
| | # 1974–1980: the first AI winter |
| | |
| | ```fact |
| | The term "AI winter" was coined by analogy with "nuclear winter." It describes a period of reduced funding, diminished interest, and widespread disillusionment with AI research. There have been two widely recognized AI winters: 1974–1980 and 1987–1993. |
| | ``` |
| | |
| | By the early 1970s, the gap between AI's promises and its delivery had become impossible to ignore. Several converging factors triggered a collapse in funding and confidence. |
| | |
| | ## The Lighthill report |
| | |
| - | In 1973, the British Science Research Council commissioned mathematician Sir James Lighthill to evaluate the state of AI research. His report, *Artificial Intelligence: A General Survey*, was devastating. Lighthill concluded that "in no part of the field have the discoveries made so far produced the major impact that was then promised." He was particularly critical of robotics and natural language processing research[5]. |
| + | In 1973, the British Science Research Council commissioned mathematician Sir James Lighthill to evaluate the state of AI research. His report, *Artificial Intelligence: A General Survey*, was devastating. Lighthill concluded that "in no part of the field have the discoveries made so far produced the major impact that was then promised." He was particularly critical of robotics and natural language processing research[[cite: J. Lighthill, "Artificial Intelligence: A General Survey," in *Artificial Intelligence: a Paper Symposium*, Science Research Council, 1973 — [full report, Chilton Computing archive](http://www.chilton-computing.org.uk/inf/literature/reports/lighthill_report/contents.htm)]]. |
| | |
| - | The Lighthill Report led the British government to end support for AI research at most UK universities. The effect was swift and severe: the University of Edinburgh's Department of Artificial Intelligence, one of the largest in the world, saw its funding decimated. The consequences persisted for nearly a decade[5]. |
| + | The Lighthill Report led the British government to end support for AI research at most UK universities. The effect was swift and severe: the University of Edinburgh's Department of Artificial Intelligence, one of the largest in the world, saw its funding decimated. The consequences persisted for nearly a decade[[cite: J. Lighthill, "Artificial Intelligence: A General Survey," in *Artificial Intelligence: a Paper Symposium*, Science Research Council, 1973 — [full report, Chilton Computing archive](http://www.chilton-computing.org.uk/inf/literature/reports/lighthill_report/contents.htm)]]. |
| | |
| | ## DARPA funding cuts |
| | |
| - | In the United States, the 1969 Mansfield Amendment required DARPA to fund only research with direct military applications, rather than basic research. AI researchers who had relied on DARPA's relatively open-ended grants suddenly found themselves needing to justify their work in military terms. Combined with the failure of DARPA-funded machine translation projects to deliver on their promises, AI funding in the US declined sharply[6]. |
| + | In the United States, the 1969 Mansfield Amendment required DARPA to fund only research with direct military applications, rather than basic research. AI researchers who had relied on DARPA's relatively open-ended grants suddenly found themselves needing to justify their work in military terms. Combined with the failure of DARPA-funded machine translation projects to deliver on their promises, AI funding in the US declined sharply[[cite: T. Haigh, ["There Was No ‘First AI Winter’"](https://doi.org/10.1145/3625833), *Communications of the ACM* 66(12), December 2023]]. |
| | |
| | ## The combinatorial explosion |
| | |
| | A more fundamental problem underlay the political ones: the programs didn't scale. Methods that worked brilliantly on small, well-defined problems (prove a theorem, solve a puzzle, parse a sentence about toy blocks) collapsed when faced with real-world complexity. The search spaces grew exponentially — a phenomenon called the **combinatorial explosion** — and the hardware of the 1970s simply could not keep up. |
| | |
| - | Richard Karp's 1972 proof that 21 problems were **NP-complete** gave mathematical rigour to what practitioners already suspected: many of the problems AI researchers were trying to solve were fundamentally intractable by brute-force search[4]. |
| + | Richard Karp's 1972 proof that 21 problems were **NP-complete** gave mathematical rigour to what practitioners already suspected: many of the problems AI researchers were trying to solve were fundamentally intractable by brute-force search[[cite: Actuaries Institute, ["A history of AI winters"](https://actuaries.digital/2023/03/20/a-history-of-ai-winters/), actuaries.digital]]. |
| | |
| | --- |
| | |
| | # 1980–1987: the expert systems boom |
| | |
| | ## XCON and the commercial promise |
| | |
| - | In 1980, John McDermott at Carnegie Mellon University built an **expert system** called R1 (later renamed XCON) for Digital Equipment Corporation (DEC). XCON's task was mundane: configure VAX computer systems based on customer orders. But its commercial value was startling. Before XCON, DEC needed teams of senior engineers to manually assemble each configuration. XCON automated the process and, by DEC's own estimates, saved the company over $40 million per year[7]. |
| + | In 1980, John McDermott at Carnegie Mellon University built an **expert system** called R1 (later renamed XCON) for Digital Equipment Corporation (DEC). XCON's task was mundane: configure VAX computer systems based on customer orders. But its commercial value was startling. Before XCON, DEC needed teams of senior engineers to manually assemble each configuration. XCON automated the process and, by DEC's own estimates, saved the company over $40 million per year[[cite: StoryOfAI, ["The Second AI Winter: How the Boom Turned to Bust (1987–1993)"](https://storyofai.net/), storyofai.net]]. |
| | |
| | XCON demonstrated that AI could deliver measurable business value. The expert systems boom followed immediately. |
| | |
| | ## The boom |
| | |
| - | Expert systems — programs that encoded human expertise as collections of IF-THEN rules — became the dominant commercial AI technology of the 1980s. Companies like Symbolics, IntelliCorp, Teknowledge, and Carnegie Group sold expert system shells, tools, and consulting services. Venture capital poured in. The AI industry grew from a few million dollars in 1980 to billions by 1985[7]. |
| + | Expert systems — programs that encoded human expertise as collections of IF-THEN rules — became the dominant commercial AI technology of the 1980s. Companies like Symbolics, IntelliCorp, Teknowledge, and Carnegie Group sold expert system shells, tools, and consulting services. Venture capital poured in. The AI industry grew from a few million dollars in 1980 to billions by 1985[[cite: StoryOfAI, ["The Second AI Winter: How the Boom Turned to Bust (1987–1993)"](https://storyofai.net/), storyofai.net]]. |
| | |
| | Much of the hardware ran on **Lisp machines** — specialized computers built for symbolic processing, costing between $70,000 and $150,000 each. Symbolics, the leading Lisp machine manufacturer, was valued at over $200 million at its peak. |
| | |
| | ## Japan's Fifth Generation |
| | |
| - | In 1982, Japan's Ministry of International Trade and Industry (MITI) launched the **Fifth Generation Computer Systems** (FGCS) project with a budget of ¥54.15 billion ($416.54 million over ten years). The project aimed to build computers based on massively parallel processing and logic programming that would achieve artificial intelligence through inference at speeds of 100 million to 1 billion logical inferences per second[8]. |
| + | In 1982, Japan's Ministry of International Trade and Industry (MITI) launched the **Fifth Generation Computer Systems** (FGCS) project with a budget of ¥54.15 billion ($416.54 million over ten years). The project aimed to build computers based on massively parallel processing and logic programming that would achieve artificial intelligence through inference at speeds of 100 million to 1 billion logical inferences per second[[cite: A. Ishikawa, ["A Retrospective and Prospects of the Fifth Generation Computer Project"](https://repositories.lib.utexas.edu/items/d920916f-8ed3-4a10-94a8-ff75f037b160), IC² Institute, University of Texas, September 1993]]. |
| | |
| | The announcement sent shockwaves through Western governments and industry. The US responded with the Strategic Computing Initiative; the UK launched the Alvey Programme; the EU created the ESPRIT programme. Fear of falling behind Japan in computing drove a massive increase in government AI spending. |
| | |
| | ```fact |
| | Japan's Fifth Generation project invested ¥54.15 billion ($416.54 million) over ten years (1982–1992). By the project's end, the field had moved away from logic programming toward statistical and neural approaches. The project's parallel inference machines never achieved commercial success, though they contributed to advances in concurrent logic programming. |
| | ``` |
| | |
| | --- |
| | |
| | # 1987–1993: the second AI winter |
| | |
| | [[File:deep-blue.jpg|right|250px|IBM's Deep Blue chess computer. Photo by James the photographer, CC BY 2.0.]] |
| | |
| | The second AI winter was worse than the first. This time, entire industries collapsed. |
| | |
| | ## Expert systems hit the wall |
| | |
| - | Expert systems turned out to be **brittle, expensive to maintain, and impossible to scale**. The knowledge acquisition bottleneck — the slow, painful process of extracting rules from human experts and encoding them — proved intractable. Systems that worked in demonstrations failed in production. Rules contradicted each other as knowledge bases grew. Updating a system with ten thousand rules was, as one researcher put it, "like editing a house of cards"[7]. |
| + | Expert systems turned out to be **brittle, expensive to maintain, and impossible to scale**. The knowledge acquisition bottleneck — the slow, painful process of extracting rules from human experts and encoding them — proved intractable. Systems that worked in demonstrations failed in production. Rules contradicted each other as knowledge bases grew. Updating a system with ten thousand rules was, as one researcher put it, "like editing a house of cards"[[cite: StoryOfAI, ["The Second AI Winter: How the Boom Turned to Bust (1987–1993)"](https://storyofai.net/), storyofai.net]]. |
| | |
| | ## The Lisp machine crash |
| | |
| - | General-purpose workstations from Sun Microsystems caught up in performance at a fraction of the price. When expert system shells like CLIPS started running acceptably on standard hardware, the Lisp machine's value proposition evaporated overnight. Symbolics' stock collapsed to a fraction of its peak. The company, along with Lisp Machines Inc. and dozens of AI startups, went bankrupt or shrank to irrelevance[7]. |
| + | General-purpose workstations from Sun Microsystems caught up in performance at a fraction of the price. When expert system shells like CLIPS started running acceptably on standard hardware, the Lisp machine's value proposition evaporated overnight. Symbolics' stock collapsed to a fraction of its peak. The company, along with Lisp Machines Inc. and dozens of AI startups, went bankrupt or shrank to irrelevance[[cite: StoryOfAI, ["The Second AI Winter: How the Boom Turned to Bust (1987–1993)"](https://storyofai.net/), storyofai.net]]. |
| | |
| | ## Government funding evaporates |
| | |
| - | DARPA, which had poured hundreds of millions into AI through the Strategic Computing Initiative, sharply cut its AI budget. The US Congressional Office of Technology Assessment reported in 1990 that the SCI had "achieved none of its original, ambitious goals." Japan's Fifth Generation project concluded in 1992 without delivering on its promises. The UK's Alvey Programme wound down[6][8]. |
| + | DARPA, which had poured hundreds of millions into AI through the Strategic Computing Initiative, sharply cut its AI budget. The US Congressional Office of Technology Assessment reported in 1990 that the SCI had "achieved none of its original, ambitious goals." Japan's Fifth Generation project concluded in 1992 without delivering on its promises. The UK's Alvey Programme wound down[[cite: T. Haigh, ["How the AI Boom Went Bust"](https://doi.org/10.1145/3634901), *Communications of the ACM* 67(2), February 2024]][[cite: A. Ishikawa, ["A Retrospective and Prospects of the Fifth Generation Computer Project"](https://repositories.lib.utexas.edu/items/d920916f-8ed3-4a10-94a8-ff75f037b160), IC² Institute, University of Texas, September 1993]]. |
| | |
| | ## The word "AI" becomes toxic |
| | |
| - | By the late 1980s, the term "artificial intelligence" had become a liability in grant applications and business plans. Venture capitalists who had poured money into AI startups saw their investments collapse. Researchers who continued working on what would later be recognized as core AI problems — statistical pattern recognition, probabilistic reasoning, optimization — rebranded their work as "machine learning," "data mining," "knowledge discovery," or "intelligent systems." The field didn't die, but it went underground[6]. |
| + | By the late 1980s, the term "artificial intelligence" had become a liability in grant applications and business plans. Venture capitalists who had poured money into AI startups saw their investments collapse. Researchers who continued working on what would later be recognized as core AI problems — statistical pattern recognition, probabilistic reasoning, optimization — rebranded their work as "machine learning," "data mining," "knowledge discovery," or "intelligent systems." The field didn't die, but it went underground[[cite: T. Haigh, ["How the AI Boom Went Bust"](https://doi.org/10.1145/3634901), *Communications of the ACM* 67(2), February 2024]]. |
| | |
| | The cultural scar of the second AI winter persisted for decades. As late as 2010, many researchers avoided the term "AI" and identified their work as "machine learning" or "pattern recognition" — fields that had quietly built impressive practical capabilities while the AI brand remained toxic. The rehabilitation of the term "artificial intelligence" would not begin until the deep learning revolution of 2012, and would not be complete until ChatGPT made "AI" the most searched technology term on Earth. |
| | |
| | ## What survived |
| | |
| | It is important to note what the second AI winter did **not** destroy. Neural network research continued, albeit with drastically reduced funding. Geoffrey Hinton, Yann LeCun, and Yoshua Bengio — later called the "godfathers of deep learning" and joint winners of the 2018 Turing Award — continued working on neural networks throughout the 1990s and 2000s, when the approach was deeply unfashionable. Reinforcement learning research produced TD-Gammon (1992), which taught itself to play backgammon at world-class level. The field of robotics advanced steadily. Natural language processing made slow but real progress using statistical methods. The seeds of every major AI breakthrough of the 2010s and 2020s were planted during the winter. |
| | |
| | --- |
| | |
| | # 1986–2011: the quiet revolution |
| | |
| | While expert systems were collapsing, a different approach to AI was slowly gathering momentum. |
| | |
| | ## Backpropagation |
| | |
| - | In 1986, David Rumelhart, Geoffrey Hinton, and Ronald Williams published "Learning representations by back-propagating errors" in *Nature*. The paper described a method for training multi-layer neural networks by propagating error signals backwards through the network, adjusting weights at each layer. Backpropagation was not entirely new — Paul Werbos had described it in his 1974 PhD thesis, and the mathematical foundations (reverse-mode automatic differentiation) dated to Seppo Linnainmaa's 1970 master's thesis — but the 1986 paper demonstrated its practical effectiveness and reached a wide audience[9]. |
| + | In 1986, David Rumelhart, Geoffrey Hinton, and Ronald Williams published "Learning representations by back-propagating errors" in *Nature*. The paper described a method for training multi-layer neural networks by propagating error signals backwards through the network, adjusting weights at each layer. Backpropagation was not entirely new — Paul Werbos had described it in his 1974 PhD thesis, and the mathematical foundations (reverse-mode automatic differentiation) dated to Seppo Linnainmaa's 1970 master's thesis — but the 1986 paper demonstrated its practical effectiveness and reached a wide audience[[cite: D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Representations by Back-Propagating Errors," *Nature* 323, October 1986, pp. 533–536 — [doi:10.1038/323533a0](https://doi.org/10.1038/323533a0)]]. |
| | |
| | Backpropagation solved the problem that had killed the perceptron: it allowed neural networks to learn **non-linear** functions by training hidden layers. Multi-layer perceptrons could now learn XOR and far more complex patterns. But the networks of the 1980s and 1990s were small — limited by available compute — and they struggled with deep architectures due to the **vanishing gradient problem** (gradients shrinking to near-zero as they propagated through many layers). |
| | |
| | ## Convolutional neural networks |
| | |
| - | In 1989, Yann LeCun at Bell Labs applied backpropagation to a convolutional neural network, creating **LeNet** — a system that could read handwritten zip codes on mail. LeNet used small, learnable filters that slid across the input image, sharing weights across spatial positions. This architectural insight — that local patterns (edges, curves, corners) are the same regardless of where they appear in an image — dramatically reduced the number of parameters and made the network practical for visual tasks[9]. |
| + | In 1989, Yann LeCun at Bell Labs applied backpropagation to a convolutional neural network, creating **LeNet** — a system that could read handwritten zip codes on mail. LeNet used small, learnable filters that slid across the input image, sharing weights across spatial positions. This architectural insight — that local patterns (edges, curves, corners) are the same regardless of where they appear in an image — dramatically reduced the number of parameters and made the network practical for visual tasks[[cite: D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Representations by Back-Propagating Errors," *Nature* 323, October 1986, pp. 533–536 — [doi:10.1038/323533a0](https://doi.org/10.1038/323533a0)]]. |
| | |
| | LeNet was deployed by the US Postal Service in the 1990s and processed millions of checks and envelopes. It was a genuine commercial success, but it was not widely recognized as a harbinger of a revolution. The neural network community remained small, underfunded, and largely marginal within the broader AI and machine learning research world. |
| | |
| | ## Long Short-Term Memory (LSTM) |
| | |
| - | In 1997, Sepp Hochreiter and Jürgen Schmidhuber published the **Long Short-Term Memory** (LSTM) architecture, a type of recurrent neural network designed to overcome the vanishing gradient problem. LSTMs used gated memory cells that could store information over long sequences — a capability that standard recurrent networks lacked. LSTMs would later become the dominant architecture for speech recognition, machine translation, and text generation, until the Transformer superseded them in 2017[9]. |
| + | In 1997, Sepp Hochreiter and Jürgen Schmidhuber published the **Long Short-Term Memory** (LSTM) architecture, a type of recurrent neural network designed to overcome the vanishing gradient problem. LSTMs used gated memory cells that could store information over long sequences — a capability that standard recurrent networks lacked. LSTMs would later become the dominant architecture for speech recognition, machine translation, and text generation, until the Transformer superseded them in 2017[[cite: D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Representations by Back-Propagating Errors," *Nature* 323, October 1986, pp. 533–536 — [doi:10.1038/323533a0](https://doi.org/10.1038/323533a0)]]. |
| | |
| | ## Statistical methods rise |
| | |
| | Through the 1990s and 2000s, AI research increasingly embraced statistical and probabilistic methods. **Support vector machines** (SVMs), introduced by Vladimir Vapnik and colleagues in the 1990s, provided mathematically rigorous classifiers with strong theoretical guarantees. **Random forests**, **Bayesian networks**, and **hidden Markov models** achieved practical success in speech recognition, spam filtering, medical diagnosis, and other domains. These methods didn't require hand-coded rules; they learned patterns from data. |
| | |
| | The success of statistical methods shifted the culture of AI research. The old symbolic AI community had valued elegance, formal logic, and knowledge representation. The new statistical community valued benchmark performance, cross-validation, and the ability to handle noisy, real-world data. This cultural shift — from "knowledge engineering" to "learning from data" — would prove permanent. |
| | |
| | ```fact |
| | The annual Conference on Neural Information Processing Systems (NeurIPS, originally NIPS) was founded in 1987. Through the 1990s and 2000s it was a modest academic gathering. By 2019, it had grown to over 13,000 attendees and corporate sponsors were paying over $100,000 for booths. The conference's growth mirrors the field's trajectory. |
| | ``` |
| | |
| | ## Landmark systems |
| | |
| | | Year | System | Achievement | |
| | |---|---|---| |
| | | 1997 | Deep Blue (IBM) | Defeated world chess champion Garry Kasparov in a six-game match | |
| | | 2002 | Roomba (iRobot) | First widely successful autonomous consumer robot | |
| | | 2005 | Stanley (Stanford) | Won the DARPA Grand Challenge, driving 132 miles autonomously | |
| | | 2009 | ImageNet (Fei-Fei Li) | Created a dataset of 14 million labeled images that would enable the deep learning revolution | |
| | | 2011 | Watson (IBM) | Defeated Jeopardy! champions Ken Jennings and Brad Rutter | |
| | | 2011 | Siri (Apple) | First widely deployed voice assistant on a consumer device | |
| | |
| - | Deep Blue's 1997 victory over Kasparov was a landmark, but it was a victory of **brute force** — the system evaluated 200 million chess positions per second using 32 parallel processors and a processing speed of 11.38 billion floating-point operations per second. It didn't "understand" chess in any cognitive sense. Kasparov himself noted the difference: "I could feel — I could smell — a new kind of intelligence across the table." The match was deeply controversial — Kasparov accused IBM of cheating after an unusual move in Game 2, and IBM refused a rematch and retired Deep Blue to the Smithsonian. But the result stood: for the first time, a machine had defeated the world's best human at a task considered a pinnacle of human intellect[10]. |
| + | Deep Blue's 1997 victory over Kasparov was a landmark, but it was a victory of **brute force** — the system evaluated 200 million chess positions per second using 32 parallel processors and a processing speed of 11.38 billion floating-point operations per second. It didn't "understand" chess in any cognitive sense. Kasparov himself noted the difference: "I could feel — I could smell — a new kind of intelligence across the table." The match was deeply controversial — Kasparov accused IBM of cheating after an unusual move in Game 2, and IBM refused a rematch and retired Deep Blue to the Smithsonian. But the result stood: for the first time, a machine had defeated the world's best human at a task considered a pinnacle of human intellect[[cite: IBM, ["Deep Blue"](https://www.ibm.com/history/deep-blue) and ["Watson, Jeopardy! Champion"](https://www.ibm.com/history/watson-jeopardy), ibm.com/history]]. |
| | |
| - | IBM Watson's 2011 Jeopardy! victory was more technically ambitious. Watson consisted of 10 racks holding 90 servers with 2,880 processor cores. It processed natural language questions, searched through millions of documents (including all of Wikipedia), and answered in under three seconds. The system defeated Ken Jennings and Brad Rutter, Jeopardy!'s two most successful contestants, in a televised match watched by millions. Watson's subsequent commercial career — IBM attempted to apply it to healthcare, customer service, and legal research — was widely seen as a disappointment, but the Jeopardy! demonstration proved that machines could handle open-domain question answering over unstructured text[10]. |
| + | IBM Watson's 2011 Jeopardy! victory was more technically ambitious. Watson consisted of 10 racks holding 90 servers with 2,880 processor cores. It processed natural language questions, searched through millions of documents (including all of Wikipedia), and answered in under three seconds. The system defeated Ken Jennings and Brad Rutter, Jeopardy!'s two most successful contestants, in a televised match watched by millions. Watson's subsequent commercial career — IBM attempted to apply it to healthcare, customer service, and legal research — was widely seen as a disappointment, but the Jeopardy! demonstration proved that machines could handle open-domain question answering over unstructured text[[cite: IBM, ["Deep Blue"](https://www.ibm.com/history/deep-blue) and ["Watson, Jeopardy! Champion"](https://www.ibm.com/history/watson-jeopardy), ibm.com/history]]. |
| | |
| | The 2005 **DARPA Grand Challenge** deserves special mention. Stanley, a modified Volkswagen Touareg built by Stanford's Racing Team under Sebastian Thrun, drove 132 miles through the Mojave Desert without human intervention. The previous year's challenge had ended in failure — no vehicle completed even 8 miles. Stanley's success launched the autonomous vehicle industry and led directly to [[Google|Google's]] self-driving car project (now Waymo). |
| | |
| | ## The compute curve |
| | |
| | A crucial development during this period was the exponential growth of available computation. Moore's Law continued to deliver more transistors per chip, but the real transformation came from an unexpected source: **GPUs** (graphics processing units). |
| | |
| | GPUs were designed for rendering video game graphics — a task that requires thousands of simple, parallel arithmetic operations (transforming vertices, shading pixels, computing lighting). Neural network training requires essentially the same type of computation: multiplying large matrices of floating-point numbers. A single modern GPU can perform trillions of these operations per second. |
| | |
| - | In 2007, NVIDIA released **CUDA** (Compute Unified Device Architecture), a programming platform that allowed researchers to run general-purpose computations on NVIDIA GPUs without needing to disguise them as graphics operations. CUDA lowered the barrier to GPU computing dramatically and created a software ecosystem that would become the foundation of the entire deep learning industry[11]. |
| + | In 2007, NVIDIA released **CUDA** (Compute Unified Device Architecture), a programming platform that allowed researchers to run general-purpose computations on NVIDIA GPUs without needing to disguise them as graphics operations. CUDA lowered the barrier to GPU computing dramatically and created a software ecosystem that would become the foundation of the entire deep learning industry[[cite: A. Krizhevsky, I. Sutskever, and G. E. Hinton, ["ImageNet Classification with Deep Convolutional Neural Networks"](https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html), *NeurIPS*, 2012]]. |
| | |
| | The numbers tell the story. A single NVIDIA A100 GPU (2020) delivers approximately 312 teraflops of half-precision computation — roughly 100,000 times the floating-point throughput of a typical 2000-era CPU. Training AlexNet in 2012 took days on two GPUs. Training GPT-4 in 2023 took months on tens of thousands of GPUs. The scaling of compute has been the single most important enabler of modern AI. |
| | |
| | --- |
| | |
| | # 2012–2017: the deep learning revolution |
| | |
| | [[File:nvidia-gpu-cluster.jpg|right|300px|An NVIDIA Tesla GPU cluster. Photo by ChrisDag, CC BY 2.0.]] |
| | |
| | ## AlexNet and ImageNet |
| | |
| - | On September 30, 2012, a deep convolutional neural network called **AlexNet** — built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton at the University of Toronto — won the ImageNet Large Scale Visual Recognition Challenge with a top-5 error rate of 15.3%, obliterating the runner-up by more than 10 percentage points[11]. |
| + | On September 30, 2012, a deep convolutional neural network called **AlexNet** — built by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton at the University of Toronto — won the ImageNet Large Scale Visual Recognition Challenge with a top-5 error rate of 15.3%, obliterating the runner-up by more than 10 percentage points[[cite: A. Krizhevsky, I. Sutskever, and G. E. Hinton, ["ImageNet Classification with Deep Convolutional Neural Networks"](https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html), *NeurIPS*, 2012]]. |
| | |
| | The result was a shock. ImageNet had been running since 2010, and previous winners had used hand-crafted feature engineering with modest improvements each year. AlexNet used a deep neural network trained on GPUs — an approach that most of the computer vision community had dismissed as impractical. The margin of victory made the result undeniable. |
| | |
| | ```fact |
| | AlexNet was not technically the first deep CNN to win a vision contest. Dan Cireșan's "DanNet" at IDSIA won four consecutive vision competitions between May 2011 and September 2012, including the first superhuman result in visual pattern recognition (IJCNN 2011 traffic sign recognition, 99.46% accuracy versus 99.22% human). But AlexNet's dominant victory on ImageNet — the field's most watched benchmark — was the tipping point that convinced the broader machine learning community. |
| | ``` |
| | |
| | ## The revolution accelerates |
| | |
| | After AlexNet, progress was extraordinary: |
| | |
| | | Year | Milestone | |
| | |---|---| |
| | | 2014 | **GANs** (Generative Adversarial Networks) proposed by Ian Goodfellow | |
| | | 2014 | **VGGNet** and **GoogLeNet** push ImageNet accuracy further | |
| | | 2015 | **ResNet** (a variant of Highway Nets) enables networks with 150+ layers | |
| | | 2016 | **AlphaGo** (DeepMind) defeats world Go champion Lee Sedol 4–1 | |
| | | 2017 | **"Attention Is All You Need"** — the Transformer architecture is published | |
| | |
| | ## AlphaGo |
| | |
| | [[File:alphago-divine-move.jpg|right|250px|Lee Sedol's "divine move" (move 78) in Game 4 against AlphaGo. Board diagram by Axd, CC BY-SA 4.0.]] |
| | |
| - | In March 2016, DeepMind's **AlphaGo** defeated the world Go champion Lee Sedol in a five-game match, winning 4–1. The result was considered at least a decade ahead of predictions. Go has a search space of approximately 10^170 legal positions (compared to ~10^47 for chess), making brute-force search impossible. AlphaGo used deep neural networks combined with Monte Carlo tree search, trained on millions of human games and then refined through self-play[12]. |
| + | In March 2016, DeepMind's **AlphaGo** defeated the world Go champion Lee Sedol in a five-game match, winning 4–1. The result was considered at least a decade ahead of predictions. Go has a search space of approximately 10^170 legal positions (compared to ~10^47 for chess), making brute-force search impossible. AlphaGo used deep neural networks combined with Monte Carlo tree search, trained on millions of human games and then refined through self-play[[cite: D. Silver et al., "Mastering the Game of Go with Deep Neural Networks and Tree Search," *Nature* 529, January 2016, pp. 484–489 — [doi:10.1038/nature16961](https://doi.org/10.1038/nature16961)]]. |
| | |
| | The match's most celebrated moment came in Game 4, when Lee Sedol played what commentators called the "divine move" (move 78) — a creative, unexpected wedge that caused AlphaGo to make errors and led to Lee Sedol's only victory. It remains one of the most famous moves in the history of Go. |
| | |
| - | In 2017, DeepMind published **AlphaGo Zero**, which learned Go entirely through self-play with no human game data, surpassed the original AlphaGo within three days of training, and overtook the strongest version within 40 days. **AlphaZero** then generalized the approach to chess and shogi, defeating the strongest existing programs in all three games[12]. |
| + | In 2017, DeepMind published **AlphaGo Zero**, which learned Go entirely through self-play with no human game data, surpassed the original AlphaGo within three days of training, and overtook the strongest version within 40 days. **AlphaZero** then generalized the approach to chess and shogi, defeating the strongest existing programs in all three games[[cite: D. Silver et al., "Mastering the Game of Go with Deep Neural Networks and Tree Search," *Nature* 529, January 2016, pp. 484–489 — [doi:10.1038/nature16961](https://doi.org/10.1038/nature16961)]]. |
| | |
| | ## The Transformer |
| | |
| - | On June 12, 2017, a team of researchers at Google published "Attention Is All You Need," introducing the **Transformer** architecture. The Transformer replaced the recurrent and convolutional layers used in previous sequence-processing models with a purely attention-based mechanism that could process all positions in a sequence simultaneously, enabling massive parallelization and far more effective training on large datasets[13]. |
| + | On June 12, 2017, a team of researchers at Google published "Attention Is All You Need," introducing the **Transformer** architecture. The Transformer replaced the recurrent and convolutional layers used in previous sequence-processing models with a purely attention-based mechanism that could process all positions in a sequence simultaneously, enabling massive parallelization and far more effective training on large datasets[[cite: A. Vaswani et al., ["Attention Is All You Need"](https://arxiv.org/abs/1706.03762), *NeurIPS*, 2017]]. |
| | |
| - | Nearly every large language model built since 2017 — GPT, BERT, T5, PaLM, Claude, Gemini, Llama — is a descendant of the Transformer. The paper's eight authors — Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin — produced what may be the single most influential machine learning paper of the 21st century[13]. |
| + | Nearly every large language model built since 2017 — GPT, BERT, T5, PaLM, Claude, Gemini, Llama — is a descendant of the Transformer. The paper's eight authors — Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin — produced what may be the single most influential machine learning paper of the 21st century[[cite: A. Vaswani et al., ["Attention Is All You Need"](https://arxiv.org/abs/1706.03762), *NeurIPS*, 2017]]. |
| | |
| | --- |
| | |
| | # 2018–2022: the age of large language models |
| | |
| | ## GPT and the scaling hypothesis |
| | |
| - | In June 2018, OpenAI published "Improving Language Understanding by Generative Pre-Training," introducing **GPT** (Generative Pre-trained Transformer). The paper demonstrated that pre-training a Transformer on a large corpus of text, then fine-tuning it on specific tasks, produced strong results across a wide range of natural language benchmarks[14]. |
| + | In June 2018, OpenAI published "Improving Language Understanding by Generative Pre-Training," introducing **GPT** (Generative Pre-trained Transformer). The paper demonstrated that pre-training a Transformer on a large corpus of text, then fine-tuning it on specific tasks, produced strong results across a wide range of natural language benchmarks[[cite: A. Radford et al., ["Improving Language Understanding by Generative Pre-Training"](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf), OpenAI, 2018; T. Brown et al., ["Language Models Are Few-Shot Learners"](https://arxiv.org/abs/2005.14165) (GPT-3), *NeurIPS*, 2020]]. |
| | |
| - | The approach scaled. GPT-2 (2019, 1.5 billion parameters) generated text so fluent that OpenAI initially withheld the full model, citing concerns about misuse. GPT-3 (2020, 175 billion parameters) demonstrated **in-context learning** — the ability to perform tasks described in the prompt without any fine-tuning. GPT-3 could translate languages, write code, answer questions, and generate essays that were often indistinguishable from human writing[14]. |
| + | The approach scaled. GPT-2 (2019, 1.5 billion parameters) generated text so fluent that OpenAI initially withheld the full model, citing concerns about misuse. GPT-3 (2020, 175 billion parameters) demonstrated **in-context learning** — the ability to perform tasks described in the prompt without any fine-tuning. GPT-3 could translate languages, write code, answer questions, and generate essays that were often indistinguishable from human writing[[cite: A. Radford et al., ["Improving Language Understanding by Generative Pre-Training"](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf), OpenAI, 2018; T. Brown et al., ["Language Models Are Few-Shot Learners"](https://arxiv.org/abs/2005.14165) (GPT-3), *NeurIPS*, 2020]]. |
| | |
| | The **scaling hypothesis** — the idea that simply making models larger, training them on more data, and using more compute would continue to produce qualitative improvements in capability — became the central organizing principle of frontier AI research. Scaling laws published by OpenAI (Kaplan et al., 2020) and later refined by DeepMind's Chinchilla paper (2022) showed that model performance improved predictably as a power law of compute, data, and parameter count. This finding transformed AI research from an art into something closer to an engineering discipline: if you knew how much compute you had, you could predict how capable the resulting model would be. |
| | |
| | The implications were profound. Frontier AI became a capital-intensive enterprise. Training GPT-4 reportedly cost over $100 million in compute alone. Building and operating the GPU clusters required for frontier training became a bottleneck that only the largest technology companies and a few well-funded startups could clear. By 2024, the cost of a single frontier training run was approaching $1 billion, and plans for $10 billion training clusters were being discussed openly. |
| | |
| | ## BERT and the NLP revolution |
| | |
| - | In October 2018, Google published **BERT** (Bidirectional Encoder Representations from Transformers), which processed text bidirectionally (considering both left and right context simultaneously). BERT achieved state-of-the-art results on eleven NLP benchmarks and was rapidly adopted across the industry. Google integrated BERT into its search engine in 2019, calling it the biggest change to search in five years[15]. |
| + | In October 2018, Google published **BERT** (Bidirectional Encoder Representations from Transformers), which processed text bidirectionally (considering both left and right context simultaneously). BERT achieved state-of-the-art results on eleven NLP benchmarks and was rapidly adopted across the industry. Google integrated BERT into its search engine in 2019, calling it the biggest change to search in five years[[cite: J. Devlin, M. Chang, K. Lee, and K. Toutanova, ["BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding"](https://arxiv.org/abs/1810.04805), *NAACL*, 2019]]. |
| | |
| | ## Key developments 2018–2022 |
| | |
| | | Year | Development | Significance | |
| | |---|---|---| |
| | | 2018 | GPT (OpenAI) | Pre-training + fine-tuning paradigm for language | |
| | | 2018 | BERT (Google) | Bidirectional understanding; transformed search | |
| | | 2019 | GPT-2 (OpenAI, 1.5B params) | Fluent text generation; staged release over misuse concerns | |
| | | 2020 | GPT-3 (OpenAI, 175B params) | In-context learning; few-shot performance | |
| | | 2020 | AlphaFold 2 (DeepMind) | Solved protein structure prediction (CASP14) | |
| | | 2021 | DALL-E (OpenAI) | Text-to-image generation | |
| | | 2021 | Codex (OpenAI) | Code generation from natural language | |
| | | 2022 | Chinchilla (DeepMind) | Showed that smaller models trained on more data outperform larger models trained on less | |
| | | 2022 | Stable Diffusion (Stability AI) | Open-source text-to-image; democratized generative AI | |
| | | 2022 | ChatGPT (OpenAI) | Consumer chatbot; 100M users in 2 months | |
| | |
| | ## AlphaFold: AI solves biology |
| | |
| - | In December 2020, DeepMind's **AlphaFold 2** achieved a median GDT score of 92.4 at CASP14, effectively solving the protein structure prediction problem that had been open for fifty years. The system predicted protein structures with accuracy comparable to experimental methods like X-ray crystallography. In 2022, DeepMind released predicted structures for nearly all known proteins — over 200 million structures — in a public database[16]. |
| + | In December 2020, DeepMind's **AlphaFold 2** achieved a median GDT score of 92.4 at CASP14, effectively solving the protein structure prediction problem that had been open for fifty years. The system predicted protein structures with accuracy comparable to experimental methods like X-ray crystallography. In 2022, DeepMind released predicted structures for nearly all known proteins — over 200 million structures — in a public database[[cite: J. Jumper et al., "Highly Accurate Protein Structure Prediction with AlphaFold," *Nature* 596, 2021, pp. 583–589 — [doi:10.1038/s41586-021-03819-2](https://doi.org/10.1038/s41586-021-03819-2)]]. |
| | |
| | AlphaFold demonstrated that deep learning could make fundamental contributions to science, not just engineering. It is widely considered one of the most important scientific achievements of the 21st century. |
| | |
| | ## ChatGPT: the inflection point |
| | |
| - | On November 30, 2022, OpenAI released **ChatGPT**, a chatbot built on GPT-3.5 and fine-tuned using reinforcement learning from human feedback (RLHF). It reached 1 million users in five days and 100 million users in two months, making it the fastest-growing consumer application in history[17]. |
| + | On November 30, 2022, OpenAI released **ChatGPT**, a chatbot built on GPT-3.5 and fine-tuned using reinforcement learning from human feedback (RLHF). It reached 1 million users in five days and 100 million users in two months, making it the fastest-growing consumer application in history[[cite: OpenAI, ["Introducing ChatGPT"](https://openai.com/index/chatgpt/), November 30, 2022]]. |
| | |
| - | ChatGPT's impact was not primarily technical — GPT-3 had been available via API for two years. The impact was **social**: for the first time, hundreds of millions of non-technical users interacted directly with a large language model. The experience of conversing with a system that could write essays, debug code, explain concepts, and generate creative text convinced the public that AI had fundamentally changed[17]. |
| + | ChatGPT's impact was not primarily technical — GPT-3 had been available via API for two years. The impact was **social**: for the first time, hundreds of millions of non-technical users interacted directly with a large language model. The experience of conversing with a system that could write essays, debug code, explain concepts, and generate creative text convinced the public that AI had fundamentally changed[[cite: OpenAI, ["Introducing ChatGPT"](https://openai.com/index/chatgpt/), November 30, 2022]]. |
| | |
| | The consequences were immediate and far-reaching: |
| | |
| | - **Google** declared a "code red" — the company's search monopoly was suddenly threatened by a conversational interface that could synthesize answers rather than listing links. |
| | - **Microsoft** invested $10 billion in OpenAI and integrated GPT-4 into Bing, Office 365 (as "Copilot"), and Azure. |
| | - **Universities** scrambled to respond to students submitting AI-generated work. Some banned ChatGPT; others integrated it into curricula. |
| | - **Venture capital** flooded into AI startups at unprecedented levels — tens of billions of dollars in total funding was deployed into AI platforms between 2022 and 2025. |
| | - **Governments** began drafting AI regulation in earnest. The EU accelerated the AI Act; the US issued executive orders; the UK hosted the first AI Safety Summit. |
| | - **The job market** shifted. Demand for AI/ML engineers exploded, while concerns about automation-driven job displacement entered mainstream political discourse. |
| | |
| | ```fact |
| | Google Trends data shows that global search interest for "AI" remained relatively stable for nearly two decades before seeing a 10x exponential increase starting in early 2023. Unlike previous technology buzzwords, the elevated interest has sustained itself, suggesting a permanent shift in public attention rather than a temporary spike. |
| | ``` |
| | |
| | --- |
| | |
| | # 2023–2025: the frontier era |
| | |
| | [[File:openai-logo.svg|right|120px|OpenAI logo. Public domain (below threshold of originality).]] |
| | |
| | ## The model race |
| | |
| | The release of ChatGPT triggered an arms race among the world's largest technology companies. |
| | |
| - | **March 2023**: OpenAI released **GPT-4**, a multimodal model (text and images) that set new records across major benchmarks, passing the bar exam at the 90th percentile. The same month, Anthropic released **Claude 1**, its first public model, emphasizing safety and helpfulness[18]. |
| + | **March 2023**: OpenAI released **GPT-4**, a multimodal model (text and images) that set new records across major benchmarks, passing the bar exam at the 90th percentile. The same month, Anthropic released **Claude 1**, its first public model, emphasizing safety and helpfulness[[cite: OpenAI, ["GPT-4 Technical Report"](https://arxiv.org/abs/2303.08774), arXiv:2303.08774, March 2023]]. |
| | |
| - | **December 2023**: Google released **Gemini**, a family of multimodal models (Ultra, Pro, Nano), positioning them as direct GPT-4 competitors. Mistral AI released **Mixtral 8x7B**, the first major open-source mixture-of-experts model[18]. |
| + | **December 2023**: Google released **Gemini**, a family of multimodal models (Ultra, Pro, Nano), positioning them as direct GPT-4 competitors. Mistral AI released **Mixtral 8x7B**, the first major open-source mixture-of-experts model[[cite: Mistral AI, ["Mixtral of experts"](https://mistral.ai/news/mixtral-of-experts), December 2023]]. |
| | |
| - | **2024**: The frontier advanced rapidly. Anthropic's **Claude 3 Opus** matched GPT-4 on most benchmarks. Google's **Gemini 1.5 Pro** introduced a 1-million-token context window. OpenAI's **GPT-4o** added native multimodal understanding (text, images, audio). Meta released **Llama 3** with open weights, continuing the push for open-source AI. OpenAI's **o1** introduced "reasoning models" that think step-by-step before answering, dramatically improving performance on mathematics and science benchmarks[18]. |
| + | **2024**: The frontier advanced rapidly. Anthropic's **Claude 3 Opus** matched GPT-4 on most benchmarks. Google's **Gemini 1.5 Pro** introduced a 1-million-token context window. OpenAI's **GPT-4o** added native multimodal understanding (text, images, audio). Meta released **Llama 3** with open weights, continuing the push for open-source AI. OpenAI's **o1** introduced "reasoning models" that think step-by-step before answering, dramatically improving performance on mathematics and science benchmarks[[cite: Google DeepMind, ["Gemini"](https://deepmind.google/technologies/gemini/), model family, December 2023]]. |
| | |
| - | **January 2025**: DeepSeek released **DeepSeek-R1**, a reasoning model matching o1's performance at a fraction of the training cost, triggering a global selloff in AI chip stocks and challenging the assumption that frontier AI required hundreds of millions of dollars in compute[18]. |
| + | **January 2025**: DeepSeek released **DeepSeek-R1**, a reasoning model matching o1's performance at a fraction of the training cost, triggering a global selloff in AI chip stocks and challenging the assumption that frontier AI required hundreds of millions of dollars in compute[[cite: D. Guo et al., ["DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning"](https://arxiv.org/abs/2501.12948), January 2025]]. |
| | |
| | ## 2025: reasoning goes mainstream |
| | |
| | The year after ChatGPT's wake-up call, the frontier converged on a new axis. OpenAI's o1 had shown that letting a model "think" — generating long internal reasoning chains before answering — unlocked capabilities that scale alone could not reach. In 2025, that insight became the industry's default: every major lab shipped reasoning models, and the question shifted from *whether* models could reason to *what they should reason about*. |
| | |
| | | Model | Lab | Date | Why it mattered | |
| | |---|---|---|---| |
| | | DeepSeek-R1 | DeepSeek | Jan 2025 | Open-weights reasoning matching o1 at a fraction of the cost — the release that crashed AI chip stocks | |
| | | o3 / o4-mini | OpenAI | Apr 2025 | Reasoning became the default mode, not a demo | |
| | | Claude 4 family | Anthropic | May–Nov 2025 | Opus 4, Opus 4.1, Sonnet 4.5, Opus 4.5 — the frontier for coding and agentic work | |
| | | GPT-5 | OpenAI | Aug 2025 | Unified model routing between fast answers and deep reasoning | |
| | | Gemini 2.5 Pro / Gemini 3 | Google | Mar / Nov 2025 | Reasoning folded into the multimodal stack | |
| | | Llama 4 | Meta | Apr 2025 | Open-weights mixture-of-experts; the Behemoth variant promised frontier scale | |
| | | Grok 4 | xAI | Jul 2025 | Trained on Colossus, then the world's largest single GPU cluster | |
| | |
| | Three developments defined the era beyond the models themselves: |
| | |
| - | **AI won its science Nobels.** In October 2024, the physics prize went to John Hopfield and Geoffrey Hinton for the neural-network foundations of machine learning, and the chemistry prize to Demis Hassabis, John Jumper, and David Baker for computational protein design and structure prediction — the first time the field's mathematics had been honored as physics and chemistry in their own right[18]. |
| + | **AI won its science Nobels.** In October 2024, the physics prize went to John Hopfield and Geoffrey Hinton for the neural-network foundations of machine learning, and the chemistry prize to Demis Hassabis, John Jumper, and David Baker for computational protein design and structure prediction — the first time the field's mathematics had been honored as physics and chemistry in their own right[[cite: Nobel Prizes 2024 — [Physics (Hopfield, Hinton)](https://www.nobelprize.org/prizes/physics/2024/summary/) and [Chemistry (Baker, Hassabis, Jumper)](https://www.nobelprize.org/prizes/chemistry/2024/summary/), nobelprize.org]]. |
| | |
| | **The agentic turn.** 2025 was the year models stopped merely answering questions and started doing work. Anthropic's Claude Code and OpenAI's Codex agent turned large language models into software engineers that could be assigned tasks — reading codebases, writing patches, running tests — and real-world software benchmarks replaced trivia-style evaluations as the scoreboard that mattered. Protocols like MCP began standardizing how models connect to tools and data. The phrase of the year was "agentic": systems that act, check their work, and act again. |
| | |
| - | **The infrastructure race.** Frontier AI became, openly, a contest of capital. OpenAI, Oracle, and SoftBank announced the **Stargate** initiative in January 2025 — a plan for half a trillion dollars of US data-center construction. xAI built the Colossus cluster in Memphis; Meta stood up **Superintelligence Labs** and paid unprecedented sums to recruit top researchers. The economics of the field hardened into the pattern the scaling hypothesis predicted: whoever could marshal the most compute, power, and talent set the pace[19]. |
| + | **The infrastructure race.** Frontier AI became, openly, a contest of capital. OpenAI, Oracle, and SoftBank announced the **Stargate** initiative in January 2025 — a plan for half a trillion dollars of US data-center construction. xAI built the Colossus cluster in Memphis; Meta stood up **Superintelligence Labs** and paid unprecedented sums to recruit top researchers. The economics of the field hardened into the pattern the scaling hypothesis predicted: whoever could marshal the most compute, power, and talent set the pace[[cite: OpenAI / SoftBank / Oracle, ["Announcing The Stargate Project"](https://openai.com/index/announcing-the-stargate-project/), January 2025]]. |
| | |
| | ```fact Honest horizon |
| | This table covers releases through late 2025. The frontier moves faster than any article can be revised — treat this section as the most perishable on the page, and check primary sources for what has happened since. |
| | ``` |
| | |
| | ## The open-source movement |
| | |
| - | A parallel development has been the rise of open-weight models. Meta's **Llama** series (2023–2025), Mistral's models, Google's **Gemma**, and DeepSeek's releases have made frontier-capable AI available to anyone with sufficient hardware. The debate over whether frontier AI should be open or closed remains one of the field's most contentious[18]. |
| + | A parallel development has been the rise of open-weight models. Meta's **Llama** series (2023–2025), Mistral's models, Google's **Gemma**, and DeepSeek's releases have made frontier-capable AI available to anyone with sufficient hardware. The debate over whether frontier AI should be open or closed remains one of the field's most contentious[[cite: Meta, ["Llama"](https://ai.meta.com/llama/), open-weight model family; T. Touvron et al., ["Llama 2: Open Foundation and Fine-Tuned Chat Models"](https://arxiv.org/abs/2307.09288), 2023]]. |
| | |
| | --- |
| | |
| | # AI ethics, safety, and regulation |
| | |
| | The rapid advance of AI capabilities has generated an equally rapid response from policymakers, ethicists, and the AI research community itself. |
| | |
| | ## The alignment problem |
| | |
| | The **alignment problem** — the challenge of ensuring that AI systems pursue goals that are beneficial to humans — has moved from a niche concern of a few researchers to a central topic of mainstream AI discourse. The problem was articulated in its modern form by Stuart Russell in his 2019 book *Human Compatible*: if we build machines that are more capable than we are, and those machines are optimizing for objectives that don't perfectly capture human values, the results could be catastrophic — not because the machine is malicious, but because it is indifferent to anything we didn't explicitly specify. |
| | |
| | The alignment problem is not hypothetical. Even current language models exhibit misalignment in everyday ways: they confidently state falsehoods ("hallucination"), they can be manipulated into producing harmful outputs ("jailbreaking"), and they sometimes pursue the letter of an instruction while violating its spirit. These are mild versions of a problem that could become severe as AI systems gain more autonomy and capability. |
| | |
| - | In May 2023, **Geoffrey Hinton**, widely regarded as the "godfather of deep learning" and a co-author of the foundational 1986 backpropagation paper, resigned from Google to speak freely about the risks of AI. Hinton warned that AI systems might become more intelligent than their creators within years rather than decades, and that there were no guaranteed methods for maintaining control. "I console myself with the normal excuse: if I hadn't done it, somebody else would have," Hinton told the *New York Times*. His departure was a watershed moment — a founder of the field publicly declaring that the technology he helped create might pose existential risks[20]. |
| + | In May 2023, **Geoffrey Hinton**, widely regarded as the "godfather of deep learning" and a co-author of the foundational 1986 backpropagation paper, resigned from Google to speak freely about the risks of AI. Hinton warned that AI systems might become more intelligent than their creators within years rather than decades, and that there were no guaranteed methods for maintaining control. "I console myself with the normal excuse: if I hadn't done it, somebody else would have," Hinton told the *New York Times*. His departure was a watershed moment — a founder of the field publicly declaring that the technology he helped create might pose existential risks[[cite: Ethics.ai, ["Timeline of AI Ethics: 1942 to Today"](https://ethics.ai/timeline); Future of Life Institute, ["Pause Giant AI Experiments: An Open Letter"](https://futureoflife.org/open-letter/pause-giant-ai-experiments/), March 2023]]. |
| | |
| - | In March 2023, the **Future of Life Institute** published an open letter calling for a six-month pause on training AI systems more powerful than GPT-4. The letter was signed by over 30,000 people, including Elon Musk, Steve Wozniak, and Yoshua Bengio. Many prominent AI researchers declined to sign, arguing that a pause was impractical, unenforceable, and would not address the underlying technical challenges. The letter nonetheless demonstrated that concerns about AI safety had moved from the academic fringe to the mainstream[20]. |
| + | In March 2023, the **Future of Life Institute** published an open letter calling for a six-month pause on training AI systems more powerful than GPT-4. The letter was signed by over 30,000 people, including Elon Musk, Steve Wozniak, and Yoshua Bengio. Many prominent AI researchers declined to sign, arguing that a pause was impractical, unenforceable, and would not address the underlying technical challenges. The letter nonetheless demonstrated that concerns about AI safety had moved from the academic fringe to the mainstream[[cite: Ethics.ai, ["Timeline of AI Ethics: 1942 to Today"](https://ethics.ai/timeline); Future of Life Institute, ["Pause Giant AI Experiments: An Open Letter"](https://futureoflife.org/open-letter/pause-giant-ai-experiments/), March 2023]]. |
| | |
| - | In November 2023, 28 countries and the European Union signed the **Bletchley Declaration** at the first AI Safety Summit, hosted at Bletchley Park — Alan Turing's wartime workplace. The declaration acknowledged the potential for "serious, even catastrophic" harm from frontier AI and committed signatory nations to cooperating on AI safety research[20]. |
| + | In November 2023, 28 countries and the European Union signed the **Bletchley Declaration** at the first AI Safety Summit, hosted at Bletchley Park — Alan Turing's wartime workplace. The declaration acknowledged the potential for "serious, even catastrophic" harm from frontier AI and committed signatory nations to cooperating on AI safety research[[cite: UK Government, ["The Bletchley Declaration"](https://www.gov.uk/government/publications/ai-safety-summit-2023-the-bletchley-declaration), AI Safety Summit, November 2023]]. |
| | |
| | ## Regulation |
| | |
| | | Date | Action | Jurisdiction | |
| | |---|---|---| |
| | | Oct 2023 | Executive Order 14110 on AI safety | United States | |
| | | Nov 2023 | Bletchley Declaration on AI safety | 28 countries + EU | |
| | | Jun 2024 | EU AI Act signed into law | European Union | |
| | | Aug 2024 | EU AI Act enters into force | European Union | |
| | | Feb 2025 | EU AI Act prohibited practices begin applying | European Union | |
| | | Jan 2025 | US revokes Executive Order 14110 | United States | |
| | | Aug 2026 | EU AI Act full application date | European Union | |
| | |
| - | The **EU AI Act** (Regulation 2024/1689) is the world's first comprehensive AI statute. It follows a risk-based approach: certain AI practices are outright prohibited (social scoring, real-time biometric surveillance in public spaces with narrow exceptions), high-risk AI systems must meet strict requirements for transparency, testing, and human oversight, and general-purpose AI models face disclosure obligations regarding training data and capabilities[21]. |
| + | The **EU AI Act** (Regulation 2024/1689) is the world's first comprehensive AI statute. It follows a risk-based approach: certain AI practices are outright prohibited (social scoring, real-time biometric surveillance in public spaces with narrow exceptions), high-risk AI systems must meet strict requirements for transparency, testing, and human oversight, and general-purpose AI models face disclosure obligations regarding training data and capabilities[[cite: [Regulation (EU) 2024/1689](https://eur-lex.europa.eu/eli/reg/2024/1689/oj) ("Artificial Intelligence Act"), Official Journal of the EU, July 12, 2024; [Executive Order 14110](https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence), October 2023]]. |
| | |
| - | The United States has taken a more volatile approach. Executive Order 14110, signed by President Biden in October 2023, required safety testing and government notification for the most powerful AI systems. The order was revoked in January 2025 under the incoming administration, which adopted a deregulatory posture emphasizing American AI competitiveness[21]. |
| + | The United States has taken a more volatile approach. Executive Order 14110, signed by President Biden in October 2023, required safety testing and government notification for the most powerful AI systems. The order was revoked in January 2025 under the incoming administration, which adopted a deregulatory posture emphasizing American AI competitiveness[[cite: [Regulation (EU) 2024/1689](https://eur-lex.europa.eu/eli/reg/2024/1689/oj) ("Artificial Intelligence Act"), Official Journal of the EU, July 12, 2024; [Executive Order 14110](https://www.federalregister.gov/documents/2023/11/01/2023-24283/safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence), October 2023]]. |
| | |
| | ## Ongoing debates |
| | |
| | The field remains deeply divided on several fundamental questions: |
| | |
| | **Existential risk.** Some researchers (Hinton, Stuart Russell, Yoshua Bengio) believe that advanced AI poses a genuine existential threat to humanity. The argument is not that AI will "decide to be evil" in some science-fiction sense, but that sufficiently capable systems optimizing for imperfectly specified objectives could cause irreversible harm at a scale and speed that humans could not respond to. Others (Yann LeCun, Andrew Ng, Meta's chief AI scientist) argue that these concerns are overblown, that current AI systems are nowhere near the threshold of autonomous danger, and that the existential risk framing distracts from more immediate and concrete harms: algorithmic bias, misinformation at scale, surveillance overreach, and job displacement. |
| | |
| | **Copyright and training data.** The *New York Times v. OpenAI* lawsuit (filed December 2023) and similar cases are testing whether training AI models on copyrighted material constitutes fair use under US law. If courts find that it does not, the economics of the entire LLM industry could be disrupted — companies would need to license or replace their training data. If courts find that it does, content creators face a world in which their work can be ingested by commercial systems without compensation. As of 2025, no definitive ruling has been issued, and the cases are working through the US court system. |
| | |
| | **Autonomous agents.** AI systems that can browse the web, execute code, manage files, and take actions in the real world — so-called "agentic AI" — emerged as a major capability in 2025. These systems raise oversight questions that existing regulatory frameworks were not designed to address. When an AI agent books a flight, sends an email, or modifies a database, who is responsible for errors? When an agent operates a computer for hours without human oversight, how are its actions audited? These questions are largely unresolved. |
| | |
| | **Concentration of power.** Frontier AI development requires billions of dollars in compute. As of 2025, only a handful of organizations — OpenAI, Anthropic, Google DeepMind, Meta, xAI, DeepSeek, and a few others — can train frontier models. This concentration of capability has no clear precedent in the history of technology. Whether it is sustainable, desirable, or dangerous is an open question. DeepSeek's demonstration that competitive models can be trained at a fraction of the expected cost has somewhat loosened this constraint, but the gap between frontier labs and everyone else remains enormous. |
| | |
| | **Consciousness and understanding.** Do large language models "understand" anything, or are they sophisticated pattern-matching engines? The question is philosophically ancient but has taken on new urgency. When a model writes a correct proof, debugs a program, or explains a nuanced concept, is there "someone home" — or is it, in philosopher John Searle's formulation, a very elaborate Chinese Room? The AI research community has no consensus, and the question may not be answerable by current methods. |
| | |
| | --- |
| | |
| | # Timeline of key milestones |
| | |
| | | Year | Milestone | |
| | |---|---| |
| | | 1950 | Turing publishes "Computing Machinery and Intelligence" | |
| | | 1956 | Dartmouth workshop; the term "artificial intelligence" is coined | |
| | | 1958 | Rosenblatt builds the Mark I Perceptron | |
| | | 1958 | McCarthy creates LISP | |
| | | 1966 | Weizenbaum creates ELIZA | |
| | | 1969 | Minsky and Papert publish *Perceptrons*; neural network funding collapses | |
| | | 1973 | Lighthill Report; UK government cuts AI funding | |
| | | 1974 | Werbos describes backpropagation in his PhD thesis | |
| | | 1980 | XCON expert system saves DEC $40M/year | |
| | | 1982 | Japan launches the Fifth Generation Computer Systems project | |
| | | 1986 | Rumelhart, Hinton, and Williams publish backpropagation for neural networks | |
| | | 1987 | Lisp machine market crashes; second AI winter begins | |
| | | 1997 | Deep Blue defeats Garry Kasparov | |
| | | 2009 | Fei-Fei Li introduces ImageNet | |
| | | 2011 | IBM Watson wins Jeopardy!; Apple releases Siri; DanNet wins first vision contests | |
| | | 2012 | AlexNet wins ImageNet; the deep learning revolution begins | |
| | | 2014 | Ian Goodfellow introduces GANs | |
| | | 2016 | AlphaGo defeats Lee Sedol | |
| | | 2017 | "Attention Is All You Need" introduces the Transformer | |
| | | 2018 | GPT and BERT published | |
| | | 2020 | GPT-3; AlphaFold 2 solves protein structure prediction | |
| | | 2022 | ChatGPT launches; reaches 100M users in two months | |
| | | 2023 | GPT-4; Claude; Gemini; Hinton warns about AI risk; EU agrees on AI Act | |
| | | 2024 | Claude 3; GPT-4o; Llama 3; EU AI Act signed; AI takes the physics and chemistry Nobels | |
| | | 2025 | DeepSeek-R1; reasoning goes mainstream; GPT-5, Claude 4, Gemini 3; the agentic coding era; EU AI Act begins enforcement | |
| | |
| | --- |
| | |
| | # What the history tells us |
| | |
| | Looking back across seven decades, several patterns emerge. |
| | |
| | **AI advances in waves, not linearly.** The field's history is not a smooth upward curve. It is a series of surges — each driven by a new approach (symbolic reasoning, expert systems, statistical learning, deep learning, large language models) — interrupted by periods of disillusionment when the approach's limitations become apparent. Each wave builds on the ruins of the previous one. Backpropagation was published in the depths of the expert systems winter. The Transformer was developed years before anyone imagined ChatGPT. The next breakthrough is likely being worked on now, in a corner of the field that the mainstream is ignoring. |
| | |
| | **Compute is the great enabler.** Turing had the ideas in 1950; the hardware to implement them took decades. The perceptron's architecture was sound; the compute to train deep versions of it didn't exist until the 2010s. The Transformer was a 2017 paper; it took thousands of GPUs and billions of dollars to turn it into GPT-4. Nearly every major AI breakthrough has been a combination of an old idea and new hardware. The researchers who developed backpropagation, convolutional networks, and attention mechanisms did so years or decades before the hardware existed to exploit their inventions. |
| | |
| | **Predictions about AI are almost always wrong.** The optimists of the 1960s predicted general AI within a generation. The pessimists of the AI winters predicted that neural networks were a dead end. The consensus of 2015 was that Go would not be solved for another decade. The consensus of 2020 was that writing fluent prose required understanding that machines lacked. The only reliable prediction about AI is that the timeline will be wrong — and the direction of the error is unpredictable. |
| | |
| | **The social consequences lag behind the technical capabilities.** Deep Blue defeated Kasparov in 1997, but the societal impact of AI didn't become a mainstream concern until ChatGPT in 2022 — a quarter century later. AlphaFold solved protein folding in 2020, but the full impact on drug discovery and biology will unfold over decades. Today's frontier models can write software and conduct research; the societal adjustments those capabilities demand have barely begun. |
| | |
| | --- |
| | |
| | # Quiz: were you paying attention? |
| | |
| | - [ ] Who coined the term "artificial intelligence" — and at what event? |
| | - [ ] What was the name of the devastating 1973 report that led Britain to cut AI funding — and what did its author conclude? |
| | - [ ] What fundamental limitation of the perceptron did Minsky and Papert demonstrate in 1969, and why did it matter so much? |
| | - [ ] What was AlexNet, and why did its 2012 result shock the computer vision community? |
| | - [ ] What fifty-year-old biology problem did AlphaFold 2 solve — and what did DeepMind do with the results? |
| | - [ ] What 2017 architecture underpins nearly all modern large language models — and who wrote the paper? |
| | |
| | ```hide Show answers |
| | 1 — **John McCarthy**, in the 1955 proposal for the **Dartmouth workshop** (summer 1956) · 2 — the **Lighthill Report** by Sir James Lighthill — he concluded "in no part of the field have the discoveries made so far produced the major impact that was then promised" · 3 — a single-layer perceptron cannot learn the **XOR function** (not linearly separable) — the result was widely misinterpreted as proving all neural networks were useless, collapsing funding for a decade · 4 — a **deep convolutional neural network** by Krizhevsky, Sutskever, and Hinton that won ImageNet 2012 with a top-5 error of 15.3%, beating the runner-up by over 10 percentage points — it convinced the field that deep learning on GPUs was the path forward · 5 — **protein structure prediction** (CASP14, median GDT 92.4) — DeepMind then released predicted structures for over 200 million proteins in a public database · 6 — the **Transformer**, from "Attention Is All You Need" by Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, and Polosukhin (Google, June 2017). |
| | ``` |
| | |
| | --- |
| | |
| | # See also |
| | |
| | - [[Machine_Learning|Machine Learning]] |
| | - [[Neural_Networks|Neural Networks]] |
| | - [[Deep_Learning|Deep Learning]] |
| | - [[Natural_Language_Processing|Natural Language Processing]] |
| | - [[Artificial_General_Intelligence|Artificial General Intelligence]] |
| | |
| | --- |
| | |
| - | # References |
| - | |
| - | [1] A. M. Turing, "Computing Machinery and Intelligence," *Mind*, vol. LIX, no. 236, pp. 433–460, October 1950. |
| - | [2] J. McCarthy, M. L. Minsky, N. Rochester, and C. E. Shannon, "A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence," August 31, 1955. Reprinted in *AI Magazine*, vol. 27, no. 4, 2006. |
| - | [3] F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," *Psychological Review*, vol. 65, no. 6, pp. 386–408, 1958. M. Minsky and S. Papert, *Perceptrons*, MIT Press, 1969. |
| - | [4] Actuaries Institute, "History of AI Winters," actuaries.asn.au. |
| - | [5] J. Lighthill, "Artificial Intelligence: A General Survey," in *Artificial Intelligence: a paper symposium*, Science Research Council, 1973. |
| - | [6] T. Haigh, "There Was No 'First AI Winter'" and "How the AI Boom Went Bust," *Communications of the ACM*, 2023–2024. |
| - | [7] StoryOfAI.net, "The Second AI Winter: How the Boom Turned to Bust (1987–1993)." |
| - | [8] A. Ishikawa, "A Retrospective and Prospects of the Fifth Generation Computer Project," University of Texas, September 1993. |
| - | [9] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning Representations by Back-Propagating Errors," *Nature*, vol. 323, pp. 533–536, October 1986. |
| - | [10] IBM, "Deep Blue," ibm.com/history/deep-blue. IBM, "Watson, Jeopardy! Champion," ibm.com/history/watson-jeopardy. |
| - | [11] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," *Advances in Neural Information Processing Systems*, 2012. |
| - | [12] D. Silver et al., "Mastering the Game of Go with Deep Neural Networks and Tree Search," *Nature*, vol. 529, pp. 484–489, January 2016. |
| - | [13] A. Vaswani et al., "Attention Is All You Need," *Advances in Neural Information Processing Systems*, 2017. |
| - | [14] A. Radford et al., "Improving Language Understanding by Generative Pre-Training," OpenAI, 2018. T. Brown et al., "Language Models Are Few-Shot Learners," *NeurIPS*, 2020. |
| - | [15] J. Devlin, M. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," *NAACL*, 2019. |
| - | [16] J. Jumper et al., "Highly Accurate Protein Structure Prediction with AlphaFold," *Nature*, vol. 596, pp. 583–589, 2021. |
| - | [17] D. Guo et al., "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning," arXiv:2501.12948, January 2025. |
| - | [18] The Nobel Prize in Physics 2024 (J. J. Hopfield, G. E. Hinton); The Nobel Prize in Chemistry 2024 (D. Baker, D. Hassabis, J. M. Jumper), nobelprize.org. |
| - | [19] The White House, "Announcing the Stargate Project," January 2025; OpenAI, Anthropic, Google DeepMind, Meta, and xAI model announcements, 2023–2025. |
| - | [20] Ethics.ai, "Timeline of AI Ethics: 1942 to Today," ethics.ai/timeline. |
| - | [21] Regulation (EU) 2024/1689, "Artificial Intelligence Act," Official Journal of the European Union, July 12, 2024. |
| | |
| | [[Category:Artificial_Intelligence]] |
| | |