This 1960s Chatbot Was a Precursor to AI. Its Maker Grew to Fear It.

In 1966, computer scientist Joseph Weizenbaum built a primitive computer program he named ELIZA. Almost immediately, he regretted his creation. 

Developed to mimic simple psychotherapy exchanges, ELIZA sparked unexpectedly deep reactions. Users opened up, shared intimate details about themselves and treated the program as if it were human. The response was so intense that even Weizenbaum’s secretary at the Massachusetts Institute of Technology (MIT) reportedly asked him to step out so she could speak with the program in private.

ELIZA is widely recognized as the world’s first chatbot, and a version of it is still available online today. Its creation made Weizenbaum a pioneer of artificial intelligence and language-processing models, programs designed to analyze and generate responses by recognizing patterns in text.

“What I had not realized is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people,” Weizenbaum recalled in his 1976 book, Computer Power and Human Reason. This phenomenon, which became known as the “ELIZA effect,” deeply disturbed him.

Until his death in 2008, he devoted his life to being one of technology’s fiercest and most vocal critics. 

The world’s first chatbot was never intended to push the limits of artificial intelligence. 

“Weizenbaum designed ELIZA more as a psychological experiment than a technological one,” explains Brian Merchant, author of Blood in the Machine: The Origins of the Rebellion Against Big Tech. “He was interested in how humans interact with machines, and he designed ELIZA as a means of investigating that relationship.” 

ELIZA worked by scanning a user’s typed input for keywords and generating responses that resembled human conversation based on pattern matching. “ELIZA shows, if nothing else, how easy it is to create and maintain the illusion of understanding,” Weizenbaum wrote. He believed that once the program’s mechanisms were revealed, the “magic” would “crumble away.”

But Merchant says that Weizenbaum was “appalled” by the way users sought emotional intimacy with the bot, especially given how basic the technology was. Even after being told that the program was rudimentary, users continued to attribute understanding, empathy and intention to ELIZA. The response revealed an unsettling insight into the human-machine dynamic. 

“Weizenbaum recognized that our desire to form relationships, and the ease with which a computer program could be built and sold to prey upon that desire, would leave people vulnerable to mass exploitation at the hands of whoever owned and operated the computer systems,” Merchant says. 

Luke Fernandez, an associate professor at Weber State University’s School of Computing, similarly points out that Weizenbaum saw humans’ “power of projection” as a potential risk factor in the evolving relationship between humans and machines.

“Weizenbaum dismissed the idea that there was anything inevitable about where technology might take us or that agency resided in technology itself,” Fernandez says. He worried that if humans placed too much stock in machines, we might jeopardize our accountability to each other.

Weizenbaum strongly believed that human judgment and reasoning carried a moral responsibility, which shaped his understanding of technology and AI. 

Forced to flee Nazi Germany as a young teen, Weizenbaum saw firsthand how the Nazis’ genocidal agenda had been aided by the scores of everyday people who looked the other way. He also recognized the powerful impact of the Nazis’ antisemitic propaganda, which dehumanized Jewish families like his in service of the Nazis’ eugenic effort to engineer a so-called perfect race. This campaign of dehumanization sowed complacency in the general public.

Given his background, Weizenbaum was especially troubled by the dehumanizing language some of his peers used when comparing human reasoning with the possibilities of AI. He bristled at MIT researcher Marvin Minsky’s description of human beings as “meat machines,” and chafed at leading technologists’ implication that future robots could present “a correction and improvement on nature.” Weizenbaum emphasized that this line of thinking had fueled some of the greatest atrocities in history.

“In the Nazi era, Jews were portrayed as vermin, a metaphor that legitimized mass murder,” Weizenbaum told German journalist Bernhard Pörksen in 1998. “Today…the idea that man is merely an information-processing machine that can be replaced by a robot is gaining substance and power.”  

Weizenbaum warned that the “immense power of an inhumane image of man” can “spread like a virus in a society," and that humanity must fiercely guard against it. 

Though Weizenbaum was never dismissive of the future capabilities of technology like AI, the potential power of machines was never his chief concern. Instead, he feared that people’s projections onto these machines would hamper their better judgment. From the 1970s through the early 2000s, he was an advocate for technology ethics, urging scientists to confront the social consequences of their work.

The risk he foresaw was not that technologies like AI would evolve into civilization-destroying monsters, but that society would look upon them as false idols. The real threat to humanity came from within, he believed; technology simply gave it an outlet.

Weizenbaum repeatedly insisted to his friends: “I’m not an AI critic. I’m a critic of society."