"Let's Sell AGI to the US, China, and Russia" ... OpenAI Considered a 'National Auction' Plan

One of the reasons Dario Amodei left OpenAI

Leak of a summary of a book covering the founding of Anthropic

"The split between Altman and Amodei was a trust issue"

[Block Media James Jung Reporter] The background behind Dario Amodei's departure from OpenAI was not solely due to philosophical differences surrounding artificial intelligence (AI) safety.

There were discussions at OpenAI about auctioning the rights to artificial general intelligence (AGI) to governments such as the United States, China, and Russia to secure funding. It has been reported that Amodei learned about this plan, referred to as the 'countries plan,' which led to a breakdown of trust between the two sides.

The 'countries plan,' which was once discussed internally at OpenAI, originated from the idea that the risks of AGI should be managed collectively by the international community. However, at a time when OpenAI was facing financial difficulties, they considered a shocking proposal to effectively auction the rights to future AGI to various governments.

Dario Amodei, who was the head of research at OpenAI, strongly opposed this plan, arguing that it could trigger an arms race in AI and lead to human rights violations.

Funding by Selling AGI to Governments

The American monthly magazine The Atlantic published excerpts from Kevin Roose's new book "The AGI Chronicles" on the 29th of last month (local time). Roose interviewed over 150 officials from OpenAI, Anthropic, and Google DeepMind and traced the process of the split between Amodei and OpenAI's management based on internal documents.

Amodei joined OpenAI, which was a non-profit research institute at the time, after leaving Google in 2016. He led research indicating that scaling up AI models leads to improved intelligence.

The conflict began in 2017 when Elon Musk's funding was halted, forcing OpenAI to find new ways to secure massive funding for AI development.

Greg Brockman, co-founder of OpenAI, and Ilya Sutskever, chief scientist, proposed the 'countries plan.' Initially, the idea was for several countries to create an international coalition to jointly manage the risks of AGI and share its benefits.

However, the nature of the proposal changed during the internal announcement phase. It was discussed that a sort of auction would be held to provide rights to future AGI technology to countries like the US, China, and Russia to secure funding. One participant recalled that it sounded like a startup-style idea to extract money from governments.

It has not been confirmed whether this proposal was directly suggested by Sam Altman, the CEO of OpenAI.

If We Have to Hand It Over to Governments, Let's Sell It

The auction idea was linked to national security. The reasoning was that if AGI becomes a reality, governments would likely take the technology from the developing companies. If the technology is going to be handed over to governments anyway, it would be better to secure funding from them in advance at a time when money is needed.

This shocking plan emerged while OpenAI was looking for new funding sources, and it is also intertwined with the current controversies surrounding AI apocalypse theories and state-led AI development, raising fundamental questions about "what private AI development companies are thinking when creating AGI technology."

Amodei strongly opposed the countries plan. In December 2017, he sent an email to Brockman, Sutskever, and other executives arguing that this plan could not only fail to reduce the arms race but could actually exacerbate it. He raised concerns that authoritarian governments securing AGI reflecting their values could lead to serious human rights violations.

He warned that such a plan could undermine OpenAI's moral legitimacy. He believed that an organization advocating for safe AGI development contradicts its founding purpose by providing powerful technology to various governments for funding.

In reality, the plan was never implemented. Later, Microsoft (MS) made a significant investment in OpenAI, causing the countries plan to be shelved.

OpenAI refuted Roose's claims, stating that they briefly considered whether OpenAI could be transformed into an international collaborative project at that time but did not consider selling AGI to China or Russia.

From the 'Countries Plan' to MS and GPT-3... Growing Distrust

The countries plan was not the sole reason for the split. Various incidents compounded the growing distrust between Amodei and the management.

There was also conflict surrounding the non-public version of GPT-3. Amodei requested a list of customers who had been approved to use the model from Altman. However, he later discovered that there was a longer list than the one he received. Amodei began to suspect that the information provided to him was being filtered.

The investment negotiations with MS were also problematic. OpenAI's charter included principles stating that if other projects emphasizing safety approached AGI development, they could halt competition and collaborate. Amodei wanted this possibility to be guaranteed in the investment contract with MS.

However, Altman explained that MS would not accept this. Amodei began to doubt how strongly Altman had raised this demand during the actual negotiations. Disagreements over individual business decisions expanded into questions of whether the management could be trusted.

In 2020, the attempt to provide rapper Kanye West with access to the beta version of GPT-3 also became controversial. After meeting West, Altman attempted to invite him to the unofficial GPT-3 beta. Jack Clark, then OpenAI's policy head, intervened, and the actual invitation did not take place.

For Amodei, the core issue was ultimately 'trust.' It is reported that he kept a separate record of instances where he felt he was 'being deceived' by Altman while working at OpenAI. According to the provided materials, one memo noted that the core issue was 'Altman himself.'

Breakdown of Trust and Separation

The conflict eventually escalated into issues regarding the organizational power structure. Amodei requested that a new product launch be subject to a veto power by the safety team as a condition for staying at OpenAI. The key issue was who should have the final decision-making authority when AI safety conflicts with business goals.

During the negotiations, Amodei and six colleagues created an unofficial Slack channel called 'BATNA.' BATNA (Best Alternative To a Negotiated Agreement) refers to the best alternative that can be chosen if negotiations break down.

The secret meeting included Daniella Amodei (Dario's cousin), Jack Clark, Sam McCandlish, Tom Brown, Chris Ola, and Jared Kaplan. They were preparing for separation while negotiating to stay at OpenAI.

Ultimately, both sides involved lawyers in what they internally referred to as 'divorce' negotiations over the terms of departure. Altman agreed to give them a chance to recruit OpenAI employees for a certain period, with the condition that they would not recruit additional employees for one year afterward. There was also a condition that neither side would publicly disparage the other.

In December 2020, during an all-hands video conference, Amodei and his colleagues announced that they would leave the company to create a new AI safety organization. By the end of the meeting, their OpenAI emails and Slack access were also blocked. Amodei founded Anthropic in 2021.

This content is provided for general informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online promotions, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets. Crypto assets are highly volatile and may result in loss. The availability of WEEX services, products, and related events may vary by region. You are responsible for ensuring that your participation is in accordance with applicable local laws and regulations.