AI insiders speak out on risks and the need for regulation

OpenAI employees demand change after the company "prioritizes profits over safety"

A stylized image of Sam Altman with the forces of good and evil on his shoulders. Image by DALL-E.

There are so many upsides to AI, from incredible advances in biotech, to tackling climate change, to helping me write emails in record time. But the race to develop the most advanced AI systems has prompted a group of insiders from leading AI firms to sound the alarm on the dangers that come with such rapid advancement.

Last Tuesday, several current and former OpenAI employees released a statement demanding the "right to warn" the public about AI's dangers without fear of retaliation. The New York Times reported that these employees have accused OpenAI of prioritizing profits and growth over safety, creating an atmosphere where voicing concerns about AI risks is increasingly difficult.

A shift in OpenAI’s mission

OpenAI was founded as a nonprofit organization with the mission "to ensure AGI benefits all of humanity," emphasizing the need to build safe and beneficial artificial general intelligence (AGI) and distribute its benefits broadly. But recent actions and reports on internal culture suggest a shift away from these ambitions.

Former OpenAI researcher Daniel Kokotajlo estimates a 70% likelihood that advanced AI could pose an existential threat to humanity. This fear, often referred to as p(doom) in AI circles, underscores the potential for catastrophic outcomes if the tech isn’t carefully managed.

The right to warn

The statement, titled "A Right to Warn about Advanced Artificial Intelligence," demands transparency and protections for whistleblowers. Endorsed by AI luminaries like Geoffrey Hinton, Yoshua Bengio, and Stuart Russell, the letter calls for:

1. Eliminating nondisparagement clauses: Allowing employees to criticize the company on risk-related concerns without fear of losing vested economic benefits.

2. Anonymous reporting: Establishing anonymous channels for raising concerns to the company’s board, regulators, and independent organizations.

3. Supporting open criticism: Fostering a culture where employees can publicly discuss risk-related issues without retaliation, as long as trade secrets are protected.

4. Whistleblower protections: Safeguarding employees who publicly share risk-related information after other processes have failed.

Voices of dissent

In a Vox article, Kokotajlo expressed his fears about OpenAI’s trajectory. He and others believe that leadership isn’t taking the risks of their technology seriously enough. The departure of key safety-focused employees like Ilya Sutskever and Jan Leike has intensified these concerns.

A clash of incentives

The core issue, as outlined by these insiders, is the misalignment of incentives. AI companies like OpenAI started with ambitious governance structures aimed at prioritizing humanity's best interests. However, the immense computational resources required for advanced AI research drove these companies to form profit-driven partnerships, like OpenAI’s collaboration with Microsoft. This shift skewed priorities toward commercialization and rapid deployment, often at the expense of thorough safety measures.

Carroll Wainwright, another former OpenAI employee, noted that the pressure to maximize profits is increasingly overshadowing mission-aligned work. The board's failed attempt to oust CEO Sam Altman last November showed the limitations of corporate governance in holding the company accountable.

The need for regulation and transparency

The call for a "right to warn" highlights a critical gap in the current oversight of AI development. Traditional whistleblower protections focus on illegal activities, but many AI risks aren’t regulated yet. Insiders, who understand the intricacies and potential hazards of these technologies, are the best people to highlight these issues. Allowing them to speak out can create a powerful incentive for companies to adhere to their public commitments on safety and ethical standards.

So, while AI offers immense potential, the race for advanced systems requires a careful balance between innovation and safety. Ensuring transparency and protecting those who raise concerns are crucial steps in creating a way for AI to develop responsibly.

Weekly disruptions

Groundbreaking AI heart attack scans could soon be rolled out across UK (Guardian) AI scans could predict your heart attack risk 10 years early. This Oxford University tech is being reviewed by the NHS and could save thousands of lives. The AI analyzes regular CT scans to find hidden signs of inflammation that traditional methods miss, which could lead to earlier treatment and prevent heart attacks.

Can We Save Coral Reefs with Machine Learning? (Human+AI) An AI program called SurfPerch is being used to analyze sounds from coral reefs to track fish populations. This helps scientists assess reef health and the effectiveness of conservation efforts. You can even help by listening to coral reef recordings on the "Calling in Our Corals" website.

Tribeca to Screen AI-Generated Short Films Created by OpenAI’s Sora (IndieWire) The Tribeca Film Festival is showcasing the world's first ever AI-generated short films created using OpenAI's new text-to-video model Sora. These films by established directors like "Nanny" director Nikyatu Jusu will be screened on June 15th. Sora allows for complex scenes with camera movement, background characters, and even different perspectives, but lacks audio and restricts violence and nudity.