FAR.AI

TechCrunch covered a discussion at our Journalism Workshop in Washington, D.C., where New York State Assemblymember Alex Bores spoke about the RAISE Act and the growing political debate around state-level AI safety efforts. The conversation highlighted why we convene these gatherings. Policymakers, researchers, and industry leaders need clear, evidence-based guidance related to advanced AI systems. Our mission is to make advanced AI safe and beneficial for everyone, and to support policymakers with rigorous, nonpartisan insights. Link to full article by TechCrunch and the Journalism Workshop 👇

That’s a wrap on the AGI Journalism Workshop, hosted by FAR.AI and the Tarbell Center for AI Journalism in Washington, D.C. Journalists, editors, and researchers came together to explore how newsrooms can better understand, communicate, and investigate the implications of AGI. ▸ Talk by Helen Toner (Center for Security and Emerging Technology (CSET)) on AGI timelines ▸ Fireside chat with Anton Korinek (UVA) & Ioana Marinescu (Penn) moderated by Tharin Pillay on AI’s impact on jobs and the economy ▸ Talks by Scott S. (CEIP) and Saif M. Khan (Institute for Progress) on U.S.–China AI policy and semiconductor export controls ▸ Technical perspectives from Adam Gleave, Evan Hubinger (Anthropic) & Jason Wolfe (OpenAI) on misalignment and detecting scheming in models ▸ Policy sessions featuring Rocco Casagrande (Deloitte), Alex Bores (NY State Assembly), Dean Ball (FAI), and Ben Buchanan (Johns Hopkins) on AI governance, biosecurity, and regulatory strategy Thank you to all speakers and attendees for advancing thoughtful reporting on AI and safety. Links in comments. 👇

Frontier AI models with openly available weights are steadily becoming more powerful and widely adopted. However, compared to proprietary models, open-weight models pose different opportunities and challenges for effective risk management. For example, they allow for more open research and testing. However, managing their risks is also challenging because they can be modified arbitrarily, used without oversight, and spread irreversibly. Addressing these challenges will be key to both realizing their benefits and mitigating their harms. In this paper, we present 16 open technical problems for open-weight model safety involving training data, training algorithms, evaluations, deployment, and ecosystem monitoring. We conclude by discussing the nascent state of the field, emphasizing that openness about research, methods, and evaluations – not just weights – will be key to building a rigorous science of open-weight model risk management. This paper was co-authored by Stephen Casper, Kyle O'Brien, Shayne Longpre, Elizabeth Seger, Kevin Klyman, Rishi Bommasani, Aniruddha N., Ilia Shumailov, Sören Mindermann, Yoshua Bengio, Steven Basart, Dan Hendrycks, Frank Rudzicz, Kellin Pelrine, Avijit Ghosh, PhD, Andrew Strait, Robert Kirk, Geoffrey Irving, Yarin Gal, Peter Henderson, Zico Kolter, and Dylan Hadfield-Menell. Link to Open Technical Problems in Open-Weight AI Model Risk Management in comments 👇

Can social trust survive when healthcare regulators often verify systems with mere screenshots? After 17 years as a federal regulator, Tina Morrison joined EQTY Lab to tackle this challenge. With 91% of enterprises facing supply chain attacks, her team embedded verifiable compute into next-generation chips, adding just 10% computational overhead. Morrison demonstrated this by verifying a complex AI workflow without accessing its data or reproducing the pipeline. Highlights: ▸ Rapid compliance with AI policies like the EU AI Act ▸ Binding policies embedded directly in next-gen Intel/NVIDIA chips ▸ Auditable safeguards proving AI outputs are genuine and secure ▸ End-to-end confidentiality while maintaining verifiability Link to the full talk from Technical Innovations for AI Policy in comments 👇

Great to partner with AI Security Institute on their inaugural Alignment Conference! Collaboration between researchers, funders, and policymakers is essential for advancing AI alignment.

If you’re attending #NeurIPS2025 or nearby, join us for the San Diego Alignment Workshop: Open Social, an informal evening of conversation and connection hosted by FAR.AI. 📍 Omni San Diego Hotel at the Ballpark (Grand Ballroom Foyer) ⏰ Monday, December 1 | 7–9PM PST Light refreshments and beverages will be provided. We’re asking everyone to RSVP in advance (by November 30) so we can plan accordingly. All NeurIPS attendees and others interested in AI safety and alignment are welcome. Bring a friend and spread the word! 👇

Congratulations to our CEO Adam Gleave on being selected as an #AI2050 Early Career Fellow! This 3-year grant from Schmidt Sciences will advance our research on AI interpretability and safety, specifically developing new methods to audit what AI systems have actually learned and remove unwanted behaviors. We're proud to be among this year’s cohort, which includes 28 researchers from 42 institutions across eight countries, who are collectively receiving more than $18 million to advance AI for the benefit of humanity. Thanks to Eric Schmidt, James Manyika, and the AI2050 team for supporting this important work on AI safety. 👇

🔊 We’re hiring a Senior Communications Manager! Join Samuel Bauer in transforming complex research into materials that inform policy, shape industry standards, and support global understanding.

Our team uncovered vulnerabilities in frontier AI models that prompted patches that now block high-impact misuse. Q3 2025 brought major research breakthroughs and deepened global collaboration in AI safety. Highlights: ▸ Red-teaming GPT-5: We partnered with OpenAI to strengthen safeguards, now significantly better at preventing CBRN misuse. ▸ Attempt to Persuade Evaluation: Our new APE benchmark exposed key weaknesses. Following disclosure, Gemini 2.5 Pro, GPT-5, and Claude Opus 4.1 all reduced persuasion behaviors. ▸ Jailbreak-Tuning Research: Our work demonstrated that leading models: DeepSeek-R1, Gemini 2.0 Flash, GPT-4.1 — remain at risk from combined jailbreak-tuning attacks. ▸ AI Safety Connect at UNGA: We convened 100+ leaders from government, labs, academia, and civil society to shape shared standards for mitigating misuse. ▸ We are currently hiring for multiple roles across the organization. ▸ Join us December 1-2 for the San Diego Alignment Workshop! Link to our newsletter in comments 👇

We will increasingly trust the information, analysis and courses of action AI provides. This has relevance in military contexts and intelligence analysis. @Steve Kelly (IST) speculates on risks from over-reliance on AI in warfare. His broader analysis covers three crucial questions: 1️⃣ How can AI help defense? 2️⃣ Will AI stabilize or destabilize geopolitics? 3️⃣ Could AIs themselves become security threats? Highlights: ▸ The U.S. is adding AI gradually, keeping humans in charge of weapons, though some defenses already use autonomy when split-second action is needed. ▸ China's military doctrine officially maintains human control, but their self-identified "five incapables" reveal gaps in commander readiness. ▸ Growing dependence on AI could erode human reasoning and decision-making across all domains over time. ▸ Future networks of AI systems may show unpredictable behaviors that create new risks. Link to Technical Innovations for AI Policy recording in comments 👇