An ex-OpenAI safety researcher says he's “terrified” by AI development's pace and that labs racing to AGI can cut corners on alignment, pushing all to speed up
and my top reasons to not panic just yet. — In the end, though, I really do think it could give AI labs license to invest less in safety www.platformer.news/deepseek-ai- ... [image] Zachary Miller / @zchrymllr.com : It's astounding to me that science fiction writers and filmmakers from decades ago were like, “What if in the future an artificial intelligence is created and becomes too smart, creates a dystopian nightmare, and kills humanity,” and the tech industry was like, “You son of a bitch, I'm in.” X: Steven Adler / @sjgadler : Some personal news: After four years working on safety across @openai, I left in mid-November. It was a wild ride with lots of chapters - dangerous capability evals, agent safety/control, AGI and online identity, etc. - and I'll miss many parts of it. Steven Adler / @sjgadler : Honestly I'm pretty terrified by the pace of AI development these days. When I think about where I'll raise a future family, or how much to save for retirement, I can't help but wonder: Will humanity even make it to that point? Steven Adler / @sjgadler : IMO, an AGI race is a very risky gamble, with huge downside. No lab has a solution to AI alignment today. And the faster we race, the less likely that anyone finds one in time. Alex Jupiter / @alexjupiter23 : Genuinely wondering what impact anyone working on “AI safety” is having right now. And I'm asking for the whole world. Roi Carthy / @roi : Dwarfed by your work on Appetite for Destruction. Steven Adler / @sjgadler : Today, it seems like we're stuck in a really bad equilibrium. Even if a lab truly wants to develop AGI responsibly, others can still cut corners to catch up, maybe disastrously. And this pushes all to speed up. I hope labs can be candid about real safety regs needed to stop this. Steven Adler / @sjgadler : As for what's next, I'm enjoying a break for a bit, but I'm curious: what do you see as the most important & neglected ideas in AI safety/policy? I'm esp excited re: control methods, scheming detection, and safety cases; feel free to DM if that overlaps your interests. Trevor Bingham / @22trevorbingham : Dario Amodei and Sam Altman and the other people helping to build AGI are all engaged in a very dangerous activity. It is exactly like your neighbor deciding to conduct some potentially lucrative chemistry experiments in their house in an effort to create a new class of very Steven Adler / @sjgadler : @CronopioMex Important to verify that the model isn't sandbagging in that case, but in principle maybe. One issue with sacrificing capabilities is that safety-defecting labs then gain an advantage by not doing this