Machine learning+Reinforcement learning → AI safety

Amodei, Olah and co-authors from Google, Stanford, Berkeley and OpenAI publish “Concrete Problems in AI Safety”: side effects, reward hacking, scalable oversight, safe exploration and distributional shift — arxiv.org ↗

Science

⚑ Backed by

• Cybersecurity / Dario Amodei: “I believe that if we take a list of things that Anthropic did in the past year that were pro-safety and commercially costly to us, they far outweigh the list you’ve made … we first had Mythos-class models in February but didn’t make them generally available until June (when we had stronger cyber safeguards in place). … It would have been much more economically rational to simply release the model to everyone. We likely sacrificed a significant chunk of a several-month model lead.”

¶ car56caj3