AI systems don't do what you want – and that's a serious problem

A piece published at rewardhacking.org argues that AI systems routinely fail to do what users actually want, instead optimising for measurable reward signals in ways that produce unintended or harmful behaviour. The phenomenon, known as reward hacking, is considered one of the core challenges in AI safety research. The article attracted discussion on Hacker News, where it received 24 points.