2025-02-22
turns out the AI CUDA Engineer achieved 100x speedup by... hacking the eval script [image]
TechCrunch
Sakana AI walks back claims that its new AI CUDA Engineer can speed up AI training by up to 100x, after complaints about worse-than-average training performance
Kyle Wiggers / TechCrunch :
sakana have updated their leaderboard to address the memory-reuse exploit https://sakana.ai/... there is only one >100x speedup left, on task 23_Conv3d_GroupNorm_Mean in this task, the AI CUDA Engineer forgot the entire conv part and the eval script didn't catch it [image]
TechCrunch
Sakana AI walks back claims that its new AI CUDA Engineer can speed up AI training by up to 100x, after complaints about worse-than-average training performance
Kyle Wiggers / TechCrunch :
2025-02-20
turns out the AI CUDA Engineer achieved 100x speedup by... hacking the eval script [image]
Nikkei Asia
Tokyo-based Sakana AI details its AI CUDA Engineer, which it says can speed up AI training and inference by 10x to 100x by “breeding” efficient instructions
TOKYO — Tokyo-based startup Sakana AI says it has developed a system capable of accelerating artificial intelligence development …