2025-09-14
Amazing post on why LLMs are non-deterministic even with a temperature of zero. TLDR; Essentially it boils down to different inference batch sizes, as some kernels are not batch-size invariant. Meaning that their output is influenced by the number of requests🤯
TechCrunch
Mira Murati's TML launches a research blog called Connectionism, and shares its work on resolving nondeterminism and achieving reproducible results from LLMs
There's been great interest in what Mira Murati's Thinking Machines Lab is building with its $2 billion in seed funding …