-
10 October 2026 · 5 min read · Part 2 of 2
A Billion-Parameter Music Model: What 23× More Parameters Bought
I scaled a text-to-music transformer from 45 million to 1 billion parameters, trained it on two GPUs, and measured what the extra parameters actually bought.
-
7 October 2026 · 9 min read · Part 1 of 2
Building a Small Transformer That Makes Music
Building a 45M-parameter text-to-music model by hand in PyTorch — EnCodec tokens, cross-attention, home-made pipeline parallelism on two GPUs, and the silent PCIe bug that nearly stopped it. With real training curves and generated audio to judge.