I rewrote
@karpathy
's llama2.c in dart, took me roughly 1.5 days, but man so much insight was gained.
had to go back and reference the original papers many times to understand what exactly was happening and force myself to understand, for example RoPE was just so intuitive to…