small-scale pretraining experiments of mine
-
BEE-spoke-data/smol_llama-101M-GQA
Text Generation • 0.1B • Updated • 2.56k • 30 -
BEE-spoke-data/smol_llama-220M-GQA
Text Generation • 0.2B • Updated • 3.26k • 13 -
BEE-spoke-data/smol_llama-220M-GQA-fineweb_edu
Text Generation • 0.2B • Updated • 55 • 1 -
BEE-spoke-data/smol_llama-81M-tied
Text Generation • 81.3M • Updated • 1.21k • 9