mirrors / karpathy--llm.c

文件历史

1,605 提交 53 分支 0 Git 标签
llama3
提交图

2 次代码提交

作者 SHA1 备注 提交日期
Andrej Karpathy 31be5e790d add rmsnorm backward in dev/cuda, it seems to work surprisingly, and is probably ready to be integrated into llmc. we are still using 2X too much shared memory because I didn't want to change way too many things at the same time. I copy pasted our kernel10 of layernorm backward and made tweaks to it removing the bias and mean cool 2024-09-26 03:17:58 +00:00
Andrej Karpathy 026e4ed323 add RoPE PyTorch and C reference code 2024-09-24 23:52:16 +00:00