 Andrej Karpathy
|
31be5e790d
|
add rmsnorm backward in dev/cuda, it seems to work surprisingly, and is probably ready to be integrated into llmc. we are still using 2X too much shared memory because I didn't want to change way too many things at the same time. I copy pasted our kernel10 of layernorm backward and made tweaks to it removing the bias and mean cool
|
2024-09-26 03:17:58 +00:00 |
|