mirrors / karpathy--llm.c

项目文件夹

karpathy--llm.c
文件
llama3
T
文件历史
Andrej Karpathy 7d945e994c reshuffle repkv a bit, i wrote it from scratch. the kernel is still correct. repkv backward looks correct. rope backward is trivial so i don't see how it's not correct, and i also checked it. basically i'm really confused right now
2024-09-27 19:25:09 +00:00
..
cbridge
add rmsnorm backward in dev/cuda, it seems to work surprisingly, and is probably ready to be integrated into llmc. we are still using 2X too much shared memory because I didn't want to change way too many things at the same time. I copy pasted our kernel10 of layernorm backward and made tweaks to it removing the bias and mean cool
2024-09-26 03:17:58 +00:00
cpu
Setting up dev/CPU area with the first matmul_forward.c
2024-05-14 18:22:39 -07:00
cuda
reshuffle repkv a bit, i wrote it from scratch. the kernel is still correct. repkv backward looks correct. rope backward is trivial so i don't see how it's not correct, and i also checked it. basically i'm really confused right now
2024-09-27 19:25:09 +00:00
data
fix tokenizer omg
2024-08-13 17:24:15 +00:00
eval
small fixes and readme
2024-06-19 01:03:36 +00:00
test
Addressed review comments
2024-06-27 00:58:19 -07:00
download_starter_pack.sh
add edu fineweb sh download script, change paths to my HF repo, add hellaswag to starter pack
2024-07-01 21:41:15 +00:00
loss_checker_ci.py
Adding CI check for exceeding loss tolerance
2024-07-13 01:14:20 -07:00
unistd.h
Move WIN32 code into unistd.h
2024-07-20 12:05:20 -07:00
vislog.ipynb
update vislog to be a bit more fancy, has GPT-2/3 perf for hellaswag
2024-06-19 01:51:09 +00:00