mirrors / karpathy--llm.c

项目文件夹

karpathy--llm.c
文件
refactor/code-to-zerocuh
T
文件历史
Andrej Karpathy 882a214ad4 Merge branch 'move-to-zero' of https://github.com/ngc92/llm.c into ngc92-move-to-zero
2024-07-15 18:59:22 +00:00
..
adamw.cuh
Make sure each parameter was rounded with a unique seed
2024-06-15 12:02:39 +00:00
attention.cuh
Refactor matmul to only use cuBLASLt and optionally use fused GELU
2024-06-30 20:45:37 +01:00
cublas_common.h
Refactor matmul to only use cuBLASLt and optionally use fused GELU
2024-06-30 20:45:37 +01:00
cuda_common.h
make windows happy
2024-06-25 21:28:52 +03:00
cuda_utils.cuh
moved loss calculation to backward
2024-06-25 21:28:52 +03:00
cudnn_att.cpp
Make cuDNN deterministic for Flash Attention backward (requires frontend >=1.5)
2024-06-27 13:33:02 +00:00
cudnn_att.h
move all kernels into a main_stream in preparation for future parallelism
2024-06-04 22:35:12 +03:00
dataloader.h
Dataloader test and ptx/sass file generator
2024-06-21 17:23:03 -07:00
encoder.cuh
Add a comment for why pass a unique seed per param
2024-06-18 17:01:03 +02:00
fused_classifier.cuh
switch losses to fp32
2024-06-26 20:48:08 +03:00
gelu.cuh
Fix guarding logic, refactor
2024-06-17 14:12:05 +02:00
global_norm.cuh
update norm computation: multi-process reduce on GPU
2024-06-25 21:28:52 +03:00
layernorm.cuh
added cudaCheck where missing for proper error checking.
2024-07-13 13:20:39 -04:00
logger.h
Add learning rate and grad norm logging
2024-06-07 17:34:42 +02:00
matmul.cuh
fix default for test_gpt2cu
2024-06-30 20:45:37 +01:00
mfu.h
Add H100 PCIe
2024-06-14 09:23:55 +02:00
outlier_detector.h
oops i was only using window size 16 for testing let's make it 128 or something
2024-06-25 01:56:04 +00:00
rand.h
Dataloader test and ptx/sass file generator
2024-06-21 17:23:03 -07:00
sampler.h
refactor part 2, moving stuff into common files so that we can later nicely separate out all the kernels as well
2024-06-03 18:10:33 +00:00
schedulers.h
small touchups and add the wsd schedule
2024-06-21 00:49:01 +00:00
tokenizer.h
made implicit includes of standard headers explicit
2024-06-04 13:22:16 +03:00
utils.h
use int instead of bool
2024-07-15 18:01:01 +00:00
zero.cuh
move cpu float reduction to zero
2024-07-12 11:12:58 +02:00