mirrors / karpathy--llm.c

项目文件夹

文件

T

文件历史

Andrej Karpathy a00a3a88ef fix bug with all_hostsname_hashes missing, re-arrange code a bit, make sure we only print on rank0

2024-04-26 18:11:09 +00:00

..

adamw.cu

fix the constness and the warning C4305: 'initializing': truncation from 'double' to 'float'

2024-04-19 11:04:58 -07:00

attention_backward.cu

one more version of attention backward for another few percent perf gains

2024-04-19 22:46:40 +03:00

attention_forward.cu

slow attention kernel that demonstrates a direct translation of the CPU code

2024-04-20 19:03:34 +03:00

classifier_fused.cu

add kernel3, which is like kernel 2 but no float4

2024-04-19 01:19:20 +00:00

common.h

added NaN-based makes for reference checks

2024-04-22 14:53:24 +03:00

crossentropy_forward.cu

nicer printing

2024-04-15 02:03:48 +03:00

crossentropy_softmax_backward.cu

fix the bug that yields cpu, gpu results mismatch

2024-04-17 21:04:45 -07:00

encoder_backward.cu

encoder backward add, we want to get rid of atomics, hmm

2024-04-23 19:06:15 +00:00

encoder_forward.cu

rename positional_forward to encoder_forward as it should have been

2024-04-23 18:19:20 +00:00

gelu_forward.cu

nicer printing

2024-04-15 02:03:48 +03:00

layernorm_backward.cu

incorporate layernorm backward kernel 2, we are now down to 73.1ms/iter on average

2024-04-22 18:44:43 +00:00

layernorm_forward.cu

fix kernel 5 of layernorm, bug from previous copy paste with loop bounds

2024-04-20 00:51:06 +00:00

matmul_backward_bias.cu

nice new kernel for bias reduce, we are down by ~1ms/iter, to 76.13ms/iter on average

2024-04-22 17:51:06 +00:00

matmul_backward.cu

fix the 0.0f initialization for double

2024-04-18 19:35:34 -07:00

matmul_forward.cu

fix indexing in matmul_forward_kernel1

2024-04-20 15:38:12 +08:00

nccl_all_reduce.cu

fix bug with all_hostsname_hashes missing, re-arrange code a bit, make sure we only print on rank0

2024-04-26 18:11:09 +00:00

README.md

might as well push the few kernels that i feel ok about so far

2024-04-09 14:34:13 +00:00

residual_forward.cu

nicer printing

2024-04-15 02:03:48 +03:00

softmax_forward.cu

fix the 0.0f initialization for double

2024-04-18 19:33:30 -07:00

trimat_forward.cu

rename to _forward for the kernel, and small fixes to docs, and avoid auto

2024-04-22 16:32:03 +00:00

README.md

dev/cuda

This directory is scratch space for developing various versions of the needed CUDA kernels. Each file develops a kernel, see the top of each file for instructions on how to compile and run each one.