mirrors / karpathy--llm.c

项目文件夹

文件

T

文件历史

Andrej Karpathy 44cd50cb25 fix all the handles and things

2024-04-12 22:18:18 +00:00

..

attention_forward.cu

sorry there should be no mallocs inside these functions, have to pass in buffer memory from outside. getting tired

2024-04-10 00:55:14 +00:00

crossentropy_forward.cu

ok crossentropy is fairly ez, just have to be careful with logs of negative numbers in debug tests. so this should be all the layers now

2024-04-10 01:13:20 +00:00

gelu_forward.cu

make gelu constant be a #define, speeds up the kernel by 1% or so

2024-04-11 15:49:01 +00:00

layernorm_forward.cu

ceildiv in layernorm

2024-04-11 18:21:34 +00:00

matmul_forward.cu

fix all the handles and things

2024-04-12 22:18:18 +00:00

positional_forward.cu

might as well push the few kernels that i feel ok about so far

2024-04-09 14:34:13 +00:00

README.md

might as well push the few kernels that i feel ok about so far

2024-04-09 14:34:13 +00:00

residual_forward.cu

might as well push the few kernels that i feel ok about so far

2024-04-09 14:34:13 +00:00

softmax_forward.cu

intra-warp reductions and inter-warp shared memory reductions for B,T,C parallelism of softmax kernel nice

2024-04-09 20:08:11 +00:00

README.md

dev/cuda

This directory is scratch space for developing various versions of the needed CUDA kernels. Each file develops a kernel, see the top of each file for instructions on how to compile and run each one.