 Ross Wheeler
|
5b95bff118
|
Dataloader test and ptx/sass file generator
Setup new test infrastructure
|
2024-06-21 17:23:03 -07:00 |
|
|
Aleksa Gordic
|
c6040eef5f
|
Even better kernel - modification of what we use for fused residual fwd
|
2024-06-17 09:53:58 +02:00 |
|
|
Aleksa Gordic
|
d5601d16da
|
Fix stuff in layernorm forward
|
2024-06-16 09:34:01 +02:00 |
|
|
lancer
|
5c90845f7a
|
update the -lcublas -lcublasLt flag in the comment
|
2024-05-09 22:04:02 -07:00 |
|
|
Andrej Karpathy
|
91aab62aad
|
fix kernel 5 of layernorm, bug from previous copy paste with loop bounds
|
2024-04-20 00:51:06 +00:00 |
|
|
Andrej Karpathy
|
49d41ae296
|
add one more kernel, allocating a block per row. bad idea if C is too low, as we have it right now
|
2024-04-19 23:24:51 +00:00 |
|
|
Andrej Karpathy
|
cb791c4ef5
|
new kernel that does a single pass over x on load, using a more clever variance formula. only very slightly faster on my A100 sadly
|
2024-04-19 22:55:30 +00:00 |
|
|
Erik Schultheis
|
99b42c1a94
|
nicer printing
|
2024-04-15 02:03:48 +03:00 |
|
|
Erik Schultheis
|
59bada94d9
|
improved testing
|
2024-04-13 17:06:52 +03:00 |
|
|
Erik Schultheis
|
7e7460cfbd
|
const'ness fixes
|
2024-04-13 13:38:11 +03:00 |
|
|
Erik Schultheis
|
7e8e038de8
|
extracted common testing functions into their own header
|
2024-04-13 13:00:05 +03:00 |
|
|
Andrej Karpathy
|
00b9fda4a1
|
ceildiv in layernorm
|
2024-04-11 18:21:34 +00:00 |
|
|
Andrej Karpathy
|
089cd5f1dd
|
make sure we also test correctness of mean and rstd and do it all block sizes
|
2024-04-11 18:12:26 +00:00 |
|
|
Andrej Karpathy
|
f28613e0b9
|
small edits to comments and variables
|
2024-04-11 18:09:54 +00:00 |
|
|
Erik Schultheis
|
4e91362fa2
|
optionally generate mean and rstd
|
2024-04-11 20:48:14 +03:00 |
|
|
Erik Schultheis
|
afddb1af5b
|
fixed block size and improved load op hint
|
2024-04-11 03:47:38 +03:00 |
|
|
Erik Schultheis
|
466723bb06
|
fully fused layer-norm kernel
|
2024-04-11 02:01:41 +03:00 |
|
|
Andrej Karpathy
|
41d5f56278
|
might as well push the few kernels that i feel ok about so far
|
2024-04-09 14:34:13 +00:00 |
|