 lancer
|
5c90845f7a
|
update the -lcublas -lcublasLt flag in the comment
|
2024-05-09 22:04:02 -07:00 |
|
|
ngc92
|
99e765de8d
|
moved bf16 boilerplate to common.h
|
2024-05-09 11:19:24 +02:00 |
|
|
Erik Schultheis
|
91bc72d0c0
|
fixup
|
2024-05-01 22:40:26 +03:00 |
|
|
Erik Schultheis
|
06980b6b2f
|
mixed precision utilities for dev/cuda
|
2024-05-01 22:35:55 +03:00 |
|
|
Andrej Karpathy
|
769c911ab6
|
small tweaks
|
2024-04-30 23:48:06 +00:00 |
|
|
Erik Schultheis
|
603d862830
|
fix up bandwidth calculation
|
2024-04-30 23:52:08 +03:00 |
|
|
Erik Schultheis
|
c3515cfe47
|
fixed potential error and generalized gelu forward
|
2024-04-30 23:45:34 +03:00 |
|
|
Andrej Karpathy
|
9141e04693
|
fixes to the PR, careful with float/floatX etc
|
2024-04-30 19:35:05 +00:00 |
|
|
Christopher Dryden
|
568615fef1
|
Added packing for gelu forwards kernel
|
2024-04-30 03:21:42 +00:00 |
|
|
Andrej Karpathy
|
5e12c66f5a
|
move packed128 to common
|
2024-04-29 22:55:28 +00:00 |
|
|
Andrej Karpathy
|
f43592954f
|
reset back to precious hyperparameters
|
2024-04-29 22:44:25 +00:00 |
|
|
Andrej Karpathy
|
ee125cffc4
|
make it imo clearer that the grid size is one quarter as large as it would otherwise be. because each thread now does 4X work
|
2024-04-29 22:40:26 +00:00 |
|
|
Andrej Karpathy
|
083b210dd4
|
small changes
|
2024-04-29 22:38:03 +00:00 |
|
|
Andrej Karpathy
|
d9506a7a0a
|
minor changes to packed128
|
2024-04-29 22:31:19 +00:00 |
|
|
Erik Schultheis
|
4872c57164
|
comments
|
2024-04-29 22:46:33 +03:00 |
|
|
Erik Schultheis
|
9f07a176e0
|
simplify to only support cs
|
2024-04-29 22:43:52 +03:00 |
|
|
Erik Schultheis
|
896b644cb9
|
slight simplification
|
2024-04-29 22:19:12 +03:00 |
|
|
Erik Schultheis
|
45252d434c
|
yet another gelu
|
2024-04-29 21:53:08 +03:00 |
|
|
Erik Schultheis
|
99b42c1a94
|
nicer printing
|
2024-04-15 02:03:48 +03:00 |
|
|
Erik Schultheis
|
fe1d8cfb47
|
fix timing output
|
2024-04-15 02:03:42 +03:00 |
|
|
Andrej Karpathy
|
82623518a4
|
fix comment block of gelu forward kernel
|
2024-04-13 17:48:53 +00:00 |
|
|
Erik Schultheis
|
59bada94d9
|
improved testing
|
2024-04-13 17:06:52 +03:00 |
|
|
Erik Schultheis
|
7e7460cfbd
|
const'ness fixes
|
2024-04-13 13:38:11 +03:00 |
|
|
Erik Schultheis
|
7e8e038de8
|
extracted common testing functions into their own header
|
2024-04-13 13:00:05 +03:00 |
|
|
Andrej Karpathy
|
f26cf00a61
|
make gelu constant be a #define, speeds up the kernel by 1% or so
|
2024-04-11 15:49:01 +00:00 |
|
|
lancer
|
3a4d783633
|
amend
|
2024-04-09 09:59:03 -07:00 |
|
|
lancer
|
73c92912b0
|
fix error: identifier M_PI is undefined
|
2024-04-09 09:51:28 -07:00 |
|
|
Andrej Karpathy
|
41d5f56278
|
might as well push the few kernels that i feel ok about so far
|
2024-04-09 14:34:13 +00:00 |
|