d6793645f5
replace cublaslt with custom matmul
2024-06-12 09:42:19 +03:00
7f3f3acf16
shared memory matmul
2024-06-11 21:03:43 +03:00
071c977c79
vectorized memory access
2024-06-11 00:46:51 +03:00
bb89efdcb1
register reuse
2024-06-11 00:41:08 +03:00
06980b6b2f
mixed precision utilities for dev/cuda
2024-05-01 22:35:55 +03:00
d48b0d2b2b
fix indexing in matmul_forward_kernel1
2024-04-20 15:38:12 +08:00
99b42c1a94
nicer printing
2024-04-15 02:03:48 +03:00
c5ee371564
make the tests own their cublas cublaslt handles and settings, instead of putting that stuff in common
2024-04-13 17:56:18 +00:00
59bada94d9
improved testing
2024-04-13 17:06:52 +03:00
7e7460cfbd
const'ness fixes
2024-04-13 13:38:11 +03:00
7e8e038de8
extracted common testing functions into their own header
2024-04-13 13:00:05 +03:00
8822e7803e
use cublaslt, which fuses bias, and also use tf32 when your GPU supports it ( #98 )
...
Fusing the bias into matmul and moving to tf32 speed up the code *a lot*, we're now down to 59ms/iter compared to 25ms/iter for pytorch compiled tf32. i.e. still off by 2.36X.
2024-04-12 15:32:07 -07:00
68d2ce0601
fix error in small typos in matmul_forward.cu
2024-04-10 13:35:20 -07:00
9f5069d3b2
Merge pull request #38 from lancerts/fix_matmul
...
fix the consistency of the transpose notation in matmul_foward.cu
2024-04-10 12:15:53 -07:00
80f52e5290
the full forward pass of GPT-2 in one file of pure CUDA
2024-04-10 18:15:55 +00:00
fbd679adec
fix the consistency of the transpose notation
2024-04-09 15:19:55 -07:00
41d5f56278
might as well push the few kernels that i feel ok about so far
2024-04-09 14:34:13 +00:00
Erik Schultheis