 Andrej Karpathy
|
90a7bfecf1
|
move fused classifier
|
2024-06-03 22:35:20 +00:00 |
|
|
Andrej
|
dd4191fbd5
|
Merge pull request #539 from karpathy/move/attention
move attention
|
2024-06-03 15:23:27 -07:00 |
|
|
Andrej Karpathy
|
2186cd6fe7
|
move attention
|
2024-06-03 22:19:47 +00:00 |
|
|
Andrej
|
3427983615
|
Merge pull request #537 from karpathy/move/gelu
move gelu to llmc
|
2024-06-03 15:08:22 -07:00 |
|
|
Andrej Karpathy
|
d8d55e0a08
|
move gelu to llmc
|
2024-06-03 21:57:02 +00:00 |
|
|
Andrej Karpathy
|
49b9776e50
|
also move the launchers
|
2024-06-03 21:37:18 +00:00 |
|
|
Andrej Karpathy
|
93546aed01
|
move encoder, kernel 1/N
|
2024-06-03 21:21:28 +00:00 |
|
|
Andrej
|
3392671826
|
Merge pull request #535 from ngc92/speed-up-compile
moved device utilities into a separate file, and follow-up cleanup
|
2024-06-03 14:11:10 -07:00 |
|
|
Erik Schultheis
|
67502c7f8e
|
moved device utilities into a separate file, and follow-up cleanup
|
2024-06-03 23:58:47 +03:00 |
|
|
Andrej
|
8cc39a794b
|
Merge pull request #528 from gordicaleksa/refactor_evalloader
Refactor eval loader - remove 2 unnecessary malloc calls per iteration
|
2024-06-03 13:54:52 -07:00 |
|
|
Andrej
|
2a6a5b2c6c
|
Merge pull request #529 from gordicaleksa/improve_start_training_print_info
Improve start training print info
|
2024-06-03 13:53:32 -07:00 |
|
|
Andrej
|
0cd9f1ac21
|
Merge pull request #530 from gordicaleksa/minor_refactor
Minor refactor: remove hardcoded val, delete unused vars...
|
2024-06-03 13:52:42 -07:00 |
|
|
Andrej
|
d17893b754
|
Merge pull request #532 from ngc92/ln-buffers
more efficient use of memory buffers for LN recomputation
|
2024-06-03 13:51:22 -07:00 |
|
|
Andrej
|
ac931455a4
|
Merge pull request #533 from karpathy/feature/refactor_part2
refactor part 2
|
2024-06-03 13:44:59 -07:00 |
|
|
Andrej Karpathy
|
001ad78fe1
|
resolve deviceProp and cudaCheck dupes
|
2024-06-03 19:54:32 +00:00 |
|
|
Andrej Karpathy
|
020b15ad77
|
small more fixes
|
2024-06-03 19:32:32 +00:00 |
|
|
Andrej Karpathy
|
5d9403479c
|
some fixes, still not done
|
2024-06-03 18:50:41 +00:00 |
|
|
Andrej Karpathy
|
804d0693c8
|
refactor part 2, moving stuff into common files so that we can later nicely separate out all the kernels as well
|
2024-06-03 18:10:33 +00:00 |
|
|
Andrej
|
e40268457f
|
Merge pull request #526 from karpathy/try/move_cudnn_to_llmc
attempt to move cudnn attention to the llmc directory
|
2024-06-03 09:32:31 -07:00 |
|
|
Erik Schultheis
|
0d16b51626
|
more efficient use of memory buffers for LN recomputation
|
2024-06-03 18:32:51 +03:00 |
|
|
Andrej Karpathy
|
6434009298
|
attempt to remove PATH_SEP from Makefile
|
2024-06-03 15:30:45 +00:00 |
|
|
Aleksa Gordic
|
42ca620306
|
Minor refactor: remove hardcoded val, delete unused vars...
|
2024-06-03 16:24:22 +02:00 |
|
|
Aleksa Gordic
|
7413a1a315
|
Remove unnecessary load filename line
|
2024-06-03 12:33:33 +02:00 |
|
|
Aleksa Gordic
|
728762ceed
|
Improve start of training print info
|
2024-06-03 12:31:33 +02:00 |
|
|
Aleksa Gordic
|
d9e867678e
|
Remove file size - not used
|
2024-06-03 12:14:58 +02:00 |
|
|
Aleksa Gordic
|
e86bd62374
|
Refactor eval loader - remove 2 unnecessary malloc calls per iteration
|
2024-06-03 11:43:33 +02:00 |
|
|
Andrej Karpathy
|
41dc65a0ef
|
attempt to move cudnn attention to the llmc directory
|
2024-06-03 03:23:04 +00:00 |
|
|
Andrej Karpathy
|
08fc45b492
|
delete copy paste code for gpt2 model init
|
2024-06-03 02:08:29 +00:00 |
|
|
Andrej Karpathy
|
c827dd2f8e
|
Merge branch 'ngc92-mfu'
|
2024-06-03 01:41:31 +00:00 |
|
|
Andrej Karpathy
|
1ac30591f2
|
add new file llmc/mfu.h for MFU calculations for different GPUs
|
2024-06-03 01:41:18 +00:00 |
|
|
Andrej
|
80e0c3b8d9
|
Merge pull request #512 from gordicaleksa/refactor_encoder_bwd_kernel
Remove redundant CPU computation in encoder bwd
|
2024-06-02 16:43:10 -07:00 |
|
|
Andrej
|
cb5eff0578
|
Merge pull request #514 from ngc92/compile-fix
fix compilation with older nvcc
|
2024-06-02 16:42:31 -07:00 |
|
|
Andrej
|
ff98d8bda7
|
Merge pull request #515 from gordicaleksa/fix_adamw_kernel3_num_threads
Use local params for num blocks in adamw_kernel3
|
2024-06-02 16:41:59 -07:00 |
|
|
Andrej
|
1ce30fdc4a
|
Merge pull request #516 from gordicaleksa/update_random_doc
Add clarification on Box-Muller
|
2024-06-02 16:37:37 -07:00 |
|
|
Andrej
|
252584954e
|
Merge pull request #519 from gordicaleksa/mem_leak
Fix mem leak
|
2024-06-02 16:34:54 -07:00 |
|
|
Andrej Karpathy
|
1e893f4e11
|
Merge branch 'gordicaleksa-fix_zero_stage1'
|
2024-06-02 23:28:22 +00:00 |
|
|
Andrej Karpathy
|
4944ac5743
|
adjust comments
|
2024-06-02 23:28:13 +00:00 |
|
|
Aleksa Gordic
|
a8a5a444bc
|
Add ; - ah c my old friend :)
|
2024-06-02 23:04:47 +02:00 |
|
|
Aleksa Gordic
|
9a455edc25
|
Improve comments
|
2024-06-02 23:03:41 +02:00 |
|
|
Aleksa Gordic
|
b4e8bd7f66
|
Fix zero stage 1 norm computation - WIP
|
2024-06-02 22:45:26 +02:00 |
|
|
Aleksa Gordic
|
df0a826ef3
|
Close file
|
2024-06-02 21:37:52 +02:00 |
|
|
Aleksa Gordic
|
30d81cce15
|
Fix mem leak
|
2024-06-02 21:28:42 +02:00 |
|
|
Aleksa Gordic
|
79cbc03ccf
|
Use local params for num blocks
|
2024-06-02 15:06:06 +02:00 |
|
|
Aleksa Gordic
|
ae9c957273
|
Add clarification on Box-Muller
|
2024-06-02 15:04:45 +02:00 |
|
|
Erik Schultheis
|
a902323f36
|
fix compilation with older nvcc
|
2024-06-02 12:40:28 +03:00 |
|
|
Aleksa Gordic
|
290c00a362
|
Remove redundant CPU computation
|
2024-06-01 18:27:24 +02:00 |
|
|
Andrej
|
a26041af12
|
Merge pull request #495 from ChrisDryden/shared_memory
Removed unnecesary shared memory due to blockreduce using static defined shared memory
|
2024-06-01 08:12:21 -07:00 |
|
|
Andrej
|
9cf8c2f135
|
Merge pull request #509 from ngc92/inplace
Inplace gelu backward
|
2024-06-01 08:10:56 -07:00 |
|
|
Andrej
|
bd0f036ee5
|
Merge pull request #511 from gordicaleksa/gordicaleksa_fix_docs
Update compile cmd in the dev/cuda README
|
2024-06-01 08:09:31 -07:00 |
|
|
Aleksa Gordic
|
8eafd40615
|
Update compile cmd in the dev/cuda README
|
2024-06-01 16:46:32 +02:00 |
|