 Andrej Karpathy
|
888c0d4be3
|
also update the other files
|
2024-05-27 21:15:09 +00:00 |
|
|
Andrej Karpathy
|
f145c1b101
|
fix the other files due to dataloader change
|
2024-05-26 17:01:32 +00:00 |
|
|
Andrej Karpathy
|
edb0df967a
|
continued changes for sharded dataloader
|
2024-05-22 11:59:09 +00:00 |
|
|
Andrej Karpathy
|
666145e7b8
|
also fix the .c file and fp32 file
|
2024-05-21 17:30:56 +00:00 |
|
|
Andrej Karpathy
|
722e5b2fe5
|
refactor how we treat datasets, because we're about to have more of them and we don't want them to clutter up root dir etc. this is only step 1, i'm about to refactor a bunch of the dataloading, how the .bin files work and are loaded, how the DataLoader works, etc. This is all needed to support good evals and training at scale
|
2024-05-20 22:43:40 +00:00 |
|
|
lancer
|
03b7323d1e
|
minor fix of the include
|
2024-05-03 17:38:39 -07:00 |
|
|
Andrej Karpathy
|
f02ba4b436
|
import updated tokenizer properly
|
2024-04-29 23:02:26 +00:00 |
|
|
Andrej Karpathy
|
c20497cfc1
|
add comments pointing to the definition of the utils functions
|
2024-04-28 23:26:58 +00:00 |
|
|
Andrej Karpathy
|
b1c80e9d7b
|
Merge branch 'split-file' of https://github.com/ngc92/llm.c into ngc92-split-file
|
2024-04-28 23:23:00 +00:00 |
|
|
Erik Schultheis
|
4a3c278f3a
|
moved checked helper functions into a separate file
|
2024-04-29 00:00:37 +03:00 |
|
|
Erik Schultheis
|
4c295c7e97
|
and even more cleanup
|
2024-04-28 23:19:36 +03:00 |
|
|
Erik Schultheis
|
ca48791522
|
as promised, cleanup enabled by padding :)
|
2024-04-28 23:19:36 +03:00 |
|
|
Andrej Karpathy
|
327eef3f22
|
incorporate faster encoder_forward kernel to fp32 CUDA version
|
2024-04-28 20:10:33 +00:00 |
|
|
Andrej Karpathy
|
835060e18d
|
padded vocab change. touched a lot of code. very stressful and error prone, but i think it is done. had to bump versions on all .bin files, invalidating the previous files. re-run the python training script to re-export the new version files. let's not do too much of things like this in the future lol. actually, fun fact i had a chance to do the padded vocab really really early in the history of llm.c development, and chose not do it, thinking i'll just do it later. i should have done it. such is life, you make mistakes, you accumulate scar tissue, and you learn, and you become better, faster, stronger. this is the mindset one must have to lead a happy and fulfilling life. it's not important that you are perfect at any point in time, it's only important that you keep improving, every day.
|
2024-04-28 18:47:03 +00:00 |
|
|
Andrej Karpathy
|
00627070aa
|
fix a really bad bug in how i was checking the gradients, where i loaded them in the old order, so yeah...
|
2024-04-27 23:17:22 +00:00 |
|
|
Andrej Karpathy
|
e5fd817de8
|
revert fp32 version to no multigpu support
|
2024-04-26 18:12:44 +00:00 |
|
|
Andrej Karpathy
|
a00a3a88ef
|
fix bug with all_hostsname_hashes missing, re-arrange code a bit, make sure we only print on rank0
|
2024-04-26 18:11:09 +00:00 |
|
|
Petr Zhizhin
|
d49e8301eb
|
[Multi-GPU] llm.c now runs on multiple GPUs with NCCL
|
2024-04-24 22:11:00 +00:00 |
|
|
Andrej Karpathy
|
095d27662b
|
checkpoint the fp32 CUDA implementation to separate file. our mainline iteration will now continue in the new (mixed precision) file
|
2024-04-23 17:21:44 +00:00 |
|