 Andrej Karpathy
|
755458d0e1
|
fix tokenizer omg
|
2024-08-13 17:24:15 +00:00 |
|
|
Andrej Karpathy
|
16635d41a2
|
attempt to fix PR
|
2024-08-13 02:13:14 +00:00 |
|
|
Aleksa Gordic
|
68624cc6ac
|
Refactor tokenize, delegate instead
|
2024-08-12 20:26:21 +02:00 |
|
|
Aleksa Gordic
|
e8739c89a0
|
Refactor magic/version header dict
|
2024-08-12 20:17:53 +02:00 |
|
|
Aleksa Gordic
|
eb1b359dae
|
Replace fire with argparse
|
2024-08-10 22:45:46 +02:00 |
|
|
Aleksa Gordic
|
50765638c0
|
Add fineweb llama 3 support
|
2024-08-10 22:35:46 +02:00 |
|
|
Aleksa Gordic
|
f8c1ff590f
|
Use fire module, update readme
|
2024-08-10 22:29:19 +02:00 |
|
|
Aleksa Gordic
|
15b76bf0fb
|
Add tiny shakespeare
|
2024-08-10 22:23:33 +02:00 |
|
|
Aleksa Gordic
|
71419cbd61
|
Update header description
|
2024-08-10 21:47:09 +02:00 |
|
|
Aleksa Gordic
|
d3151c4ace
|
Add llama 3 tokenization and data loader
|
2024-08-10 21:25:57 +02:00 |
|
|
Andrej Karpathy
|
1566d6b9f6
|
revert makefile change
|
2024-07-30 20:07:54 +00:00 |
|
|
mspronesti
|
574ef9b87d
|
chore(permute): use common.h utils for validation and benchmarking
|
2024-07-29 00:19:31 +01:00 |
|
|
mspronesti
|
a49da7d917
|
make: generalize TARGETS definition
|
2024-07-29 00:19:03 +01:00 |
|
|
Andrej
|
b4623bc5e7
|
Merge pull request #712 from indianspeedster/dev-cuda
Added permute kernel in dev/cuda
|
2024-07-28 06:08:20 -10:00 |
|
|
mspronesti
|
c6e74ea322
|
fix(dev/cuda): memory leaks
|
2024-07-26 00:33:51 +01:00 |
|
|
indianspeedster
|
a80bce65e5
|
Added compilation steps in makefile for permute.cu
|
2024-07-25 18:02:06 -04:00 |
|
|
indianspeedster
|
8952c524af
|
added kernel to permute in dev/cuda
|
2024-07-25 17:35:29 -04:00 |
|
|
Ross Wheeler
|
9dc920a66f
|
Move WIN32 code into unistd.h
|
2024-07-20 12:05:20 -07:00 |
|
|
Ross Wheeler
|
56621bba0e
|
Adding CI check for exceeding loss tolerance
LLMC-26
|
2024-07-13 01:14:20 -07:00 |
|
|
Andrej Karpathy
|
48fee049cb
|
add edu fineweb sh download script, change paths to my HF repo, add hellaswag to starter pack
|
2024-07-01 21:41:15 +00:00 |
|
|
Andrej Karpathy
|
ea36d04e39
|
small cosmetic fixes to fineweb data file
|
2024-06-30 15:52:51 +00:00 |
|
|
Andrej
|
2daaf6ab83
|
Merge pull request #629 from rosslwheeler/feature/tests
CI Dataloader test and ptx/sass file generator
|
2024-06-29 15:17:31 -07:00 |
|
|
Andrej Karpathy
|
ff7a90bd8d
|
Merge branch 'add_edu_fineweb_support' of https://github.com/eliebak/llm.c into eliebak-add_edu_fineweb_support
|
2024-06-28 19:35:44 +00:00 |
|
|
Ross Wheeler
|
d6d08be3c0
|
Addressed review comments
Explicit precision set for tests
|
2024-06-27 00:58:19 -07:00 |
|
|
rosslwheeler
|
267efc9a0e
|
Merge branch 'karpathy:master' into feature/tests
|
2024-06-26 22:38:21 -07:00 |
|
|
Andrej Karpathy
|
e12d090483
|
resolve merge conflicts
|
2024-06-26 04:05:46 +00:00 |
|
|
Ross Wheeler
|
2e6fd9b150
|
Add unistd.h for Windows cuDNN build
Small fix - move WIN32 change into dev.unistd.h
Add USE_CUDNN=1 build for Windows for CI
|
2024-06-25 10:36:59 -07:00 |
|
|
rosslwheeler
|
2e32e25e37
|
Merge branch 'karpathy:master' into feature/tests
|
2024-06-24 23:05:55 -07:00 |
|
|
Andrej Karpathy
|
569f9f16f9
|
add outlier detector, test for it, and start tracking z score of the loss, for now only print it
|
2024-06-25 01:37:31 +00:00 |
|
|
Andrej Karpathy
|
2543b62b88
|
Merge branch 'streams-io' of https://github.com/ngc92/llm.c into ngc92-streams-io
|
2024-06-23 00:48:05 +00:00 |
|
|
Ross Wheeler
|
5b95bff118
|
Dataloader test and ptx/sass file generator
Setup new test infrastructure
|
2024-06-21 17:23:03 -07:00 |
|
|
Andrej Karpathy
|
2677d82dcb
|
add shell script that downloads the .bin files of FineWeb100B directly
|
2024-06-20 22:39:09 +00:00 |
|
|
Andrej Karpathy
|
0bc8753bc0
|
update vislog to be a bit more fancy, has GPT-2/3 perf for hellaswag
|
2024-06-19 01:51:09 +00:00 |
|
|
Andrej Karpathy
|
b29478e20b
|
add starter pack .sh script for faster quickstart
|
2024-06-19 01:44:36 +00:00 |
|
|
Andrej
|
4577d58cba
|
Merge pull request #611 from rosslwheeler/feature/dataloader_test
Bug fixes and warning fixes for test_dataloader.c
|
2024-06-18 18:08:05 -07:00 |
|
|
Andrej
|
88f0543c38
|
Merge pull request #594 from karpathy/feature/hf_eval
add scripts to export to HF and run Eleuther evals
|
2024-06-18 18:04:14 -07:00 |
|
|
Andrej Karpathy
|
1a66151a80
|
small fixes and readme
|
2024-06-19 01:03:36 +00:00 |
|
|
Ross Wheeler
|
24923829c4
|
Adding suggested changes
Ternary change to test_dataloader and header cleanup
|
2024-06-18 10:38:33 -07:00 |
|
|
Erik Schultheis
|
3ebd5abe82
|
fixup tests
|
2024-06-18 19:58:03 +03:00 |
|
|
Erik Schultheis
|
188e7274e6
|
add unit test to CI
|
2024-06-18 13:23:56 +03:00 |
|
|
Ross Wheeler
|
30095bcb0b
|
Fixes on accessing past memory buffer
|
2024-06-17 21:37:35 -07:00 |
|
|
rosslwheeler
|
4e0221de0d
|
Merge branch 'karpathy:master' into feature/dataloader_test
|
2024-06-17 18:18:23 -07:00 |
|
|
Ross Wheeler
|
37023dd9a1
|
Cleanup for data loader
|
2024-06-17 18:15:29 -07:00 |
|
|
Andrej
|
f104009bc7
|
Merge pull request #600 from gordicaleksa/add_faster_layernorm_kernel
Use faster kernel for LayerNorm forward
|
2024-06-17 14:19:22 -07:00 |
|
|
Andrej Karpathy
|
14d9cd671e
|
resolve merge conflict in staate file between use_master_weights and DataLoader should_shuffle
|
2024-06-17 19:16:42 +00:00 |
|
|
Andrej Karpathy
|
de53e71729
|
small touchups
|
2024-06-17 18:55:16 +00:00 |
|
|
Aleksa Gordic
|
9b5ec8f719
|
Set v memory to random floats >=0
|
2024-06-17 17:44:00 +02:00 |
|
|
Aleksa Gordic
|
a0a8795be1
|
Fix guarding logic, refactor
|
2024-06-17 14:12:05 +02:00 |
|
|
Aleksa Gordic
|
c6040eef5f
|
Even better kernel - modification of what we use for fused residual fwd
|
2024-06-17 09:53:58 +02:00 |
|
|
Andrej
|
13c893ffec
|
Merge pull request #561 from lancerts/fix-warning
Fix the compiler warnings and errors
|
2024-06-16 09:25:51 -07:00 |
|