 Andrej Karpathy
|
c20497cfc1
|
add comments pointing to the definition of the utils functions
|
2024-04-28 23:26:58 +00:00 |
|
|
Erik Schultheis
|
4a3c278f3a
|
moved checked helper functions into a separate file
|
2024-04-29 00:00:37 +03:00 |
|
|
Andrej Karpathy
|
b7972ff928
|
make padded vocab fixes in the .c code as well, i missed it in the previous PR, should satisfy the CI now
|
2024-04-28 19:33:21 +00:00 |
|
|
Andrej Karpathy
|
9c2209254b
|
also improve situation in the c code
|
2024-04-26 22:27:22 +00:00 |
|
|
Coenraad
|
2efac6670a
|
Make msvc compile by adding the ifdefs back in gelu_backward, fixes #200
|
2024-04-22 23:17:06 +02:00 |
|
|
Andrej
|
734f65f33e
|
Merge pull request #192 from rosslwheeler/const_fixes
const changes to help with Microsoft compiler OpenMP 5.2 support
|
2024-04-21 10:42:15 -07:00 |
|
|
Andrej Karpathy
|
adac21e97b
|
add a small comment as to what this is
|
2024-04-20 19:45:34 +00:00 |
|
|
Coenraad
|
f534e4bdfe
|
Fixes -Ofast optimizations breaking model by skipping them for gelu_backward
|
2024-04-20 21:33:00 +02:00 |
|
|
Ross Wheeler
|
e564222113
|
const changes to support OpenMP 5.2 MSVC work-around
const changes to support OpenMP MSVC work-around
|
2024-04-20 02:07:12 -07:00 |
|
|
Andrej Karpathy
|
5bdbc45806
|
only do 5 validation batches by default, we really don't need much precision here for this little script with current defaults
|
2024-04-20 00:45:13 +00:00 |
|
|
Andrej Karpathy
|
27ccf0b595
|
fix bugs, must use size_t instead of int, this was bad. and also init cpu_losses to NULL if we want to free it later
|
2024-04-15 22:20:44 +00:00 |
|
|
Coenraad
|
7064e36472
|
more original typos fixed
|
2024-04-15 20:22:33 +02:00 |
|
|
Andrej Karpathy
|
bcb5228ab9
|
also update the .c file in addition to .cu with recent changes, e.g. the tokenizer and the B*T sampling, and update the README to match
|
2024-04-15 02:32:26 +00:00 |
|
|
Andrej Karpathy
|
3652832073
|
update the .c code, not just .cu code. oh copy paste, starting to bite us haha. also fflush when printing tokens
|
2024-04-15 02:13:49 +00:00 |
|
|
Andrej Karpathy
|
dec54904b4
|
one more simple clarifying comment
|
2024-04-11 22:24:38 +00:00 |
|
|
Andrej Karpathy
|
d680fbad25
|
one more comment on attention
|
2024-04-11 21:49:14 +00:00 |
|
|
Andrej Karpathy
|
9710163a60
|
did one pass of additional comments, potentially helpful, as i think a lot of people may be looking at this for the first time
|
2024-04-11 21:45:46 +00:00 |
|
|
Andrej Karpathy
|
1751af1607
|
inline docs for sizes
|
2024-04-11 16:04:14 +00:00 |
|
|
Andrej Karpathy
|
f26cf00a61
|
make gelu constant be a #define, speeds up the kernel by 1% or so
|
2024-04-11 15:49:01 +00:00 |
|
|
Andrej Karpathy
|
80f52e5290
|
the full forward pass of GPT-2 in one file of pure CUDA
|
2024-04-10 18:15:55 +00:00 |
|
|
karpathy
|
3bcb9ba7d2
|
remove unused variables
|
2024-04-08 15:10:51 -07:00 |
|
|
Rickard Hallerbäck
|
b2228d0e83
|
Minor correction for openmp check
|
2024-04-08 22:33:47 +02:00 |
|
|
karpathy
|
e8e1628632
|
first commit of just the reference cpu fp32 gpt2 training
|
2024-04-08 12:41:21 -07:00 |
|