aedf3a5cdd
changes to readme for multigpu. i think this is ready now for v0 merge to master nice
2024-04-26 18:20:15 +00:00
8389fbaa0c
Merge branch 'multi_gpu_support' of https://github.com/PeterZhizhin/llm.c into PeterZhizhin-multi_gpu_support
2024-04-26 15:55:15 +00:00
d49e8301eb
[Multi-GPU] llm.c now runs on multiple GPUs with NCCL
2024-04-24 22:11:00 +00:00
0ee2d4a965
Add llm.go in description
2024-04-24 17:48:49 -04:00
e44ed15639
small updates on intent for fp32 and non-fp32 CUDA versions
2024-04-24 19:44:12 +00:00
b2c779d40e
Merge pull request #231 from Saimirbaci/zig-port-ref
...
Added ref to the zig port
2024-04-24 10:00:08 -07:00
c5f1db6020
fix important typo on precision
2024-04-23 17:43:19 +00:00
ed6387a695
updates to readme, and introduce the test fp32 cuda file too
2024-04-23 17:41:42 +00:00
7f279cd6cb
Adding Zig port ref
2024-04-23 13:37:53 -04:00
451c1a4c99
update readme to point to SoU
2024-04-22 19:18:03 +00:00
38857ff1d0
Added llm.metal to notable forks
...
This is a fork of llm.c designed to take full advantage of Apple Silicon via the Metal framework. Just got the forward pass working and will start on the backward pass. I wrote a minimal C wrapper API over the Objective-C Metal API that was needed to set up and run compute shaders. The rest of the code is pure-C and Metal Shading Language. I'm trying to mirror the original repo as closely as possible both in code and project philosophy.
2024-04-22 09:01:21 -07:00
43ffd2af8a
add Rust port to notable forks
2024-04-21 19:23:50 -02:30
7fa9f7b140
document one way to do experiments / sweeps as just an example
2024-04-20 19:39:12 +00:00
7df2d4d92b
also requirements for quick start
2024-04-20 00:59:32 +00:00
38fd42d3e3
also add a quick start CPU section
2024-04-20 00:41:13 +00:00
da4c5c79ca
very quick start section add
2024-04-20 00:35:35 +00:00
c6bd590b47
typo
2024-04-19 14:17:13 -07:00
6cd1354b3e
C# port ref
2024-04-19 13:59:36 -07:00
1f08134fb9
Added link to Mojo port in Notable Forks section
2024-04-20 01:44:50 +05:30
810b0dd90f
adjust header
2024-04-19 17:19:26 +00:00
f7fb3e67d5
update readme
2024-04-19 17:16:48 +00:00
70170006ca
adjust readme with latest speed
2024-04-18 22:50:52 +00:00
eef92c3fbe
make comment on comparison to PyTorch runtime with kernels as they are right now
2024-04-18 04:53:09 +00:00
33a4985d19
change readme again to keep it up to date
2024-04-18 04:39:59 +00:00
79586880b7
adjust readme now that the full training loop works in cud
2024-04-17 17:15:59 +00:00
66c30fb55f
-fno-finite-math-only for almost 2x speed up
2024-04-16 12:49:54 +02:00
a44b0ce0f0
Merge pull request #142 from dagelf/master
...
Update Makefile with -march=native for 25%+ CPU speedup
2024-04-15 12:01:25 -07:00
480b638606
few more words on what i want the repo to be atm
2024-04-15 18:55:50 +00:00
ecde56c1c8
Add -march=native to README
2024-04-15 20:32:27 +02:00
cc5931bfdb
add clarification on notable forks to readme
2024-04-15 03:43:43 +00:00
82761ff29b
confirm that linux and macos disagree, putting in the macos version as before
2024-04-15 02:38:23 +00:00
bcb5228ab9
also update the .c file in addition to .cu with recent changes, e.g. the tokenizer and the B*T sampling, and update the README to match
2024-04-15 02:32:26 +00:00
534b93a732
erase the old kernel to save LOC and do some renaming and adjust dev/cuda code to add the new largeC kernel as kernel 7
2024-04-13 22:02:31 +00:00
c70fbd059b
refine comment on time
2024-04-13 21:09:25 +00:00
ca242a3d0b
get rid of spurious handle variable creation, remnant from before, and update README
2024-04-13 21:04:49 +00:00
6b49ed1c0b
incorporate the fastest kernels and adjust readme
2024-04-13 02:58:23 +00:00
3ab2039e32
update timings
2024-04-12 23:47:15 +00:00
6396e393e3
add links to discussions
2024-04-12 01:11:29 +00:00
3aaed922a3
-O3 cannot go with -Ofast
...
Signed-off-by: soldy <4786022+soldy@users.noreply.github.com >
2024-04-11 16:27:59 +01:00
919b2200a0
Merge pull request #20 from varunlakkur/master
...
Fix a typo
2024-04-10 12:21:07 -07:00
add9e77daa
mention compilation issue that seems to be common. we may want to resolve this inside the Makefile though
2024-04-10 19:11:44 +00:00
80f52e5290
the full forward pass of GPT-2 in one file of pure CUDA
2024-04-10 18:15:55 +00:00
327705d03f
mention ongoing cuda work in main readme
2024-04-09 18:29:24 +00:00
69c1f179a8
Fix a typo
2024-04-09 15:02:33 +05:30
c39de59168
attach a simple tutorial
2024-04-08 16:26:38 -07:00
2790d263a6
oops fix readme bug
2024-04-08 12:42:44 -07:00
e8e1628632
first commit of just the reference cpu fp32 gpt2 training
2024-04-08 12:41:21 -07:00
Andrej Karpathy