7f4e95a68d
Merge pull request #183 from AlpinDale/patch-2
...
Enhance GPU `bfloat16` support check
2023-07-24 02:11:34 -07:00
5486baa59a
Merge branch 'main' into main
2023-07-24 02:07:28 -07:00
76d86645b8
Merge pull request #219 from abhilash1910/main
...
Enable Qlora scripts on Intel GPUs
2023-07-24 01:25:03 -07:00
82c730a48a
Merge pull request #143 from bubundas17/patch-1
...
Fixed Local .jsonl dataset loading
2023-07-24 01:15:44 -07:00
a9644b4154
xpu support
2023-07-19 04:01:03 -07:00
a17d93e6fd
xpu support
2023-07-19 00:47:53 -07:00
61dfbe287e
Updated qlora.py
...
Updated the way embeddings are frozen. First load the base model, then resize tokenizer, then do prepare_for_kbit_training to freeze the model's original weights, then LoRA-fy and then train.
2023-07-19 02:03:36 +03:00
5111741384
freezing embeddings and known issue
2023-07-18 04:22:33 -07:00
0fc40da9dc
Enhances GPU bfloat16 support check
2023-06-22 16:06:06 +00:00
af90b31d5a
add trust_remote_code for AutoTokenizer
2023-06-19 00:44:14 +08:00
9c6e450468
Fixed Local .jsonl dataset loading
2023-06-08 00:18:53 +05:30
3da535abdf
Merge pull request #101 from dameikle/main
...
Added use_auth_token to model arguments to load private models
2023-06-06 11:49:29 -07:00
f4ee7fd786
Update qlora.py
...
Co-authored-by: Quentin Lhoest <42851186+lhoestq@users.noreply.github.com >
2023-06-06 11:31:24 -07:00
00e5ad2a75
pandas is better at reading lines
2023-06-03 19:46:39 -04:00
bed4b0d863
typo
2023-06-03 19:42:49 -04:00
e69303d556
introduce LOCAL_RANK mapping and optiojn to leave input data as-is
2023-06-03 19:30:39 -04:00
986c500c5b
Respect cache_dir passed by user
...
When the user overrides default `cache_dir` we should respect it not only for tokenizer but also for the model
2023-06-01 13:40:55 +02:00
8ee760d80e
Added use_auth_token to model arguments to load private models
2023-06-01 06:14:32 +01:00
a45718eec2
fix tokenization issues
2023-05-31 19:52:43 -07:00
5f618e7e90
oasst1 and bug fixes
2023-05-31 18:16:00 -07:00
94e50419e3
Merge pull request #86 from dred0n/feature-local-training-data
...
add support local training data
2023-05-31 14:27:33 -07:00
d9f201c2b6
Merge pull request #79 from KKcorps/resume_patch
...
Bug Fix: Resume training from checkpoints
2023-05-31 14:22:00 -07:00
7e1e81445b
Merge pull request #44 from KKcorps/empty_adapter_patch
...
Bug Fix: 443 Bytes `adapter_model.bin` files
2023-05-30 11:56:00 -07:00
ace51b6e34
add jsonl local dataset support
2023-05-30 10:45:11 -05:00
2e21c16297
add csv, tsv support
2023-05-30 09:43:49 -05:00
c06d702e5f
add support local training set support
2023-05-30 08:52:20 -05:00
163adf26a6
Only load adapter weights on resume
2023-05-29 14:57:21 +05:30
e31aedd4ba
Suppress pad_token warning message
2023-05-28 14:00:12 +02:00
f1d1b2901a
Remove resetting old state to fix empty adapters
2023-05-26 22:02:16 +05:30
8532e0117d
Check for LlamaTokenizerFast rather than infer type from path name. Fix cases where non-standard llama model path names gets bypassed in tokenizer check. The tokenizer is init with use_fast=True and qlora requires >4.29.2 transformers so the only possible tokenizer is LlamaTokenizerFast.
2023-05-25 08:56:14 +00:00
9f684a4c8c
Merge pull request #2 from scottlogic-alex/support-mpt
...
support trusting remote code (to unpickle MPT-based models)
2023-05-22 11:19:44 -07:00
7e4b88cbaf
remove broken logger.log() call (was not specifying which log level to use). put the message in the error instead.
2023-05-22 19:01:20 +01:00
765455f2d6
support trusting remote code (to unpickle MPT-based models)
2023-05-22 18:34:31 +01:00
132e0510f0
rename and cleanup
2023-05-12 12:14:23 -07:00
Artidoro Pagnoni