839b4d7161
Fix issues: #274 , #280 ( #282 )
...
* Fix issue with reading directories in `iterate_files()` (#280 )
* Add directory checking logic in `iterate_files()` (#274 )
* Added tests for #282 , #274 , #280
---------
Co-authored-by: Simon Willison <swillison@gmail.com >
2023-09-18 23:14:30 -07:00
b4ec54ef19
NotImplementedError for system prompts with OpenAI completion models, refs #284
...
Signed-off-by: Simon Willison <swillison@gmail.com >
2023-09-18 22:51:22 -07:00
f76b2120e4
Revert "Handle system prompts for completion models, refs #284 "
...
This reverts commit 4eed871cf1#288
2023-09-18 22:44:38 -07:00
4eed871cf1
Handle system prompts for completion models, refs #284
2023-09-18 22:36:38 -07:00
fcff36c6bc
completion: true to register completion models, refs #284
2023-09-18 22:17:26 -07:00
9c7792dce5
Removed rogue raise
2023-09-18 22:05:00 -07:00
4fea46113f
logprobs support for OpenAI completion models, refs #284
2023-09-18 22:04:28 -07:00
ae14ce4add
LLM_OPENAI_SHOW_RESPONSES=1 debug trick, closes #286
2023-09-18 21:13:42 -07:00
4d18da4e11
Bump default gpt-3.5-turbo-instruct max tokens to 256, refs #284
2023-09-18 20:29:39 -07:00
4d46ebaa32
OpenAI completion models including gpt-3.5-turbo-instruct, refs #284
2023-09-18 18:34:32 -07:00
33dee4762e
llm embed-multi --batch-size option, closes #273
2023-09-13 16:33:27 -07:00
b9478e6a17
batch_size= argument to embed_multi(), refs #273
2023-09-13 16:24:04 -07:00
6c43948325
llm.user_dir() creates directory if needed, closes #275
...
Would have fixed this bug too:
- https://github.com/simonw/llm-sentence-transformers/issues/9
2023-09-13 15:58:18 -07:00
4952a8d119
llm similar --binary, closes #269
2023-09-12 11:23:31 -07:00
9e529bb36a
Wrap !multi in single quotes, for consistency with exit/quit
2023-09-12 10:45:02 -07:00
f54f2c659d
response.__str__ method, closes #268
2023-09-12 10:36:29 -07:00
9c33d30843
llm chat !multi support, closes #267
2023-09-12 09:31:20 -07:00
22a59f795e
llm collections defaults to llm collections list, close #265
2023-09-11 23:08:11 -07:00
591ad6f571
Revert "Reuse embeddings for hashed content, --store now works on second run - closes #224 "
...
This reverts commit 267e2ea999https://github.com/simonw/llm/issues/224#issuecomment-1715014393
2023-09-11 22:56:52 -07:00
267e2ea999
Reuse embeddings for hashed content, --store now works on second run - closes #224
2023-09-11 22:44:22 -07:00
52cec1304b
Binary embeddings ( #254 )
...
* Binary embeddings support, refs #253
* Write binary content to content_blob, with tests - refs #253
* supports_text and supports_binary embedding validation, refs #253
2023-09-11 18:58:44 -07:00
f07b291a78
change type of cli "embed -c" to be STR not FILE ( #263 )
...
* change type of cli "embed -c" to be STR not FILE
The type of the "content" argument was erroneously set to be FILE with a "type=click.Path(...)" argument, which content is a string.
---------
Co-authored-by: Simon Willison <swillison@gmail.com >
2023-09-11 18:17:34 -07:00
5ba34dbe36
llm embed-db is now llm collections, refs #229
2023-09-10 14:24:27 -07:00
6012f31e36
llm plugins --all option, closes #259
2023-09-10 14:18:16 -07:00
b84f6f9ffd
Docs for LLM_LOAD_PLUGINS, closes #256
2023-09-10 14:12:37 -07:00
df32d7685d
Updated error message for invalid or missing embedding model ( #257 )
...
* Updated error message for missing embedding model
---------
Co-authored-by: Simon Willison <swillison@gmail.com >
2023-09-10 11:56:29 -07:00
5912bd47c0
--no-stream option for llm chat, closes #248
2023-09-10 11:47:38 -07:00
ae7f4f6de7
llm chat -o/--option - refs #244
2023-09-10 11:14:28 -07:00
17e6402908
Fix bug with llm chat -c and API keys, closes #247
2023-09-05 19:26:21 -07:00
5495112d9f
Initial tests for llm chat, refs #231
2023-09-04 23:36:25 -07:00
969a5d3364
Initial prototype of llm chat, refs #231
2023-09-04 23:36:25 -07:00
78a0e9bd44
llm --files --encoding option and latin-1 fallback, closes #225
2023-09-04 12:28:31 -07:00
a54743f228
Fix to get embedding set_alias to work
...
Refs:
- https://github.com/simonw/llm-sentence-transformers/issues/4
- https://github.com/simonw/llm/issues/192
2023-09-03 18:15:45 -07:00
2e90a30c4f
Add embedding model aliases to llm aliases, refs #192
2023-09-03 17:55:56 -07:00
3bf781fba2
Duplicate content is only embedded once, closes #217
2023-09-03 17:39:11 -07:00
0eda99e91c
Default embedding model finishing touches, closes #222
2023-09-03 17:21:47 -07:00
0da1ed7d98
--remove-default for llm embed-models default, refs #222
2023-09-03 16:40:00 -07:00
c8c0f80441
--prefix for llm embed-multi, refs #215
2023-09-03 16:40:00 -07:00
2440eb4f48
Help tweak and ran cog
2023-09-03 16:40:00 -07:00
3d379fbddc
In progress llm embed-multi, refs #215
2023-09-03 16:40:00 -07:00
156bed7c65
Populate content_hash with embed_multi, refs #217
2023-09-03 14:43:29 -07:00
53fc9cbab7
Fixed hookspec docstrings
2023-09-03 14:04:23 -07:00
213e0b0c75
embed-db delete-collection command and .delete() method, closes #219
2023-09-03 12:55:48 -07:00
87af2ddda9
De-register functions after migration, refs #217
...
Also refs https://github.com/simonw/sqlite-utils/issues/589
2023-09-03 12:33:06 -07:00
573659c84d
Remove obsolete debugging code
2023-09-03 10:58:46 -07:00
a5d6b580ba
Store content_hash in embeddings table, refs #217
...
Uses new migrations feature from https://github.com/simonw/sqlite-migrate/issues/9
2023-09-03 10:50:51 -07:00
26332045dd
llm embed --metadata option, closes #209
2023-09-03 07:43:23 -07:00
73a9043108
Store updated timestamp on embeddings, closes #211
2023-09-02 20:40:33 -07:00
51488c579b
Collection now defaults to in-memory DB, closes #213
2023-09-02 15:43:20 -07:00
e49c816e18
Drop llm.EmbeddingModel.embedding_size property, closes #208
2023-09-02 08:45:17 -07:00
e. alvarez