 Georgi Gerganov
|
e093dd2382
tests : re-enable tokenizer tests (#8611)
|
1 éve |
|
Georgi Gerganov
|
0c27e6f62e
ggml : fix loongson compile warnings (#7537)
|
1 éve |
 Haoxiang Fei
|
f99e1e456e
llama : lookup word in vocab before doing BPE merges (#7193)
|
1 éve |
 Ren Xuancheng
|
229ffff872
llama : add BPE pre-tokenization for Qwen2 (#7114)
|
1 éve |
 DAN™
|
889bdd7686
command-r : add BPE pre-tokenization (#7063)
|
1 éve |
|
Georgi Gerganov
|
92139b90af
tests : add test-tokenizer-0.sh + fix some tokenizers (#7036)
|
1 éve |
|
Georgi Gerganov
|
f4ab2a4147
llama : fix BPE pre-tokenization (#6920)
|
1 éve |
 Olivier Chafik
|
7593639ce3
`main`: add --json-schema / -j flag (#6659)
|
1 éve |
 Clint Herron
|
57dd02c44b
Tests: Added integration tests for GBNF parser (#6472)
|
1 éve |
|
Olivier Chafik
|
5b7b0ac8df
json-schema-to-grammar improvements (+ added to server) (#5978)
|
1 éve |
 Xuan Son Nguyen
|
11b12de39b
llama : add llama_chat_apply_template() (#5538)
|
1 éve |
 crasm
|
413e7b0559
ci : add model tests + script wrapper (#4586)
|
2 éve |
|
Georgi Gerganov
|
c918fe8dca
metal : create autorelease pool during library build (#4970)
|
2 éve |
 Cuong Trinh Manh
|
97bbca6e85
cmake : fix ld warning duplicate libraries libllama.a (#4671)
|
2 éve |
 manikbhandari
|
ea5497df5d
gpt2 : Add gpt2 architecture integration (#4555)
|
2 éve |
|
Georgi Gerganov
|
fe680e3d10
sync : ggml (new ops, tests, backend, etc.) (#4359)
|
2 éve |
 Galunid
|
36eed0c42c
stablelm : StableLM support (#3586)
|
2 éve |
|
Galunid
|
daab3d7f45
Add more tokenizer tests (#3742)
|
2 éve |
 goerch
|
9e70cc0322
Add test for MPT tokenization (#3728)
|
2 éve |
|
goerch
|
ff5a3f0c09
Work on the BPE tokenizer (#3252)
|
2 éve |
|
Georgi Gerganov
|
ec893798b7
llama : custom attention mask + parallel decoding + no context swaps (#3228)
|
2 éve |
|
goerch
|
71ca2fad7d
whisper : tokenizer fix + re-enable tokenizer test for LLaMa (#3096)
|
2 éve |
 Cebtenzzre
|
849408957c
tests : add a C compliance test (#2848)
|
2 éve |
|
Georgi Gerganov
|
edd4c14817
llama : more tokenizer fixes (#2810)
|
2 éve |
|
Georgi Gerganov
|
cf658adc83
llm : add Falcon support (#2717)
|
2 éve |
|
Georgi Gerganov
|
6381d4e110
gguf : new file format with flexible meta data (beta) (#2398)
|
2 éve |
 drbh
|
7cf54e1f74
tests : adds simple llama grammar tests (#2618)
|
2 éve |
|
drbh
|
ee77efea2a
test : add simple grammar parsing tests (#2594)
|
2 éve |
 Eve
|
81844fbcfd
tests : Fix compilation warnings (Linux/GCC) (#2451)
|
2 éve |
 wzy
|
b1f4290953
cmake : install targets (#2256)
|
2 éve |
