 Georgi Gerganov
|
c3f8d58356
tests : test-tokenizer-0.sh print more info (#7402)
|
1 rok temu |
 jaime-m-p
|
d7e852c1bc
Tokenizer SPM fixes for phi-3 and llama-spm (bugfix) (#7425)
|
1 rok temu |
|
jaime-m-p
|
917dc8cfa6
Tokenizer SPM fixes for phi-3 and llama-spm (#7375)
|
1 rok temu |
|
Georgi Gerganov
|
fabf30b4c4
llama : remove Persimmon (#7408)
|
1 rok temu |
 Anas Ahouzi
|
6aade19ee7
Add StableLM2 pre-tokenizer (#7349)
|
1 rok temu |
|
Georgi Gerganov
|
b49a13dd2f
convert : fix set_vocab_sentencepiece (#6866)
|
1 rok temu |
 Aarni Koskela
|
d273c1402b
py : convert-hf-to-gguf-update improvements (#7340)
|
1 rok temu |
 amd-lalithnc
|
e18bc6aaf3
convert : fix Qwen/Qwen-7b conversion (#7308)
|
1 rok temu |
 compilade
|
ee52225067
convert-hf : support direct Q8_0 conversion (#7234)
|
1 rok temu |
 Joan Fontanals
|
9aa672490c
llama : rename jina tokenizers to v2 (#7249)
|
1 rok temu |
|
compilade
|
5a419926b0
convert-hf : support bfloat16 conversion (#7158)
|
1 rok temu |
 Joan Fontanals
|
b83cc3f5b3
llama : add Jina Embeddings architecture (#6826)
|
1 rok temu |
|
Georgi Gerganov
|
9cb317f77e
ggml : full ALiBi support (#7192)
|
1 rok temu |
|
compilade
|
f98eb31c51
convert-hf : save memory with lazy evaluation (#7075)
|
1 rok temu |
 Ren Xuancheng
|
229ffff872
llama : add BPE pre-tokenization for Qwen2 (#7114)
|
1 rok temu |
 DAN™
|
4cd621c26d
convert : add BPE pre-tokenization for DBRX (#7132)
|
1 rok temu |
 nopperl
|
b6aa670203
Fix OLMo HF to GGUF conversion (#6910)
|
1 rok temu |
|
DAN™
|
889bdd7686
command-r : add BPE pre-tokenization (#7063)
|
1 rok temu |
|
Georgi Gerganov
|
92139b90af
tests : add test-tokenizer-0.sh + fix some tokenizers (#7036)
|
1 rok temu |
 Brian
|
a2ac89d6ef
convert.py : add python logging instead of print() (#6511)
|
1 rok temu |
 Bartowski
|
60325fa56f
Remove .attention from skipped tensors to match more accurately (#7051)
|
1 rok temu |
|
Georgi Gerganov
|
952d03dbea
convert : use utf8 encoding (#7000)
|
1 rok temu |
|
Georgi Gerganov
|
f4ab2a4147
llama : fix BPE pre-tokenization (#6920)
|
1 rok temu |
 Christian Zhou-Zheng
|
3055a41805
convert : fix conversion of some BERT embedding models (#6937)
|
1 rok temu |
 Junyang Lin
|
3fec68be4e
convert : add support of codeqwen due to tokenizer (#6707)
|
1 rok temu |
 liuwei-git
|
c8297c6af5
llama : add phi3 support (#6852)
|
1 rok temu |
 Pedro Cuenca
|
b97bc3966e
llama : support Llama 3 HF conversion (#6745)
|
1 rok temu |
|
nopperl
|
9958c81b79
Implement the OLMo architecture (#6741)
|
1 rok temu |
 Zheng.Deng
|
facb8b56f8
convert : fix autoawq gemma (#6704)
|
1 rok temu |
 Ashish
|
dbceec87c0
llama : add StableLM2 12B (#6635)
|
1 rok temu |
