Gabriel Larson 4a4f426944 model : add Kimi-K2 support (#14654) 6 mesi fa
..
templates 4a4f426944 model : add Kimi-K2 support (#14654) 6 mesi fa
.editorconfig 6381d4e110 gguf : new file format with flexible meta data (beta) (#2398) 2 anni fa
ggml-vocab-aquila.gguf ff5a3f0c09 Work on the BPE tokenizer (#3252) 2 anni fa
ggml-vocab-baichuan.gguf daab3d7f45 Add more tokenizer tests (#3742) 2 anni fa
ggml-vocab-bert-bge.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-bert-bge.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-bert-bge.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-command-r.gguf 889bdd7686 command-r : add BPE pre-tokenization (#7063) 1 anno fa
ggml-vocab-command-r.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-command-r.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-deepseek-coder.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-deepseek-coder.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-deepseek-coder.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-deepseek-llm.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-deepseek-llm.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-deepseek-llm.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-falcon.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-falcon.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-falcon.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-gpt-2.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-gpt-2.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-gpt-2.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-gpt-neox.gguf daab3d7f45 Add more tokenizer tests (#3742) 2 anni fa
ggml-vocab-llama-bpe.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-llama-bpe.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-llama-bpe.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-llama-spm.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-llama-spm.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-llama-spm.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-mpt.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-mpt.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-mpt.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-nomic-bert-moe.gguf aa50ba462f tests : improve UGM tokenizer test coverage (#13773) 7 mesi fa
ggml-vocab-phi-3.gguf 3b38d48609 Per token attributes (#7685) 1 anno fa
ggml-vocab-phi-3.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-phi-3.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-qwen2.gguf 229ffff872 llama : add BPE pre-tokenization for Qwen2 (#7114) 1 anno fa
ggml-vocab-qwen2.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-qwen2.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-refact.gguf 92139b90af tests : add test-tokenizer-0.sh + fix some tokenizers (#7036) 1 anno fa
ggml-vocab-refact.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-refact.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-starcoder.gguf f4ab2a4147 llama : fix BPE pre-tokenization (#6920) 1 anno fa
ggml-vocab-starcoder.gguf.inp 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa
ggml-vocab-starcoder.gguf.out 07e4351ce6 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 7 mesi fa