Home Esplora Aiuto
Accedi
cturan
/
llama.cpp
mirror da https://github.com/cturan/llama.cpp
1
0
Forka 0
File Problemi 0 Wiki
Albero (Tree): 68ff663a04
Rami (Branch) Tag
llama.cpp
/
examples
/
llava
Georgi Gerganov 68ff663a04 repo : update links to new url (#11886) 11 mesi fa
..
android 1c641e6aac `build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809) 1 anno fa
CMakeLists.txt 1ec208083c llava: add quantization for the visual projector LLAVA, Qwen2VL (#11644) 11 mesi fa
MobileVLM-README.md e665744317 llava : fix the script error in MobileVLM README (#9054) 1 anno fa
README-glmedge.md 0cec062a63 llama : add support for GLM-Edge and GLM-Edge-V series models (#10573) 11 mesi fa
README-minicpmo2.6.md 68ff663a04 repo : update links to new url (#11886) 11 mesi fa
README-minicpmv2.5.md 68ff663a04 repo : update links to new url (#11886) 11 mesi fa
README-minicpmv2.6.md d565bb2fd5 llava : support MiniCPM-V-2.6 (#8967) 1 anno fa
README-quantize.md 1ec208083c llava: add quantization for the visual projector LLAVA, Qwen2VL (#11644) 11 mesi fa
README.md e235b267a2 py : switch to snake_case (#8305) 1 anno fa
clip-quantize-cli.cpp 1ec208083c llava: add quantization for the visual projector LLAVA, Qwen2VL (#11644) 11 mesi fa
clip.cpp 1ec208083c llava: add quantization for the visual projector LLAVA, Qwen2VL (#11644) 11 mesi fa
clip.h 0cec062a63 llama : add support for GLM-Edge and GLM-Edge-V series models (#10573) 11 mesi fa
convert_image_encoder_to_gguf.py 511636df0c ci : reduce severity of unused Pyright ignore comments (#9697) 1 anno fa
glmedge-convert-image-encoder-to-gguf.py 0cec062a63 llama : add support for GLM-Edge and GLM-Edge-V series models (#10573) 11 mesi fa
glmedge-surgery.py 0cec062a63 llama : add support for GLM-Edge and GLM-Edge-V series models (#10573) 11 mesi fa
llava-cli.cpp afa8a9ec9b llama : add `llama_vocab`, functions -> methods, naming (#11110) 1 anno fa
llava.cpp 0cec062a63 llama : add support for GLM-Edge and GLM-Edge-V series models (#10573) 11 mesi fa
llava.h 3071c0a5f2 llava : support MiniCPM-V-2.5 (#7599) 1 anno fa
llava_surgery.py e235b267a2 py : switch to snake_case (#8305) 1 anno fa
llava_surgery_v2.py 3fd62a6b1c py : type-check all Python scripts with Pyright (#8341) 1 anno fa
minicpmv-cli.cpp 3e3357fd77 llava : support Minicpm-omni (#11289) 1 anno fa
minicpmv-convert-image-encoder-to-gguf.py 3e3357fd77 llava : support Minicpm-omni (#11289) 1 anno fa
minicpmv-surgery.py 3e3357fd77 llava : support Minicpm-omni (#11289) 1 anno fa
qwen2_vl_surgery.py 4ddd199f6f llava : Allow locally downloaded models for QwenVL (#10833) 1 anno fa
qwen2vl-cli.cpp afa8a9ec9b llama : add `llama_vocab`, functions -> methods, naming (#11110) 1 anno fa
requirements.txt d3ae0ee8d7 py : fix requirements check '==' -> '~=' (#8982) 1 anno fa

README-glmedge.md

GLMV-EDGE

Currently this implementation supports glm-edge-v-2b and glm-edge-v-5b.

Usage

Build with cmake or run make llama-llava-cli to build it.

After building, run: ./llama-llava-cli to see the usage. For example:

./llama-llava-cli -m model_path/ggml-model-f16.gguf --mmproj model_path/mmproj-model-f16.gguf --image img_path/image.jpg -p "<|system|>\n system prompt <image><|user|>\n prompt <|assistant|>\n"

note: A lower temperature like 0.1 is recommended for better quality. add --temp 0.1 to the command to do so. note: For GPU offloading ensure to use the -ngl flag just like usual

GGUF conversion

  1. Clone a GLMV-EDGE model (2B or 5B). For example:

    git clone https://huggingface.co/THUDM/glm-edge-v-5b or https://huggingface.co/THUDM/glm-edge-v-2b

  2. Use glmedge-surgery.py to split the GLMV-EDGE model to LLM and multimodel projector constituents:

    python ./examples/llava/glmedge-surgery.py -m ../model_path

  3. Use glmedge-convert-image-encoder-to-gguf.py to convert the GLMV-EDGE image encoder to GGUF:

    python ./examples/llava/glmedge-convert-image-encoder-to-gguf.py -m ../model_path --llava-projector ../model_path/glm.projector --output-dir ../model_path

  4. Use examples/convert_hf_to_gguf.py to convert the LLM part of GLMV-EDGE to GGUF:

    python convert_hf_to_gguf.py ../model_path

Now both the LLM part and the image encoder are in the model_path directory.