瀏覽代碼

Fix convert-llama-ggmlv3-to-gguf.py vocab conversion (#2698)

When converting without metadata, the hex value for bytes entries weren't 0 padded to 2 digits.
Kerfuffle 2 年之前
父節點
當前提交
930523c8e1
共有 1 個文件被更改,包括 1 次插入2 次删除
  1. 1 2
      convert-llama-ggmlv3-to-gguf.py

+ 1 - 2
convert-llama-ggmlv3-to-gguf.py

@@ -236,8 +236,7 @@ class GGMLToGGUF:
             if len(vbytes) == 0:
             if len(vbytes) == 0:
                 tt = 3 # Control
                 tt = 3 # Control
             elif tokid >= 3 and tokid <= 258 and len(vbytes) == 1:
             elif tokid >= 3 and tokid <= 258 and len(vbytes) == 1:
-                hv = hex(vbytes[0])[2:].upper()
-                vbytes = bytes(f'<0x{hv}>', encoding = 'UTF-8')
+                vbytes = bytes(f'<0x{vbytes[0]:02X}>', encoding = 'UTF-8')
                 tt = 6 # Byte
                 tt = 6 # Byte
             else:
             else:
                 vbytes = vbytes.replace(b' ', b'\xe2\x96\x81')
                 vbytes = vbytes.replace(b' ', b'\xe2\x96\x81')