Przeglądaj źródła

gguf-py: enable reading non-native endian files (#12081)

Currently self.byte_order is never used.
Actually use it to byteswap read data to
allow reading big endian files on little endian systems
and vice versa.

Now it's possible to convert little-endian model
into a big-endian model and back
on a little-endian system.
Aleksei Nikiforov 10 miesięcy temu
rodzic
commit
3567ee3a94
1 zmienionych plików z 5 dodań i 3 usunięć
  1. 5 3
      gguf-py/gguf/gguf_reader.py

+ 5 - 3
gguf-py/gguf/gguf_reader.py

@@ -146,9 +146,11 @@ class GGUFReader:
         itemsize = int(np.empty([], dtype = dtype).itemsize)
         end_offs = offset + itemsize * count
         arr = self.data[offset:end_offs].view(dtype=dtype)[:count]
-        if override_order is None:
-            return arr
-        return arr.view(arr.dtype.newbyteorder(override_order))
+        if override_order is not None:
+            return arr.view(arr.dtype.newbyteorder(override_order))
+        if self.byte_order == 'S':
+            return arr.view(arr.dtype.newbyteorder(self.byte_order))
+        return arr
 
     def _push_field(self, field: ReaderField, skip_sum: bool = False) -> int:
         if field.name in self.fields: