Jelajahi Sumber

scripts: corrected encoding when getting chat template (#11866) (#11907)

Signed-off-by: MoonRide303 <moonride303@gmail.com>
MoonRide303 11 bulan lalu
induk
melakukan
5137da7b8c
1 mengubah file dengan 1 tambahan dan 1 penghapusan
  1. 1 1
      scripts/get_chat_template.py

+ 1 - 1
scripts/get_chat_template.py

@@ -21,7 +21,7 @@ def get_chat_template(model_id, variant=None):
         # Use huggingface_hub library if available.
         # Allows access to gated models if the user has access and ran `huggingface-cli login`.
         from huggingface_hub import hf_hub_download
-        with open(hf_hub_download(repo_id=model_id, filename="tokenizer_config.json")) as f:
+        with open(hf_hub_download(repo_id=model_id, filename="tokenizer_config.json"), encoding="utf-8") as f:
             config_str = f.read()
     except ImportError:
         import requests