Explorar el Código

model-conversion : add token ids to prompt token output [no ci] (#17863)

This commit adds the token ids to the printed prompt outputs.

The motivation for this is that is can be useful to see the actual token
ids alongside the token strings for debugging.
Daniel Bevenius hace 1 mes
padre
commit
2fa51c19b0
Se han modificado 1 ficheros con 1 adiciones y 1 borrados
  1. 1 1
      examples/model-conversion/logits.cpp

+ 1 - 1
examples/model-conversion/logits.cpp

@@ -144,7 +144,7 @@ int main(int argc, char ** argv) {
             return 1;
             return 1;
         }
         }
         std::string s(buf, n);
         std::string s(buf, n);
-        printf("%s", s.c_str());
+        printf("%s (%d)", s.c_str(), id);
     }
     }
     printf("\n");
     printf("\n");