This is the llama3 tokenizer with an added <think> and </think> tokens.
Created using this notebook https://colab.research.google.com/drive/15OyZuxGTung7ZwtrYcRw0ItN6d2FFH_H?usp=sharing
Downloads last month
-
Downloads are not tracked for this model.
How to track