Upload tokenizer files (vocab, config, README)

Files changed (2) hide show

README.md CHANGED Viewed

@@ -15,6 +15,6 @@ This tokenizer was trained using a Python-only pipeline (no `transformers` or `t
 ```python
 from transformers import PreTrainedTokenizerFast
-tokenizer = PreTrainedTokenizerFast.from_pretrained("your-username/tokenizer-name")
 ```

 ```python
 from transformers import PreTrainedTokenizerFast
+tokenizer = PreTrainedTokenizerFast.from_pretrained("goabonga/wikitext-2-raw-v1")
 ```

vocab.json CHANGED Viewed

The diff for this file is too large to render. See raw diff