Hugging Face's logo

ByteDance-Seed
/
Seed-OSS-36B-Instruct

Text Generation
Transformers
Safetensors
seed_oss
vllm
conversational
Model card Files
xet
Community
32

Prevent tokenizer calls from returning `token_type_ids`

#26
by joaogante - opened
base: refs/heads/main
←
from: refs/pr/26
Discussion Files changed
+4
-0
joaogante

Hi there 👋 transformers maintainer here.

Adding this extra tokenization config class will prevent the tokenizer from returning token_type_ids, which are unused by SeedOss 🤗 The default value for this flag is ["input_ids", "token_type_ids", "attention_mask"]

 Prevent tokenizer calls from returning `token_type_ids`1f44e2ee
joaogante

Also opened a PR in your other SeedOss models:

  1. https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Base-woSyn/discussions/7
  2. https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Base/discussions/5
joaogante

cc @Fazzie 🤗

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment