HuggingFaceTB
/

SmolLM2-135M

Text Generation

text-generation-inference

Model card Files Files and versions

What MMLU (cloze) used for evaluation.

#2

by akhilfau - opened Nov 13, 2024

Did you guys used below dataset or some other one from hugging face.
https://huggingface.co/datasets/cais/mmlu

Hugging Face Smol Models Research org Nov 23, 2024

We use this dataset https://huggingface.co/datasets/lighteval/mmlu (cf this code https://github.com/huggingface/cosmopedia/blob/653acad4f6146ce1043e2ca8792c671947e83ce0/evaluation/lighteval_tasks.py#L280)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment