Lighteval

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.13.0).

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

Available tasks

Browse and inspect tasks available in LightEval.

List all tasks:

lighteval tasks list

Extract tasks details:

lighteval tasks dump

Store the tasks details in a JSON file:

lighteval tasks dump > tasks.json

Inspect specific tasks

Inspect a task to view its config, metrics, and requirements:

lighteval tasks inspect <task_name>

Example:

lighteval tasks inspect truthfulqa:mc

Update on GitHub

←Available Metrics EvaluationTracker→