#
LiteRT

LiteRT (previously known as TensorFlow Lite) is a high-performance runtime designed for on-device machine learning.

The Optimum library exports a model to LiteRT for many architectures.

The benefits of exporting to LiteRT include the following.

- Low-latency, privacy-focused, no internet connectivity required, and reduced model size and power consumption for on-device machine learning.
- Broad platform, model framework, and language support.
- Hardware acceleration for GPUs and Apple Silicon.

Export a Transformers model to LiteRT with the Optimum CLI.

Run the command below to install Optimum and the exporters module for LiteRT.

    pip install optimum[exporters-tf]

Refer to the Export a model to TFLite with optimum.exporters.tflite guide for all available arguments or with the command below.

    optimum-cli export tflite --help

Set the `--model` argument to export a from the Hub.

    optimum-cli export tflite --model google-bert/bert-base-uncased --sequence_length 128 bert_tflite/

You should see logs indicating the progress and showing where the resulting `model.tflite` is saved.

    Validating TFLite model...
    	-[✓] TFLite model output names match reference model (logits)
    	- Validating TFLite Model output "logits":
    		-[✓] (1, 128, 30522) matches (1, 128, 30522)
    		-[x] values not close enough, max diff: 5.817413330078125e-05 (atol: 1e-05)
    The TensorFlow Lite export succeeded with the warning: The maximum absolute difference between the output of the reference model and the TFLite exported model is not within the set tolerance 1e-05:
    - logits: max diff = 5.817413330078125e-05.
     The exported model was saved at: bert_tflite

For local models, make sure the model weights and tokenizer files are saved in the same directory, for example `local_path`. Pass the directory to the `--model` argument and use `--task` to indicate the task a model can perform. If `--task` isn’t provided, the model architecture without a task-specific head is used.

    optimum-cli export tflite --model local_path --task question-answering google-bert/bert-base-uncased --sequence_length 128 bert_tflite/

< > Update on GitHub