#
How to use OpenVINO for inference

🤗 Optimum provides Stable Diffusion pipelines compatible with OpenVINO. You can now easily perform inference with OpenVINO Runtime on a variety of Intel processors (see the full list of supported devices).

##
Installation

Install 🤗 Optimum Intel with the following command:

    pip install --upgrade-strategy eager optimum["openvino"]

The `--upgrade-strategy eager` option is needed to ensure `optimum-intel` is upgraded to its latest version.

##
Stable Diffusion

###
Inference

To load an OpenVINO model and run inference with OpenVINO Runtime, you need to replace `StableDiffusionPipeline` with `OVStableDiffusionPipeline`. In case you want to load a PyTorch model and convert it to the OpenVINO format on-the-fly, you can set `export=True`.

    from optimum.intel import OVStableDiffusionPipeline

    model_id = "runwayml/stable-diffusion-v1-5"
    pipeline = OVStableDiffusionPipeline.from_pretrained(model_id, export=True)
    prompt = "sailing ship in storm by Rembrandt"
    image = pipeline(prompt).images[0]

    # Don't forget to save the exported model
    pipeline.save_pretrained("openvino-sd-v1-5")

To further speed up inference, the model can be statically reshaped :

    # Define the shapes related to the inputs and desired outputs
    batch_size, num_images, height, width = 1, 1, 512, 512

    # Statically reshape the model
    pipeline.reshape(batch_size, height, width, num_images)
    # Compile the model before inference
    pipeline.compile()

    image = pipeline(
        prompt,
        height=height,
        width=width,
        num_images_per_prompt=num_images,
    ).images[0]

In case you want to change any parameters such as the outputs height or width, you’ll need to statically reshape your model once again.

###
Supported tasks

Task | Loading Class
--- | ---
`text-to-image` | `OVStableDiffusionPipeline`
`image-to-image` | `OVStableDiffusionImg2ImgPipeline`
`inpaint` | `OVStableDiffusionInpaintPipeline`

You can find more examples in the optimum documentation.

##
Stable Diffusion XL

###
Inference

    from optimum.intel import OVStableDiffusionXLPipeline

    model_id = "stabilityai/stable-diffusion-xl-base-1.0"
    pipeline = OVStableDiffusionXLPipeline.from_pretrained(model_id, export=True)
    prompt = "sailing ship in storm by Rembrandt"
    image = pipeline(prompt).images[0]

To further speed up inference, the model can be statically reshaped as showed above. You can find more examples in the optimum documentation.

###
Supported tasks

Task | Loading Class
--- | ---
`text-to-image` | `OVStableDiffusionXLPipeline`
`image-to-image` | `OVStableDiffusionXLImg2ImgPipeline`