File size: 2,753 Bytes
677252f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
base_model: black-forest-labs/FLUX.1-Kontext-dev
library_name: diffusers
base_model_relation: quantized
tags:
- quantization
---
# Flux-Kontext-dev model outputs using BnB&Hqq 4bit quantization
<td style="text-align: center;">
Original Input<br>
<medium-zoom background="rgba(0,0,0,.7)"><img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png" alt="Original Input"></medium-zoom>
</td>
<td style="text-align: center;">
BnB 4-bit (DiT) & Hqq 4-bit (T5)<br>
<medium-zoom background="rgba(0,0,0,.7)"><img src="kontext.1-dev_dit_bnb_4bit_t5_hqq_4bit.png" alt="BnB 4-bit (DiT) & Hqq 4-bit (T5) Output"></medium-zoom>
</td>
# Usage with Diffusers
To use this quantized FLUX.1 [dev] checkpoint, you need to install the 🧨 diffusers, transformers, bitsandbytes and hqq library:
```
pip install git+https://github.com/huggingface/diffusers.git@main # add support for `FluxKontextPipeline`
pip install transformers>=4.53.1 # add support for hqq quantized model in diffusers pipeline
pip install -U bitsandbytes
pip install -U hqq
```
After installing the required library, you can run the following script:
```python
import torch
from diffusers import FluxKontextPipeline
from diffusers.utils import load_image
pipe = FluxKontextPipeline.from_pretrained("HighCWu/FLUX.1-Kontext-dev-bnb-hqq-4bit", torch_dtype=torch.bfloat16)
pipe.to("cuda")
input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png")
image = pipe(
image=input_image,
prompt="Add a hat to the cat",
guidance_scale=2.5,
num_inference_steps=28,
generator=torch.Generator("cuda").manual_seed(0),
).images[0]
image.save(f"kontext.1-dev.png")
```
# How to generate this quantized checkpoint ?
This checkpoint was created with the following script using "black-forest-labs/FLUX.1-Kontext-dev" checkpoint:
```python
import torch
assert torch.cuda.is_available() # force initialization of cuda
from diffusers import FluxKontextPipeline
from diffusers import BitsAndBytesConfig as DiffusersBitsAndBytesConfig
from diffusers.quantizers import PipelineQuantizationConfig
from transformers import HqqConfig as TransformersHqqConfig
pipeline_quant_config = PipelineQuantizationConfig(
quant_mapping={
"transformer": DiffusersBitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16),
"text_encoder_2": TransformersHqqConfig(nbits=4, group_size=64),
}
)
pipe = FluxKontextPipeline.from_pretrained(
"black-forest-labs/FLUX.1-Kontext-dev",
quantization_config=pipeline_quant_config,
torch_dtype=torch.bfloat16
)
pipe.save_pretrained("FLUX.1-Kontext-dev-bnb-hqq-4bit")
```
|