README.md

---
base_model: Alpha-VLLM/Lumina-Image-2.0
library_name: diffusers
license: apache-2.0
instance_prompt: a puppy, yarn art style
widget:
- text: a puppy in a pond, yarn art style
  output:
    url: yarn_lora.png
- text: a puppy in a pond, yarn art style (dark env)
  output:
    url: yarn_lora_You_are_an_assistant_designed_to_generate_superior_images_with_a_dark_overall_theme.png
- text: a puppy in a pond, yarn art style (shiny env)
  output:
    url: yarn_lora_You_are_an_assistant_designed_to_generate_superior_images_with_a_bright_and_shiny_overall_.png
tags:
- text-to-image
- diffusers-training
- diffusers
- lora
- lumina2
- lumina2-diffusers
- template:sd-lora
---

# Lumina2 DreamBooth LoRA - trained-lumina2-lora-yarn

<Gallery />

## Model description

These are `trained-lumina2-lora-yarn` DreamBooth LoRA weights for [Alpha-VLLM/Lumina-Image-2.0](https://hf.co/Alpha-VLLM/Lumina-Image-2.0).

The weights were trained using [DreamBooth](https://dreambooth.github.io/) with the [Lumina2 diffusers trainer](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_lumina2.md).


## Trigger words

You should use `yarn art style` to trigger the image generation.

The following `system_prompt` was also used used during training (ignore if `None`): None.

## Download model

[Download the *.safetensors LoRA]({repo_id}/tree/main) in the Files & versions tab.

## Use it with the [🧨 diffusers library](https://github.com/huggingface/diffusers)

```py
import torch
from diffusers import Lumina2Text2ImgPipeline

pipe = Lumina2Text2ImgPipeline.from_pretrained(
    "Alpha-VLLM/Lumina-Image-2.0", torch_dtype=torch.bfloat16
).to("cuda")

pipe.load_lora_weights("trained-lumina2-lora-yarn")
prompt = "a puppy in a pond, yarn art style"

image = pipe(
    prompt, 
    negative_prompt="bad quality, worse quality, degenerate quality", 
    guidance_scale=6,
    num_inference_steps=35, 
    generator=torch.manual_seed(0)
).images[0]
```

For more details, including weighting, merging and fusing LoRAs, check the [documentation on loading LoRAs in diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters).

## Results

The model benefits from `system_prompt`. Here is a comparison across different system prompts:

<table>
  <thead>
    <tr>
      <th>No system prompt</th>
      <th>"Dark surrounding"<br>system prompt</th>
      <th>"Sunny surrounding"<br>system prompt</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><img src="yarn_lora.png" alt="No system prompt image" width="200"></td>
      <td><img src="yarn_lora_You_are_an_assistant_designed_to_generate_superior_images_with_a_dark_overall_theme.png" alt="Dark surrounding image" width="200"></td>
      <td><img src="yarn_lora_You_are_an_assistant_designed_to_generate_superior_images_with_a_bright_and_shiny_overall_.png" alt="Sunny surrounding image" width="200"></td>
    </tr>
    <tr>
      <td colspan="3">
        <div style="text-align: center; font-weight: bold;">
          <b>Original prompt</b>: <i>a puppy in a pond, yarn art style</i>
        </div>
      </td>
    </tr>
  </tbody>
</table>

<details>
<summary>Code</summary>

```py
import torch
from diffusers import Lumina2Text2ImgPipeline

pipe = Lumina2Text2ImgPipeline.from_pretrained(
    "Alpha-VLLM/Lumina-Image-2.0", torch_dtype=torch.bfloat16
).to("cuda")


system_prompts = [
    None, 
    "You are an assistant designed to generate superior images with a dark overall theme.",
    "You are an assistant designed to generate superior images with a bright and shiny overall theme."
]

pipe.load_lora_weights("trained-lumina2-lora-yarn")
prompt = "a puppy in a pond, yarn art style"

for sp in system_prompts:
    filename = "yarn_lora"
    image = pipe(
        prompt, 
        negative_prompt="bad quality, worse quality, degenerate quality",
        system_prompt=sp, 
        guidance_scale=6,
        num_inference_steps=35, 
        generator=torch.manual_seed(0)
    ).images[0]
    if sp:
        filename += "_" + "_".join(sp.split(" ")).replace(",", "").replace(".", "")
        filename = filename[:100]
    
    image.save(f"{filename}.png")

```

</details>