iamlucaconti commited on
Commit
a66b9b7
·
verified ·
1 Parent(s): deb6a84

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +74 -1
README.md CHANGED
@@ -12,4 +12,77 @@ GitHub: https://github.com/iamlucaconti/InstructPix2Pix
12
 
13
  Given an input image and an accompanying edit instruction, our model generates the corresponding modification directly. Unlike approaches that rely on detailed textual descriptions of both the source and target images, our method requires only the instruction (`ediit_prompt`) and performs the edit in a single forward pass, without per-example inversion or additional fine-tuning. An example generated by our model is shown below:
14
 
15
- ![Home Alone](images/home_alone.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
  Given an input image and an accompanying edit instruction, our model generates the corresponding modification directly. Unlike approaches that rely on detailed textual descriptions of both the source and target images, our method requires only the instruction (`ediit_prompt`) and performs the edit in a single forward pass, without per-example inversion or additional fine-tuning. An example generated by our model is shown below:
14
 
15
+ <img src='https://github.com/iamlucaconti/InstructPix2Pix/raw/main/images/home_alone.png'/>
16
+
17
+ ## How to use our model
18
+
19
+ To edit an image using our model:
20
+
21
+ ```python
22
+ import os
23
+ import torch
24
+ import requests
25
+ import matplotlib.pyplot as plt
26
+ from PIL import Image, ImageOps
27
+ from io import BytesIO
28
+ from diffusers import StableDiffusionInstructPix2PixPipeline
29
+
30
+
31
+ def download_image(url: str, resize: bool = False, resolution: int = 512) -> Image.Image:
32
+ # Download and open the image
33
+ image = Image.open(BytesIO(requests.get(url, stream=True).content))
34
+ # Fix orientation issues from EXIF metadata
35
+ image = ImageOps.exif_transpose(image).convert("RGB")
36
+
37
+ if resize:
38
+ w, h = image.size
39
+ if w > h:
40
+ new_w = resolution
41
+ new_h = int(h * resolution / w)
42
+ else:
43
+ new_h = resolution
44
+ new_w = int(w * resolution / h)
45
+ image = image.resize((new_w, new_h))
46
+
47
+ return image
48
+
49
+
50
+ # Parameters
51
+ pretrained_model_name_or_path = "iamlucaconti/instruct-pix2pix-model" # Custom Pix2Pix model
52
+ image_url = "<URL OF IMAGE TO EDIT>"
53
+ prompt = "<YOUR EDIT PROMPT>" # Instructional edit prompt
54
+ num_inference_steps = 50 # More steps = higher quality but slower
55
+ image_guidance_scale = 2.0 # Strength of adherence to input image
56
+ guidance_scale = 3.0 # Strength of adherence to text prompt
57
+ seed = 0 # Random seed (for reproducibility)
58
+ output_path = "output.png" # File to save result
59
+
60
+
61
+ # Load pipeline
62
+ pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained(
63
+ pretrained_model_name_or_path,
64
+ torch_dtype=torch.float16,
65
+ safety_checker=None
66
+ ).to("cuda")
67
+
68
+ # Load image
69
+ image = download_image(image_url, resize=True)
70
+
71
+ # Set seed
72
+ generator = torch.Generator("cuda").manual_seed(seed)
73
+
74
+
75
+ # Generate the edited image
76
+ edited_image = pipe(
77
+ prompt=prompt,
78
+ image=image,
79
+ num_inference_steps=num_inference_steps,
80
+ image_guidance_scale=image_guidance_scale,
81
+ guidance_scale=guidance_scale,
82
+ generator=generator
83
+ ).images[0]
84
+
85
+
86
+ # Save
87
+ edited_image.save(output_path)
88
+ ```