---
tags:
- text-to-image
- lora
- diffusers
- template:diffusion-lora
widget:
- output:
    url: images/input_image.jpg
  text: Original Image
- output:
    url: images/result_base_model.jpg
  text: change the face to face segmentation mask
- output:
    url: images/result_lora_model.jpg
  text: change the face to face segmentation mask
base_model:
- Qwen/Qwen-Image-Edit
instance_prompt: null
license: mit
pipeline_tag: image-to-image
---
# Qwen-Image-Lora-Faceseg

<Gallery />

## Model description 

# Face Segmentation Model Description
## Overview
This is a LoRA fine-tuned face segmentation model based on Qwen-VL (Qwen Vision-Language) architecture, specifically designed to transform facial images into precise segmentation masks. The model leverages the powerful multimodal capabilities of Qwen-VL and enhances it through Parameter-Efficient Fine-Tuning (PEFT) using LoRA (Low-Rank Adaptation) technique.
## Model Architecture
- Base Model: Qwen-Image-Edit (built on Qwen-VL foundation)
- Fine-tuning Method: LoRA (Low-Rank Adaptation)
- Task: Image-to-Image translation (Face → Segmentation Mask)
- Input: RGB facial images
- Output: Binary&#x2F;grayscale segmentation masks highlighting facial regions
## Training Configuration
- Dataset: 20 carefully curated face segmentation samples
- Training Steps: 900-1000 steps
- Prompt: &quot;change the image from the face to the face segmentation mask&quot;
- Precision Options:
   - BF16 precision for high-quality results
   - FP4 quantization for memory-efficient deployment
## Key Features
1. High Precision Segmentation: Accurately identifies and segments facial boundaries with fine detail preservation
2. Memory Efficient: FP4 quantized version maintains competitive quality while significantly reducing memory footprint
3. Fast Inference: Optimized for real-time applications with 20 inference steps
4. Robust Performance: Handles various lighting conditions and facial orientations
5. Parameter Efficient: Only trains LoRA adapters (~1M parameters) while keeping base model frozen
## Technical Specifications
- Inference Steps: 20
- CFG Scale: 2.5
- Input Resolution: Configurable (typically 512x512)
- Model Size: Base model + ~1M LoRA parameters
- Memory Usage:
   - BF16 version: Higher memory, best quality
   - FP4 version: 75% memory reduction, competitive quality
## Use Cases
- Identity Verification: KYC (Know Your Customer) applications
- Privacy Protection: Face anonymization while preserving facial structure
- Medical Applications: Facial analysis and dermatological assessments
- AR&#x2F;VR Applications: Real-time face tracking and segmentation
- Content Creation: Automated face masking for video editing
## Performance Highlights
- Accuracy: Significantly improved boundary detection compared to base model
- Detail Preservation: Maintains fine facial features in segmentation masks
- Consistency: Stable segmentation quality across different input conditions
- Efficiency: FP4 quantization achieves 4x memory savings with minimal quality loss
## Deployment Options
- High-Quality Mode: BF16 precision for maximum accuracy
- Efficient Mode: FP4 quantization for resource-constrained environments
- Real-time Applications: Optimized inference pipeline for low-latency requirements
This model represents a practical solution for face segmentation tasks, offering an excellent balance between accuracy, efficiency, and deployability across various hardware configurations

## Example:
Control Images
![input_image.jpg](https:&#x2F;&#x2F;cdn-uploads.huggingface.co&#x2F;production&#x2F;uploads&#x2F;641af68ea5f876fe30c38508&#x2F;sPFRuwzgdMjUNWkL84jLl.jpeg)

Edited Image with Qwen-Image-Edit by promot
&#x60;change the face to face segmentation mask&#x60;

![result_base_model.jpg](https:&#x2F;&#x2F;cdn-uploads.huggingface.co&#x2F;production&#x2F;uploads&#x2F;641af68ea5f876fe30c38508&#x2F;v20z6hctGEY_DdP5WtFFv.jpeg)

After Lora Finetune with same prompt

![result_lora_model.jpg](https:&#x2F;&#x2F;cdn-uploads.huggingface.co&#x2F;production&#x2F;uploads&#x2F;641af68ea5f876fe30c38508&#x2F;pE6F_FSSfdxphfrfiZjeu.jpeg)

## Code
Lora Finetune of Qwen-Image-Edit Code here:
https:&#x2F;&#x2F;github.com&#x2F;tsiendragon&#x2F;qwen-image-finetune



## Download model


[Download](/TsienDragon/qwen-image-edit-lora-face-segmentation/tree/main) them in the Files & versions tab.