数据集 / HiDream-ai/HiDream-E1-Full

HiDream-ai/HiDream-E1-Full 已完整同步

HiDream-E1 Demo

HiDream-E1 is an image editing model built on HiDream-I1.

For more features and to experience the full capabilities of our product, please visit https://vivago.ai/.

Project Updates

Quick Start

Please make sure you have installed Flash Attention and latest Diffusers. We recommend CUDA versions 12.4 for the manual installation.

pip install -r requirements.txt
pip install -U flash-attn --no-build-isolation
pip install -U git+https://github.com/huggingface/diffusers.git

Then you can run the inference scripts to generate images:

python ./inference.py

Alternatively, you can use the model in your own code:

import torch
from transformers import PreTrainedTokenizerFast, LlamaForCausalLM
from pipeline_hidream_image_editing import HiDreamImageEditingPipeline
from PIL import Image

# Load the tokenizer and text encoder
tokenizer_4 = PreTrainedTokenizerFast.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
text_encoder_4 = LlamaForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    output_hidden_states=True,
    output_attentions=True,
    torch_dtype=torch.bfloat16,
)

# Load the HiDream pipeline
pipe = HiDreamImageEditingPipeline.from_pretrained(
    "HiDream-ai/HiDream-E1-Full",
    tokenizer_4=tokenizer_4,
    text_encoder_4=text_encoder_4,
    torch_dtype=torch.bfloat16,
)

# Load and prepare input image
test_image = Image.open("your_image.jpg")
test_image = test_image.resize((768, 768))

# Move pipeline to GPU
pipe = pipe.to("cuda", torch.bfloat16)

# Generate edited image
image = pipe(
    prompt = 'Editing Instruction: Convert the image into a Ghibli style. Target Image Description: A person in a light pink t-shirt with short dark hair, depicted in a Ghibli style against a plain background.',
    negative_prompt = "low resolution, blur",
    image = test_image,
    guidance_scale=5.0,
    image_guidance_scale=4.0,
    num_inference_steps=28,
    generator=torch.Generator("cuda").manual_seed(3),
).images[0]

# Save output image
image.save("output.jpg")

Note

The inference script will try to automatically download meta-llama/Llama-3.1-8B-Instruct model files. You need to agree to the license of the Llama model on your HuggingFace account and login using huggingface-cli login in order to use the automatic downloader.

Note

The model accepts instructions in the following format:

Editing Instruction: {instruction}. Target Image Description: {description}

Example:

Editing Instruction: Convert the image into a Ghibli style. Target Image Description: A person in a light pink t-shirt with short dark hair, depicted in a Ghibli style against a plain background.

To refine your instructions, use the provided script:

python ./instruction_refinement.py --src_image ./test.jpeg --src_instruction "convert the image into a Ghibli style"

The instruction refinement script requires a VLM API key - you can either run vllm locally or use OpenAI's API.

Gradio Demo

We also provide a Gradio demo for interactive image editing. You can run the demo with:

pytho

37 个文件

浏览文件

数据集版权信息

本数据集的许可证为 MIT License。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证