Open
Description
Describe the bug
FluxPipeline fails to load LoRA weights (https://civitai.com/models/876464):
Reproduction
from diffusers import FluxPipeline
from torch import bfloat16
prompt_from_repo = "Beautiful female woodelf in shiny iron and silver armor, full body view, full body shot, feet and arms seen in the view, hyper photorealistic, ultra photorealistic, looking straight into camera, Elf, pointy ears, photorealistic, fighting pose, shoulder length wavy red hair, full body, athletic figure, beautiful green eyes, 8k, ultra realistic, hyper realistic, highly detailed, daylight, fantasy style, high contrast, colorful polychromatic, (Intricately Designed face: 1.3), dramatic lighting, wet and sweaty skin, dirty skin, Cinematic colorful lighting, realistic body and face, Photograph Taken on Nikon D750, Intricate, Elegant, Digital Illustration, Scenic, Hyper-Realistic, Unreal Engine, CryEngine, Octane Render, Artgerm, WLOP, Greg Rutkowski, 8k ultra high resolution concept art, hyper-defined, sharp focus, echoing, (hyperdetailed body and face: 1.3), asymmetric balance, Stunning and amazing.,Acrylian4,Lineart style:Charcoal lineart and white chalk lineart,Tannis, Tanis, Esmeralda , piercing multicolor colorful eye color,black ink lineart, dynamic pose, Amazing artwork, a Masterpiece painting"
negative_prompt_from_repo = "score_6,score_5,score_4, , poor artist, bad artist, ugly, poorly drawn, poorly detailed, warped, morphed, uncanny, horror, disfigured, score_4, score_5, score_6, source_furry, wrong hand, bad hands, bad anatomy, fewer digits, bad perspective, bad proportions, bad arm, extra arms, cross-eyed, logo, censored, blurry, lowres, artistic error, watermark, jaggy lines, deformed, monochrome, bad eyes embedding:EasyNegative"
lora_id = "D1-3105/lora_876464__issue11659__DEV-2308"
model_id = "black-forest-labs/FLUX.1-dev"
pipeline = FluxPipeline.from_pretrained(model_id, cache_dir="train_model/huggingface", torch_dtype=bfloat16).to("cuda")
pipeline.load_lora_weights(lora_id)
assert pipeline.get_active_adapters()
image = pipeline(
prompt=prompt_from_repo,
negative_prompt=negative_prompt_from_repo,
width = 768,
height = 1152,
seed=442779123,
steps=25,
guidance_scale=25,
).images[0]
Logs
No LoRA keys associated to FluxTransformer2DModel found with the prefix='transformer'. This is safe to ignore if LoRA state dict didn't originally have any FluxTransformer2DModel related params. You can also try specifying `prefix=None` to resolve the warning. Otherwise, open an issue if you think it's unexpected: https://github.com/huggingface/diffusers/issues/new
No LoRA keys associated to CLIPTextModel found with the prefix='text_encoder'. This is safe to ignore if LoRA state dict didn't originally have any CLIPTextModel related params. You can also try specifying `prefix=None` to resolve the warning. Otherwise, open an issue if you think it's unexpected: https://github.com/huggingface/diffusers/issues/new
System Info
diffusers==0.33.1