Skip to content

Wan 2.2 First vs Second stage #12019

@vladmandic

Description

@vladmandic

Wan 2.2 consists of two transformer stages.
Diffusers currently allows to run either fully on first stage or at the boundary switch to second stage (that concept somewhat resembles old-sdxl-refiner).

However, after some testing of Wan2.2, its far more useful to be able to skip first stage and run on second stage only - and right now that is not possible in Diffusers as transformers cannot be None
It should be possible to run pipeline with either transformers or transformers_2 to none with very little effort.

Also add some error handling as right now if boundary is reached, it attempts to execute second stage regardless if its loaded or not.

Also, due to extreme memory requirements, it would be highly beneficial to be able to load second stage only when its needed, not before processing even starts. Which means there should be a break-of-a-sort when first stage completes and before switching processing to second stage - perhaps via callback?

cc @yiyixuxu @a-r-r-o-w

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions