Add FluxPAGPipeline with support for PAG

[tongyu0924]

What does this PR do?

This PR adds support for Perturbed Attention Guidance (PAG) to the FluxPipeline

Fixes #11488

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[DN6]

@tongyu0924 Is this ready for review? If so, could you move the pipeline under src/diffusers/pipelines/pag please and make the changes needed to import the pipeline?

[tongyu0924]

@tongyu0924 Is this ready for review? If so, could you move the pipeline under src/diffusers/pipelines/pag please and make the changes needed to import the pipeline?

Done! The pipeline is now under src/diffusers/pipelines/pag. It's ready for review

[DN6]

Thank you @tongyu0924 👍🏽! Could we add the pipeline to the necessary init files

PAG Module:
https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/pag/__init__.py

Pipelines Module:

diffusers/src/diffusers/pipelines/__init__.py

Line 175 in 049082e

_import_structure["pag"].extend(

Diffusers main init

diffusers/src/diffusers/__init__.py

Line 341 in 049082e

"AnimateDiffPAGPipeline",

And then could you please add a fast test for the pipeline, similar to how it has been done here

diffusers/tests/pipelines/pag/test_pag_sd3.py

Line 26 in 049082e

class StableDiffusion3PAGPipelineFastTests(unittest.TestCase, PipelineTesterMixin):

[tongyu0924]

Thank you @tongyu0924 👍🏽! Could we add the pipeline to the necessary init files

PAG Module: https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/pag/__init__.py

Pipelines Module:

diffusers/src/diffusers/pipelines/__init__.py

Line 175 in 049082e

_import_structure["pag"].extend(

Diffusers main init

diffusers/src/diffusers/__init__.py

Line 341 in 049082e

"AnimateDiffPAGPipeline",

And then could you please add a fast test for the pipeline, similar to how it has been done here

diffusers/tests/pipelines/pag/test_pag_sd3.py

Line 26 in 049082e

class StableDiffusion3PAGPipelineFastTests(unittest.TestCase, PipelineTesterMixin):

I've added the pipeline to the necessary __init__.py files and added a fast test following the structure in test_pag_sd3.py.

[DN6]

Sorry for the delay here @tongyu0924. I think we just need define the correct PAG Attn Processors for this pipeline and we should be good to go 👍🏽

Thanks for your patience.

[DN6]

Could we remove this commented section please.

[DN6]

Here we would need to define a PAGFluxAttnProcessor_2_0 similar to how it is done for SD3's JointAttnProcessor

diffusers/src/diffusers/models/attention_processor.py

Line 1506 in f46abfe

class PAGJointAttnProcessor2_0:

And since the pipeline supports true CFG we would also need to add a PAGCFGFluxAttnProcessor_2_0

diffusers/src/diffusers/models/attention_processor.py

Line 1662 in f46abfe

class PAGCFGJointAttnProcessor2_0:

[nitinmukesh]

Please update the following if possible

diffusers/src/diffusers/pipelines/pag/pipeline_pag_flux.py

EXAMPLE_DOC_STRING = """
Examples:
```py
>>> import torch
>>> from diffusers import FluxPipeline

    >>> pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16)
    >>> pipe.to("cuda")
    >>> prompt = "A cat holding a sign that says hello world"
    >>> # Depending on the variant being used, the pipeline call will slightly vary.
    >>> # Refer to the pipeline documentation for more details.
    >>> image = pipe(prompt, num_inference_steps=4, guidance_scale=0.0).images[0]
    >>> image.save("flux.png")
    ```

"""