SD Forge Attention Couple

This is an Extension for the Forge Webui, which allows you to generate couples target conditioning at different regions. No more color bleeds or mixed features!

Support Forge, reForge, and Forge Classic; but not Automatic1111 Webui

Support both SD1 & SDXL; but not SD3 or Flux

Showcase

• Trying to generate "jesus christ arguing with santa claus"

with Forge Couple	without Forge Couple
Distinct and separate characters	Features mixed between characters

How to Use

Important

Think of this Extension as a LLM for prompts: the defined regions are more like "suggestions" rather than strict boundaries. Don't expect the results to follow the regions with pixel-perfect precision. Additionally, the effectiveness of this Extension depends on how well the Checkpoint follows to the prompt. If the Checkpoint doesn't understand the composition, it won't generate the desired result accurately.

Tip

As shown in the various examples, even if a region only contains 1 subject, it's usually still better to prompt for the total amount of subjects first.

Index

• Basic Mode
  • Tile Direction
• Advanced Mode
• Mask Mode
• Misc.
  • Global Effect
  • Compatibility
  • Separator
  • Common Prompts
  • LoRA
• API
• FAQ

Basic Mode

The Basic mode works by dividing the image into multiple "tiles" where each tile corresponds to a "line" of the positive prompt. Therefore, simply prompt more lines to generate more regions.

2girls, blonde twintails, cyan eyes, white serafuku, standing, waving, looking at viewer, smile
2girls, black long hair, red eyes, dark school uniform, standing, crossed arms, looking away

Tile Direction

In the Basic mode (only), you can choose dividing the image into columns or rows.

• Horizontal: First / Last line corresponds to the Left / Right -most region
• Vertical: First / Last line corresponds to the Top / Bottom -most region

Vertical Direction

galaxy, stars, milky way
blue sky, clouds
sunrise, lens flare
ocean, waves
beach, sand
pavement, road

Advanced Mode

Were these automated and equally-sized tiles not sufficient for your needs? Now you can manually specify each regions!

Important

The entire image must contain weights

• Entries:
  • Each row contains a range for x axis, a range for y axis, a weight, as well as the corresponding line of prompt
    • x axis is from left to right; y axis is from top to bottom
  • The range should be within 0.0 ~ 1.0, representing the percentage of the full width/height (eg. 0.0 to 1.0 would span across the entire axis)
  • The weight is capped at 0.0 ~ 5.0
  • The prompt is only for quick reference; in case it went out of sync, the actual generation is still based on the prompt field

Note

The mappings are not sent when using the Send to img2img function, click the Pull from txt2img to manually transfer the mappings

• Control:

  • Click on a row to select it, highlighting its bounding box
    • Click on the same row again to deselect it
  • When a row is selected, click the 🆕 button above / below to insert a new row above / below
    • If holding Shift, it will also insert a new empty line to the prompts
  • When a row is selected, click the ❌ button to delete it
    • If holding Shift, it will also delete the corresponding line of prompt
  • Click the Default Mapping button to reset the mappings

• Presets:

  • You can save the current mapping data, and load them again in the future

• Draggable Region:

  • When a bounding box is highlighted, simply drag the box around to reposition the region; drag the edges / corners to resize the region

• Background:

  • Click the 📂 button to load a image as the background of the mapping
  • Click the ⏏ button to load the img2img input image as the background
  • Click the 🗑 button to clear the background

Advanced Mode UI

Generation Result

a cinematic photo of a couple, from side, outdoors
couple photo, man, black tuxedo
couple photo, woman, white dress
wedding photo, holding flower bouquet together
sunset, golden hour, lens flare

Mask Mode

Were these bounding boxes still too rigid for you...? Now you can also manually draw the areas for each regions!

Important

The entire image must contain weights

• Canvas:
  • Click the Create Empty Canvas button to generate a blank canvas to draw on
  • Only pure white (255, 255, 255) pixels count towards the mask, other colors are simply discarded
    • This also means you can use other colors as the "eraser"
  • Click the Save Mask button to save the image as a new layer of masks
  • When a layer is selected:
    • Click Load Mask to load the mask into canvas
    • Click Override Mask to save the image and override the selected layer of mask
  • Click the Reset All Masks button to clear all the data

Note

The masks are not sent when using the Send to img2img function, click the Pull from txt2img to manually transfer the masks (the weights are not sent...)

• Entries:

  • Each row contains a preview of the layer, the corresponding line of prompt, and the weight for the layer
    • The prompt is only for quick reference; in case it went out of sync, the actual generation is still based on the prompt field
  • Click the preview image to select the layer
  • Use the arrow buttons to quickly re-order the layers
  • Click the ❌ button to delete the layer

• Uploads:

  • Use the Upload Background to upload an image as reference to draw masks on
    • The image will be darkened, thus not counting towards the mask
  • Use the Upload Mask to upload an image as a mask that can directly be saved
    • Mainly for when you prepare the masks in external programs

Note

Do not draw a mask that is too small, having just a dot of conditioning doesn't do much...

Warning

For Forge Classic (Gradio 3) users, avoid pasting images. Instead manually upload or simply drag & drop the images. Using Ctrl + V might send the image to the Canvas, thus breaking the Extension...

Mask Mode UI

Generation Result

cinematic photo of a dungeon
glowing lit lamps
treasure chest

Other Parameters

These options are not limited to a specific mode

Global Effect

In Basic and Mask modes (only), you can set either the first line or the last line of the positive prompt as the "background," affecting the entire image instead of just one region. Useful for specifying styles or quality tags.

Compatibility Toggle

When this is enabled, this Extension will not function during the Hires. Fix pass to improve the compatibility with other Extensions. (Recommended)

Important

This Extension currently does not support Hires. fix prompts...

Couple Separator

By default when the field is left empty, this Extension uses the newline character (\n) as the separator to determine "lines" of the prompts. You may also specify other words as the separator instead.

• To keep the custom separator within its own line, you can add \n before and after the word

  • eg. \nMySep\n

• Examples

  • (left empty)
  • foo
  • \nbar\n
  • \n\n

Common Prompts

If you have multiple characters that share the same outfits, poses, or expressions, you can now simplify the process via Common Prompts - No more copying and pasting the same lines over and over!

0. To enable, select a syntax between { } or < > first
1. First, define an unique key (eg. cloth)
2. Second, follow up with a : (eg. cloth:)
3. Third, follow up with your common prompts (eg. cloth:t-shirt, jacket, jeans)
4. Then, surround the whole thing with your chosen brackets (eg. {cloth:t-shirt, jacket, jeans})
5. Finally, you can now use the key to recall the common prompts in other lines (ie. {cloth})

• TL;DR: If you have {foo:bar} in your prompt, every occurrence of {foo} (and the original {foo:bar}) will be replaced with bar during the generation

Important

• The key has to be unique
• You can have more than 1 common prompt at the same time
• Each bracket can only contain one key

Tip

You can enable Debug to check if it is working as intended in the console

score_9, score_8_up, score_7_up, source_anime, high quality, best quality, masterpiece, {subject:2girls}, <lora:hasunosora_pony:0.8>,
{subject}, murano sayaka, low twintails, {cloth:brown dress, white sailor collar, neckerchief, standing, arms behind back, looking at viewer, smile, blush, classroom}
{subject}, hinoshita kaho, medium hair, {cloth}

LoRA Support

LoRA that contains multiple subjects is easier to generate multiple characters. Using different LoRAs in different regions is also possible, though it depends on how well the LoRAs' concepts work together...

API

For usages with API, please refer to the Wiki

Troubleshooting

Frequently Asked Questions

• Generation gets interrupted at 1st step

  • In Forge, when raising an Error from an Extension, it only gets caught while the generation continues, leading to ForgeCouple failing "silently." To work around this, ForgeCouple now interrupts the generation when an error occurs. Check the Console logs to see what went wrong...

• Not Enough Lines in Prompt

  • In Basic mode, you need at least 2 lines of prompts for it to tile; 3 in case you enable Global Effect

• Number of Couples and Masks mismatched

  • Similarly, the number of lines in prompts should match the number of regions defined in Advanced and Mask modes

Important

Empty lines are still counted; ensure you do not leave an empty line at the end; if you want to have an empty line between each region for clarity, adjust the Couple Separator

• Image must contain weights on all pixels

  • As mentioned in Advanced and Mask, the entire image must contain weights. This error occurs when you didn't fill the whole image. The easiest way to achieve this:
    • Advanced: Create a layer that covers the entire image (ie. 0.0, 1.0, 0.0, 1.0)
    • Mask: Use the Global Effect

• Incompatible Extension

  • Certain Extensions, such as sd-dynamic-prompts, will also process the prompts before/during generation. These may break the Couple Separator and/or Common Prompts as a result.

TypeError: 'NoneType'

For users that get the following error:

RuntimeError: shape '[X, Y, 1]' is invalid for input of size Z
shape '[X, Y, 1]' is invalid for input of size Z
*** Error completing request
    ...
    Traceback (most recent call last):
        ...
        res = list(func(*args, **kwargs))
    TypeError: 'NoneType' object is not iterable

1. Go to Settings -> Optimizations, and enable Pad prompt/negative prompt
2. Set the Width and Height to multiple of 64

---

Special Thanks

• Credits to the original author, laksjdjf, whose ComfyUI Node I referenced to port into Forge