ComfyUI

ComfyUI

by

Open WebUI supports image generation through ComfyUI backend. This guide will help you set up this integration.

ComfyUI offers an alternative interface for managing and interacting with image generation models. Learn more or download it from its GitHub page. Below are the steps to set up ComfyUI alongside your existing tools.

Initial Setup

  1. Download and Extract: Download the ComfyUI software package from GitHub and extract it to your preferred directory.
  2. Start ComfyUI: Run the following command to launch ComfyUI:
   python main.py

For systems with lower VRAM, use the following command to optimize memory usage:

   python main.py --lowvram
  1. Docker Installation: For a Docker-based WebUI setup with preset environment variables, run:
   docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e COMFYUI_BASE_URL=http://host.docker.internal:7860/ -e ENABLE_IMAGE_GENERATION=True -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Setting Up Open WebUI with ComfyUI

Step 1: Configure Open WebUI Settings

  1. Go to Admin Panel in Open WebUI.
  2. Click Settings, then select the Images tab.
  3. In the Image Generation Engine field, select ComfyUI.
  4. Enter the API URL where ComfyUI is running, following this format: http://<your_comfyui_address>:8188/.
  5. Set the environment variable COMFYUI_BASE_URL to this address to ensure it’s persistent in the WebUI.

Step 2: Verify Connection and Enable Image Generation

  1. Make sure ComfyUI is running, then verify the connection with Open WebUI.
  2. Once verified, toggle Image Generation (Experimental). You’ll now see additional options.

Step 3: Configure ComfyUI Settings and Import Workflow

  1. Enable developer mode in ComfyUI by clicking the gear icon above the Queue Prompt button and enabling Dev Mode.
  2. Export the desired workflow from ComfyUI using the Save (API Format) button. This will download the file as workflow_api.json.
  3. In Open WebUI, click Click here to upload a workflow.json file, and select the workflow_api.json to import the workflow.
  4. After importing, map the ComfyUI Workflow Nodes according to the imported node IDs.

Info: You may need to adjust some Input Keys in Open WebUI’s ComfyUI Workflow Nodes to match your workflow’s node IDs (e.g., rename seed to noise_seed).

Tip: Workflows utilizing Flux models may require multiple node IDs. Separate these IDs with commas in the node entry field (e.g., 1 or 1, 2).

Click Save to apply the settings and start generating images with ComfyUI in Open WebUI!

FLUX.1 Model Setup

  1. Download Model Checkpoints: Download either the FLUX.1-schnell or FLUX.1-dev model from the Black Forest Labs HuggingFace page.
  2. Place the model checkpoints in both the models/checkpoints and models/unet directories in ComfyUI. Alternatively, you can create a symbolic link between the two directories.
  3. VAE Model: Download the ae.safetensors VAE and place it in the models/vae directory.
  4. CLIP Model: Download the clip_l.safetensors file and place it in the models/clip directory.
  5. T5XXL Model: Download either the t5xxl_fp16.safetensors or t5xxl_fp8_e4m3fn.safetensors model and place it in the models/clip directory.

By completing these steps, you’ll have successfully integrated ComfyUI with Open WebUI, enabling the use of FLUX.1 models for image generation.

Read related articles and guides: