Speed comparison between Torch + CUDA + xFormers versions and TensorRT vs xFormers for Stable Diffusion XL (SDXL)
Added 2023-12-10 19:57:18 +0000 UTCJoin discord and tell me your discord username to get a special rank : SECourses Discord
Full TensorRT Tutorial is here (42 minutes, 32 chapters) : Double Your Stable Diffusion Inference Speed with RTX Acceleration TensorRT: A Comprehensive Guide
Speed comparison between Torch + CUDA + xFormers versions and TensorRT vs xFormers for Stable Diffusion XL (SDXL)
I have Automatic1111 SD Web UI to compare
1st image xFormers 23 + Torch 2.1.1 + Cuda 121
2: xFormers 22 + Torch 2.0.1 + Cuda 118
3: Torch 2.1.1 + Cuda 121 + TensorRT
4: Torch 2.0.1 + Cuda 118 + TensorRT
1152x896 pixels with 1.5x high res fix (1728x1344)
30 steps first pass
30 steps high res fix pass
Windows 10
RTX 3090 TI
When TensorRT is used, xFormers is disabled and not used
Full prompt:
Fantasy Forest with Mythical Creatures: A mystical forest filled with mythical creatures, magical trees, sharp focus, intricate, cinematic, full color, and radiant magical lights.
Steps: 30, Sampler: DPM++ 2M SDE Karras, CFG scale: 8, Seed: 1093049346, Size: 1152x896, Model hash: 0724518c6b, Model: juggernautXL_v7Rundiffusion, Denoising strength: 0.7, Hypertile U-Net: True, Hypertile U-Net max depth: 1, Hypertile VAE: True, Hires upscale: 1.5, Hires upscaler: Latent, Version: v1.7.0-RC-5-gf92d6149
Comments
great
Furkan Gözükara
2024-01-04 21:17:05 +0000 UTCyes both - I have to update all of my tensorRT again but that is ok. Hopefully, I won't have to update again in awhile.
Ec Jep
2024-01-04 21:16:33 +0000 UTCok now working right?
Furkan Gözükara
2024-01-04 21:15:12 +0000 UTCon my fresh install: now my settings are: version: v1.7.0 • python: 3.10.11 • torch: 2.1.2+cu121 • xformers: 0.0.23.post1 • gradio: 3.41.2 • checkpoint: 6ce0161689 on my original SD install: I performed these steps with error above and it now works.. uninstalled torch from .bat commandline, installed pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121, and updated xformers from 0.0.23 to 0.0.23.post1, deleted tensorRT extension and reinstalled with your install .bat file and it works again latest config: version: v1.7.0-285-g15ec54dd • python: 3.10.11 • torch: 2.1.2+cu121 • xformers: 0.0.23.post1 • gradio: 3.41.2 • checkpoint: a58017353f
Ec Jep
2024-01-04 20:15:20 +0000 UTCbasically I want to install the latest version of torch and xformers to get latest speed improvements. I was using tensorRT successfully and it really helps - I don't know how I missed the patreon update a few months ago..
Ec Jep
2023-12-11 17:19:09 +0000 UTCtrying to do similar comparison and ran into torch update error. I was on xformers 0.0.20 and torch 2.0.1. I updated xformers to 0.0.23 with pip install xformers==0.0.23 and it also update torch to 2.1.1 After install I get this error "RuntimeError: Torch is not able to use GPU; add --skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check" Any suggestions to how to perform these updates and ensure the proper dependencies? Thx
Ec Jep
2023-12-11 17:15:06 +0000 UTC
