This is not a model and has no preview. You will see the speed change in the console after installation.
Nvidia Cuda 12.2 libs + Increases processing speed on ~5-10%
Support: SDXL, 1.5
update for CUDA 12.2.2 and cuDNN 8.9.5.29 (!) (09.2023)
add and replace files in dir:
AUTOMATIC1111 - webui\venv\Lib\site-packages\torch\lib
Comfyui - python_embedded\Lib\site_packages\torch\lib
InvokeAi - InvokeAi\.venv\Lib\site-packages\torch\lib (for cuDNN only)
(You can make backup before replacing and compare before and after.)
Description
This is not a model and has no preview. You will see the speed change in the console after installation.
Nvidia Cuda 12.2 libs + Increases processing speed on ~5-10%
Support: SDXL, 1.5
update 12.2, 8.9.5.29 (09.2023)
add and replace files in dir:
webui\venv\Lib\site-packages\torch\lib
Comfyui, AUTOMATIC1111
FAQ
Comments (21)
Can you link to the original download page?
Why not download this from Nvidia Site?
File names differ from original installation (new versions), i know from experience this may prevent stuff from working.
Which one(s) are needed to copy or update from lib/bin/others into torch ?
I installed these files and checked them on my AUTOMATIC1111, then copied them from it and posted them here. This works and does not cause conflicts. If you want, you can make a backup copy.
webui\venv\Lib\site-packages\torch\lib
@WebH I've actually got the package from nvidia a few weeks ago and trying to match the two. For instance, i can't find any of the cudnn*.dll files in theirs. I'm not completely paranoid, just carefully stepping on eggs :P
It runs if put among the other ones but if i rename one of the originals, say, cublas64_11.dll to something else, torch fails to load.
@ze_thriller My archive contains several updated libraries, including cuDNN 8.9.5.29. Make a replacement with a backup copy only in directory \torch\lib
Now that's interesting. Trying to see which dll is loaded (use https://learn.microsoft.com/en-us/sysinternals/downloads/listdlls) and i got this:
> Listdlls64 python.exe | grep cublas64
0x00000000a04c0000 0x54a5000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cublas64_11.dll
0x000000008a6e0000 0x5d37000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cublas64_12.dll
> Listdlls64 python.exe | grep cudnn
0x00000000c8e90000 0x4b000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cudnn64_8.dll
0x0000000039590000 0x77df000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cudnn_adv_infer64_8.dll
0x0000000033fb0000 0x55df000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cudnn_ops_infer64_8.dll
0x000000002cfd0000 0x6fd2000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cudnn_adv_train64_8.dll
0x0000000028ca0000 0x4327000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cudnn_ops_train64_8.dll
0x00000000062f0000 0x229a4000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cudnn_cnn_infer64_8.dll
0x00000000fee30000 0x74bf000 F:\automatic1111\stable-diffusion-webui\venv\lib\site-packages\torch\lib\cudnn_cnn_train64_8.dll
TL;DR: both old --and-- new one are loaded ?
@ze_thriller cudnn is new anyway, the 12.2 library may be partially used. I just made a replacement and compared the speed results and posted it for everyone. You can check.
How about if you do submit a PR on the a1111's Github repo so that the upgrade is automatically performed upon next run from the files / dlls downloaded straight from the nvidia site? I think everybody will appreciate and love you for that :)
Can confirm it did help speed up the inferencing/generating time although for a small percentage (which is already very good considering i use an MX series card and not an RTX card)
Are you guys running this on the latest commit? Cause I am not and it slows me down by about 50%.
I use AUTOMATIC1111 1.6.0 on rtx 3060 ti. This works correctly for me.
Its working for me at A1111. I had 15.5it/s with euler A and even after patch its 17it/s.
Went from average 11.5 its to 12.2 its....so..did something but not 5-10! :) 3090!
11.5 - 100%
12.2 - X
This is an increase of 6.1%.
That is, more than 5% for you :)
@WebH When you put it like that :P I was talking about a real meaningful speed up, sure, but is it really noticeable outside of statistics and numbers in the CLI? Not really! :) But thanks for this of course, something is always something
For anyone having trust issues you can download cudnn 8.9.5.29 files from Nvidia here: https://developer.nvidia.com/rdp/cudnn-archive
Note: you will be asked to either create or login to a developer account (it's free).
Copy files from /bin folder to the /lib folder in your torch installation (as per description by OP).
The zip from OP also has cublas files (not present in the cudnn archive from NVidia). Not sure if they are needed (my Automatic1111 works just fine without them).
However, if you need them files for anything I'm not aware of you can get them from Cuda Toolkit installation archive from https://developer.nvidia.com/cuda-downloads
(it's an .exe but you can uzip it like any other archive and you find cublas files in /cublas/bin).
An important thing I discovered is that if you decide to use the CUBLAS files, you should not replace the existing ones, but just add the new ones.
For example, instead of replacing cublas64_11.dll with cublas64_12.dll, just add cublas64_12.dll.
If you remove the old ones, you will get a "Torch is not able to use GPU" error upon startup and A1111 will not run.
Updates have already been released. Please upload the new version. Thank you for your work!