📂 Connecting to Google Drive...
Mounted at /content/drive

💿 Checking dataset...
📁MyDrive/Loras/sed/dataset
📈 Found 404 images with 2 repeats, equaling 808 steps.
📉 Divide 808 steps by 4 batch size to get 202.0 steps per epoch.
🔮 There will be 20 epochs, for around 4040 total training steps.

🏭 Installing trainer...

59 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: Skipping acquire of configured file 'main/source/Sources' as repository 'https://r2u.stat.illinois.edu/ubuntu jammy InRelease' does not seem to provide it (sources.list entry misspelt?)
The following additional packages will be installed:
  libaria2-0 libc-ares2 python3-pip-whl python3-setuptools-whl
The following NEW packages will be installed:
  aria2 libaria2-0 libc-ares2 python3-pip-whl python3-setuptools-whl
  python3.10-venv
0 upgraded, 6 newly installed, 0 to remove and 59 not upgraded.
Need to get 3,994 kB of archives.
After this operation, 8,194 kB of additional disk space will be used.
Selecting previously unselected package libc-ares2:amd64.
(Reading database ... 122363 files and directories currently installed.)
Preparing to unpack .../0-libc-ares2_1.18.1-1ubuntu0.22.04.3_amd64.deb ...
Unpacking libc-ares2:amd64 (1.18.1-1ubuntu0.22.04.3) ...
Selecting previously unselected package libaria2-0:amd64.
Preparing to unpack .../1-libaria2-0_1.36.0-1_amd64.deb ...
Unpacking libaria2-0:amd64 (1.36.0-1) ...
Selecting previously unselected package aria2.
Preparing to unpack .../2-aria2_1.36.0-1_amd64.deb ...
Unpacking aria2 (1.36.0-1) ...
Selecting previously unselected package python3-pip-whl.
Preparing to unpack .../3-python3-pip-whl_22.0.2+dfsg-1ubuntu0.7_all.deb ...
Unpacking python3-pip-whl (22.0.2+dfsg-1ubuntu0.7) ...
Selecting previously unselected package python3-setuptools-whl.
Preparing to unpack .../4-python3-setuptools-whl_68.1.2-2~jammy3_all.deb ...
Unpacking python3-setuptools-whl (68.1.2-2~jammy3) ...
Selecting previously unselected package python3.10-venv.
Preparing to unpack .../5-python3.10-venv_3.10.12-1~22.04.15_amd64.deb ...
Unpacking python3.10-venv (3.10.12-1~22.04.15) ...
Setting up python3-setuptools-whl (68.1.2-2~jammy3) ...
Setting up python3-pip-whl (22.0.2+dfsg-1ubuntu0.7) ...
Setting up libc-ares2:amd64 (1.18.1-1ubuntu0.22.04.3) ...
Setting up libaria2-0:amd64 (1.36.0-1) ...
Setting up python3.10-venv (3.10.12-1~22.04.15) ...
Setting up aria2 (1.36.0-1) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.8) ...
/sbin/ldconfig.real: /usr/local/lib/libtbbbind.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero_v2.so.0 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc.so.2 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc_proxy.so.2 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtcm_debug.so.1 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libur_adapter_level_zero.so.0 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtcm.so.1 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libumf.so.1 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libur_adapter_opencl.so.0 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbb.so.12 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_5.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_0.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libur_loader.so.0 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libhwloc.so.15 is not a symbolic link

Cloning into '/content/trainer'...
remote: Enumerating objects: 388, done.
remote: Total 388 (delta 0), reused 0 (delta 0), pack-reused 388 (from 1)
Receiving objects: 100% (388/388), 7.80 MiB | 6.36 MiB/s, done.
Resolving deltas: 100% (223/223), done.
HEAD is now at 47ef6ed added installer for linux python 3.12 added a log-like output of the warmup steps in the console
Submodule 'sd_scripts' (https://github.com/kohya-ss/sd-scripts) registered for path 'sd_scripts'
Cloning into '/content/trainer/sd_scripts'...
Submodule path 'sd_scripts': checked out 'e89653975ddf429cdf0c0fd268da0a5a3e8dba1f'
creating venv and installing requirements
Looking in indexes: https://download.pytorch.org/whl/cu124
Collecting torch==2.5.1
  Downloading https://download-r2.pytorch.org/whl/cu124/torch-2.5.1%2Bcu124-cp310-cp310-linux_x86_64.whl (908.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 908.3/908.3 MB 1.6 MB/s eta 0:00:00
Collecting torchvision==0.20.1
  Downloading https://download-r2.pytorch.org/whl/cu124/torchvision-0.20.1%2Bcu124-cp310-cp310-linux_x86_64.whl (7.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.3/7.3 MB 87.4 MB/s eta 0:00:00
Collecting triton==3.1.0
  Downloading https://download-r2.pytorch.org/whl/triton-3.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (209.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.5/209.5 MB 6.8 MB/s eta 0:00:00
Collecting nvidia-cuda-cupti-cu12==12.4.127
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (13.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.8/13.8 MB 101.1 MB/s eta 0:00:00
Collecting nvidia-cudnn-cu12==9.1.0.70
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl (664.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 664.8/664.8 MB 1.0 MB/s eta 0:00:00
Collecting nvidia-nvjitlink-cu12==12.4.127
  Downloading https://download.pytorch.org/whl/cu124/nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (21.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.1/21.1 MB 69.6 MB/s eta 0:00:00
Collecting filelock
  Downloading filelock-3.29.0-py3-none-any.whl (39 kB)
Collecting nvidia-curand-cu12==10.3.5.147
  Downloading https://download.pytorch.org/whl/cu124/nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl (56.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.3/56.3 MB 11.4 MB/s eta 0:00:00
Collecting nvidia-nvtx-cu12==12.4.127
  Downloading https://download.pytorch.org/whl/cu124/nvidia_nvtx_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (99 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.1/99.1 KB 17.6 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu12==12.4.127
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (883 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 883.7/883.7 KB 72.0 MB/s eta 0:00:00
Collecting fsspec
  Downloading fsspec-2026.4.0-py3-none-any.whl (203 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 203.4/203.4 KB 9.4 MB/s eta 0:00:00
Collecting nvidia-nccl-cu12==2.21.5
  Downloading https://download.pytorch.org/whl/nvidia_nccl_cu12-2.21.5-py3-none-manylinux2014_x86_64.whl (188.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 188.7/188.7 MB 6.6 MB/s eta 0:00:00
Collecting jinja2
  Downloading https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl (134 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.9/134.9 KB 20.2 MB/s eta 0:00:00
Collecting sympy==1.13.1
  Downloading sympy-1.13.1-py3-none-any.whl (6.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.2/6.2 MB 92.3 MB/s eta 0:00:00
Collecting typing-extensions>=4.8.0
  Downloading https://download.pytorch.org/whl/typing_extensions-4.15.0-py3-none-any.whl (44 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.6/44.6 KB 6.7 MB/s eta 0:00:00
Collecting networkx
  Downloading networkx-3.4.2-py3-none-any.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 87.2 MB/s eta 0:00:00
Collecting nvidia-cusparse-cu12==12.3.1.170
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl (207.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 207.5/207.5 MB 5.9 MB/s eta 0:00:00
Collecting nvidia-cusolver-cu12==11.6.1.9
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl (127.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 127.9/127.9 MB 7.9 MB/s eta 0:00:00
Collecting nvidia-cublas-cu12==12.4.5.8
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl (363.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 363.4/363.4 MB 2.3 MB/s eta 0:00:00
Collecting nvidia-cufft-cu12==11.2.1.3
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl (211.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 211.5/211.5 MB 6.0 MB/s eta 0:00:00
Collecting nvidia-cuda-nvrtc-cu12==12.4.127
  Downloading https://download.pytorch.org/whl/cu124/nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (24.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 24.6/24.6 MB 50.2 MB/s eta 0:00:00
Collecting pillow!=8.3.*,>=5.3.0
  Downloading pillow-12.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (7.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 124.2 MB/s eta 0:00:00
Collecting numpy
  Downloading numpy-2.2.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.8/16.8 MB 83.9 MB/s eta 0:00:00
Collecting mpmath<1.4,>=1.1.0
  Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 KB 3.9 MB/s eta 0:00:00
Collecting MarkupSafe>=2.0
  Downloading https://download.pytorch.org/whl/markupsafe-3.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (20 kB)
Installing collected packages: mpmath, typing-extensions, sympy, pillow, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, MarkupSafe, fsspec, filelock, triton, nvidia-cusparse-cu12, nvidia-cudnn-cu12, jinja2, nvidia-cusolver-cu12, torch, torchvision
Successfully installed MarkupSafe-3.0.3 filelock-3.29.0 fsspec-2026.4.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.4.2 numpy-2.2.6 nvidia-cublas-cu12-12.4.5.8 nvidia-cuda-cupti-cu12-12.4.127 nvidia-cuda-nvrtc-cu12-12.4.127 nvidia-cuda-runtime-cu12-12.4.127 nvidia-cudnn-cu12-9.1.0.70 nvidia-cufft-cu12-11.2.1.3 nvidia-curand-cu12-10.3.5.147 nvidia-cusolver-cu12-11.6.1.9 nvidia-cusparse-cu12-12.3.1.170 nvidia-nccl-cu12-2.21.5 nvidia-nvjitlink-cu12-12.4.127 nvidia-nvtx-cu12-12.4.127 pillow-12.2.0 sympy-1.13.1 torch-2.5.1+cu124 torchvision-0.20.1+cu124 triton-3.1.0 typing-extensions-4.15.0
Looking in indexes: https://download.pytorch.org/whl/cu124
Collecting xformers==0.0.29.post1
  Downloading https://download.pytorch.org/whl/cu124/xformers-0.0.29.post1-cp310-cp310-manylinux_2_28_x86_64.whl (44.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.1/44.1 MB 12.3 MB/s eta 0:00:00
Requirement already satisfied: torch==2.5.1 in ./venv/lib/python3.10/site-packages (from xformers==0.0.29.post1) (2.5.1+cu124)
Requirement already satisfied: numpy in ./venv/lib/python3.10/site-packages (from xformers==0.0.29.post1) (2.2.6)
Requirement already satisfied: fsspec in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (2026.4.0)
Requirement already satisfied: networkx in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (3.4.2)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127)
Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.5.8)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (9.1.0.70)
Requirement already satisfied: filelock in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (3.29.0)
Requirement already satisfied: jinja2 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (3.1.6)
Requirement already satisfied: sympy==1.13.1 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (1.13.1)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (2.21.5)
Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (10.3.5.147)
Requirement already satisfied: triton==3.1.0 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (3.1.0)
Requirement already satisfied: typing-extensions>=4.8.0 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (4.15.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127)
Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (11.2.1.3)
Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (11.6.1.9)
Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.3.1.170)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch==2.5.1->xformers==0.0.29.post1) (12.4.127)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./venv/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.1->xformers==0.0.29.post1) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./venv/lib/python3.10/site-packages (from jinja2->torch==2.5.1->xformers==0.0.29.post1) (3.0.3)
Installing collected packages: xformers
Successfully installed xformers-0.0.29.post1
Obtaining file:///content/trainer/sd_scripts (from -r requirements.txt (line 45))
  Preparing metadata (setup.py) ... done
Collecting accelerate==0.33.0
  Downloading accelerate-0.33.0-py3-none-any.whl (315 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 315.1/315.1 KB 6.1 MB/s eta 0:00:00
Collecting transformers==4.44.0
  Downloading transformers-4.44.0-py3-none-any.whl (9.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.5/9.5 MB 94.0 MB/s eta 0:00:00
Collecting diffusers[torch]==0.25.0
  Downloading diffusers-0.25.0-py3-none-any.whl (1.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 63.1 MB/s eta 0:00:00
Collecting ftfy==6.1.1
  Downloading ftfy-6.1.1-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.1/53.1 KB 9.4 MB/s eta 0:00:00
Collecting opencv-python==4.8.1.78
  Downloading opencv_python-4.8.1.78-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.7/61.7 MB 11.5 MB/s eta 0:00:00
Collecting einops==0.7.0
  Downloading einops-0.7.0-py3-none-any.whl (44 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.6/44.6 KB 8.0 MB/s eta 0:00:00
Collecting pytorch-lightning==1.9.0
  Downloading pytorch_lightning-1.9.0-py3-none-any.whl (825 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 825.8/825.8 KB 69.6 MB/s eta 0:00:00
Collecting bitsandbytes==0.44.0
  Downloading bitsandbytes-0.44.0-py3-none-manylinux_2_24_x86_64.whl (122.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 122.4/122.4 MB 8.4 MB/s eta 0:00:00
Collecting prodigyopt==1.0
  Downloading prodigyopt-1.0-py3-none-any.whl (5.5 kB)
Collecting lion-pytorch==0.0.6
  Downloading lion_pytorch-0.0.6-py3-none-any.whl (4.2 kB)
Collecting schedulefree==1.4
  Downloading schedulefree-1.4.tar.gz (22 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tensorboard
  Downloading tensorboard-2.20.0-py3-none-any.whl (5.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.5/5.5 MB 121.7 MB/s eta 0:00:00
Collecting safetensors==0.4.4
  Downloading safetensors-0.4.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (435 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 435.5/435.5 KB 53.0 MB/s eta 0:00:00
Collecting altair==4.2.2
  Downloading altair-4.2.2-py3-none-any.whl (813 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 813.6/813.6 KB 70.9 MB/s eta 0:00:00
Collecting easygui==0.98.3
  Downloading easygui-0.98.3-py2.py3-none-any.whl (92 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.7/92.7 KB 19.4 MB/s eta 0:00:00
Collecting toml==0.10.2
  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting voluptuous==0.13.1
  Downloading voluptuous-0.13.1-py3-none-any.whl (29 kB)
Collecting huggingface-hub==0.24.5
  Downloading huggingface_hub-0.24.5-py3-none-any.whl (417 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 417.5/417.5 KB 54.1 MB/s eta 0:00:00
Collecting imagesize==1.4.1
  Downloading imagesize-1.4.1-py2.py3-none-any.whl (8.8 kB)
Collecting rich==13.7.0
  Downloading rich-13.7.0-py3-none-any.whl (240 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.6/240.6 KB 39.6 MB/s eta 0:00:00
Collecting sentencepiece==0.2.0
  Downloading sentencepiece-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 100.2 MB/s eta 0:00:00
Collecting psutil
  Downloading psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl (155 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.6/155.6 KB 28.5 MB/s eta 0:00:00
Collecting pyyaml
  Downloading pyyaml-6.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (770 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 770.3/770.3 KB 74.8 MB/s eta 0:00:00
Requirement already satisfied: torch>=1.10.0 in ./venv/lib/python3.10/site-packages (from accelerate==0.33.0->-r requirements.txt (line 1)) (2.5.1+cu124)
Collecting numpy<2.0.0,>=1.17
  Downloading numpy-1.26.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 81.4 MB/s eta 0:00:00
Collecting packaging>=20.0
  Using cached packaging-26.2-py3-none-any.whl (100 kB)
Collecting tokenizers<0.20,>=0.19
  Downloading tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 82.5 MB/s eta 0:00:00
Collecting regex!=2019.12.17
  Downloading regex-2026.5.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (794 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 794.1/794.1 KB 55.7 MB/s eta 0:00:00
Collecting requests
  Downloading requests-2.34.2-py3-none-any.whl (73 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 73.1/73.1 KB 8.9 MB/s eta 0:00:00
Requirement already satisfied: filelock in ./venv/lib/python3.10/site-packages (from transformers==4.44.0->-r requirements.txt (line 2)) (3.29.0)
Collecting tqdm>=4.27
  Downloading tqdm-4.67.3-py3-none-any.whl (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.4/78.4 KB 11.3 MB/s eta 0:00:00
Collecting importlib-metadata
  Downloading importlib_metadata-9.0.0-py3-none-any.whl (27 kB)
Requirement already satisfied: Pillow in ./venv/lib/python3.10/site-packages (from diffusers[torch]==0.25.0->-r requirements.txt (line 3)) (12.2.0)
Collecting wcwidth>=0.2.5
  Downloading wcwidth-0.7.0-py3-none-any.whl (110 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110.8/110.8 KB 20.8 MB/s eta 0:00:00
Requirement already satisfied: fsspec[http]>2021.06.0 in ./venv/lib/python3.10/site-packages (from pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (2026.4.0)
Collecting torchmetrics>=0.7.0
  Downloading torchmetrics-1.9.0-py3-none-any.whl (983 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 983.4/983.4 KB 90.2 MB/s eta 0:00:00
Collecting lightning-utilities>=0.4.2
  Downloading lightning_utilities-0.15.3-py3-none-any.whl (31 kB)
Requirement already satisfied: typing-extensions>=4.0.0 in ./venv/lib/python3.10/site-packages (from pytorch-lightning==1.9.0->-r requirements.txt (line 8)) (4.15.0)
Collecting toolz
  Downloading toolz-1.1.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.1/58.1 KB 10.2 MB/s eta 0:00:00
Collecting jsonschema>=3.0
  Downloading jsonschema-4.26.0-py3-none-any.whl (90 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90.6/90.6 KB 16.4 MB/s eta 0:00:00
Collecting entrypoints
  Downloading entrypoints-0.4-py3-none-any.whl (5.3 kB)
Collecting pandas>=0.18
  Downloading pandas-2.3.3-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (12.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 96.3 MB/s eta 0:00:00
Requirement already satisfied: jinja2 in ./venv/lib/python3.10/site-packages (from altair==4.2.2->-r requirements.txt (line 16)) (3.1.6)
Collecting markdown-it-py>=2.2.0
  Downloading markdown_it_py-4.2.0-py3-none-any.whl (91 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 91.7/91.7 KB 17.0 MB/s eta 0:00:00
Collecting pygments<3.0.0,>=2.13.0
  Downloading pygments-2.20.0-py3-none-any.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 96.7 MB/s eta 0:00:00
Collecting werkzeug>=1.0.1
  Downloading werkzeug-3.1.8-py3-none-any.whl (226 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 226.5/226.5 KB 37.6 MB/s eta 0:00:00
Collecting grpcio>=1.48.2
  Downloading grpcio-1.80.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (6.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 124.9 MB/s eta 0:00:00
Collecting markdown>=2.6.8
  Downloading markdown-3.10.2-py3-none-any.whl (108 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 108.2/108.2 KB 20.9 MB/s eta 0:00:00
Collecting absl-py>=0.4
  Downloading absl_py-2.4.0-py3-none-any.whl (135 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.8/135.8 KB 23.3 MB/s eta 0:00:00
Requirement already satisfied: setuptools>=41.0.0 in ./venv/lib/python3.10/site-packages (from tensorboard->-r requirements.txt (line 13)) (68.1.2)
Collecting tensorboard-data-server<0.8.0,>=0.7.0
  Downloading tensorboard_data_server-0.7.2-py3-none-manylinux_2_31_x86_64.whl (6.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 125.1 MB/s eta 0:00:00
Collecting protobuf!=4.24.0,>=3.19.6
  Downloading protobuf-7.35.0-cp310-abi3-manylinux2014_x86_64.whl (327 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 327.1/327.1 KB 25.9 MB/s eta 0:00:00
Collecting aiohttp!=4.0.0a0,!=4.0.0a1
  Downloading aiohttp-3.13.5-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 82.0 MB/s eta 0:00:00
Collecting referencing>=0.28.4
  Downloading referencing-0.37.0-py3-none-any.whl (26 kB)
Collecting jsonschema-specifications>=2023.03.6
  Downloading jsonschema_specifications-2025.9.1-py3-none-any.whl (18 kB)
Collecting rpds-py>=0.25.0
  Downloading rpds_py-0.30.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (390 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 390.5/390.5 KB 49.1 MB/s eta 0:00:00
Collecting attrs>=22.2.0
  Downloading attrs-26.1.0-py3-none-any.whl (67 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.5/67.5 KB 12.8 MB/s eta 0:00:00
Collecting mdurl~=0.1
  Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting python-dateutil>=2.8.2
  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 229.9/229.9 KB 34.8 MB/s eta 0:00:00
Collecting pytz>=2020.1
  Downloading pytz-2026.2-py2.py3-none-any.whl (510 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 510.1/510.1 KB 36.5 MB/s eta 0:00:00
Collecting tzdata>=2022.7
  Downloading tzdata-2026.2-py2.py3-none-any.whl (349 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 349.3/349.3 KB 47.5 MB/s eta 0:00:00
Requirement already satisfied: triton==3.1.0 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (3.1.0)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (9.1.0.70)
Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (11.2.1.3)
Requirement already satisfied: sympy==1.13.1 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (1.13.1)
Requirement already satisfied: networkx in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (3.4.2)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (2.21.5)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127)
Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (11.6.1.9)
Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.5.8)
Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.3.1.170)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127)
Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (10.3.5.147)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (12.4.127)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./venv/lib/python3.10/site-packages (from sympy==1.13.1->torch>=1.10.0->accelerate==0.33.0->-r requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: markupsafe>=2.1.1 in ./venv/lib/python3.10/site-packages (from werkzeug>=1.0.1->tensorboard->-r requirements.txt (line 13)) (3.0.3)
Collecting zipp>=3.20
  Downloading zipp-4.1.0-py3-none-any.whl (10 kB)
Collecting urllib3<3,>=1.26
  Downloading urllib3-2.7.0-py3-none-any.whl (131 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 131.1/131.1 KB 21.9 MB/s eta 0:00:00
Collecting charset_normalizer<4,>=2
  Downloading charset_normalizer-3.4.7-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (216 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 216.9/216.9 KB 35.0 MB/s eta 0:00:00
Collecting idna<4,>=2.5
  Downloading idna-3.15-py3-none-any.whl (72 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.3/72.3 KB 13.5 MB/s eta 0:00:00
Collecting certifi>=2023.5.7
  Downloading certifi-2026.5.20-py3-none-any.whl (134 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.1/134.1 KB 24.5 MB/s eta 0:00:00
Collecting yarl<2.0,>=1.17.0
  Downloading yarl-1.24.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (106 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.7/106.7 KB 18.1 MB/s eta 0:00:00
Collecting aiohappyeyeballs>=2.5.0
  Downloading aiohappyeyeballs-2.6.2-py3-none-any.whl (15 kB)
Collecting async-timeout<6.0,>=4.0
  Downloading async_timeout-5.0.1-py3-none-any.whl (6.2 kB)
Collecting frozenlist>=1.1.1
  Downloading frozenlist-1.8.0-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (219 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 219.5/219.5 KB 31.9 MB/s eta 0:00:00
Collecting aiosignal>=1.4.0
  Downloading aiosignal-1.4.0-py3-none-any.whl (7.5 kB)
Collecting multidict<7.0,>=4.5
  Downloading multidict-6.7.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (243 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 243.3/243.3 KB 39.1 MB/s eta 0:00:00
Collecting propcache>=0.2.0
  Downloading propcache-0.5.2-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.1/60.1 KB 10.7 MB/s eta 0:00:00
Collecting six>=1.5
  Downloading six-1.17.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: schedulefree
  Building wheel for schedulefree (pyproject.toml) ... done
  Created wheel for schedulefree: filename=schedulefree-1.4-py3-none-any.whl size=39333 sha256=32eb21d3392e366de7e1fb1b2208f3175b36302a1ca6a3dcc2a6e41f90d8e70c
  Stored in directory: /root/.cache/pip/wheels/04/4f/8e/15e0ac6c13041e94078d9e4f88d14c2cd00ec198bc0fc78e91
Successfully built schedulefree
Installing collected packages: voluptuous, sentencepiece, pytz, library, easygui, zipp, werkzeug, wcwidth, urllib3, tzdata, tqdm, toolz, toml, tensorboard-data-server, six, schedulefree, safetensors, rpds-py, regex, pyyaml, pygments, psutil, protobuf, propcache, prodigyopt, packaging, numpy, multidict, mdurl, markdown, imagesize, idna, grpcio, frozenlist, entrypoints, einops, charset_normalizer, certifi, attrs, async-timeout, aiohappyeyeballs, absl-py, yarl, tensorboard, requests, referencing, python-dateutil, opencv-python, markdown-it-py, lightning-utilities, importlib-metadata, ftfy, aiosignal, rich, pandas, jsonschema-specifications, huggingface-hub, aiohttp, torchmetrics, tokenizers, lion-pytorch, jsonschema, diffusers, bitsandbytes, accelerate, transformers, pytorch-lightning, altair
  Running setup.py develop for library
  Attempting uninstall: numpy
    Found existing installation: numpy 2.2.6
    Uninstalling numpy-2.2.6:
      Successfully uninstalled numpy-2.2.6
Successfully installed absl-py-2.4.0 accelerate-0.33.0 aiohappyeyeballs-2.6.2 aiohttp-3.13.5 aiosignal-1.4.0 altair-4.2.2 async-timeout-5.0.1 attrs-26.1.0 bitsandbytes-0.44.0 certifi-2026.5.20 charset_normalizer-3.4.7 diffusers-0.25.0 easygui-0.98.3 einops-0.7.0 entrypoints-0.4 frozenlist-1.8.0 ftfy-6.1.1 grpcio-1.80.0 huggingface-hub-0.24.5 idna-3.15 imagesize-1.4.1 importlib-metadata-9.0.0 jsonschema-4.26.0 jsonschema-specifications-2025.9.1 library-0.0.0 lightning-utilities-0.15.3 lion-pytorch-0.0.6 markdown-3.10.2 markdown-it-py-4.2.0 mdurl-0.1.2 multidict-6.7.1 numpy-1.26.4 opencv-python-4.8.1.78 packaging-26.2 pandas-2.3.3 prodigyopt-1.0 propcache-0.5.2 protobuf-7.35.0 psutil-7.2.2 pygments-2.20.0 python-dateutil-2.9.0.post0 pytorch-lightning-1.9.0 pytz-2026.2 pyyaml-6.0.3 referencing-0.37.0 regex-2026.5.9 requests-2.34.2 rich-13.7.0 rpds-py-0.30.0 safetensors-0.4.4 schedulefree-1.4 sentencepiece-0.2.0 six-1.17.0 tensorboard-2.20.0 tensorboard-data-server-0.7.2 tokenizers-0.19.1 toml-0.10.2 toolz-1.1.0 torchmetrics-1.9.0 tqdm-4.67.3 transformers-4.44.0 tzdata-2026.2 urllib3-2.7.0 voluptuous-0.13.1 wcwidth-0.7.0 werkzeug-3.1.8 yarl-1.24.2 zipp-4.1.0
Processing /content/trainer/custom_scheduler
  Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for LoraEasyCustomOptimizer, since package 'wheel' is not installed.
Installing collected packages: LoraEasyCustomOptimizer
  Running setup.py install for LoraEasyCustomOptimizer ... done
Successfully installed LoraEasyCustomOptimizer-1.0.0
Collecting starlette
  Downloading starlette-1.0.0-py3-none-any.whl (72 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.7/72.7 KB 2.8 MB/s eta 0:00:00
Collecting uvicorn[standard]
  Downloading uvicorn-0.47.0-py3-none-any.whl (71 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.3/71.3 KB 13.0 MB/s eta 0:00:00
Requirement already satisfied: requests in ./venv/lib/python3.10/site-packages (from -r ../requirements.txt (line 3)) (2.34.2)
Collecting dadaptation
  Downloading dadaptation-3.2.tar.gz (13 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting wandb
  Downloading wandb-0.27.0-py3-none-manylinux_2_28_x86_64.whl (27.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 27.3/27.3 MB 20.1 MB/s eta 0:00:00
Collecting pyngrok
  Downloading pyngrok-8.1.2-py3-none-any.whl (25 kB)
Collecting pycloudflared
  Downloading pycloudflared-0.3.0-py3-none-any.whl (6.7 kB)
Collecting scipy
  Downloading scipy-1.15.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (37.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 37.7/37.7 MB 22.2 MB/s eta 0:00:00
Collecting came-pytorch
  Downloading came_pytorch-0.1.3-py3-none-any.whl (6.2 kB)
Collecting lycoris-lora==3.0.1.dev10
  Downloading lycoris_lora-3.0.1.dev10.tar.gz (51 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.8/51.8 KB 9.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Collecting pytorch_optimizer==3.1.2
  Downloading pytorch_optimizer-3.1.2-py3-none-any.whl (187 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 187.2/187.2 KB 29.5 MB/s eta 0:00:00
Collecting wheel
  Using cached wheel-0.47.0-py3-none-any.whl (32 kB)
Requirement already satisfied: einops in ./venv/lib/python3.10/site-packages (from lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (0.7.0)
Requirement already satisfied: toml in ./venv/lib/python3.10/site-packages (from lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (0.10.2)
Requirement already satisfied: torch in ./venv/lib/python3.10/site-packages (from lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (2.5.1+cu124)
Requirement already satisfied: tqdm in ./venv/lib/python3.10/site-packages (from lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (4.67.3)
Requirement already satisfied: numpy in ./venv/lib/python3.10/site-packages (from pytorch_optimizer==3.1.2->-r ../requirements.txt (line 11)) (1.26.4)
Collecting anyio<5,>=3.6.2
  Downloading anyio-4.13.0-py3-none-any.whl (114 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.4/114.4 KB 19.2 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=4.10.0 in ./venv/lib/python3.10/site-packages (from starlette->-r ../requirements.txt (line 1)) (4.15.0)
Collecting h11>=0.8
  Downloading h11-0.16.0-py3-none-any.whl (37 kB)
Collecting click>=7.0
  Downloading click-8.4.0-py3-none-any.whl (116 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 116.1/116.1 KB 16.7 MB/s eta 0:00:00
Collecting websockets>=10.4
  Downloading websockets-16.0-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (183 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 183.8/183.8 KB 25.5 MB/s eta 0:00:00
Collecting python-dotenv>=0.13
  Downloading python_dotenv-1.2.2-py3-none-any.whl (22 kB)
Requirement already satisfied: pyyaml>=5.1 in ./venv/lib/python3.10/site-packages (from uvicorn[standard]->-r ../requirements.txt (line 2)) (6.0.3)
Collecting uvloop>=0.15.1
  Downloading uvloop-0.22.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (3.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 90.0 MB/s eta 0:00:00
Collecting httptools>=0.6.3
  Downloading httptools-0.7.1-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (440 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 440.9/440.9 KB 48.8 MB/s eta 0:00:00
Collecting watchfiles>=0.20
  Downloading watchfiles-1.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (457 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 457.1/457.1 KB 51.3 MB/s eta 0:00:00
Requirement already satisfied: charset_normalizer<4,>=2 in ./venv/lib/python3.10/site-packages (from requests->-r ../requirements.txt (line 3)) (3.4.7)
Requirement already satisfied: urllib3<3,>=1.26 in ./venv/lib/python3.10/site-packages (from requests->-r ../requirements.txt (line 3)) (2.7.0)
Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.10/site-packages (from requests->-r ../requirements.txt (line 3)) (3.15)
Requirement already satisfied: certifi>=2023.5.7 in ./venv/lib/python3.10/site-packages (from requests->-r ../requirements.txt (line 3)) (2026.5.20)
Requirement already satisfied: packaging in ./venv/lib/python3.10/site-packages (from wandb->-r ../requirements.txt (line 5)) (26.2)
Collecting platformdirs
  Downloading platformdirs-4.9.6-py3-none-any.whl (21 kB)
Collecting gitpython!=3.1.29,>=1.0.0
  Downloading gitpython-3.1.50-py3-none-any.whl (212 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 212.5/212.5 KB 26.9 MB/s eta 0:00:00
Requirement already satisfied: protobuf!=5.28.0,!=5.29.0,<8,>4.21.0 in ./venv/lib/python3.10/site-packages (from wandb->-r ../requirements.txt (line 5)) (7.35.0)
Collecting pydantic<3,>=2.6
  Downloading pydantic-2.13.4-py3-none-any.whl (472 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 472.3/472.3 KB 16.4 MB/s eta 0:00:00
Collecting sentry-sdk>=2.0.0
  Downloading sentry_sdk-2.60.0-py3-none-any.whl (475 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 475.6/475.6 KB 58.3 MB/s eta 0:00:00
Collecting tomli
  Downloading tomli-2.4.1-py3-none-any.whl (14 kB)
Collecting exceptiongroup>=1.0.2
  Downloading exceptiongroup-1.3.1-py3-none-any.whl (16 kB)
Collecting gitdb<5,>=4.0.1
  Downloading gitdb-4.0.12-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 KB 11.7 MB/s eta 0:00:00
Collecting typing-inspection>=0.4.2
  Downloading typing_inspection-0.4.2-py3-none-any.whl (14 kB)
Collecting annotated-types>=0.6.0
  Downloading annotated_types-0.7.0-py3-none-any.whl (13 kB)
Collecting pydantic-core==2.46.4
  Downloading pydantic_core-2.46.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 114.6 MB/s eta 0:00:00
Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (10.3.5.147)
Requirement already satisfied: filelock in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.29.0)
Requirement already satisfied: fsspec in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (2026.4.0)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127)
Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.5.8)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (2.21.5)
Requirement already satisfied: triton==3.1.0 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.1.0)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127)
Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (11.6.1.9)
Requirement already satisfied: jinja2 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.1.6)
Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.3.1.170)
Requirement already satisfied: sympy==1.13.1 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (1.13.1)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (9.1.0.70)
Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (11.2.1.3)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (12.4.127)
Requirement already satisfied: networkx in ./venv/lib/python3.10/site-packages (from torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.4.2)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in ./venv/lib/python3.10/site-packages (from sympy==1.13.1->torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (1.3.0)
Collecting smmap<6,>=3.0.1
  Downloading smmap-5.0.3-py3-none-any.whl (24 kB)
Requirement already satisfied: MarkupSafe>=2.0 in ./venv/lib/python3.10/site-packages (from jinja2->torch->lycoris-lora==3.0.1.dev10->-r ../requirements.txt (line 10)) (3.0.3)
Using legacy 'setup.py install' for lycoris-lora, since package 'wheel' is not installed.
Building wheels for collected packages: dadaptation
  Building wheel for dadaptation (pyproject.toml) ... done
  Created wheel for dadaptation: filename=dadaptation-3.2-py3-none-any.whl size=23290 sha256=b432ddb7ade49274eec2586f531f54f9336acc0719ef9ae4f6b5a6c5055bf8d2
  Stored in directory: /root/.cache/pip/wheels/d0/03/6d/feba04df15ef39d9ac4e3504058ac2a88fb2ef9183ba92b111
Successfully built dadaptation
Installing collected packages: wheel, websockets, uvloop, typing-inspection, tomli, smmap, sentry-sdk, scipy, python-dotenv, pyngrok, pydantic-core, platformdirs, httptools, h11, exceptiongroup, dadaptation, click, annotated-types, uvicorn, pydantic, pycloudflared, gitdb, anyio, watchfiles, starlette, gitpython, wandb, pytorch_optimizer, lycoris-lora, came-pytorch
  Running setup.py install for lycoris-lora ... done
Successfully installed annotated-types-0.7.0 anyio-4.13.0 came-pytorch-0.1.3 click-8.4.0 dadaptation-3.2 exceptiongroup-1.3.1 gitdb-4.0.12 gitpython-3.1.50 h11-0.16.0 httptools-0.7.1 lycoris-lora-3.0.1.dev10 platformdirs-4.9.6 pycloudflared-0.3.0 pydantic-2.13.4 pydantic-core-2.46.4 pyngrok-8.1.2 python-dotenv-1.2.2 pytorch_optimizer-3.1.2 scipy-1.15.3 sentry-sdk-2.60.0 smmap-5.0.3 starlette-1.0.0 tomli-2.4.1 typing-inspection-0.4.2 uvicorn-0.47.0 uvloop-0.22.1 wandb-0.27.0 watchfiles-1.2.0 websockets-16.0 wheel-0.47.0
completed installing
Found existing installation: rich 13.7.0
Uninstalling rich-13.7.0:
  Successfully uninstalled rich-13.7.0

✅ Installation finished in 330 seconds.

🔄 Getting model...

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
28731f|OK  |   176MiB/s|//content/downloaded_model.safetensors

Status Legend:
(OK):download completed.

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
af4d91|OK  |   239MiB/s|//content/sdxl_vae.safetensors

Status Legend:
(OK):download completed.


📄 Config saved to /content/drive/MyDrive/Loras/sed/training_config.toml
📄 Dataset config saved to /content/drive/MyDrive/Loras/sed/dataset_config.toml

⭐ Starting trainer...

rich is not installed, using basic logging
Loading settings from /content/drive/MyDrive/Loras/sed/training_config.toml...
/content/drive/MyDrive/Loras/sed/training_config
rich is not installed, using basic logging
tokenizer_config.json: 100% 905/905 [00:00<00:00, 7.81MB/s]
vocab.json: 961kB [00:00, 115MB/s]
merges.txt: 525kB [00:00, 144MB/s]
special_tokens_map.json: 100% 389/389 [00:00<00:00, 3.41MB/s]
tokenizer.json: 2.22MB [00:00, 197MB/s]
tokenizer_config.json: 100% 904/904 [00:00<00:00, 5.95MB/s]
vocab.json: 862kB [00:00, 157MB/s]
merges.txt: 525kB [00:00, 148MB/s]
special_tokens_map.json: 100% 389/389 [00:00<00:00, 3.48MB/s]
tokenizer.json: 2.22MB [00:00, 191MB/s]
Loading dataset config from /content/drive/MyDrive/Loras/sed/dataset_config.toml
prepare images.
get image size from name of cache files
100% 404/404 [00:00<00:00, 320927.81it/s]
set image size from cache files: 0/404
found directory /content/drive/MyDrive/Loras/sed/dataset contains 404 image files
read caption:  75% 304/404 [00:53<00:01, 54.88it/s]neither caption file nor class tokens are found. use empty caption for /content/drive/MyDrive/Loras/sed/dataset/309.jpeg / キャプションファイルもclass tokenも見つかりませんでした。空のキャプションを使用します: /content/drive/MyDrive/Loras/sed/dataset/309.jpeg
read caption: 100% 404/404 [00:53<00:00,  7.56it/s] 
No caption file found for 1 images. Training will continue without captions for these images. If class token exists, it will be used. / 1枚の画像にキャプションファイルが見つかりませんでした。これらの画像についてはキャプションなしで学習を続行します。class tokenが存在する場合はそれを使います。
/content/drive/MyDrive/Loras/sed/dataset/309.jpeg
808 train images with repeating.
0 reg images.
no regularization images / 正則化画像が見つかりませんでした
[Dataset 0]
  batch_size: 4
  resolution: (1024, 1024)
  enable_bucket: True
  network_multiplier: 1.0
  min_bucket_reso: 256
  max_bucket_reso: 4096
  bucket_reso_steps: 64
  bucket_no_upscale: False

  [Subset 0 of Dataset 0]
    image_dir: "/content/drive/MyDrive/Loras/sed/dataset"
    image_count: 404
    num_repeats: 2
    shuffle_caption: True
    keep_tokens: 1
    keep_tokens_separator: 
    caption_separator: ,
    secondary_separator: None
    enable_wildcard: False
    caption_dropout_rate: 0.0
    caption_dropout_every_n_epochs: 0
    caption_tag_dropout_rate: 0.0
    caption_prefix: None
    caption_suffix: None
    color_aug: False
    flip_aug: False
    face_crop_aug_range: None
    random_crop: False
    token_warmup_min: 1
    token_warmup_step: 0
    alpha_mask: False
    custom_attributes: {}
    is_reg: False
    class_tokens: None
    caption_extension: .txt


[Dataset 0]
loading image sizes.
100% 404/404 [00:01<00:00, 354.86it/s]
make buckets
number of images (including repeats) / 各bucketの画像枚数（繰り返し回数を含む）
bucket 0: resolution (832, 1216), count: 2
bucket 1: resolution (1024, 1024), count: 804
bucket 2: resolution (1152, 896), count: 2
mean ar error (without repeats): 7.977811976167784e-05
preparing accelerator
accelerator device: cuda
loading model for process 0/1
load StableDiffusion checkpoint: /content/downloaded_model.safetensors
building U-Net
loading U-Net from checkpoint
U-Net: <All keys matched successfully>
building text encoders
loading text encoders from checkpoint
text encoder 1: <All keys matched successfully>
text encoder 2: <All keys matched successfully>
building VAE
loading VAE from checkpoint
VAE: <All keys matched successfully>
load VAE: /content/sdxl_vae.safetensors
additional VAE loaded
Enable SDPA for U-Net
import network module: networks.lora
[Dataset 0]
caching latents with caching strategy.
caching latents...
100% 404/404 [10:18<00:00,  1.53s/it]
create LoRA network. base dim (rank): 32, alpha: 16
neuron dropout: p=None, rank dropout: p=None, module dropout: p=None
apply LoRA to Conv2d with kernel size (3,3). dim (rank): 32, alpha: 16.0
create LoRA for Text Encoder 1:
create LoRA for Text Encoder 2:
create LoRA for Text Encoder: 88 modules.
create LoRA for U-Net: 788 modules.
enable LoRA for text encoder: 88 modules
enable LoRA for U-Net: 788 modules
prepare optimizer, data loader etc.
use 8-bit AdamW optimizer | {'weight_decay': 0.1, 'betas': [0.9, 0.99]}
override steps. steps for 20 epochs is / 指定エポックまでのステップ数: 4060
enable full fp16 training.
running training / 学習開始
  num train images * repeats / 学習画像の数×繰り返し回数: 808
  num reg images / 正則化画像の数: 0
  num batches per epoch / 1epochのバッチ数: 203
  num epochs / epoch数: 20
  batch size per device / バッチサイズ: 4
  gradient accumulation steps / 勾配を合計するステップ数 = 1
  total optimization steps / 学習ステップ数: 4060
steps:   0% 0/4060 [00:00<?, ?it/s]unet dtype: torch.float16, device: cuda:0
text_encoder [0] dtype: torch.float16, device: cuda:0
text_encoder [1] dtype: torch.float16, device: cuda:0

epoch 1/20
epoch is incremented. current_epoch: 0, epoch: 1
steps:   5% 203/4060 [34:57<11:04:16, 10.33s/it, avr_loss=0.0936]
saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-01.safetensors

epoch 2/20
epoch is incremented. current_epoch: 1, epoch: 2
steps:  10% 406/4060 [1:09:54<10:29:08, 10.33s/it, avr_loss=0.0929]
saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-02.safetensors

epoch 3/20
epoch is incremented. current_epoch: 2, epoch: 3
steps:  15% 609/4060 [1:44:46<9:53:45, 10.32s/it, avr_loss=0.101]
saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-03.safetensors

epoch 4/20
epoch is incremented. current_epoch: 3, epoch: 4
steps:  20% 812/4060 [2:19:48<9:19:12, 10.33s/it, avr_loss=0.103]
saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-04.safetensors

epoch 5/20
epoch is incremented. current_epoch: 4, epoch: 5
steps:  25% 1015/4060 [2:54:49<8:44:28, 10.33s/it, avr_loss=0.089]
saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-05.safetensors

epoch 6/20
epoch is incremented. current_epoch: 5, epoch: 6
steps:  30% 1218/4060 [3:29:52<8:09:42, 10.34s/it, avr_loss=0.0877]
saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-06.safetensors

epoch 7/20
epoch is incremented. current_epoch: 6, epoch: 7
steps:  35% 1421/4060 [4:04:52<7:34:45, 10.34s/it, avr_loss=0.0962]
saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-07.safetensors

epoch 8/20
epoch is incremented. current_epoch: 7, epoch: 8
steps:  40% 1624/4060 [4:39:53<6:59:49, 10.34s/it, avr_loss=0.0971]
saving checkpoint: /content/drive/MyDrive/Loras/sed/output/sed-08.safetensors

epoch 9/20
epoch is incremented. current_epoch: 8, epoch: 9
steps:  43% 1756/4060 [5:02:40<6:37:08, 10.34s/it, avr_loss=0.0931]