Others

Update, May 31: added v1.1 with nested directory support, hopefully faster conversion, and general fixes. See the version notes for all changes and a known issue with reproducible checksums.

I wanted an easy way to convert .pt (PyTorch/PickleTensors) and .bin files for Textual Inversions and VAEs to the Safetensors format. DiffusionDalmation on GitHub has a Jupyter/Colab notebook (MIT license) that handled .pt files but not .bin files because of missing data in the .bin files that I had. Hugging Face has a function in the Safetensors repo (Apache license) that handles .bin, and probably .pt but I liked the training metadata details from the notebook version.

WARNING: code within files will be executed when the models are loaded - any malicious code will be executed, too. Do not run this on your own machine with untrusted/unscanned files containing pickle imports.

I started with pieces of both scripts and reworked it into a script that will try to convert both types as individual files or a directory of files. The .safetensors gets a new file hash but they are functionally identical from my testing. I have only tested on PyTorch 2 with SD1.5 TIs and VAEs though. It works on Windows with CPU or CUDA, but has theoretical support for the MacOS Metal backend and will fall back to using CPU. Buy me a Mac and I'll test it there. ;P

Assuming that you're in a trusted environment converting models you trust, you can activate an existing venv and run it from there, or set up a new venv with requirements.txt - the convert.bat script should handle this for you on Windows.

Using convert.bat:

V:\sd-info\safetensors-converter\convert.bat .

Or reusing an existing venv from the automatic1111 web UI and running the script directly:

V:\stable-diffusion-webui\venv\Scripts\activate
python V:\sd-info\safetensors-converter\bin-pt_to_safetensors.py .
deactivate

• You can pass '.' in as the content_path to convert anything in the current directory, or provide the full path to a file or directory. It now recurses through subdirectories and convert what it finds.

• If you get an error on a specific file, it may just have the wrong extension, e.g. try renaming .bin to .pt or the other way around (my .pt VAEs needed to be named .bin).

• Safetensors metadata will be added detailing the original format and shape of the tensor (vectors, dimensions).

• The .pt conversion will display and save metadata about the training model, hash, and steps, when available.

• The .bin conversion does not provide those extra details, the format seems to lack that data.

• Post-conversion, the script will check file sizes and compare that the output tensors match the original. It will throw an error if the file has changed too much or there's a mismatch.

• The script should halt whenever there's an error, and will overwrite any existing .safetensors files with the same base name as the original file.

Before/after file size and hashes for a few example TIs:

$ ls -lGR
.:
total 0
drwxr-xr-x 1 user 0 May 31 01:20 negatives/
drwxr-xr-x 1 user 0 May 31 01:20 nobodies/
drwxr-xr-x 1 user 0 May 31 01:20 sliders/

./negatives:
total 456
-rw-r--r-- 1 user 231339 Apr 30 19:59 ng_deepnegative_v1_75t.pt
-rw-r--r-- 1 user 230672 May 31 01:20 ng_deepnegative_v1_75t.safetensors

./nobodies:
total 8
-rw-r--r-- 1 user 3931 May 16 06:21 LulaCipher.bin
-rw-r--r-- 1 user 3232 May 31 01:20 LulaCipher.safetensors

./sliders:
total 104
-rw-r--r-- 1 user 50036 May 20 10:35 AS-MidAged.pt
-rw-r--r-- 1 user 49328 May 31 01:20 AS-MidAged.safetensors

$ for i in */*; do sha256sum ${i}; done
54e7e4826d53949a3d0dde40aea023b1e456a618c608a7630e3999fd38f93245 *negatives/ng_deepnegative_v1_75t.pt
4cc4143ba4ca9d534dfaaad6027a130e9ba2c3e2de186225b953f11bf4077af1 *negatives/ng_deepnegative_v1_75t.safetensors
433c565251ac13398000595032c436eb361634e80e581497d116f224083eb468 *nobodies/LulaCipher.bin
b96963cb3b7106f544926606969e1289662bac088ceea6abd58714d0530486bc *nobodies/LulaCipher.safetensors
d9a9546a597ad34497d4a5a24624478df056b5a9426a1934efdbfd65177b120d *sliders/AS-MidAged.pt
459d6f71b6b64653e018e6791b2c43e6af7fb4794b9f457625547f35e513b809 *sliders/AS-MidAged.safetensors