This is a readme file designed to provide a simple guide on how to use AingDiffusion models effectively.
AingDiffusion (pronounced as "Ah-eeng Diffusion") is a combination of various anime models. It is a powerful model that can generate high-quality anime images. The term "aing" is derived from the informal Sundanese language spoken in West Java, Indonesia. It translates to "I" or "my." The name signifies that this model produces images that align with my personal taste.
This tutorial can be applied to other models as well and is not specific to AingDiffusion. Before proceeding, please ensure that you have the latest version of AUTOMATIC1111's Stable Diffusion WebUI installed. To begin, follow these steps:
Save the desired model to the models
directory, located at [YOUR_SDWEBUI_DIR]\models\Stable-diffusion
. For example, if you are using the Stable Diffusion model, save it to [YOUR_SDWEBUI_DIR]\models\Stable-diffusion
.
Save the VAE (Variational Autoencoder) in the [YOUR_SDWEBUI_DIR]\models\VAE
directory.
After launching the WebUI, navigate to the Settings tab.
Once in the settings tab, click on Stable Diffusion
on the left side of the page. This will take you to the Stable Diffusion settings. Scroll down and locate the setting labeled "Clip Skip." Set its value to 2
.
On the same page, you will find a setting called SD VAE
. Select the VAE you downloaded from AingDiffusion. If the VAE does not appear in the list, click the Refresh
button next to the list box to update the VAE list.
Switch to the "Sampler Parameters" tab. On the left side of the page, click on Sampler Parameters
. Scroll down until you find the setting labeled "Eta noise seed delta." Set its value to 31337
. This number originated from the leaked NovelAI model and is still commonly used.
On the same page, locate the setting labeled "Eta (noise multiplier) for ancestral samplers." Change its value to 0.667
.
This setting will improve the performance of ancestral samplers such as Euler a.
Please note that this guide does not provide an exhaustive list of settings for optimizing SD-WebUI for image generation. There are many other potential improvements that can be made by adjusting the settings. Please feel free to explore and experiment with different configurations to achieve the desired results.
So, you're wondering about how to create pseudo-3D parallax gifs. Luckily, it's very easy!
I'll be specifically going about using this on existing images, as you're probably not trying to animate every single thing your SD generates, although it would be possible with Option 2.
Although I'll be mainly talking about generated images, this exact process works with just about any picture.
Attached are the two images I've showcased for you to play around with, I'd just ask for you not to re-upload them. Otherwise, generation data is available in the still (non-animated) images posted.
Great results in seconds, external tool
Option number 1 is using an external tool, the LeiaPix Converter
. It can be found here.
You have to create an account on the site, but the tool is (currently) completely free to use. All you have to do is drop your image into the site after making an account and wait for the processing to complete. You can then play around with the options to the left, which are all completely self-explanatory.
Under the "Depth Map" tab to the far left you can override the automatically generated depth map, which is quite finicky.
Takes a few minutes to run and results are slightly less good, but runs locally
Note: this should work with Collab, but I have absolutely no experience with that so this guide focuses on local installations of SD using AUTOMATIC1111s WebUI.
If you're like me and don't want to rely on external tools which may or not stay free to use, you can use an extension for the WebUI called Depthmap Script
.
In your WebUI, open the the extensions tab, head to Install from URL
and paste the following into the first input field: https://github.com/thygate/stable-diffusion-webui-depthmap-script.git
Then just hit Install
, wait for the process to complete (have an eye on the output of your Terminal window running the .bat
-Script - if you're on Windows), then just head back to the Installed
tab and hit Apply and restart UI
.
After reloading, there is a new Tab in your WebUI, labeled Depth
.
On the left side, select the Single Image tab and drop your picture into the box.
Choose a model to generate your depth map, I prefer zoedepth
in the indoor and outdoor variety.
Uncheck BOOST
if you don't have a lot of VRAM. BOOST
gives better results, but is very VRAM intensive and couldn't handle 512px² on my 8GB VRAM card. You can try; watch your terminal output for VRAM errors.
If you you've unchecked BOOST
, set the resolution. If you're working with smaller pictures (eg. 512px²), select Match input size
. If you're getting VRAM issues, uncheck Match input size
and set the resolution manually. Match the aspect ratio of your input.
Uncheck Save DepthMap
, check Generate 3D inpainted mesh
(and optionally Generate 4 demo videos
, although I found those not to be very helpful).
Hit the Generate
-Button under the input image, not to the right.
Monitor your terminal output. The script will download your chosen model, than do the depth-analysis. On my somewhat okay graphics card, this takes around 3-5 minutes for 512px² images.
After the process is complete, head to the Generate video
tab on the right. The input mesh should automatically fill, otherwise you'll have to point it to the path of the generated mesh. In my case (Windows, custom saving folders), this would for example be outputs/extras-images\depthmap-0004.obj
.
Now you can start playing around with the settings! For noticeable-but-subtle effects, I use 300 frames, 24fps, circle trajectory, -0.004, 0.0, -0.005 translation, 0.03, 0.03, 0.05, 0.03 cropping, no dolly
. For faster animations, you could lower the number of frames or raise the framerate; for more noticeable movements increase the translation amounts. I'd advise against large translation motions however, because the inpainted corner will become extremely obvious.
Hit Generate Video
. It takes just a few seconds (depending on frame count), so trial-and-error parameters are feasible. The output is in your extras
folder.
Note:
After you've got a video you're happy with, you can safely clean up the depth map and mesh from your extras
folder.
You can automatically create those meshes during render in txt2img and img2img using the scripts
dropdown. However, this will create a depth map every time you render something, which really is just wasted time and electricity in my opinion as opposed to doing it once with the one image you actually want to animate.
Have fun bringing your GPUs creations to life!